Anubis-based solutions
For server load issues caused by AI crawler abuse, SHA256 proof-of-workload protection for Anubis can be implemented with the following steps:
- Deployment Architecture: Before deploying Anubis as a reverse proxy on a web server, all requests need to pass PoW authentication first.
- Calculation parameter configuration: The default nonce value (16 bits) can be adjusted, the higher the value the stronger the protection but may affect the user experience.
- Log Monitoring: By
docker logsAnalyze blocking data to optimize protection strategies
Alternatives include:
- Cloudflare's Bot Management Service (for scenarios requiring SEO)
- Nginx flow limiting module (requires manual configuration of IP rules)
This answer comes from the articleAnubis: Interfering with AI Crawler Crawling by Proof of WorkloadThe
May not be reproduced without permission:AI productivity tools " How to solve the problem of excessive server resource consumption of a website due to AI crawlers?
English 
简体中文 
日本語 
Deutsch 
Português do Brasil