Anubis is an open-source tool developed by the Canadian TecharoHQ team that specializes in protecting websites from AI crawlers. The core mechanism of the tool is to add SHA256 proof-of-work (Proof-of-Work) challenges to HTTP requests, requiring visitors to complete specific computational tasks before gaining access.
Its main role is reflected in three areas:
- Protect against AI crawlers: Blocking automated crawler access through computationally intensive tasks
- Saving server resources: Prevent low-value crawler requests from consuming server performance
- content protection: for sites that want to limit content to search engines
The tool is written in Go, supports Docker deployment, and has been open sourced on GitHub. The design is inspired by the response to the abuse of AI crawlers that ignore the robots.txt protocol, which is called a "nuclear" solution by developers.
This answer comes from the articleAnubis: Interfering with AI Crawler Crawling by Proof of WorkloadThe
May not be reproduced without permission:AI productivity tools " What is Anubis and what is its main function?
English 
简体中文 
日本語 
Deutsch 
Português do Brasil