Block Archive.org in nginxI just learned that Archive.org does not honor robots.txt, so I blocked them in nginx.conf
as follows:
if ($http_user_agent ~ (ia_archiver|archive\.org)) {
return 403;
}
I just learned that Archive.org does not honor robots.txt, so I blocked them in nginx.conf
as follows:
if ($http_user_agent ~ (ia_archiver|archive\.org)) {
return 403;
}