fix: Disallow bots from executing iddiff (#147)

ietf-tools · Jan 9, 2023 · 56b2e80 · 56b2e80
1 parent e9b8b99
commit 56b2e80
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/www/robots.txt b/www/robots.txt
@@ -1,8 +1,10 @@
 User-agent: *
 Allow: /
+Allow: /iddiff$
 Disallow: /api/
 Disallow: /api2/
 Disallow: /diff
+Disallow: /iddiff?*
 
 Sitemap: https://author-tools.ietf.org/sitemap.xml