Explorar o código

Added Some constraints to robots file

Yoginth %!s(int64=7) %!d(string=hai) anos
pai
achega
7f6dbf3926
Modificáronse 1 ficheiros con 6 adicións e 1 borrados
  1. 6 1
      public/robots.txt

+ 6 - 1
public/robots.txt

@@ -8,14 +8,19 @@ Disallow: /admin
 Disallow: /*/*/forks
 Disallow: /*/*/stars
 Disallow: /*/*/pulls
-Disallow: /*/*/archive/
+Disallow: /*/*/archive
+Disallow: /*/*/settings
 Disallow: /*/*/raw/*
+Disallow: /*/*/wiki/_new
+Disallow: /*/*/wiki/*/_edit
 Disallow: /*/*/commits
 Disallow: /*/followers
 Disallow: /*/following
 Disallow: /*/*/src/*
+Disallow: /*/*/_edit
 Disallow: /*/*/issues/new
 Disallow: /*/*/issues?
+Disallow: /*/*/compare
 Disallow: /*?
 
 Sitemap: https://gitote.in/sitemap.xml