robots.txt update: fix warnings, disallow /news;
This commit is contained in:
tmshv committed
1 parent
da79a17374
commit
534ce3f2e3
1 file changed
+1
-3
+1
-3
@@ -1,7 +1,5 @@
|
||||
User-agent: *
|
||||
Disallow:
|
||||
Disallow: /news
|
||||
Allow: /
|
||||
Host: art.shlisselburg.org
|
||||
Sitemap: http://art.shlisselburg.org/sitemap.xml
|
||||
Crawl-delay: 3
|
||||
Crawl-rate: 1/3
|
||||
Reference in new issue
Block a user