feat: added robots.txt file for testing purposes
All checks were successful
continuous-integration/drone/push Build is passing

This commit is contained in:
surtur 2020-07-08 15:02:41 +02:00
parent d8d9948982
commit 45fc12c2f2
Signed by: wanderer
GPG Key ID: 19CE1EC1D9E0486D

10
static/robots.txt Normal file
View File

@ -0,0 +1,10 @@
User-agent: AdsBot*
Disallow: /
User-agent: Mediapartners*
Disallow: /
User-Agent: *
Disallow: /404.html
Disallow: /categories/
Sitemap: https://dotya.ml/sitemap.xml