1
1
mirror of https://gitlab.archlinux.org/archlinux/infrastructure.git synced 2025-01-18 08:06:16 +01:00
infrastructure/roles/archwiki/files/robots.txt
Jakub Klinkovský f276e10d67
archwiki: update robots.txt
- The /index.php? rule disallows all URLs with this prefix, so using
  longer prefixes does not make sense.
- Allow indexing of the Help: namespace which contains relevant text.
- Keep the intention of disallowing special namespaces, including the
  File: alias for Image: and using the /title/ path and colon separator
  for the namespace prefix.
2024-01-03 13:26:43 +01:00

9 lines
182 B
Plaintext

User-agent: *
Disallow: /index.php?
Disallow: /skins/
Disallow: /title/File:
Disallow: /title/Image:
Disallow: /title/MediaWiki:
Disallow: /title/Special:
Disallow: /title/Template: