Christian Heusel f562b4e927 aurweb: Create robots.txt ... This should i.e. forbid crawlers to index all of the git diffs which put's unneccessary load on the server and is not really of benefit to be indexed anyways. Link: https://gitlab.archlinux.org/archlinux/infrastructure/-/issues/610 Reviewed-by: Sven-Hendrik Haase <svenstaro@gmail.com> Reviewed-by: Levente Polyak <anthraxx@archlinux.org> Signed-off-by: Christian Heusel <christian@heusel.eu>		2024-08-03 00:50:14 +02:00
..
main.yml	aurweb: Create robots.txt	2024-08-03 00:50:14 +02:00