We want to roll out HTTP/3 slowly, so this adds the necessary plumbing
and makes it possible to enable it per host.
Instead of adding the conditional logic to each nginx template, the 443
listen config is moved out into a snippet which is managed by the nginx
role.
HTTP/3 uses QUIC which is built on UDP. UDP is connectionless and
therefore reuseport[1][2] must be used to ensure that UDP packets for
the same QUIC connection is directed to the same worker. reuseport can
only be enabled once, so a default_server is added to the
"inventory_hostname vhost" for SSL/QUIC (reuseport is only enabled for
the latter). ssl_reject_handshake[3] is enabled as that allows enabling
SSL/QUIC without specifying a certificate.
[1] https://nginx.org/en/docs/http/ngx_http_core_module.html#listen
[2] https://lwn.net/Articles/542629/
[3] http://nginx.org/en/docs/http/ngx_http_ssl_module.html#ssl_reject_handshake
Ref #606
This role will still handle setting up nginx and rsyncd, due to specific
configuration requirements these services have.
We're also effectively relieving build.archlinux.org of rsyncd duties as
it is not something it should be doing anyway.
These are used to signal the start of the document in a stream of many
documents. As Ansible only supports one YAML document per file this is
unnecessary. About a third of our YAML documents already lacked these.
We had a GeoIP mirror in the past based on nginx and its GeoIP module,
but it didn't perform very well, due to the high latency (asking a
central server for the package and then redirected to the closest
mirror).
One of the reasons for offering this service, is so we can relieve
mirror.pkgbuild.com which is burning a ton of traffic (50TB/month),
likely due to it being the default mirror in our Docker image. Another
reason is so we can offer a link to our arch-boxes images in libosinfo
(used by gnome-boxes, virt-install and virt-manager), with good enough
performance for most users.
This time we take a different approach and use a DNS based solution,
which means the latency penalty is only paid once (the first DNS
request). The downside is that the mirrors must have a valid certificate
for the same domain name, which makes using third-party mirrors a
challenge. So for now, we are just using the sponsored mirorrs
controlled by the DevOps team.
Fix #101
The sponsored mirrors have a ton of storage, but mirror.pkgbuild.com
doesn't, so debug packages aren't synced to it.
[1] {america,asia,europe}.mirror.pkgbuild.com
Using a temporary directory outside of /srv/ftp was meant to protect
against incomplete files from being synced by downstream mirrors. It
does not achieve this to much effect though; each file gets uploaded
to the temporary directory but then immediately moved under a .~tmp~
directory at its target location (.~tmp~ because of --delay-updates,
otherwise the file would be renamed to its final path).
The `--delay-updates` option by itself sufficiently protects against
temp files being transferred to downstream mirrors; when used by the
receiver, it automatically adds an exclude rule for ~.tmp~, behaving
exactly like we want it to. As such, the `--temp-dir` option doesn't
provide any further benefit and can be removed.
- Replace --delete-after with more efficient --delete-delay.
- Move "-p" together with the other short options.
- Remove reference to empty ${VERBOSE} variable.
This reverts commit 75f9ca3cc64df0653a4541be5e2334244fe24ca5.
This should be fixed in rsync versions newer than 3.2.3. In Arch the fix
has been shipped in the rsync 3.2.3-4 package, which our own mirrors now
have been updated to.
[1] https://github.com/WayneD/rsync/issues/192
A extra access_log entry was added with the following commands:
$ cd roles
$ grep -lr access_log | xargs -P 1 -n 1 sed -i '/access_log/ s/\(.*\)\( \)\(\(reduced\|main\);$\)/\1 \3\n\1.json json_\3/'
yaml: truthy value should be one of [false, true] (truthy)
yaml: wrong indentation: expected 4 but found 2 (indentation)
yaml: too few spaces before comment (comments)
yaml: missing starting space in comment (comments)
yaml: too many blank lines (1 > 0) (empty-lines)
yaml: too many spaces after colon (colons)
yaml: comment not indented like content (comments-indentation)
yaml: no new line character at the end of file (new-line-at-end-of-file)
load-failure: Failed to load or parse file
parser-error: couldn't resolve module/action 'hosts'. This often indicates a misspelling, missing collection, or incorrect module path.
It should make it easier to change how the certificates is issued.
Ex: If we want to switch to ECDSA certificates in the future or replace
certbot with something else.
The script is currently hosted on my private server, but still linked in
the wiki[1]. This way, we can update the wiki to link to this copy of
the script as an official source.
[1] https://wiki.archlinux.org/index.php/DeveloperWiki:NewMirrors
Signed-off-by: Florian Pritz <bluewind@xinu.at>
The discovery script now uses a regex and no longer cares where exactly
accounting is enabled. Follow systemd upstream by enabling it by
default.
Signed-off-by: Florian Pritz <bluewind@xinu.at>
For proxy/fastcgi/uwsgi blocks, logging is still set to the old format,
but for everything else (= static data) a reduced format is used that
excludes items that no longer make sense (request_time, remote_user) and
those that are personal information all the time (remote_addr, http_x_forwarded_for).
Signed-off-by: Florian Pritz <bluewind@xinu.at>
Disable firewall because python2 module is not avaible and the tasks
fail which makes the playbooks fail and leads to handlers not being run.
https://github.com/ansible/ansible/issues/24855
Signed-off-by: Florian Pritz <bluewind@xinu.at>
