Sebastian Nagel 871e4e61d2 Merge pull request #430 from sebastian-nagel/cc-390-114-robots-closing-rule-group ... [Robots.txt] Close groups of rules as defined in RFC 9309		2023-07-12 10:35:48 +02:00
..
domains	EffectiveTldFinder to log loading of public suffix list, fixes #284	2020-02-17 16:41:25 +01:00
filters	[Robots.txt] Path analyse bug with url-decode if allow/disallow path contains escaped wild-card characters	2023-05-12 14:19:35 +02:00
mimetypes	Minor changes + applied formatting pre 0.10 release	2018-06-05 11:33:27 +01:00
robots	Merge pull request #430 from sebastian-nagel/cc-390-114-robots-closing-rule-group	2023-07-12 10:35:48 +02:00
sitemaps	[Sitemaps] Disable support for DTDs in sitemaps by default	2022-03-02 16:03:13 +01:00
utils	Query parameters normalization	2021-09-21 12:02:00 +02:00
CrawlerCommons.java	Fix license headers + applied formatting. Fixes #108	2016-06-30 11:45:08 +01:00