mirror/crawler-commons
1
0
Fork 0
mirror of https://github.com/crawler-commons/crawler-commons synced 2024-09-27 02:23:09 +02:00
Code Issues
e16d022dea
crawler-commons/src
History
Sebastian Nagel e16d022dea [Robots] Maximum values (crawl-delay and warnings): document and make visible, 
fixes #276
- change visibility (private -> public) of DEFAULT_MAX_CRAWL_DELAY and DEFAULT_MAX_WARNINGS
- document methods to set/get max crawl delay and max warnings
2020-07-15 17:47:32 +02:00
..
main [Robots] Maximum values (crawl-delay and warnings): document and make visible, 2020-07-15 17:47:32 +02:00
test Improvements to BasicURLNormalizer (#292) 2020-06-22 13:51:39 +01:00