mirror of
https://github.com/crawler-commons/crawler-commons
synced 2024-09-23 17:33:23 +02:00
0c75e75496
* Add main to SimpleRobotRulesParser for testing - implement toString() for robot rules - fix line breaks in comments * Do not detect MIME type as Tika dependency has been removed |
||
---|---|---|
.. | ||
main | ||
test |