# Uses of Class crawlercommons.robots.BaseRobotRules (Crawler-commons 0.10 API)
Date: 2018-06-05

## Uses of Class
crawlercommons.robots.BaseRobotRules

Packages that use BaseRobotRules
crawlercommons.robots: The robots package contains all of the robots.txt rule inference, parsing and utilities contained within Crawler Commons.

### Uses of BaseRobotRules in crawlercommons.robots

Subclasses of BaseRobotRules in crawlercommons.robots
`class` `SimpleRobotRules` Result from parsing a single robots.txt file - which means we get a set of rules, and a crawl-delay.

Methods in crawlercommons.robots that return BaseRobotRules
`abstract BaseRobotRules` BaseRobotsParser.`failedFetch(int httpStatusCode)` The fetch of robots.txt failed, so return rules appropriate give the HTTP status code.
`abstract BaseRobotRules` BaseRobotsParser.`parseContent(String url, byte[] content, String contentType, String robotNames)` Parse the robots.txt file in content, and return rules appropriate for processing paths by userAgent.