# Uses of Class crawlercommons.robots.BaseRobotRules (Crawler-commons 1.4 API)

## Uses of Class
crawlercommons.robots.BaseRobotRules

Packages that use BaseRobotRules
crawlercommons.robots: The robots package contains all of the robots.txt rule inference, parsing and utilities contained within Crawler Commons.

### Uses of BaseRobotRules in crawlercommons.robots

Subclasses of BaseRobotRules in crawlercommons.robots
`class` `SimpleRobotRules` Result from parsing a single robots.txt file – a set of allow/disallow rules to check whether a given URL is allowed, and optionally a Crawl-delay and Sitemap URLs.

Methods in crawlercommons.robots that return BaseRobotRules
`abstract BaseRobotRules` BaseRobotsParser.`failedFetch​(int httpStatusCode)` The fetch of robots.txt failed, so return rules appropriate for the given HTTP status code.
`abstract BaseRobotRules` BaseRobotsParser.`parseContent​(String url, byte[] content, String contentType, String robotNames)` Deprecated. since 1.4 - replaced by `BaseRobotsParser.parseContent(java.lang.String,byte[],java.lang.String,java.util.Collection<java.lang.String>)`.
`abstract BaseRobotRules` BaseRobotsParser.`parseContent​(String url, byte[] content, String contentType, Collection<String> robotNames)` Parse the robots.txt file in content, and return rules appropriate for processing paths by userAgent.