# Overview (Crawler-commons 1.0 API)
Date: 2019-03-19

# Crawler-commons 1.0 API

Packages
crawlercommons
crawlercommons.domains: Classes contained within the domains package relate to the definition of Top Level Domain's, various domain registrars and the effective handling of such domains.
crawlercommons.filters: The filters package contains code and resources for URL filtering.
crawlercommons.filters.basic
crawlercommons.mimetypes
crawlercommons.robots: The robots package contains all of the robots.txt rule inference, parsing and utilities contained within Crawler Commons.
crawlercommons.sitemaps: Sitemaps package provides all classes relevant to focused sitemap parsing, url definition and processing.
crawlercommons.sitemaps.extension
crawlercommons.sitemaps.sax
crawlercommons.sitemaps.sax.extension