- get(String) - Method in class crawlercommons.fetcher.BaseFetcher
-
- get(String, Payload) - Method in class crawlercommons.fetcher.BaseFetcher
-
Get the content stored in the resource referenced by
- get(String, Payload) - Method in class crawlercommons.fetcher.file.SimpleFileFetcher
-
- get(String, Payload) - Method in class crawlercommons.fetcher.http.SimpleHttpFetcher
-
- get(Object) - Method in class crawlercommons.fetcher.Payload
-
- getAbortReason() - Method in exception crawlercommons.fetcher.AbortedFetchException
-
- getAcceptLanguage() - Method in class crawlercommons.fetcher.http.BaseHttpFetcher
-
- getAgentName() - Method in class crawlercommons.fetcher.http.UserAgent
-
Obtain the just the user agent name
- getAssignedDomain(String) - Static method in class crawlercommons.url.EffectiveTldFinder
-
This method uses the effective TLD to determine which component of a FQDN
is the NIC-assigned domain name.
- getBaseUrl() - Method in class crawlercommons.fetcher.FetchedResult
-
- getBaseUrl() - Method in class crawlercommons.sitemaps.SiteMap
-
- getCause() - Method in exception crawlercommons.fetcher.BaseFetchException
-
- getChangeFrequency() - Method in class crawlercommons.sitemaps.SiteMapURL
-
Return the URL's change frequency
- getConnectionTimeout() - Method in class crawlercommons.fetcher.http.SimpleHttpFetcher
-
- getContent() - Method in class crawlercommons.fetcher.FetchedResult
-
- getContentLength() - Method in class crawlercommons.fetcher.FetchedResult
-
- getContentType() - Method in class crawlercommons.fetcher.FetchedResult
-
- getCookies() - Method in class crawlercommons.fetcher.http.LocalCookieStore
-
Returns an immutable array of cookies
that this HTTP state
currently contains.
- getCrawlDelay() - Method in class crawlercommons.robots.BaseRobotRules
-
- getDefaultMaxContentSize() - Method in class crawlercommons.fetcher.BaseFetcher
-
- getDomain() - Method in class crawlercommons.url.EffectiveTldFinder.EffectiveTLD
-
- getEffectiveTLD(String) - Static method in class crawlercommons.url.EffectiveTldFinder
-
- getEffectiveTLDs() - Static method in class crawlercommons.url.EffectiveTldFinder
-
- getError() - Method in exception crawlercommons.sitemaps.UnknownFormatException
-
public method, callable by exception catcher.
- getExpanded() - Method in class crawlercommons.fetcher.EncodingUtils.ExpandedResult
-
- getFetchedUrl() - Method in class crawlercommons.fetcher.FetchedResult
-
- getFetchTime() - Method in class crawlercommons.fetcher.FetchedResult
-
- getFullDateFormat() - Static method in class crawlercommons.sitemaps.AbstractSiteMap
-
- getHeaders() - Method in class crawlercommons.fetcher.FetchedResult
-
- getHostAddress() - Method in class crawlercommons.fetcher.FetchedResult
-
- getHttpHeaders() - Method in exception crawlercommons.fetcher.HttpFetchException
-
- getHttpStatus() - Method in exception crawlercommons.fetcher.HttpFetchException
-
- getHttpVersion() - Method in class crawlercommons.fetcher.http.SimpleHttpFetcher
-
- getInstance() - Static method in class crawlercommons.url.EffectiveTldFinder
-
- getKeepAliveDuration(HttpResponse, HttpContext) - Method in class crawlercommons.fetcher.http.SimpleHttpFetcher.MyConnectionKeepAliveStrategy
-
- getLastModified() - Method in class crawlercommons.sitemaps.AbstractSiteMap
-
- getLastModified() - Method in class crawlercommons.sitemaps.SiteMapURL
-
Return when this URL was last modified.
- getLocalizedMessage() - Method in exception crawlercommons.fetcher.BaseFetchException
-
- getMaxConnectionsPerHost() - Method in class crawlercommons.fetcher.http.BaseHttpFetcher
-
- getMaxContentSize(String) - Method in class crawlercommons.fetcher.BaseFetcher
-
- getMaxFetchTime() - Static method in class crawlercommons.robots.RobotUtils
-
- getMaxRedirects() - Method in class crawlercommons.fetcher.http.BaseHttpFetcher
-
- getMaxRetryCount() - Method in class crawlercommons.fetcher.http.SimpleHttpFetcher
-
- getMaxThreads() - Method in class crawlercommons.fetcher.http.BaseHttpFetcher
-
- getMessage() - Method in exception crawlercommons.fetcher.BaseFetchException
-
- getMessage() - Method in exception crawlercommons.fetcher.HttpFetchException
-
- getMimeTypeFromContentType(String) - Static method in class crawlercommons.fetcher.BaseFetcher
-
- getMinResponseRate() - Method in class crawlercommons.fetcher.http.BaseHttpFetcher
-
Return the minimum response rate.
- getNewBaseUrl() - Method in class crawlercommons.fetcher.FetchedResult
-
- getNumRedirects() - Method in class crawlercommons.fetcher.FetchedResult
-
- getNumWarnings() - Method in class crawlercommons.robots.SimpleRobotRulesParser
-
- getPayload() - Method in class crawlercommons.fetcher.FetchedResult
-
- getPLD(String) - Static method in class crawlercommons.url.PaidLevelDomain
-
Extract the PLD (paid-level domain) from the hostname.
- getPLD(URL) - Static method in class crawlercommons.url.PaidLevelDomain
-
Extract the PLD (paid-level domain) from the URL.
- getPriority() - Method in class crawlercommons.sitemaps.SiteMapURL
-
Return this URL's priority (a value between [0.0 - 1.0]).
- getReason() - Method in exception crawlercommons.fetcher.RedirectFetchException
-
- getReasonPhrase() - Method in class crawlercommons.fetcher.FetchedResult
-
- getRedirectedUrl() - Method in exception crawlercommons.fetcher.RedirectFetchException
-
- getRedirectMode() - Method in class crawlercommons.fetcher.http.BaseHttpFetcher
-
- getResponseRate() - Method in class crawlercommons.fetcher.FetchedResult
-
- getRobotRules(BaseHttpFetcher, BaseRobotsParser, URL) - Static method in class crawlercommons.robots.RobotUtils
-
Externally visible, static method for use in tools and for testing.
- getSitemap(URL) - Method in class crawlercommons.sitemaps.SiteMapIndex
-
Returns the Sitemap that has the given URL.
- getSitemaps() - Method in class crawlercommons.robots.BaseRobotRules
-
- getSitemaps() - Method in class crawlercommons.sitemaps.SiteMapIndex
-
- getSiteMapUrls() - Method in class crawlercommons.sitemaps.SiteMap
-
- getSocketTimeout() - Method in class crawlercommons.fetcher.http.SimpleHttpFetcher
-
- getStackTrace() - Method in exception crawlercommons.fetcher.BaseFetchException
-
- getStatusCode() - Method in class crawlercommons.fetcher.FetchedResult
-
- getType() - Method in class crawlercommons.sitemaps.AbstractSiteMap
-
- getUrl() - Method in exception crawlercommons.fetcher.BaseFetchException
-
- getUrl() - Method in class crawlercommons.sitemaps.AbstractSiteMap
-
- getUrl() - Method in class crawlercommons.sitemaps.SiteMapURL
-
Return the URL.
- getUserAgent() - Method in class crawlercommons.fetcher.http.BaseHttpFetcher
-
- getUserAgentString() - Method in class crawlercommons.fetcher.http.UserAgent
-
Obtain a String representing the user agent characteristics.
- getValidMimeTypes() - Method in class crawlercommons.fetcher.BaseFetcher
-
- getVersion() - Static method in class crawlercommons.CrawlerCommons
-
- valueOf(String) - Static method in enum crawlercommons.fetcher.AbortedFetchReason
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum crawlercommons.fetcher.http.BaseHttpFetcher.RedirectMode
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum crawlercommons.fetcher.RedirectFetchException.RedirectExceptionReason
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum crawlercommons.robots.SimpleRobotRules.RobotRulesMode
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum crawlercommons.sitemaps.AbstractSiteMap.SitemapType
-
Returns the enum constant of this type with the specified name.
- valueOf(String) - Static method in enum crawlercommons.sitemaps.SiteMapURL.ChangeFrequency
-
Returns the enum constant of this type with the specified name.
- values() - Static method in enum crawlercommons.fetcher.AbortedFetchReason
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum crawlercommons.fetcher.http.BaseHttpFetcher.RedirectMode
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Method in class crawlercommons.fetcher.Payload
-
- values() - Static method in enum crawlercommons.fetcher.RedirectFetchException.RedirectExceptionReason
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum crawlercommons.robots.SimpleRobotRules.RobotRulesMode
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum crawlercommons.sitemaps.AbstractSiteMap.SitemapType
-
Returns an array containing the constants of this enum type, in
the order they are declared.
- values() - Static method in enum crawlercommons.sitemaps.SiteMapURL.ChangeFrequency
-
Returns an array containing the constants of this enum type, in
the order they are declared.