# BaseHttpFetcher (Crawler-commons 0.6-SNAPSHOT API)
Date: 2015-05-26

crawlercommons.fetcher.http

## Class BaseHttpFetcher

- java.lang.Object
- - crawlercommons.fetcher.BaseFetcher
  - - crawlercommons.fetcher.http.BaseHttpFetcher

All Implemented Interfaces:
  Serializable

Direct Known Subclasses:
  SimpleHttpFetcher

    public abstract class BaseHttpFetcher
    extends BaseFetcher

See Also:
  Serialized Form

### Nested Class Summary

Nested Classes
`static class` `BaseHttpFetcher.RedirectMode`

### Field Summary

Fields
`protected String` `_acceptLanguage`
`protected int` `_maxConnectionsPerHost`
`protected int` `_maxRedirects`
`protected int` `_maxThreads`
`protected int` `_minResponseRate`
`protected BaseHttpFetcher.RedirectMode` `_redirectMode`
`protected UserAgent` `_userAgent`
`static String` `DEFAULT_ACCEPT_LANGUAGE`
`static int` `DEFAULT_MAX_CONNECTIONS_PER_HOST`
`static int` `DEFAULT_MAX_REDIRECTS`
`static int` `DEFAULT_MIN_RESPONSE_RATE`
`static BaseHttpFetcher.RedirectMode` `DEFAULT_REDIRECT_MODE`
`static int` `NO_MIN_RESPONSE_RATE`
`static int` `NO_REDIRECTS`

### Fields inherited from class crawlercommons.fetcher.BaseFetcher

`_defaultMaxContentSize, _maxContentSizes, _validMimeTypes, DEFAULT_MAX_CONTENT_SIZE`

### Constructor Summary

Constructors
Constructor and Description
`BaseHttpFetcher(int maxThreads, UserAgent userAgent)`

### Method Summary

Methods
`String` `getAcceptLanguage()`
`int` `getMaxConnectionsPerHost()`
`int` `getMaxRedirects()`
`int` `getMaxThreads()`
`int` `getMinResponseRate()` Return the minimum response rate.
`BaseHttpFetcher.RedirectMode` `getRedirectMode()`
`UserAgent` `getUserAgent()`
`void` `setAcceptLanguage(String acceptLanguage)`
`void` `setMaxConnectionsPerHost(int maxConnectionsPerHost)`
`void` `setMaxRedirects(int maxRedirects)`
`void` `setMinResponseRate(int minResponseRate)`
`void` `setRedirectMode(BaseHttpFetcher.RedirectMode mode)`

### Methods inherited from class crawlercommons.fetcher.BaseFetcher

`abort, addValidMimeType, addValidMimeTypes, get, get, getDefaultMaxContentSize, getMaxContentSize, getMimeTypeFromContentType, getValidMimeTypes, setDefaultMaxContentSize, setMaxContentSize, setValidMimeTypes`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### NO_MIN_RESPONSE_RATE

    public static final int NO_MIN_RESPONSE_RATE

See Also:
  Constant Field Values

#### NO_REDIRECTS

    public static final int NO_REDIRECTS

See Also:
  Constant Field Values

#### DEFAULT_MIN_RESPONSE_RATE

    public static final int DEFAULT_MIN_RESPONSE_RATE

See Also:
  Constant Field Values

#### DEFAULT_MAX_CONNECTIONS_PER_HOST

    public static final int DEFAULT_MAX_CONNECTIONS_PER_HOST

See Also:
  Constant Field Values

#### DEFAULT_MAX_REDIRECTS

    public static final int DEFAULT_MAX_REDIRECTS

See Also:
  Constant Field Values

#### DEFAULT_ACCEPT_LANGUAGE

    public static final String DEFAULT_ACCEPT_LANGUAGE

See Also:
  Constant Field Values

#### DEFAULT_REDIRECT_MODE

    public static final BaseHttpFetcher.RedirectMode DEFAULT_REDIRECT_MODE

#### _maxThreads

    protected int _maxThreads

#### _userAgent

    protected UserAgent _userAgent

#### _maxRedirects

    protected int _maxRedirects

#### _maxConnectionsPerHost

    protected int _maxConnectionsPerHost

#### _minResponseRate

    protected int _minResponseRate

#### _acceptLanguage

    protected String _acceptLanguage

#### _redirectMode

    protected BaseHttpFetcher.RedirectMode _redirectMode

### Constructor Detail

#### BaseHttpFetcher

    public BaseHttpFetcher(int maxThreads,
                   UserAgent userAgent)

### Method Detail

#### getMaxThreads

    public int getMaxThreads()

#### getUserAgent

    public UserAgent getUserAgent()

#### setMaxConnectionsPerHost

    public void setMaxConnectionsPerHost(int maxConnectionsPerHost)

#### getMaxConnectionsPerHost

    public int getMaxConnectionsPerHost()

#### setMinResponseRate

    public void setMinResponseRate(int minResponseRate)

#### getMinResponseRate

    public int getMinResponseRate()

Return the minimum response rate. If the speed at which bytes are being returned from the server drops below this, the fetch of that page will be aborted.

Returns:
  bytes/second

#### setAcceptLanguage

    public void setAcceptLanguage(String acceptLanguage)

#### getAcceptLanguage

    public String getAcceptLanguage()

#### setMaxRedirects

    public void setMaxRedirects(int maxRedirects)

#### getMaxRedirects

    public int getMaxRedirects()

#### setRedirectMode

    public void setRedirectMode(BaseHttpFetcher.RedirectMode mode)

#### getRedirectMode

    public BaseHttpFetcher.RedirectMode getRedirectMode()