# BasicURLNormalizer (Crawler-commons 0.8 API)
Date: 2017-06-07

crawlercommons.filters.basic

## Class BasicURLNormalizer

- java.lang.Object
- - crawlercommons.filters.URLFilter
  - - crawlercommons.filters.basic.BasicURLNormalizer

    public class BasicURLNormalizer
    extends URLFilter

Code borrowed from Apache Nutch. Converts URLs to a normal form:

- remove dot segments in path: `/./` or `/../`
- remove default ports, e.g. 80 for protocol `http://`
- normalize percent-encoding in URL paths

### Field Summary

Fields
`static org.slf4j.Logger` `LOG`

### Constructor Summary

Constructors
Constructor and Description
`BasicURLNormalizer()`

### Method Summary

All Methods Static Methods Instance Methods Concrete Methods
`String` `filter(String urlString)` Returns a modified version of the input URL or null if the URL should be removed
`static void` `main(String[] args)`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### LOG

    public static final org.slf4j.Logger LOG

### Constructor Detail

#### BasicURLNormalizer

    public BasicURLNormalizer()

### Method Detail

#### filter

    public String filter(String urlString)

Description copied from class: `URLFilter`
Returns a modified version of the input URL or null if the URL should be removed

Specified by:
  `filter` in class `URLFilter`
Parameters:
  `urlString` - a URL string to check against filter(s)
Returns:
  a filtered URL

#### main

    public static void main(String[] args)
                     throws IOException

Throws:
  `IOException`