mirror of
https://github.com/crawler-commons/crawler-commons
synced 2024-06-18 17:49:20 +02:00
Merge pull request #421 from sebastian-nagel/cc-308-url-normalizer-empty-query
[BasicNormalizer] Query parameters normalization in BasicURLNormalizer
This commit is contained in:
commit
7a95069f0e
|
@ -1,6 +1,7 @@
|
||||||
Crawler-Commons Change Log
|
Crawler-Commons Change Log
|
||||||
|
|
||||||
Current Development 1.4-SNAPSHOT (yyyy-mm-dd)
|
Current Development 1.4-SNAPSHOT (yyyy-mm-dd)
|
||||||
|
- [BasicNormalizer] Query parameters normalization in BasicURLNormalizer (aecio, sebastian-nagel) #308
|
||||||
- [Robots.txt] Handle allow/disallow directives containing unescaped Unicode characters (sebastian-nagel, Richard Zowalla, aecio) #389
|
- [Robots.txt] Handle allow/disallow directives containing unescaped Unicode characters (sebastian-nagel, Richard Zowalla, aecio) #389
|
||||||
- Improve readability of robots.txt unit tests (sebastian-nagel, Richard Zowalla) #383
|
- Improve readability of robots.txt unit tests (sebastian-nagel, Richard Zowalla) #383
|
||||||
- Upgrade project to use Java 11 (Avi Hayun, Richard Zowalla, aecio, sebastian-nagel) #320
|
- Upgrade project to use Java 11 (Avi Hayun, Richard Zowalla, aecio, sebastian-nagel) #320
|
||||||
|
|
|
@ -142,6 +142,9 @@ http:///////, http:/
|
||||||
http://example.com?,http://example.com/
|
http://example.com?,http://example.com/
|
||||||
http://example.com?a=1,http://example.com/?a=1
|
http://example.com?a=1,http://example.com/?a=1
|
||||||
|
|
||||||
|
# empty query #308
|
||||||
|
http://example.com/?,http://example.com/
|
||||||
|
|
||||||
# normalizing percent escapes #263
|
# normalizing percent escapes #263
|
||||||
https://www.last.fm/music/Prefuse+73/_/90%+of+My+Mind+Is+With+You,https://www.last.fm/music/Prefuse+73/_/90%25+of+My+Mind+Is+With+You
|
https://www.last.fm/music/Prefuse+73/_/90%+of+My+Mind+Is+With+You,https://www.last.fm/music/Prefuse+73/_/90%25+of+My+Mind+Is+With+You
|
||||||
|
|
||||||
|
|
Can't render this file because it has a wrong number of fields in line 3.
|
Loading…
Reference in New Issue