mirror/ crawler-commons
1
0
Fork 0
mirror of https://github.com/crawler-commons/crawler-commons synced 2024-05-24 03:26:04 +02:00
Code Issues
471 Commits 8 Branches 15 Tags 5.8 MiB
Commit Graph

107 Commits

Author SHA1 Message Date
Sebastian Nagel b5b500f58b [maven-release-plugin] prepare release crawler-commons-1.2 2021-10-06 22:24:00 +02:00
Sebastian Nagel 0493878f80
Sitemaps: avoid calling java.net.URL::equals in equals method of sitemaps and sitemap extensions (#326) 
* Sitemaps: avoid calling java.net.URL::equals in equals method of sitemaps and sitemap extensions
(fixes #322)
- compare URL strings to avoid that java.net.URL::equals triggers unwanted and potentially slow
  DNS lookups to resolve the host part. Replace:
  - Objects::equals in equals methods of sitemap extensions
  - URL::equals and URL::hashCode in SiteMapIndex and SiteMapURL
- enable check for URL::equals and URL::hashCode in Forbidden API Checker

* Sitemaps: avoid calling java.net.URL::equals in equals method of sitemaps and sitemap extensions
- avoid NPEs in equals and hashCode methods

* Sitemaps: avoid calling java.net.URL::equals in equals method of sitemaps and sitemap extensions
- avoid NPE, return null as before if null is passed to SitemapIndex::getSitemap
 2021-10-06 12:07:02 +03:00
Richard Zowalla 35f0638069 introduces download-maven-plugin to download the 'effective_tld_names.dat' during the maven build 
ensures no cached files are used for the release profile
 2020-08-06 14:47:58 +02:00
Julien Nioche d2c162f1cc [maven-release-plugin] prepare for next development iteration 2020-06-25 09:53:20 +01:00
Julien Nioche 24e9e87db4 [maven-release-plugin] prepare release crawler-commons-1.1 2020-06-25 09:52:34 +01:00
Sebastian Nagel f7cc8b694a Make mvn run parameterized tests, fixes #273 
- make test classes public
- upgrade Maven surefire plugin to recent version
- depend on junit-jupiter-engine
 2019-11-12 13:49:40 +01:00
Avi Hayun 3597dee5a7 Updated the tests to use parameterizedTest, thus making the test file more readable and each normalized pair is now a test in itself, thus making it much easier to identify exactly failure points 2019-07-16 21:17:25 +03:00
Avi Hayun 5bda3636d2
Upgraded to Junit v5.5 (#250) 
* Upgraded to Junit v5.5
Updated the annotations and assertions accordingly

* Removed unneeded before and after

* This is a technical debt

Fixing a styling issue I caused about 4 years ago

Details can be found here: https://github.com/crawler-commons/crawler-commons/pull/82

* Fixed according to @sebastian-nagel code review
 2019-07-15 21:29:03 +03:00
Julien Nioche 7c6a3f2ccd [maven-release-plugin] prepare for next development iteration 2019-03-19 22:07:14 +00:00
Julien Nioche 5504239c6d [maven-release-plugin] prepare release crawler-commons-1.0 2019-03-19 22:06:56 +00:00
Julien Nioche 6d7f3a3fbf Revert "[maven-release-plugin] prepare release crawler-commons-1.0" 
This reverts commit 2be967020e.
 2019-03-19 22:02:19 +00:00
Julien Nioche 9a5236c484 [maven-release-plugin] prepare for next development iteration 2019-03-16 21:31:53 +00:00
Julien Nioche 2be967020e [maven-release-plugin] prepare release crawler-commons-1.0 2019-03-16 21:31:34 +00:00
Sebastian Nagel 9318de951f Use the Java 8 date and time API (java.time.*) to parse dates in sitemaps (#217) 
* Use the Java 8 date and time API (java.time.*) to parse dates in sitemaps
- use thread-safe DateTimeFormatter instead of ThreadLocal<DateFormat>
- simplify parsing of RSS publication dates
- remove obsolete regex pattern to catch dates with time zone
  but without seconds (covered by DateTimeFormatter.ISO_OFFSET_DATE_TIME)
- extend unit tests

* Fix Javadoc error and warnings, update change log

* Remove obsolete dependency to jaxb-api
- import of javax.xml.bind.DatatypeConverter has been removed
  by updating to Java 8 date and time API
 2018-09-24 10:09:58 +01:00
Julien Nioche 1c23d5966e [maven-release-plugin] prepare for next development iteration 2018-06-05 11:36:19 +01:00
Julien Nioche 939fbf439e [maven-release-plugin] prepare release crawler-commons-0.10 2018-06-05 11:36:12 +01:00
Julien Nioche e25309d26c
Add JAX-B dependencies to POM (#207) 
* Add JAX-B dependencies to POM, fixes #196

* mentioned in CHANGES.txt
 2018-05-03 11:04:03 +01:00
Ken Krugler 2cff74427d Remove Tika dependency 2018-03-27 15:40:50 -07:00
Julien Nioche 5132651a6e Upgraded Tika to 1.17, fixes #189 2018-03-20 17:20:07 +00:00
Julien Nioche 535f96236c [maven-release-plugin] prepare for next development iteration 2017-10-27 11:13:15 +01:00
Julien Nioche 9a54f53c14 [maven-release-plugin] prepare release crawler-commons-0.9 2017-10-27 11:13:07 +01:00
Julien Nioche 5e60792a0b Upgraded Tika 1.16, fixes #175 2017-09-08 14:34:17 +01:00
Julien Nioche 7ea41105d7 [maven-release-plugin] prepare for next development iteration 2017-06-07 16:09:32 +01:00
Julien Nioche 58f8e22dc2 [maven-release-plugin] prepare release crawler-commons-0.8 2017-06-07 16:09:25 +01:00
Julien Nioche f8c20c8e4a Upgraded Tika 1.15 2017-06-02 12:22:44 +01:00
Julien Nioche 4ba1295c17 Update forbiddenapis to v2.0. Fixes #99 2017-03-20 15:58:55 +00:00
Sebastian Nagel 9a49088ec8 Add new committer 2017-01-16 15:47:44 +01:00
Ken Krugler 49d71ad295 Issue #130 
Remove unused HttpClient code & resources
 2016-11-28 08:45:09 -08:00
Julien Nioche 2df1374eb4 [maven-release-plugin] prepare for next development iteration 2016-11-21 14:49:47 +00:00
Julien Nioche 7ef3599487 [maven-release-plugin] prepare release crawler-commons-0.7 2016-11-21 14:49:43 +00:00
Lewis John McGibbney 8814bed160 Upgrade to JDK 1.8 2016-09-26 15:20:39 -07:00
Julien Nioche 4eec816179 Upgraded Tika core to 1.13, fixes #122 2016-09-19 09:23:43 +01:00
Julien Nioche a809f7abac Added organization and inception year to pom; changed details for jnioche 2015-12-03 11:33:55 +00:00
Aecio Santos f2bf9300e6 Upgrades httpclient to version 4.5.1 (fixes #84) 
and do not ignore test failures during maven build
 2015-10-09 14:08:39 -04:00
Lewis John McGibbney d7ed6a742c Upgrade to Jdk1.7 - remove license header at pom.xml and improve logging implementations. 2015-09-07 15:20:00 -04:00
Lewis John McGibbney ba5906ec40 Upgrade to JDK 1.7 compiler version and introduce Maven forbidden API's plugin 2015-09-06 13:55:26 -04:00
Julien Nioche f155148216 Upgraded Tika 1.10 #89 2015-08-20 15:35:38 +01:00
Julien Nioche 22206f3a43 [maven-release-plugin] prepare for next development iteration 2015-05-27 16:38:05 +01:00
Julien Nioche 39d076a13b [maven-release-plugin] prepare release crawler-commons-0.6 2015-05-27 16:38:01 +01:00
Julien Nioche 2394b6713a Removed tagBase from maven-release-plugin configuration 2015-05-27 16:36:05 +01:00
Julien Nioche ee4a936066 Revert "[maven-release-plugin] prepare release crawler-commons-0.6" 
This reverts commit 3b09a9ba52.
 2015-05-27 16:16:54 +01:00
Julien Nioche 3b09a9ba52 [maven-release-plugin] prepare release crawler-commons-0.6 2015-05-27 16:05:02 +01:00
Julien Nioche e8ec75e019 Reverted failed release + changed groupId 2015-05-27 12:16:18 +01:00
Julien Nioche d115f158b2 [maven-release-plugin] prepare for next development iteration 2015-05-26 10:58:35 +01:00
Julien Nioche 8328e554d4 [maven-release-plugin] prepare release crawler-commons-0.6 2015-05-26 10:58:31 +01:00
Lewis John McGibbney e8065d5372 simplify pom file #77 2015-05-14 12:05:37 -07:00
Julien Nioche 0a5d9d338a maven-java-formatter-plugin 2015-04-30 13:52:44 +01:00
Julien Nioche 1647b90f7d Upgraded to Tika 1.8 fixes #71 2015-04-22 13:19:02 +01:00
Julien Nioche a28a78c942 changed groupId to crawler-commons 2015-04-22 11:02:49 +01:00
Julien Nioche 2195fb7f7e replaced references to code.google with github equivalents 2015-04-17 10:25:52 +01:00