mirror/ crawler-commons
1
0
Fork 0
mirror of https://github.com/crawler-commons/crawler-commons synced 2024-05-30 04:56:04 +02:00
Code Issues

CC 12 Substantiate Javadoc

This commit is contained in:
lewis.mcgibbney@gmail.com 2013-01-28 02:47:01 +00:00
parent 2f34db8056
commit 51bb23f01c
6 changed files with 140 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
src/main/java/crawlercommons/fetcher/file/package.html Normal file
View File

@ -0,0 +1,25 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<body>
<package>
This package includes the {@link crawlercommons.fetcher.file.SimpleFileFetcher} code which extends the {@link crawlercommons.fetcher.BaseFetcher}.
Classes in this package concern fetching the file protocol.
</package>
</body>
<html>

24
src/main/java/crawlercommons/fetcher/http/package.html Normal file
View File

@ -0,0 +1,24 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<body>
This package concerns the fetching of files over the HTTP protocol:
Extending from {@link crawlercommons.fetcher.http.BaseHttpFetcher} (which itself extends {@link crawlercommons.fetcher.BaseFetcher}) the
{@link crawlercommons.fetcher.http.SimpleHttpFetcher} provides the Crawler Commons HTTP fetching implementation.
</body>
<html>

23
src/main/java/crawlercommons/fetcher/package.html Normal file
View File

@ -0,0 +1,23 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<body>
The main fetching package within Crawler Commons, this package defines base fetching
and encoding classes, Enum's to determine reasoning behind typical fetching behaviour
as well as the base Exceptions which may be used.
</body>
<html>

22
src/main/java/crawlercommons/robots/package.html Normal file
View File

@ -0,0 +1,22 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<body>
The robots package contains all of the robots.txt rule inference, parsing and utilities contained within Crawler Commons.
</body>
<html>

24
src/main/java/crawlercommons/sitemaps/package.html Normal file
View File

@ -0,0 +1,24 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<body>
Sitemaps package provides all classes relevant to focused sitemap parsing,
url definition and processing. Additional Enum's include support for various
sitemap formats as well as change frequency control.
</body>
<html>

22
src/main/java/crawlercommons/url/package.html Normal file
View File

@ -0,0 +1,22 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<html>
<body>
Classes contained within the url package relate to the definition of Top Level
Domain's, various domain registrars and the effective handling of such domains.
</body>
<html>