Adrián Chaves
8a3b15eb91
Document how to select dynamically-loaded content
2019-06-24 16:06:21 +02:00
Adrián Chaves
b1063d9b2c
Use the description from README.rst on index.rst
2019-03-08 17:22:49 +01:00
Raphael Wuillemier
af555cab23
Added general guide for developer tools instead of Firefox and Firebug-sections
2018-08-22 14:15:53 +02:00
Valdir Stumm Junior
80b160d0d7
include references to scrapy subreddit in the docs
2017-05-29 14:56:49 -03:00
Paul Tremberth
11fe3751cf
DOC Remove "Ubuntu" section from sidebar/ToC
2016-11-16 11:55:09 +01:00
Elias Dorneles
31260cf02f
mentions stackoverflow as help channel ( fixes #2255 )
2016-09-16 16:05:36 -03:00
orangain
4be4ef038e
DOC: Add captions to toctrees which appear in sidebar
2015-12-12 16:21:00 +09:00
Pablo Hoffman
b84b3585f2
remove empty experimental section from doc
2015-05-01 01:53:51 -03:00
Julia Medina
d47a7edc65
Update Logging topic on docs
2015-04-22 17:24:40 -03:00
Pablo Hoffman
06e1ca9ef2
Merge pull request #1177 from Curita/remove-djangoitem
...
Remove djangoitem
2015-04-21 15:35:47 -03:00
Pablo Hoffman
e4122cdd79
Merge pull request #1150 from eliasdorneles/docs-files-pipeline
...
Documenting Files Pipeline together with Images Pipeline
2015-04-21 15:28:47 -03:00
Julia Medina
6da091f162
Remove djangoitem topic from index
2015-04-20 20:19:28 -03:00
Elias Dorneles
a1f3b3c7f0
images.rst -> media-pipeline.rst
2015-04-15 14:26:08 -03:00
Elias Dorneles
dce48b8654
documenting Files Pipeline together with Images Pipeline
2015-04-11 13:57:55 -03:00
Pablo Hoffman
1a12922b77
improve scrapy deploy documentation
2015-04-09 17:39:01 -03:00
Pablo Hoffman
d8184a7239
Merge pull request #1124 from rdowinton/deployment-doc
...
Added deployment section covering scrapyd-deploy and shub
2015-04-09 16:53:25 -03:00
Richard Dowinton
2d142d6401
Added deployment section covering scrapyd-deploy and shub
2015-03-31 12:17:31 +01:00
Pablo Hoffman
bb4c922d85
Merge pull request #1081 from scrapy/dict-items
...
Allow spiders to return dicts.
2015-03-27 15:19:27 -03:00
Mikhail Korobov
f16a33f34e
DOC change structure of spider docs:
...
* start with scrapy.Spider, then mention spider arguments,
then describe generic spiders;
* change wording regarding start_urls/start_requests;
* show an example of start_requests vs start_urls;
* show an example of dicts as items;
* as defining Item is an optional step now, docs for Items are
moved below Spider docs.
2015-03-19 05:25:15 +05:00
Shadab Zafar
5a58d64131
Fix some redirection links in documentation
...
Fixes #606
2015-03-18 19:41:26 -03:00
Mikhail Korobov
a07b4353d6
DOC reorder topics
...
* duplicate "topics/commands" link is removed;
* Request/Response docs are moved to "Basic Concepts";
* settings docs are moved to "Basic Concepts";
* exceptions docs are moved to "Basic Concepts";
* "signals" and "exporters" docs are moved to "Extending Scrapy";
* "Reference" section is dropped because it is empty now.
2015-01-22 22:58:10 +05:00
Mikhail Korobov
f18ac02987
remove duplicated link extractors link
...
Check http://doc.scrapy.org/en/latest/topics/link-extractors.html - two menu items are highlighted at the left.
2013-12-28 05:40:10 +05:00
Pablo Hoffman
76087e336a
add scrapy bench command for benchmarking, with documentation
2013-05-16 13:15:25 -03:00
Pablo Hoffman
1e2ee76df2
add documentation topics: Broad Crawls & Common Practies
2012-12-26 14:02:13 -02:00
Pablo Hoffman
c380910b40
Merge pull request #167 from alexcepoi/sep-017
...
Spider contracts (SEP-017)
2012-09-28 13:57:07 -07:00
Alex Cepoi
11d29c7005
SEP-017 contracts: add tests and minor improvements
2012-09-21 00:12:46 +02:00
Pablo Hoffman
b46b5a6ef0
Documented AutoThrottle extension and added to extensions available by
...
default. Also deprecated concurrency and delay settings, in favour of
using the standard Scrapy ones.
2012-09-20 18:52:57 -03:00
Pablo Hoffman
a874964ad4
renamed 'XPath Selectors' title to just 'Selectors'
2012-09-13 15:24:44 -03:00
Alex Cepoi
bf8dc61fb7
SEP-017 contracts: pretty-printing and docs
2012-09-10 23:17:27 +02:00
Pablo Hoffman
70f8e517a1
promoted DjangoItem to main contrib
2012-08-29 11:23:11 -03:00
Pablo Hoffman
babfc6e79b
Updated documentation after singleton removal changes.
...
Also removed some unused code and made some minor additional
refactoring.
2012-08-28 18:35:57 -03:00
Alexandru Cepoi
f4faa19e31
added docs topic debugging spiders
2012-06-21 20:03:33 +02:00
Pablo Hoffman
7cca916ed5
added release notes to official documentation, including all release notes since Scrapy 0.7
2012-04-11 15:53:23 -03:00
Pablo Hoffman
431441cb52
updated documentation to remove references to old issue tracker and mercurial repos
2011-09-25 13:06:24 -03:00
Pablo Hoffman
40f7075f11
added initial documentation about suspend and resume crawls
2011-09-02 13:12:27 -03:00
Pablo Hoffman
bfda9ec319
added clarification about scrapy versioning including the recently adopted odd/even versioning scheme
...
--HG--
rename : docs/api-stability.rst => docs/versioning.rst
2011-07-12 19:53:23 -03:00
Pablo Hoffman
cd85c12c33
Some Link extractor improvements:
...
* added support for ignoring common file extensions that are not followed if
they occur in links
* fixed link extractor documentation issues
* slighly improved performance of applying filters
* added link to link extractors doc from documentation index
2011-05-18 12:32:34 -03:00
Pablo Hoffman
bf73002428
removed googledir example, replaced by dirbot project on github. updated docs accordingly
2011-04-28 02:28:39 -03:00
Pablo Hoffman
37e9c5d78e
Added new Scrapy service with support for:
...
* multiple projects
* uploading scrapy projects as Python eggs
* scheduling spiders using a JSON API
Documentation is added along with the code.
Closes #218 .
--HG--
rename : debian/scrapy-service.default => debian/scrapyd.default
rename : debian/scrapy-service.dirs => debian/scrapyd.dirs
rename : debian/scrapy-service.install => debian/scrapyd.install
rename : debian/scrapy-service.lintian-overrides => debian/scrapyd.lintian-overrides
rename : debian/scrapy-service.postinst => debian/scrapyd.postinst
rename : debian/scrapy-service.postrm => debian/scrapyd.postrm
rename : debian/scrapy-service.upstart => debian/scrapyd.upstart
rename : extras/scrapy.tac => extras/scrapyd.tac
2010-09-03 15:54:42 -03:00
Pablo Hoffman
e2ed27e4fd
Added documentation for Ubuntu packages. Refs #211
2010-08-23 21:28:32 -03:00
Pablo Hoffman
94ead94bf6
Improved documentation of Scrapy command-line tool
...
--HG--
rename : docs/topics/cmdline.rst => docs/topics/commands.rst
2010-08-19 00:04:52 -03:00
Pablo Hoffman
34554da201
Deprecated scrapy-ctl.py command in favour of simpler "scrapy" command. Closes #199 . Also updated documenation accordingly and added convenient scrapy.bat script for running from Windows.
...
--HG--
rename : debian/scrapy-ctl.1 => debian/scrapy.1
rename : docs/topics/scrapy-ctl.rst => docs/topics/cmdline.rst
2010-08-18 19:48:32 -03:00
Pablo Hoffman
e741a807d2
Added new Feed exports extension with documentation and storage tests. Closes #197 .
...
Also deprecated File export pipeline (to be removed in Scrapy 0.11).
Still need to add tests for FeedExport main extension code.
2010-08-17 14:27:48 -03:00
Pablo Hoffman
6a33d6c4d0
* Added Scrapy Web Service with documentation and tests.
...
* Marked Web Console as deprecated.
* Removed Web Console documentation to discourage its use.
2010-06-09 13:46:22 -03:00
Pablo Hoffman
076cdfd585
Added documentation about contributing to Scrapy
2010-05-17 20:10:46 -03:00
Pablo Hoffman
99a876754c
Improved "What else?" section of "Scrapy at a glance" overview
2010-03-20 20:24:18 -03:00
Pablo Hoffman
9d50604d24
added |version| to documentation title
2009-12-12 16:51:59 -02:00
Pablo Hoffman
bd481751d8
moved images pipeline documentation to stable doc
...
--HG--
rename : docs/experimental/images.rst => docs/topics/images.rst
2009-10-07 22:57:25 -02:00
Pablo Hoffman
be0f2beef0
more cleanup to scheduler middelware doc, and permanentely moved to experimental doc
2009-09-11 13:27:31 -03:00
Pablo Hoffman
f3240748cb
changed link to scheduler middleware doc, now in experimental
2009-09-11 12:03:23 -03:00
