BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 19:23:48 +00:00
Code Releases Wiki Activity
4,543 Commits 39 Branches 119 Tags
Commit Graph

4543 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Maik Röder
721d8d5dae Update form.py 
Typo fixed
 2015-02-04 11:38:37 +01:00
Mikhail Korobov
6419f76bbc Merge pull request #1029 from dangra/contribute-to-master 
DOC contribute to master branch
 2015-02-03 02:10:47 +05:00
Nicolás Alejandro Ramírez Quiros
8a3b9b6131 Merge pull request #1011 from SudShekhar/master 
Extension example fix to something that makes more sense
 2015-01-30 15:45:52 -02:00
Sudhanshu Shekhar
e42a1ac1a1 Reset items_scraped instead of item_count 
items_scraped is the counter that needs to be reset each time we have scraped a specific number of items in the code instead of item_count (which represents the specific number of items needed before a message is logged). Updating the source code to reflect this.
Removed some irrelevant words from the log message.
Signed-off-by: Sudhanshu Shekhar <sudshekhar02@gmail.com>
 2015-01-30 23:13:06 +05:30
Daniel Graña
f0bdc14522 Tentative attention message about what document to read for contributions 2015-01-29 15:40:12 -02:00
Daniel Graña
074b4a9315 Contribute to master branch 
ref #975
 2015-01-29 14:53:42 -02:00
Mikhail Korobov
a07b4353d6 DOC reorder topics 
* duplicate "topics/commands" link is removed;
* Request/Response docs are moved to "Basic Concepts";
* settings docs are moved to "Basic Concepts";
* exceptions docs are moved to "Basic Concepts";
* "signals" and "exporters" docs are moved to "Extending Scrapy";
* "Reference" section is dropped because it is empty now.
 2015-01-22 22:58:10 +05:00
Jonas Tingeborn
bd5d99a2d2 add gzip compression to filesystem http cache backend 2015-01-21 20:18:11 +01:00
Nicolás Alejandro Ramírez Quiros
2183984832 Merge pull request #1018 from barraponto/dynamic_item_practice_docs 
Updates documentation on dynamic item classes.
 2015-01-19 18:12:02 -02:00
Capi Etheriel
4bc14da59e Updates documentation on dynamic item classes. 
Fixes #398
 2015-01-19 17:21:56 -02:00
Mikhail Korobov
283d6a5344 DOC a couple more references are fixed 2015-01-19 22:07:03 +05:00
Mikhail Korobov
73e6b35622 DOC fix a reference 2015-01-19 22:02:46 +05:00
Daniel Graña
5bcb652410 Merge pull request #1017 from Curita/external-url-fix 
Don't rely on external urls for tests
 2015-01-19 13:57:18 -02:00
Julia Medina
d68615a5af Test the parse command locally instead of against an external url 2015-01-19 10:28:25 -03:00
Daniel Graña
f3110aaed1 Merge pull request #999 from Curita/fix-985 
Patch hanging HTTPConnectionPool.closeCachedConnections call
 2014-12-31 01:52:44 -02:00
Julia Medina
09ba4ff68a Patches Twisted issue while closing the connection pool on HTTPDownloadHandler 2014-12-30 19:53:50 -03:00
Nicolás Alejandro Ramírez Quiros
79c3274884 Merge pull request #987 from nyov/nyov/resolver 
t.i.b.ThreadedResolver is now a new-style class
 2014-12-17 11:28:59 -02:00
nyov
880c9e52b5 t.i.b.ThreadedResolver is now a new-style class 
since 2009 / twisted-9.0.0
663d669dce
 2014-12-16 19:28:39 +00:00
Pablo Hoffman
78954bc7f4 Merge pull request #978 from immerrr/fix-s3-authorization-for-quoted-paths 
S3DownloadHandler: fix auth for requests with quoted paths/query params
 2014-12-15 17:15:11 -02:00
Artur Gaspar
22247cf791 move restrict_css argument to end of argument list in link extractors for backwards compatibility, use keyword arguments in link extractor super().__init__() calls 2014-12-15 09:18:15 -02:00
Artur Gaspar
b0730a1d16 documentation for CSS support in link extractors 2014-12-11 18:22:08 -02:00
Artur Gaspar
403fc686b8 tests for CSS support in link extractors 2014-12-11 18:20:30 -02:00
Artur Gaspar
d4cb03eded add CSS support for link extractors 2014-12-11 16:45:46 -02:00
immerrr
82b187f283 S3DownloadHandler: fix auth for requests with quoted paths/query params 2014-12-11 18:14:36 +03:00
Mikhail Korobov
c485a05540 Merge pull request #976 from aufziehvogel/fixed_mailsender_vartypes 
fixed the variable types in mailsender documentation
 2014-12-11 03:02:33 +05:00
Stefan
3602fc4fcb fixed the variable types in mailsender documentation 2014-12-10 22:48:09 +01:00
Pablo Hoffman
1b03b12996 Merge pull request #961 from ldmberman/task_977_request_dropped 
An attempt to resolve #957, add signal to be sent when request is dropped by the scheduler
 2014-12-02 19:46:43 -02:00
Lev Berman
fdb6bb07c0 #977 - test dropping requests 2014-11-28 10:53:33 +03:00
Lev Berman
e04b0aff74 An attempt to resolve #977, add signal to be sent when request is dropped by the scheduler 2014-11-27 15:10:15 +03:00
Pablo Hoffman
c31fb87335 Merge pull request #954 from kalessin/int-download-timeout 
Force to read DOWNLOAD_TIMEOUT as int (for example to pass using environment variable)
 2014-11-26 17:14:18 -02:00
Pablo Hoffman
dedea72774 Merge pull request #946 from tpeng/limit-response-size 
avoid download large response
 2014-11-25 17:57:55 -02:00
tpeng
cd19382754 attemp to fix travis fails 2014-11-25 14:20:25 +01:00
Daniel Graña
8d8e1b2c0c mitmproxy 0.10.1 needs netlib 0.10.1 too 2014-11-21 12:15:02 -02:00
Daniel Graña
314db3db8b pin mitmproxy 0.10.1 as >0.11 does not work with tests 2014-11-21 10:54:43 -02:00
Martin Olveyra
7910fa0172 Force to read DOWNLOAD_TIMEOUT as int (for example to pass using 
environment variable)
 2014-11-21 01:09:32 -02:00
tpeng
a69f042d10 add 2 more test cases and minor doc fixes 2014-11-19 15:31:07 +01:00
tpeng
fa84730e70 avoid download large response 
introduce DOWNLOAD_MAXSIZE and DOWNLOAD_WARNSIZE in settings and
download_maxsize/download_warnsize in spider/request meta, so
downloader stop downloading as soon as the received data exceed the
limit. also check the twsisted response's length in advance to stop
downloading as early as possible.
 2014-11-12 12:28:02 +01:00
Pablo Hoffman
ed84231b60 Merge pull request #944 from JeffPaine/patch-1 
Update docs copyright year range
 2014-11-10 14:03:00 -02:00
Jeff Paine
b422312a38 Update docs copyright year range 2014-11-09 21:08:27 -05:00
Lazar-T
13f83f0da0 typo 2014-11-10 06:28:41 +05:00
HalfCrazy
b21a28cc9a Afterwords->Afterwards 2014-11-10 06:28:09 +05:00
Daniel Graña
2c67bd6c57 pywin32 is required by Twisted. closes #937 
see:
* http://twistedmatrix.com/trac/ticket/6032
* https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2028
 2014-11-05 23:05:54 -02:00
Daniel Graña
6cb8995731 Update install.rst 
fixes #937
 2014-11-05 22:50:52 -02:00
Pablo Hoffman
efe589c643 Merge pull request #882 from ahlen/feature/csvfeed-quotechar 
[MRG+1] Allow to specify the quotechar in CSVFeedSpider
 2014-11-04 11:32:59 -02:00
Lazar-T
38dcf50cd6 comma instead of fullstop 2014-10-25 09:19:50 +06:00
Pablo Hoffman
675fd5ba04 Merge pull request #898 from scrapy/download-timeout 
[MRG] DOC document download_timeout
 2014-10-24 16:52:42 -02:00
Pablo Hoffman
5de6a11fda Merge pull request #925 from Digenis/master 
a leftover for.15 compatibility
 2014-10-22 18:23:19 -02:00
Nikolaos-Digenis Karagiannis
2227805619 Compatibility with .15 leftover 2014-10-22 21:48:08 +03:00
Pablo Hoffman
0dce283459 Merge pull request #893 from kmike/less-ads 
[MRG] DOC simplify extension docs
 2014-10-21 17:13:59 -02:00
Daniel Graña
44cbbecb44 Merge pull request #914 from brunsgaard/master 
Deleted bin folder from root, fixes #913
 2014-10-09 14:58:36 -02:00