BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 19:03:53 +00:00
Code Releases Wiki Activity
5,253 Commits 39 Branches 119 Tags
Commit Graph

5253 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mikhail Korobov
916141a08f TST pin pytest to 2.7.3 2015-09-21 21:12:52 +05:00
Mikhail Korobov
181f8d27d7 Merge pull request #1501 from scrapy/disable-log-on-startproject 
disable log on startproject command
 2015-09-20 02:47:41 +05:00
Daniel Graña
51b1249033 Revert "test suite requires recent queuelib to pass because it depends on queuelib test suite" 
This reverts commit a3390afc66134e77f98ae3bfae7bc23479ed8566.
 2015-09-18 02:27:07 -03:00
Daniel Graña
a3390afc66 test suite requires recent queuelib to pass because it depends on queuelib test suite 2015-09-17 19:42:29 -03:00
Daniel Graña
a57efdb008 Add PyPI download stats badge 2015-09-17 14:07:57 -03:00
Pablo Hoffman
22327ef47b disable log on startproject command 2015-09-17 01:05:52 -03:00
Daniel Graña
3c596dcf46 Merge pull request #1467 from dacjames/master 
add support for a nested loaders
 2015-09-16 22:03:19 -03:00
Mikhail Korobov
09f388c328 Merge pull request #1497 from haiiiiiyun/fix_typo_middlware 
fix typo middlware
 2015-09-16 15:39:24 +05:00
hy
14f7f22555 fix typos in downloader-middleware.rst and exceptions.rst, middlware -> middleware 2015-09-16 16:59:23 +08:00
Daniel Collins
036109e7de updte nested loader documentation 2015-09-15 23:49:35 -07:00
nyov
ecbfe4bd66 drop deprecated "optional_features" set 2015-09-15 09:00:08 +00:00
Daniel Graña
5279da9916 Merge pull request #1496 from scrapy/default-template 
[MRG+1] style fixes for settings.py created by `scrapy startproject`
 2015-09-14 17:14:15 -03:00
Mikhail Korobov
3e13740a57 Merge pull request #1481 from mlyundin/develop 
cleanup
 2015-09-15 00:36:50 +05:00
Daniel Graña
4d1c5c3d32 Merge pull request #1488 from scrapy/py3-redirect-downloader-mware 
[MRG+1] PY3 redirect downloader mware
 2015-09-14 02:20:13 -03:00
Mikhail Korobov
5a13022713 style fixes for settings.py created by scrapy startproject 2015-09-13 21:46:20 +05:00
Elias Dorneles
430e90f4b2 minor refactor on metarefresh redirect mware test 2015-09-13 12:29:41 -03:00
Elias Dorneles
b06d0706e6 refactoring redirect logic 2015-09-13 11:49:59 -03:00
Elias Dorneles
81950f773d made encoding conversion more explicit, added test for header with utf-8 encoding replicating what browsers do 2015-09-11 18:51:48 -03:00
Elias Dorneles
179c4588ca adding test for latin1 location 2015-09-10 19:36:47 -03:00
Elias Dorneles
defa899135 PY3 port redirect middleware 2015-09-10 16:31:59 -03:00
Elias Dorneles
44bd01d51c preparatory refactor in redirect mware 2015-09-10 16:19:56 -03:00
Daniel Graña
a7787628ff Merge pull request #1263 from chekunkov/readable-middlewares-list-in-logs 
Make list of enabled middlewares more readable
 2015-09-10 12:09:26 -03:00
Alexander Chekunkov
563b1500bf Make list of enabled middlewares more readable 2015-09-10 14:59:55 +00:00
Daniel Graña
9a64d8ff97 fix scrapy squeue tests after recent changes to queuelib 2015-09-09 15:55:55 -03:00
Mikhail Korobov
600f27c39b Merge pull request #1350 from scrapy/installation-docs-update 
Updates in Installation and Contributing sections in the docs
 2015-09-09 17:54:35 +05:00
Julia Medina
eb7b9d506f Add note to ubuntu install section about debian compatibility 2015-09-08 18:49:44 -03:00
Mikhail Lyundin
be7821a482 Optimization - avoid temporary list objects, unnecessary function call 2015-09-08 22:10:24 +03:00
Daniel Graña
6490cb534e Merge pull request #1477 from mlyundin/master 
equal_attributes function optimization
 2015-09-07 14:32:27 -03:00
Daniel Collins
311d5cd495 split nested_loader into seperate methods 2015-09-04 13:15:48 -07:00
Daniel Collins
88c92cb68b provide documentation for nested loaders 2015-09-04 13:15:48 -07:00
Daniel Collins
425e35ee90 removed commented code and add test for error handling 2015-09-04 13:15:48 -07:00
Daniel Collins
1f95af3c07 add support for a nested loaders 2015-09-04 13:15:48 -07:00
Daniel Graña
70fa3ae9a5 Merge pull request #1479 from scrapy/twisted-15.4 
fixed compatibility with Twisted 15.4.0
 2015-09-04 14:10:59 -03:00
Daniel Graña
16e63040e9 Merge pull request #1480 from scrapy/travis-run-once 
don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch
 2015-09-04 14:09:21 -03:00
Mikhail Korobov
d30e539049 don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch 2015-09-04 22:06:46 +05:00
Mikhail Korobov
faf9265c91 fixed compatibility with Twisted 15.4.0 2015-09-04 20:50:48 +05:00
Mikhail Lyundin
d022d3cb9e equal_attributes function optimization 2015-09-04 18:21:41 +03:00
Nicolás Alejandro Ramírez Quiros
6ae8963256 Merge pull request #1476 from mlyundin/master 
[MRG+1] Avoid creation of temporary list object in iflatten
 2015-09-02 17:31:04 -03:00
Mikhail Lyundin
d9fddabe95 Avoid creation of temporary list object in iflatten 2015-09-02 23:12:36 +03:00
Daniel Graña
dd473145f2 Merge pull request #1473 from ArturGaspar/master 
[MRG+1] Support for returning deferreds in middlewares
 2015-09-02 13:04:26 -03:00
Robert Weindl
833efbfcbd Update tutorial.rst 
Add missing "scrapy." prefix.
 2015-09-02 20:35:23 +05:00
Artur Gaspar
dd83f6123d Test for same response object in downloader middleware test. 2015-09-02 11:36:15 -03:00
Artur Gaspar
2748b38592 Test for not calling the download function when downloader middleware returns a response in process_request. 2015-09-02 11:15:37 -03:00
Artur Gaspar
1f4af9d81e Test for robots.txt middleware for logger not being called when the request for robots.txt is ignored. 2015-09-02 10:33:53 -03:00
Artur Gaspar
668e5fd257 Test for robots.txt middleware for processing a request for which the robots.txt parser is ready. 2015-09-02 10:23:24 -03:00
Artur Gaspar
a6a629e707 Call actual error logger in robots.txt middleware tests. 2015-09-02 10:23:24 -03:00
Artur Gaspar
9ce9a293a6 Always check robots.txt before making another request in RobotsTxtMiddleware. 2015-09-02 10:23:24 -03:00
Daniel Graña
af0873746f Merge pull request #1289 from pawelmhm/dont_retry_400 
[MRG+1] [settings/default_settings.py] dont retry 400
 2015-09-02 00:39:30 -03:00
Daniel Graña
3919ad64c5 Merge pull request #1470 from scrapy/py3-middlewares 
[MRG] port some downloader middlewares to Python 3
 2015-09-02 00:17:37 -03:00
Daniel Graña
bc499cb552 Merge pull request #1472 from olafdietsche/incomplete-submit-button 
[MRG+1] Incomplete submit button
 2015-09-02 00:16:42 -03:00