BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 17:04:07 +00:00
Code Releases Wiki Activity
5,235 Commits 39 Branches 119 Tags
Commit Graph

5235 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Elias Dorneles
179c4588ca adding test for latin1 location 2015-09-10 19:36:47 -03:00
Elias Dorneles
defa899135 PY3 port redirect middleware 2015-09-10 16:31:59 -03:00
Elias Dorneles
44bd01d51c preparatory refactor in redirect mware 2015-09-10 16:19:56 -03:00
Daniel Graña
a7787628ff Merge pull request #1263 from chekunkov/readable-middlewares-list-in-logs 
Make list of enabled middlewares more readable
 2015-09-10 12:09:26 -03:00
Alexander Chekunkov
563b1500bf Make list of enabled middlewares more readable 2015-09-10 14:59:55 +00:00
Daniel Graña
9a64d8ff97 fix scrapy squeue tests after recent changes to queuelib 2015-09-09 15:55:55 -03:00
Mikhail Korobov
600f27c39b Merge pull request #1350 from scrapy/installation-docs-update 
Updates in Installation and Contributing sections in the docs
 2015-09-09 17:54:35 +05:00
Julia Medina
eb7b9d506f Add note to ubuntu install section about debian compatibility 2015-09-08 18:49:44 -03:00
Mikhail Lyundin
be7821a482 Optimization - avoid temporary list objects, unnecessary function call 2015-09-08 22:10:24 +03:00
Daniel Graña
6490cb534e Merge pull request #1477 from mlyundin/master 
equal_attributes function optimization
 2015-09-07 14:32:27 -03:00
Daniel Collins
311d5cd495 split nested_loader into seperate methods 2015-09-04 13:15:48 -07:00
Daniel Collins
88c92cb68b provide documentation for nested loaders 2015-09-04 13:15:48 -07:00
Daniel Collins
425e35ee90 removed commented code and add test for error handling 2015-09-04 13:15:48 -07:00
Daniel Collins
1f95af3c07 add support for a nested loaders 2015-09-04 13:15:48 -07:00
Daniel Graña
70fa3ae9a5 Merge pull request #1479 from scrapy/twisted-15.4 
fixed compatibility with Twisted 15.4.0
 2015-09-04 14:10:59 -03:00
Daniel Graña
16e63040e9 Merge pull request #1480 from scrapy/travis-run-once 
don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch
 2015-09-04 14:09:21 -03:00
Mikhail Korobov
d30e539049 don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch 2015-09-04 22:06:46 +05:00
Mikhail Korobov
faf9265c91 fixed compatibility with Twisted 15.4.0 2015-09-04 20:50:48 +05:00
Mikhail Lyundin
d022d3cb9e equal_attributes function optimization 2015-09-04 18:21:41 +03:00
Nicolás Alejandro Ramírez Quiros
6ae8963256 Merge pull request #1476 from mlyundin/master 
[MRG+1] Avoid creation of temporary list object in iflatten
 2015-09-02 17:31:04 -03:00
Mikhail Lyundin
d9fddabe95 Avoid creation of temporary list object in iflatten 2015-09-02 23:12:36 +03:00
Daniel Graña
dd473145f2 Merge pull request #1473 from ArturGaspar/master 
[MRG+1] Support for returning deferreds in middlewares
 2015-09-02 13:04:26 -03:00
Robert Weindl
833efbfcbd Update tutorial.rst 
Add missing "scrapy." prefix.
 2015-09-02 20:35:23 +05:00
Artur Gaspar
dd83f6123d Test for same response object in downloader middleware test. 2015-09-02 11:36:15 -03:00
Artur Gaspar
2748b38592 Test for not calling the download function when downloader middleware returns a response in process_request. 2015-09-02 11:15:37 -03:00
Artur Gaspar
1f4af9d81e Test for robots.txt middleware for logger not being called when the request for robots.txt is ignored. 2015-09-02 10:33:53 -03:00
Artur Gaspar
668e5fd257 Test for robots.txt middleware for processing a request for which the robots.txt parser is ready. 2015-09-02 10:23:24 -03:00
Artur Gaspar
a6a629e707 Call actual error logger in robots.txt middleware tests. 2015-09-02 10:23:24 -03:00
Artur Gaspar
9ce9a293a6 Always check robots.txt before making another request in RobotsTxtMiddleware. 2015-09-02 10:23:24 -03:00
Daniel Graña
af0873746f Merge pull request #1289 from pawelmhm/dont_retry_400 
[MRG+1] [settings/default_settings.py] dont retry 400
 2015-09-02 00:39:30 -03:00
Daniel Graña
3919ad64c5 Merge pull request #1470 from scrapy/py3-middlewares 
[MRG] port some downloader middlewares to Python 3
 2015-09-02 00:17:37 -03:00
Daniel Graña
bc499cb552 Merge pull request #1472 from olafdietsche/incomplete-submit-button 
[MRG+1] Incomplete submit button
 2015-09-02 00:16:42 -03:00
Artur Gaspar
ca83a0b028 Support for returning deferreds in downloader middleware methods. 2015-09-01 13:22:43 -03:00
Olaf Dietsche
937277e859 Make sure value attribute values are set. 2015-09-01 12:44:04 +02:00
Olaf Dietsche
2c28b53cc0 Add tests with incomplete buttons 2015-09-01 12:35:49 +02:00
Mikhail Korobov
c44cafe4f5 PY3 fix UserAgentMiddleware tests 2015-09-01 04:11:33 +05:00
Mikhail Korobov
3a9c73bc5d PY3 fix DownloaderStats middleware tests 2015-09-01 04:09:15 +05:00
Mikhail Korobov
78a4cd0f1c PY3 fix HttpAuthMiddleware tests 2015-09-01 04:00:26 +05:00
Mikhail Korobov
179a4409fb PY3 enable DownloadTimeoutMiddleware tests 2015-09-01 03:58:25 +05:00
Mikhail Korobov
3cf1911a92 PY3 fix DefaultHeadersMiddleware tests 2015-09-01 03:57:05 +05:00
Mikhail Korobov
45101829a5 PY3 fix AjacCrawlable middleware tests 2015-09-01 03:49:52 +05:00
Daniel Graña
b876755f1c Merge pull request #1469 from olafdietsche/form-submit-button-plain 
Form submit button plain
 2015-08-31 19:30:41 -03:00
Olaf Dietsche
121d7535be Allow button cotrols to submit form 2015-08-31 19:58:16 +02:00
Olaf Dietsche
e5f26078fa Add test: submit form with button control 2015-08-31 19:57:45 +02:00
Daniel Graña
0018caf0b6 Merge pull request #1461 from scrapy/py3-linkextractors 
[MRG] assorted Python 3 porting
 2015-08-31 11:59:30 -03:00
Daniel Graña
b4e382c867 Merge pull request #1318 from yarikoptic/fix-docs-explicit-rtd-theme-path 
[MRG+1] DOC(ENH): specify path to rtd theme explicitly
 2015-08-31 10:42:02 -03:00
Mikhail Korobov
44bfcbcf0f TST split LinkExtractorTestCase.test_extraction into several methods; remove duplicated test 2015-08-31 00:49:38 +05:00
Mikhail Korobov
f7052413e0 PY3 raise an exception if bytes are passed as url to Link constructor 2015-08-28 23:04:02 +05:00
Mikhail Korobov
9bfe6ece59 Merge branch 'master' into py3-linkextractors 
Conflicts:
	scrapy/linkextractors/lxmlhtml.py
	tests/test_linkextractors.py
 2015-08-28 04:53:32 +05:00
Mikhail Korobov
ff24cbbc47 PY3 depth, offsite and referer spider middlewares; Crawler 2015-08-28 04:11:30 +05:00