BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 22:04:05 +00:00
Code Releases Wiki Activity
5,226 Commits 39 Branches 119 Tags
Commit Graph

5226 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Daniel Graña
6490cb534e Merge pull request #1477 from mlyundin/master 
equal_attributes function optimization
 2015-09-07 14:32:27 -03:00
Daniel Collins
311d5cd495 split nested_loader into seperate methods 2015-09-04 13:15:48 -07:00
Daniel Collins
88c92cb68b provide documentation for nested loaders 2015-09-04 13:15:48 -07:00
Daniel Collins
425e35ee90 removed commented code and add test for error handling 2015-09-04 13:15:48 -07:00
Daniel Collins
1f95af3c07 add support for a nested loaders 2015-09-04 13:15:48 -07:00
Daniel Graña
70fa3ae9a5 Merge pull request #1479 from scrapy/twisted-15.4 
fixed compatibility with Twisted 15.4.0
 2015-09-04 14:10:59 -03:00
Daniel Graña
16e63040e9 Merge pull request #1480 from scrapy/travis-run-once 
don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch
 2015-09-04 14:09:21 -03:00
Mikhail Korobov
d30e539049 don't run tests twice on Travis if a PR is made from a scrapy/scrapy branch 2015-09-04 22:06:46 +05:00
Mikhail Korobov
faf9265c91 fixed compatibility with Twisted 15.4.0 2015-09-04 20:50:48 +05:00
Mikhail Lyundin
d022d3cb9e equal_attributes function optimization 2015-09-04 18:21:41 +03:00
Nicolás Alejandro Ramírez Quiros
6ae8963256 Merge pull request #1476 from mlyundin/master 
[MRG+1] Avoid creation of temporary list object in iflatten
 2015-09-02 17:31:04 -03:00
Mikhail Lyundin
d9fddabe95 Avoid creation of temporary list object in iflatten 2015-09-02 23:12:36 +03:00
Daniel Graña
dd473145f2 Merge pull request #1473 from ArturGaspar/master 
[MRG+1] Support for returning deferreds in middlewares
 2015-09-02 13:04:26 -03:00
Robert Weindl
833efbfcbd Update tutorial.rst 
Add missing "scrapy." prefix.
 2015-09-02 20:35:23 +05:00
Artur Gaspar
dd83f6123d Test for same response object in downloader middleware test. 2015-09-02 11:36:15 -03:00
Artur Gaspar
2748b38592 Test for not calling the download function when downloader middleware returns a response in process_request. 2015-09-02 11:15:37 -03:00
Artur Gaspar
1f4af9d81e Test for robots.txt middleware for logger not being called when the request for robots.txt is ignored. 2015-09-02 10:33:53 -03:00
Artur Gaspar
668e5fd257 Test for robots.txt middleware for processing a request for which the robots.txt parser is ready. 2015-09-02 10:23:24 -03:00
Artur Gaspar
a6a629e707 Call actual error logger in robots.txt middleware tests. 2015-09-02 10:23:24 -03:00
Artur Gaspar
9ce9a293a6 Always check robots.txt before making another request in RobotsTxtMiddleware. 2015-09-02 10:23:24 -03:00
Daniel Graña
af0873746f Merge pull request #1289 from pawelmhm/dont_retry_400 
[MRG+1] [settings/default_settings.py] dont retry 400
 2015-09-02 00:39:30 -03:00
Daniel Graña
3919ad64c5 Merge pull request #1470 from scrapy/py3-middlewares 
[MRG] port some downloader middlewares to Python 3
 2015-09-02 00:17:37 -03:00
Daniel Graña
bc499cb552 Merge pull request #1472 from olafdietsche/incomplete-submit-button 
[MRG+1] Incomplete submit button
 2015-09-02 00:16:42 -03:00
Artur Gaspar
ca83a0b028 Support for returning deferreds in downloader middleware methods. 2015-09-01 13:22:43 -03:00
Olaf Dietsche
937277e859 Make sure value attribute values are set. 2015-09-01 12:44:04 +02:00
Olaf Dietsche
2c28b53cc0 Add tests with incomplete buttons 2015-09-01 12:35:49 +02:00
Mikhail Korobov
c44cafe4f5 PY3 fix UserAgentMiddleware tests 2015-09-01 04:11:33 +05:00
Mikhail Korobov
3a9c73bc5d PY3 fix DownloaderStats middleware tests 2015-09-01 04:09:15 +05:00
Mikhail Korobov
78a4cd0f1c PY3 fix HttpAuthMiddleware tests 2015-09-01 04:00:26 +05:00
Mikhail Korobov
179a4409fb PY3 enable DownloadTimeoutMiddleware tests 2015-09-01 03:58:25 +05:00
Mikhail Korobov
3cf1911a92 PY3 fix DefaultHeadersMiddleware tests 2015-09-01 03:57:05 +05:00
Mikhail Korobov
45101829a5 PY3 fix AjacCrawlable middleware tests 2015-09-01 03:49:52 +05:00
Daniel Graña
b876755f1c Merge pull request #1469 from olafdietsche/form-submit-button-plain 
Form submit button plain
 2015-08-31 19:30:41 -03:00
Olaf Dietsche
121d7535be Allow button cotrols to submit form 2015-08-31 19:58:16 +02:00
Olaf Dietsche
e5f26078fa Add test: submit form with button control 2015-08-31 19:57:45 +02:00
Daniel Graña
0018caf0b6 Merge pull request #1461 from scrapy/py3-linkextractors 
[MRG] assorted Python 3 porting
 2015-08-31 11:59:30 -03:00
Daniel Graña
b4e382c867 Merge pull request #1318 from yarikoptic/fix-docs-explicit-rtd-theme-path 
[MRG+1] DOC(ENH): specify path to rtd theme explicitly
 2015-08-31 10:42:02 -03:00
Mikhail Korobov
44bfcbcf0f TST split LinkExtractorTestCase.test_extraction into several methods; remove duplicated test 2015-08-31 00:49:38 +05:00
Mikhail Korobov
f7052413e0 PY3 raise an exception if bytes are passed as url to Link constructor 2015-08-28 23:04:02 +05:00
Mikhail Korobov
9bfe6ece59 Merge branch 'master' into py3-linkextractors 
Conflicts:
	scrapy/linkextractors/lxmlhtml.py
	tests/test_linkextractors.py
 2015-08-28 04:53:32 +05:00
Mikhail Korobov
ff24cbbc47 PY3 depth, offsite and referer spider middlewares; Crawler 2015-08-28 04:11:30 +05:00
Mikhail Korobov
d5984bbea9 PY3 port scrapy.spiders 2015-08-28 04:11:30 +05:00
Mikhail Korobov
f2edbd05de PY3 port LinkExtractor 
* tests for other link extractors are moved to test_linkextractors_deprecated.py
* in Python 3 Link is converted to use native strings for urls
* minor cleanups
 2015-08-28 04:11:30 +05:00
Mikhail Korobov
f46a450080 refactor test_linkextractors 
* rename LinkExtractorTestCase to BaseSgmlLinkExtractorTestCase
* add BaseLinkExtractorTestCase link extractor tests can inherit from
  and decouple it from SgmlLinkExtractor
* add an extra check for deny_extensions
* xfail test_restrict_xpaths_with_html_entities for LxmlLinkExtractor explicitly
 2015-08-28 04:11:30 +05:00
Julia Medina
aa31811cfd Merge pull request #1214 from rgtk/link-rel 
[MRG+1] Support link rel attribute with multiple values
 2015-08-27 19:57:54 -03:00
Mikhail Korobov
9bfab53075 Merge pull request #1462 from tagatac/master 
minor: scrapy.Spider docs grammar
 2015-08-28 02:42:37 +05:00
David Tagatac
08162a15d8 minor: scrapy.Spider docs grammar 2015-08-27 17:37:16 -04:00
Rafał Gutkowski
cb3007c066 support link rel attribute with multiple values 2015-08-27 20:13:47 +02:00
Mikhail Korobov
8ecc4544b3 enable console tests in PY3 2015-08-27 21:37:24 +05:00
Mikhail Korobov
9616d91e4a Merge pull request #1444 from cyberplant/bpython_support 
[MRG +1] bpython support
 2015-08-27 21:28:05 +05:00