Capi Etheriel
|
50fa46d183
|
Document CrawlSpider.parse_start_urls method
|
2013-06-09 04:03:20 -03:00 |
|
Pablo Hoffman
|
5a097cc192
|
log optional features available on startup
|
2013-06-04 19:25:26 -03:00 |
|
Daniel Graña
|
8ea041528f
|
Merge pull request #318 from scrapy/http11
Add HTTP 1.1 download handler
|
2013-06-04 14:34:54 -07:00 |
|
Daniel Graña
|
e7acc2171d
|
show deprecation warning for HttpDownloadHandler
|
2013-06-04 17:28:43 -03:00 |
|
Daniel Graña
|
cf10474b8b
|
improve twisted version tests and enable HTTP11 handler by default if supported
|
2013-06-04 16:59:35 -03:00 |
|
Daniel Graña
|
bb806fa0f8
|
restore http10 as default handler and skip tests if http1.1 is not supported
|
2013-06-04 16:36:21 -03:00 |
|
Daniel Graña
|
8b0c9466d6
|
add required files to support twisted 11.1 (precise)
|
2013-06-04 15:35:31 -03:00 |
|
Daniel Graña
|
7729f939e9
|
implement download timeouts based on deferred cancellation
|
2013-06-03 19:18:45 -03:00 |
|
Pablo Hoffman
|
4ac0eea55d
|
Merge pull request #316 from nramirezuy/closespider/errorcount
standarizes close_spider to use signals
|
2013-05-31 12:20:58 -07:00 |
|
nramirezuy
|
b61a4e719c
|
standarizes close_spider to use signals
|
2013-05-31 15:54:17 -03:00 |
|
Daniel Graña
|
b4fca90bba
|
merge 0.16.5 release notes
|
2013-05-30 18:49:00 -03:00 |
|
Daniel Graña
|
58dc16f48d
|
obey request method when scrapy deploy is redirected to a new endpoint
|
2013-05-30 18:34:59 -03:00 |
|
Pablo Hoffman
|
0effe139e5
|
Merge pull request #311 from nramirezuy/logstats-6017
removed multispider support from logstats
|
2013-05-30 08:47:33 -07:00 |
|
nramirezuy
|
14e382ae48
|
test_closespider fixed for python 2.6
|
2013-05-30 10:57:06 -03:00 |
|
Pablo Hoffman
|
461eace4af
|
Merge pull request #313 from nramirezuy/closespider-6017
drop multi-spider support from CloseSpider extension
|
2013-05-30 06:34:52 -07:00 |
|
nramirezuy
|
971ef5c78e
|
removed multispider support, test added
|
2013-05-28 17:39:37 -03:00 |
|
Pablo Hoffman
|
2bb2ba66f6
|
simplify scrapy.tests.mockserver
|
2013-05-28 15:12:38 -03:00 |
|
Pablo Hoffman
|
f35266e6a5
|
use os.pathsep
|
2013-05-28 15:10:05 -03:00 |
|
nramirezuy
|
96b8fb69be
|
removed multispider support from logstats
|
2013-05-28 14:45:02 -03:00 |
|
Pablo Hoffman
|
98aa3efea7
|
Merge pull request #312 from nramirezuy/cmd/bech/test
added test for bench command
|
2013-05-28 08:46:31 -07:00 |
|
nramirezuy
|
1ed5643eb2
|
added test for bench command
|
2013-05-28 12:42:50 -03:00 |
|
Pablo Hoffman
|
a4b5bfbb5e
|
Merge pull request #309 from amferraz/patch-1
Add FAQ entry referencing Request.meta usage
|
2013-05-27 09:07:48 -07:00 |
|
cacovsky
|
8007762890
|
Add FAQ entry referencing Request.meta usage
|
2013-05-27 13:02:17 -03:00 |
|
Nicolás Alejandro Ramírez Quiros
|
7874eef7af
|
Merge pull request #307 from DeaconDesperado/raise-format
Raise a usage error when an invalid or unrecognized output format is entered on Command line
|
2013-05-23 12:50:17 -07:00 |
|
Mark Grey
|
485a954571
|
raise on unrecognized format
|
2013-05-23 15:11:45 -04:00 |
|
Pablo Hoffman
|
845c64b89d
|
add benchmarking to 0.18 release notes
|
2013-05-17 10:38:42 -03:00 |
|
Pablo Hoffman
|
ca12886acb
|
update copyright notes
|
2013-05-16 15:05:52 -03:00 |
|
Pablo Hoffman
|
8e49fed918
|
minor improvements to benchmarking doc
|
2013-05-16 13:23:13 -03:00 |
|
Pablo Hoffman
|
76087e336a
|
add scrapy bench command for benchmarking, with documentation
|
2013-05-16 13:15:25 -03:00 |
|
Pablo Hoffman
|
5c40741d65
|
added context manager for mock server, moved test spiders into a separate module (scrapy.tests.spiders)
|
2013-05-16 13:01:02 -03:00 |
|
Pablo Hoffman
|
214bcdf3be
|
Merge pull request #301 from tpeng/fix_telnet_doc
change manager in telnet variable to crawler
|
2013-05-08 07:00:52 -07:00 |
|
tpeng
|
edbe525d19
|
change manager in telnet variable to crawler
|
2013-05-08 11:58:15 +08:00 |
|
Pablo Hoffman
|
99f8d5733a
|
added mock tests for retries (more to come)
|
2013-05-06 20:04:15 -03:00 |
|
Pablo Hoffman
|
a7b7c89216
|
improved logging of downloader errors, when they contain tracebacks
|
2013-05-06 20:03:54 -03:00 |
|
Pablo Hoffman
|
db195ee4df
|
added more methods to mockserver
|
2013-05-06 20:03:17 -03:00 |
|
Pablo Hoffman
|
c2561ca160
|
use get_testenv() shortcut instead of get_pythonpath()
|
2013-05-06 17:35:09 -03:00 |
|
Pablo Hoffman
|
14e7121674
|
make sure the mockserver forked from tests uses the current installation of scrapy, not the system one
|
2013-05-06 17:27:59 -03:00 |
|
Pablo Hoffman
|
cf4d4bc094
|
added mock server test for DOWNLOAD_TIMEOUT
|
2013-05-06 14:47:24 -03:00 |
|
Daniel Graña
|
495acba223
|
agents requires an instance of contextFactory
|
2013-05-06 12:56:13 -03:00 |
|
Daniel Graña
|
3e62ce35a0
|
cleanup http connection pool on engine stop
|
2013-05-06 12:56:13 -03:00 |
|
Daniel Graña
|
334ad71a2e
|
adapt for singletons removals
|
2013-05-06 12:56:13 -03:00 |
|
Daniel Graña
|
ba6545555f
|
empty bodies does not require a body producer
|
2013-05-06 12:56:13 -03:00 |
|
Daniel Graña
|
db232da068
|
close pool connections before finishing tests
|
2013-05-06 12:56:13 -03:00 |
|
Daniel Graña
|
b34606030d
|
remove duplicate context factory handling for non-ssl support in http1.0
|
2013-05-06 12:56:13 -03:00 |
|
Daniel Graña
|
0a26170086
|
move ssl context factory to its own module and implement a non-ssl version that warns about pyopenssl support
|
2013-05-06 12:52:58 -03:00 |
|
Daniel Graña
|
ab3407289c
|
enable persistent connections
|
2013-05-06 12:50:14 -03:00 |
|
Daniel Graña
|
e4fe7c63b0
|
add http connection pool and custom ssl context factory
|
2013-05-06 12:50:13 -03:00 |
|
Daniel Graña
|
a7a354f982
|
http11 cleanup
|
2013-05-06 12:45:27 -03:00 |
|
paul
|
ef03603869
|
Restore handling of HTTPS
|
2013-05-06 12:45:27 -03:00 |
|
paul
|
46341d5275
|
Renamed downloader to Http11DownloadHandler and some refactoring
Only for HTTP, not HTTPS
Test on expected body length instead of request method (HEAD case)
|
2013-05-06 12:45:27 -03:00 |
|