Daniel Graña
5b5dd679b0
Add 0.18.4 release notes
...
Conflicts:
docs/news.rst
2013-10-10 01:04:35 -02:00
Daniel Graña
aa6fb7daaa
IPython refuses to update the namespace. fix #396
...
IPython embedding code borrowed from https://github.com/mitsuhiko/werkzeug/pull/85
2013-10-10 00:57:14 -02:00
Daniel Graña
7b1288ba54
Fix AlreadyCalledError replacing a request in shell command. closes #407
2013-10-09 23:50:35 -02:00
Daniel Graña
5eb429999e
Fix start_requests lazyness and early hangs
...
- Removes new public methods added by #330 to Crawler and CrawlerProcess
- Add test for start_requests lazy evaluation
- Fix and test hangs when start_requests erroed before returning the generator
- Add test when start_requests fails while generating requests
- Simplify Crawler and CrawlerProcess implementation taking in count
that only one spider can be attached per Crawler. As required by SEP-019
- "scrapy settings" command do not require starting a Crawler anymore
2013-10-09 22:58:10 -02:00
Pablo Hoffman
2bd5002fb4
Merge pull request #416 from redapple/attrnamespaces
...
.remove_namespaces(): remove namespaces on elements' attributes
2013-10-09 15:34:08 -07:00
Pablo Hoffman
e1683ddf9b
fix doc typo
2013-10-09 17:24:12 -02:00
Pablo Hoffman
ce82d22c9e
Merge pull request #423 from nramirezuy/defaultheaders-multispider
...
defaultheaders multi spider support removed
2013-10-09 09:03:33 -07:00
nramirezuy
332bf3b6b7
defaultheaders multi spider support removed
2013-10-09 13:50:15 -02:00
Pablo Hoffman
7ec01799d5
Merge pull request #419 from nramirezuy/robotstxt-multispider
...
robotstxt mid multi spider support removed
2013-10-09 07:43:11 -07:00
Pablo Hoffman
9ad736d7ce
Merge pull request #420 from nramirezuy/httpauth-multispider
...
httpauth mid multispider support removed
2013-10-08 21:06:49 -07:00
Pablo Hoffman
27d3cbb64f
Merge pull request #421 from nramirezuy/useragent-multispider
...
useragent mid multi spider support removed
2013-10-08 21:06:13 -07:00
Pablo Hoffman
37b01a229c
Merge pull request #422 from nramirezuy/downloadtimeout-multispider
...
downloadtimeout mid multi spider support removed
2013-10-08 21:05:06 -07:00
nramirezuy
1d09b6e1f2
downloadtimeout mid multi spider support removed
2013-10-08 19:03:26 -02:00
nramirezuy
fbb0ad6419
useragent mid multi spider support removed
2013-10-08 18:18:49 -02:00
nramirezuy
f56f547738
httpauth mid multispider support removed
2013-10-08 18:00:25 -02:00
nramirezuy
10646f1768
robotstxt mid multi spider support removed
2013-10-08 16:38:08 -02:00
Pablo Hoffman
3501c2025e
remove no longer existent examples from doc_files used in bdist_rpm. closes GH-417
2013-10-08 15:18:45 -02:00
Paul Tremberth
d806184bb6
.remove_namespaces(): remove namespaces on elements' attributes
2013-10-08 00:01:44 +02:00
Pablo Hoffman
8b9526a8f6
Merge pull request #400 from irgmedeiros/patch-2
...
Update the second code example
2013-10-07 07:57:18 -07:00
Pablo Hoffman
86c6e9433f
remove minor reference to 'scrapy server' command
2013-10-04 14:37:55 -03:00
Daniel Graña
aad90ec5a2
Add 0.18.3 release notes
...
Conflicts:
docs/news.rst
2013-10-03 12:56:25 -03:00
Daniel Graña
479b4942bd
fix regression on lazy evaluation of start requests
2013-10-03 12:49:16 -03:00
Pablo Hoffman
662eac11fa
added pypi version badge to README
2013-10-03 12:47:22 -03:00
Pablo Hoffman
ba7b75d299
Merge pull request #410 from duendex/httpsMockServer
...
Adds HTTPS support to the MockServer.
2013-10-02 14:05:44 -07:00
duendex
2519b3031e
Adds HTTPS support to the MockServer.
2013-10-02 17:35:28 -03:00
Daniel Graña
0c63c9539d
Merge pull request #409 from kalessin/imagesfield
...
allow to set source and destination fields in files and images pipelines, and added tests
2013-10-02 13:31:26 -07:00
Martin Olveyra
e81120e683
allow to set source and destination fields in files and images pipelines, and added tests
2013-10-02 18:23:13 -02:00
Pablo Hoffman
37c24e01d7
document bindaddress request meta
2013-10-02 17:13:17 -03:00
Pablo Hoffman
a9c3519897
updated required twisted version to 10.0
2013-10-01 14:07:38 -03:00
Pablo Hoffman
fe9ae1d46a
Merge pull request #406 from nopper/master
...
Handling inconsistencies among Twisted releases
2013-10-01 09:59:00 -07:00
Francesco Piccinno
cc8a93f185
Handling inconsistencies among Twisted releases
2013-10-01 14:36:44 +02:00
Pablo Hoffman
56275b1b9c
Merge pull request #404 from darkrho/djangoitem-docs
...
Add a section to DjangoItem docs page regarding setting up Django's settings.
2013-09-30 11:15:32 -07:00
Rolando Espinoza
d6e3eae527
docs: added section regarding setting up django's settings.
2013-09-30 09:58:10 -04:00
Rolando Espinoza
0cc1d870db
docs: minor tidy up sample code and missing shell prompts.
2013-09-30 09:58:10 -04:00
Mikhail Korobov
36cb54d302
Merge pull request #402 from LorenDavie/master
...
Corrected typo.
2013-09-29 14:21:04 -07:00
Loren Davie
8af0e89e85
Corrected typo.
2013-09-29 17:06:46 -04:00
Pablo Hoffman
1a90ee51c0
Merge pull request #401 from LorenDavie/master
...
Added dynamic creation of item classes to practices.rst. Issue
2013-09-29 07:20:52 -07:00
Loren Davie
f49f5724d5
Added dynamic creation of item classes to practices.rst.
2013-09-28 09:00:48 -04:00
irgmedeiros
9b50409986
Update the second code example
...
Update the second code example to reflect the last change in the first example.
2013-09-27 18:22:33 -03:00
Pablo Hoffman
780ef128d2
Merge pull request #399 from irgmedeiros/patch-1
...
Update practices.rst
2013-09-27 14:03:51 -07:00
irgmedeiros
d9e0fdc9aa
Update practices.rst
...
With this modification scrapy runs the spider with project settings. The previous example ran only with default settings resulting in ignoring all user settings as pipelines for example.
2013-09-27 17:56:30 -03:00
Daniel Graña
265910aae6
Merge pull request #363 from taikano/sitemap_alternate
...
also fetch alternate URLs from sitemaps, see #360
2013-09-26 09:15:02 -07:00
Pablo Hoffman
12280c2a95
fix sphinx references in doc
2013-09-25 15:13:17 -03:00
Mikhail Korobov
b74e5aa551
fix ITEM_PIPELINES setting handling
...
* item_pipelines variable was unused so the fallback didn't work;
* added support for fallback in case of ITEM_PIPELINIES defined as set.
2013-09-25 20:37:06 +06:00
Daniel Graña
61e89d8247
Merge pull request #393 from scrapy/itempipe-dict
...
Make ITEM_PIPELINE setting a dict
2013-09-23 14:12:26 -07:00
Pablo Hoffman
fc388f4636
Make ITEM_PIPELINE setting a dict
...
This is for consistency with how spider and downloader middlewares are
defined. ITEM_PIPELINE_BASE was also added and both remain empty.
Backwards compatibility is kept (with a warning) with list-based
ITEM_PIPELINES.
2013-09-23 17:50:43 -03:00
Pablo Hoffman
b1d1a36a1e
add note about enclosing urls with quotes when running from command-line. closes GH-384
2013-09-18 18:01:28 -03:00
Mikhail Korobov
bb8cb41777
Merge pull request #391 from amferraz/patch-2
...
Update request-response.rst
2013-09-18 07:59:46 -07:00
cacovsky
71b320914a
Update request-response.rst
...
Fix small doc typo (too many backticks)
2013-09-18 11:45:25 -03:00
Pablo Hoffman
ee8464b471
Merge pull request #390 from kmike/mock-in-tox
...
mock is required for running tests
2013-09-17 15:51:54 -07:00
