Pablo Hoffman
296db1dc09
log (just once) when duplicate requests are filtered out. closes #105 , #249
2013-03-12 19:28:43 -03:00
Pablo Hoffman
a4507f62c3
sep-019: fixed typo
2013-03-12 18:44:09 -03:00
Pablo Hoffman
b2b256dc37
added nicolas to sep-019 authors, changed main title formatting for sep-019 and sep-020
2013-03-10 14:22:08 -03:00
Pablo Hoffman
b11a1326b9
Merge pull request #264 from stav/sep
...
add new sep-20 and update sep-19 with table at the top
2013-03-08 11:23:15 -08:00
Steven Almeroth
e66db6fa56
add new sep-20 and update sep-19 with table at the top
2013-03-08 13:04:11 -06:00
Pablo Hoffman
eeb69d2f70
added #260 to release notes
2013-03-08 11:59:38 -02:00
Pablo Hoffman
42499c72b1
updated AUTHORS
2013-03-08 11:57:54 -02:00
Pablo Hoffman
b5b37197bf
Merge pull request #260 from llonchj/entry_point
...
Commands module
2013-03-08 05:49:08 -08:00
Pablo Hoffman
b560f4d1c1
sep-019: remove TODO section (settled down with class method)
2013-03-08 09:57:12 -02:00
Pablo Hoffman
a44757179b
sep-019: other minor fixes
2013-03-07 19:38:12 -02:00
Pablo Hoffman
c7add1e5bd
sep-019: fix typos
2013-03-07 19:24:44 -02:00
Pablo Hoffman
d806acdf50
sep-019: minor tide up
2013-03-07 12:39:40 -02:00
Pablo Hoffman
6f24b46115
added SEP-019 (per-spider settings) - first draft
2013-03-07 12:34:17 -02:00
Pablo Hoffman
a027da0a2c
Some changes to scrapy deploy command:
...
- use branch names when constructing HG/GIT versions
- added -d flag for debugging (keeps build directory)
- improved command output (redirected setup.py stderr)
2013-03-06 11:41:34 -02:00
Jordi Llonch
5b118ff4ab
added documentation (experimental feature)
2013-03-06 06:36:23 +11:00
Jordi Llonch
d9261f6c54
pluggable sub-commands for scrapy comand-line
2013-03-06 05:25:38 +11:00
Pablo Hoffman
19d0942c74
Merge branch 'shell' of git://github.com/stav/scrapy into stav-shell
2013-03-04 02:14:01 -02:00
Pablo Hoffman
3c8eef99cb
docs/contributing: added note explaining what Scrapy contrib is
2013-03-04 01:35:17 -02:00
Pablo Hoffman
7dd360f39f
Merge pull request #257 from stav/cleanups
...
doc: fix typo in spider middleware
2013-03-03 19:17:46 -08:00
Steven Almeroth
81111dd39a
fetch command should catch IgnoreRequest exception
2013-03-02 20:13:14 -06:00
Steven Almeroth
f62b6660d4
doc: fix typo in spider middleware
2013-03-02 19:46:31 -06:00
Pablo Hoffman
d5d944fa44
Log overriden Scrapy settings when Scrapy starts.
...
This is useful for debugging, to quickly find out which settings where
used for a specific spider run. dict settings are omitted for brevity.
DEBUG level is used for consistency with "Enabled
extensions/middlewares" lines (which share a similar purpose).
2013-02-28 11:35:11 -02:00
Pablo Hoffman
8f3a509d44
remove debugging code
2013-02-27 03:52:55 -02:00
Pablo Hoffman
3a6b80259d
arg_to_iter: replaced double call to isinstance with a single one. refs #248
2013-02-27 03:51:23 -02:00
Pablo Hoffman
2bbd92742b
arg_to_iter: treat items the same way as dicts (ie. non iterables). fixes #248
2013-02-27 02:39:31 -02:00
Pablo Hoffman
7400ceb1ed
added 502 to RETRY_HTTP_CODES
2013-02-22 19:12:59 -02:00
Pablo Hoffman
e3f50b97b2
made scrapy/item.py pep8 compliant
2013-02-20 11:38:50 -02:00
Pablo Hoffman
a038f46859
doc: fixed rst title
2013-02-14 11:11:17 -02:00
Pablo Hoffman
22edc44c6c
doc: remove links to diveintopython.org, which is no longer available. closes #246
2013-02-14 11:09:40 -02:00
Pablo Hoffman
aeb7fbe221
Merge pull request #240 from llonchj/spider_log
...
spider.log to pass keyword arguments into twisted log.msg
2013-02-12 18:13:26 -08:00
Pablo Hoffman
07669947c7
Merge pull request #244 from morty/master
...
scrapy genspider allows you to create a spider module with the same name as the project
2013-02-12 18:11:49 -08:00
Tom Mortimer-Jones
2efd859525
Added check so that genspider cannot create a spider with the same name as the project.
2013-02-12 12:26:38 +00:00
Pablo Hoffman
1ff8b4f831
updated release notes with previous commit
2013-02-12 00:59:25 -02:00
Rolando Espinoza La fuente
5971333a93
added --pdb command option to enable pdb debugger on failure.
2013-02-11 12:34:29 -04:00
Pablo Hoffman
d9043ffa6e
reverted previous changed as it broke tests
2013-02-11 11:43:25 -02:00
Pablo Hoffman
2df010a972
Merge pull request #241 from zuhao/master
...
Fix url_has_any_extension bug
2013-02-11 05:29:14 -08:00
Zuhao Wan
27ca25472f
Fix url_has_any_extension bug
2013-02-11 17:19:31 +08:00
Jordi Llonch
6d4b764f29
spider.log to pass keyword arguments into twisted log.msg
2013-02-11 05:00:06 +11:00
Daniel Graña
c0a7040f16
add PHONYs so build does not match build/ path
2013-02-08 15:07:00 -02:00
Daniel Graña
910effd145
get scrapy version from package data
2013-02-06 11:44:26 -02:00
Daniel Graña
79cb0318d0
update travis-ci conf
2013-02-06 10:36:08 -02:00
Daniel Graña
5db45b3825
remove scrapyd, it was migrated to its own repository
2013-02-06 05:24:07 +00:00
Daniel Graña
33ca295129
Merge pull request #235 from whodatninja/patch-1
...
Fix typo labeling attrs type bool instead of list
2013-02-05 17:27:10 -08:00
whodatninja
8e3b5baac5
Fix typo labeling attrs type bool instead of list
2013-02-05 15:10:41 -05:00
Daniel Graña
f293d08e7a
Merge pull request #234 from darkrho/httpcache-dbm-module-fullpath
...
allow to use full path in HTTPCACHE_DBM_MODULE setting
2013-02-01 09:01:27 -08:00
Rolando Espinoza La fuente
d00e6cf1d7
tests: dummydbm.error should be KeyError.
2013-02-01 12:52:59 -04:00
Rolando Espinoza La fuente
6ed44a3a67
httpcache: added tests for custom dbm module.
...
And removed the break in the line to improve readability as there
is no strict 80-chars line width convention.
2013-02-01 12:41:52 -04:00
Rolando Espinoza La fuente
a2602df90b
httpcache: allow to import submodules within packages as HTTPCACHE_DBM_MODULE setting.
2013-01-31 23:49:36 -04:00
Daniel Graña
e5edb8ec3e
Merge branch 'dangra/issue-24'
2013-01-30 16:55:53 -02:00
Daniel Graña
3af240f5f9
Merge branch 'dangra/issue-12'
2013-01-30 16:55:16 -02:00
