Julia Medina
d3d61fde87
Fix a recent mock regression by not using it in a test
2015-07-10 19:58:38 -03:00
Demelziraptor
9a15fcf89a
interpreting application/x-json as TextResponse
2015-07-02 19:56:30 +02:00
Daniel Graña
07f4f12e8b
Merge pull request #1313 from eliasdorneles/support-empty-passwd-for-http-proxy
...
[MRG+1] Support empty password for http_proxy config
2015-06-30 19:58:16 -03:00
Mikhail Korobov
ea4b175c2d
Merge pull request #1254 from scrapy/bugfix-1232
...
[MRG +1] Unquote request path before passing to FTPClient, it already escape paths
2015-06-24 18:04:45 +05:00
Elias Dorneles
eb8fed4ecd
support empty password for http_proxy config
2015-06-23 00:43:15 -03:00
Berker Peksag
8a48d9c6a8
Ignore ScrapyDeprecationWarning warnings properly.
...
Conflicts:
tests/test_utils_deprecate.py
2015-06-09 18:22:03 +03:00
Mikhail Korobov
d047665c02
make "settings" argument optional for Crawler, CrawlerRunner and CrawlerProcess
2015-06-06 03:23:13 +05:00
Mikhail Korobov
cc2f3e1b46
TST a test case to show custom_settings doesn't always work
2015-06-03 04:26:20 +05:00
Julia Medina
ffc7b7fd6c
Add helper to update deprecated class paths
2015-06-01 17:01:33 -03:00
Marven Sanchez
bb3ebf13f9
Add tests for RFC2616 policy enhancements
...
Add `scrapy/downloadermiddlewares/httpcache.py` to `tests/py3-ignores.txt
2015-06-01 18:20:12 +08:00
Mikhail Korobov
aa6a72707d
fixed CrawlerProcess when settings are passed as dicts
...
See https://github.com/scrapy/scrapy/pull/1156
2015-05-30 06:59:15 +05:00
Daniel Graña
ebe889a663
Unquote request path before passing to FTPClient, it already escape paths
2015-05-23 20:50:30 -03:00
Mikhail Korobov
cc2258b2bb
Merge pull request #1145 from bosnj/master
...
[MRG+1] default return value for extract_first
2015-05-21 22:03:54 +05:00
Daniel Graña
ee59112480
Merge pull request #1224 from scrapy/fix-empty-feed-export-fields
...
[MRG] fixed FEED_EXPORT_FIELDS handling (see #1223 )
2015-05-19 16:36:05 -03:00
Mikhail Korobov
7a5b5ec4d6
TST remove unnecessary lines from py3-ignores
...
scrapy/contrib is already skipped - see https://github.com/scrapy/scrapy/pull/1165
2015-05-19 00:57:39 +05:00
Mikhail Korobov
9b0ca1b7a0
drop support for FEED_EXPORT_FIELD=[] meaning "no fields"
2015-05-18 17:13:25 +05:00
Mikhail Korobov
9fb318338b
support FEED_EXPORT_FIELDS=[]
2015-05-18 16:44:02 +05:00
Mikhail Korobov
e1efd19175
TST, DOC document that Scrapy only infers field names for CSV
2015-05-18 16:43:23 +05:00
Mikhail Korobov
8d9e3b7e8d
TST add a test for JSON lines item exporter
2015-05-18 16:43:23 +05:00
Julia Medina
6dccb3a9b3
Replace FailureFormatter with direct exc_info conversions in log calls
2015-05-15 05:46:14 -03:00
Julia Medina
157cec7f64
Replace logger.exception calls for Python compatibility
2015-05-15 01:26:45 -03:00
Daniel Graña
5e5e44e91e
Merge pull request #1228 from nramirezuy/item_fields_defined_on_fields
...
item fields defined on fields class attribute fix
2015-05-14 23:09:24 -03:00
nramirezuy
773ea5a5ef
item fields defined on fields class attribute fix
2015-05-13 15:51:23 -03:00
Julia Medina
d3f576a816
Move scrapy/spider.py to scrapy/spiders/__init__.py
2015-05-09 04:20:09 -03:00
bosnj
8ae05478be
added docs and test case, fixed handling empty string vs None
2015-05-04 21:22:17 +02:00
Victoria Terenina (torymur)
667864ae94
testing robotstxt error
2015-04-30 16:28:57 +03:00
Julia Medina
b827097c91
Rename scrapy/statscol.py to scrapy/statscollectors.py
2015-04-29 21:36:40 -03:00
Julia Medina
9a3e3ba505
Move scrapy/contrib remaining top-level files to scrapy/extensions
2015-04-29 21:27:19 -03:00
Julia Medina
54a4ce069c
Rename scrapy/dupefilter.py to scrapy/dupefilters.py
2015-04-29 21:27:19 -03:00
Julia Medina
fc346cba4d
Move scrapy/contrib/spiders to scrapy/spiders
2015-04-29 21:27:19 -03:00
Julia Medina
f5bdf64f9c
Rename scrapy/squeue.py to scrapy/squeues.py
2015-04-29 21:27:19 -03:00
Julia Medina
645cdcbf9e
Rename scrapy/loader/processor.py to scrapy/loader/processors.py
2015-04-29 21:27:19 -03:00
Julia Medina
180272c092
Move scrapy/contrib/spidermiddleware to scrapy/spidermiddlewares
2015-04-29 21:26:35 -03:00
Julia Medina
8021df18d4
Move scrapy/contrib/pipeline to scrapy/pipelines
2015-04-29 21:26:35 -03:00
Julia Medina
b47228ada8
Move scrapy/contrib/loader to scrapy/loader
2015-04-29 21:24:30 -03:00
Julia Medina
cf064b1437
Move scrapy/contrib/linkextractors to scrapy/linkextractors
2015-04-29 21:24:30 -03:00
Julia Medina
7804b3d778
Move scrapy/contrib/exporter to scrapy/exporters
2015-04-29 21:24:30 -03:00
Julia Medina
d7c444fefb
Move scrapy/contrib/downloadermiddleware to scrapy/downloadermiddlewares
2015-04-29 21:24:30 -03:00
Mikhail Korobov
fbb1078f58
Merge pull request #1060 from Curita/python-logging
...
[MRG+1] Python logging
2015-04-29 23:20:34 +05:00
Julia Medina
cc4c31e426
Relax SpiderLoader interface check
2015-04-23 15:08:04 -03:00
Julia Medina
6d1205063c
Add a filter to replace '__name__' loggers with 'scrapy'
2015-04-22 17:24:41 -03:00
Julia Medina
c174d78f12
Deprecate scrapy/log.py
2015-04-22 17:24:40 -03:00
Julia Medina
6acb3848fb
Stdout redirect in configure_logging
2015-04-22 17:24:40 -03:00
Julia Medina
1c8708eb82
Create a logger for every Spider and adapt Spider.log to log through it
2015-04-22 17:24:40 -03:00
Julia Medina
ac40ef611a
Custom handler to count log level occurrences in a crawler
2015-04-22 17:24:40 -03:00
Julia Medina
b75556ef79
Add a logging filter to mimic Twisted's log.err formating for Failures
2015-04-22 17:24:40 -03:00
Julia Medina
6f9b423215
Restructure LogFormatter to comply with std logging calls
2015-04-22 17:24:40 -03:00
Julia Medina
c2d716807a
Use LogCapture in testfixtures package for tests
...
This allows to remove `get_testlog` helper, `flushLoggedErrors` from
twisted.trial.unittest.TestCase and Twisted log observers created for
each test on conftest.py.
2015-04-22 17:24:40 -03:00
Julia Medina
7a958f90be
Replace scrapy.log calls for their equivalents in the logging std module
...
Changes:
- Each module takes 'scrapy' logger and logs through it
- Lazy string evaluation in all log messages
- Added missing log messages in scrapy/core/engine.py
- Contextual data such as crawler or spider instances, and failures
2015-04-22 17:24:39 -03:00
Daniel Graña
571bf68d7d
Merge pull request #353 from nramirezuy/item-multi_inherit
...
[MRG] Item multi inheritance fix
2015-04-22 05:37:41 -03:00
