Julia Medina
7804b3d778
Move scrapy/contrib/exporter to scrapy/exporters
2015-04-29 21:24:30 -03:00
Julia Medina
6b4c00cc9b
scrapy/contrib/downloadermiddleware shims
2015-04-29 21:24:30 -03:00
Julia Medina
d7c444fefb
Move scrapy/contrib/downloadermiddleware to scrapy/downloadermiddlewares
2015-04-29 21:24:30 -03:00
Pablo Hoffman
9441761aef
Merge pull request #1196 from mineo/patch-1
...
Remove a duplicate word
2015-04-29 17:48:19 -03:00
Wieland Hoffmann
de6501ed1b
Remove a duplicate word
2015-04-29 22:31:48 +02:00
Pablo Hoffman
3d2b74a6ff
Merge pull request #1188 from eliasdorneles/favoring_web_scraping_over_screen_scraping
...
[MRG+1] Favoring web scraping over screen scraping in the descriptions
2015-04-29 16:49:43 -03:00
Mikhail Korobov
fbb1078f58
Merge pull request #1060 from Curita/python-logging
...
[MRG+1] Python logging
2015-04-29 23:20:34 +05:00
Daniel Graña
5eb098a939
Merge pull request #1168 from scrapy/service-identity
...
install service_identity package in tests to prevent warnings
2015-04-28 23:48:58 -03:00
Elias Dorneles
3d3633f3d2
favoring web scraping over screen scraping in the descriptions
2015-04-25 11:20:20 -03:00
Mikhail Korobov
fa1039f5b2
Merge pull request #1187 from Curita/relax-spiderloader-check
...
Relax SpiderLoader interface check
2015-04-24 03:40:10 +05:00
Julia Medina
cc4c31e426
Relax SpiderLoader interface check
2015-04-23 15:08:04 -03:00
Julia Medina
1d8f8221e6
Add backward compatibility to LogFormatter
2015-04-22 17:27:24 -03:00
Julia Medina
4858af4e94
Fix backward compatible functions in scrapy.log
2015-04-22 17:27:24 -03:00
Julia Medina
7a92dae4c8
Change Scrapy log output through docs
2015-04-22 17:27:24 -03:00
Julia Medina
6d1205063c
Add a filter to replace '__name__' loggers with 'scrapy'
2015-04-22 17:24:41 -03:00
Julia Medina
4f54ca3294
Change 'scrapy' logger for '__name__' on every module
2015-04-22 17:24:41 -03:00
Julia Medina
69a3d58110
Basic example on manually configuring log handlers
2015-04-22 17:24:41 -03:00
Julia Medina
bd0b639b21
Fix logging usage across docs
2015-04-22 17:24:41 -03:00
Julia Medina
4811d16f1d
Update logger attr and log method in the Spiders topic on docs
2015-04-22 17:24:41 -03:00
Julia Medina
d47a7edc65
Update Logging topic on docs
2015-04-22 17:24:40 -03:00
Julia Medina
ccdd8bfbcc
Parametrize log formatting strings
2015-04-22 17:24:40 -03:00
Julia Medina
21b9f377d6
Deprecate more frequently used functions from scrapy/log.py
2015-04-22 17:24:40 -03:00
Julia Medina
c174d78f12
Deprecate scrapy/log.py
2015-04-22 17:24:40 -03:00
Julia Medina
6acb3848fb
Stdout redirect in configure_logging
2015-04-22 17:24:40 -03:00
Julia Medina
ffd97f2f1b
Set root handlers based on settings in configure_logging
2015-04-22 17:24:40 -03:00
Julia Medina
1c8708eb82
Create a logger for every Spider and adapt Spider.log to log through it
2015-04-22 17:24:40 -03:00
Julia Medina
ac40ef611a
Custom handler to count log level occurrences in a crawler
2015-04-22 17:24:40 -03:00
Julia Medina
b75556ef79
Add a logging filter to mimic Twisted's log.err formating for Failures
2015-04-22 17:24:40 -03:00
Julia Medina
8baad55267
New scrapy/utils/log.py file with basic log helpers
...
There are two functions, `configure_logging` and `log_scrapy_info` which
intend to replace scrapy.log.start and scrapy.log.scrapy_info
respectively.
Creating new functions makes evident the backward incompatible change of
using another logging system, and since the Python logging module is a
standard builtin, additional helpers make sense to be on a scrapy/utils
file.
2015-04-22 17:24:40 -03:00
Julia Medina
6f9b423215
Restructure LogFormatter to comply with std logging calls
2015-04-22 17:24:40 -03:00
Julia Medina
c2d716807a
Use LogCapture in testfixtures package for tests
...
This allows to remove `get_testlog` helper, `flushLoggedErrors` from
twisted.trial.unittest.TestCase and Twisted log observers created for
each test on conftest.py.
2015-04-22 17:24:40 -03:00
Julia Medina
7a958f90be
Replace scrapy.log calls for their equivalents in the logging std module
...
Changes:
- Each module takes 'scrapy' logger and logs through it
- Lazy string evaluation in all log messages
- Added missing log messages in scrapy/core/engine.py
- Contextual data such as crawler or spider instances, and failures
2015-04-22 17:24:39 -03:00
Daniel Graña
571bf68d7d
Merge pull request #353 from nramirezuy/item-multi_inherit
...
[MRG] Item multi inheritance fix
2015-04-22 05:37:41 -03:00
nramirezuy
7871acd665
Item multi inherit fixed
2015-04-21 18:10:52 -03:00
Daniel Graña
ded3f9cb0e
Merge pull request #1185 from scrapy/fix-spiderloader-backwards-compatibility
...
[MRG +1]fixed backwards compatibility for SPIDER_MANAGER_CLASS option
2015-04-21 17:17:14 -03:00
Mikhail Korobov
c3d3a94914
fixed backwards compatibility for SPIDER_MANAGER_CLASS option
2015-04-22 00:29:40 +05:00
Pablo Hoffman
0a5bbbaed3
Merge pull request #1159 from scrapy/feed-export-fields
...
[MRG+1] FEED_EXPORT_FIELDS option
2015-04-21 15:54:24 -03:00
Pablo Hoffman
4c4eb4f77d
Merge pull request #1180 from eliasdorneles/tutorial-improvements
...
Some improvements for Scrapy tutorial
2015-04-21 15:49:39 -03:00
Daniel Graña
e034947b54
Merge pull request #1166 from scrapy/spider-loader
...
[MRG+1] rename SpiderManager to SpiderLoader
2015-04-21 15:42:17 -03:00
Pablo Hoffman
06e1ca9ef2
Merge pull request #1177 from Curita/remove-djangoitem
...
Remove djangoitem
2015-04-21 15:35:47 -03:00
Pablo Hoffman
e4122cdd79
Merge pull request #1150 from eliasdorneles/docs-files-pipeline
...
Documenting Files Pipeline together with Images Pipeline
2015-04-21 15:28:47 -03:00
Elias Dorneles
d63567531d
change data extraction in crawl example to be consistent with tutorial, removed statement implying mandatory usage of Item
2015-04-21 11:30:48 -03:00
Elias Dorneles
f7da69d116
fixing example CSS expr
2015-04-21 11:19:10 -03:00
Elias Dorneles
ff007afb9d
expanded crawling primer with examples, and applied other suggestions from the review
2015-04-21 10:57:44 -03:00
Mikhail Korobov
c8ccb7c5b9
Merge pull request #722 from Digenis/loader-iteration
...
[MRG +1]ItemLoader.load_item: iterate over copy of fields
2015-04-21 12:34:59 +05:00
Elias Dorneles
595146e158
some improvements for Scrapy tutorial
2015-04-20 21:10:07 -03:00
Julia Medina
0bf1ec160d
Merge branch 'nyov/remove-deploy' from pull request #1102
2015-04-20 20:36:33 -03:00
nyov
7ae37d61f9
remove scrapy deploy command
...
(closes #1027 , #1095 , #1102 )
2015-04-20 20:35:06 -03:00
Julia Medina
6da091f162
Remove djangoitem topic from index
2015-04-20 20:19:28 -03:00
Nikolaos-Digenis Karagiannis
d39722f6a3
ItemLoader.load_item: iterate over copy of fields
2015-04-20 11:05:56 +03:00
