Julia Medina
4811d16f1d
Update logger attr and log method in the Spiders topic on docs
2015-04-22 17:24:41 -03:00
Julia Medina
d47a7edc65
Update Logging topic on docs
2015-04-22 17:24:40 -03:00
Julia Medina
ccdd8bfbcc
Parametrize log formatting strings
2015-04-22 17:24:40 -03:00
Julia Medina
21b9f377d6
Deprecate more frequently used functions from scrapy/log.py
2015-04-22 17:24:40 -03:00
Julia Medina
c174d78f12
Deprecate scrapy/log.py
2015-04-22 17:24:40 -03:00
Julia Medina
6acb3848fb
Stdout redirect in configure_logging
2015-04-22 17:24:40 -03:00
Julia Medina
ffd97f2f1b
Set root handlers based on settings in configure_logging
2015-04-22 17:24:40 -03:00
Julia Medina
1c8708eb82
Create a logger for every Spider and adapt Spider.log to log through it
2015-04-22 17:24:40 -03:00
Julia Medina
ac40ef611a
Custom handler to count log level occurrences in a crawler
2015-04-22 17:24:40 -03:00
Julia Medina
b75556ef79
Add a logging filter to mimic Twisted's log.err formating for Failures
2015-04-22 17:24:40 -03:00
Julia Medina
8baad55267
New scrapy/utils/log.py file with basic log helpers
...
There are two functions, `configure_logging` and `log_scrapy_info` which
intend to replace scrapy.log.start and scrapy.log.scrapy_info
respectively.
Creating new functions makes evident the backward incompatible change of
using another logging system, and since the Python logging module is a
standard builtin, additional helpers make sense to be on a scrapy/utils
file.
2015-04-22 17:24:40 -03:00
Julia Medina
6f9b423215
Restructure LogFormatter to comply with std logging calls
2015-04-22 17:24:40 -03:00
Julia Medina
c2d716807a
Use LogCapture in testfixtures package for tests
...
This allows to remove `get_testlog` helper, `flushLoggedErrors` from
twisted.trial.unittest.TestCase and Twisted log observers created for
each test on conftest.py.
2015-04-22 17:24:40 -03:00
Julia Medina
7a958f90be
Replace scrapy.log calls for their equivalents in the logging std module
...
Changes:
- Each module takes 'scrapy' logger and logs through it
- Lazy string evaluation in all log messages
- Added missing log messages in scrapy/core/engine.py
- Contextual data such as crawler or spider instances, and failures
2015-04-22 17:24:39 -03:00
Daniel Graña
571bf68d7d
Merge pull request #353 from nramirezuy/item-multi_inherit
...
[MRG] Item multi inheritance fix
2015-04-22 05:37:41 -03:00
nramirezuy
7871acd665
Item multi inherit fixed
2015-04-21 18:10:52 -03:00
Daniel Graña
ded3f9cb0e
Merge pull request #1185 from scrapy/fix-spiderloader-backwards-compatibility
...
[MRG +1]fixed backwards compatibility for SPIDER_MANAGER_CLASS option
2015-04-21 17:17:14 -03:00
Mikhail Korobov
c3d3a94914
fixed backwards compatibility for SPIDER_MANAGER_CLASS option
2015-04-22 00:29:40 +05:00
Pablo Hoffman
0a5bbbaed3
Merge pull request #1159 from scrapy/feed-export-fields
...
[MRG+1] FEED_EXPORT_FIELDS option
2015-04-21 15:54:24 -03:00
Pablo Hoffman
4c4eb4f77d
Merge pull request #1180 from eliasdorneles/tutorial-improvements
...
Some improvements for Scrapy tutorial
2015-04-21 15:49:39 -03:00
Daniel Graña
e034947b54
Merge pull request #1166 from scrapy/spider-loader
...
[MRG+1] rename SpiderManager to SpiderLoader
2015-04-21 15:42:17 -03:00
Pablo Hoffman
06e1ca9ef2
Merge pull request #1177 from Curita/remove-djangoitem
...
Remove djangoitem
2015-04-21 15:35:47 -03:00
Pablo Hoffman
e4122cdd79
Merge pull request #1150 from eliasdorneles/docs-files-pipeline
...
Documenting Files Pipeline together with Images Pipeline
2015-04-21 15:28:47 -03:00
Elias Dorneles
d63567531d
change data extraction in crawl example to be consistent with tutorial, removed statement implying mandatory usage of Item
2015-04-21 11:30:48 -03:00
Elias Dorneles
f7da69d116
fixing example CSS expr
2015-04-21 11:19:10 -03:00
Elias Dorneles
ff007afb9d
expanded crawling primer with examples, and applied other suggestions from the review
2015-04-21 10:57:44 -03:00
Mikhail Korobov
c8ccb7c5b9
Merge pull request #722 from Digenis/loader-iteration
...
[MRG +1]ItemLoader.load_item: iterate over copy of fields
2015-04-21 12:34:59 +05:00
Elias Dorneles
595146e158
some improvements for Scrapy tutorial
2015-04-20 21:10:07 -03:00
Julia Medina
0bf1ec160d
Merge branch 'nyov/remove-deploy' from pull request #1102
2015-04-20 20:36:33 -03:00
nyov
7ae37d61f9
remove scrapy deploy command
...
(closes #1027 , #1095 , #1102 )
2015-04-20 20:35:06 -03:00
Julia Medina
6da091f162
Remove djangoitem topic from index
2015-04-20 20:19:28 -03:00
Nikolaos-Digenis Karagiannis
d39722f6a3
ItemLoader.load_item: iterate over copy of fields
2015-04-20 11:05:56 +03:00
Nikolaos-Digenis Karagiannis
017fb25bee
loader test with processors that use item's values
2015-04-20 11:04:31 +03:00
Daniel Graña
16a1a938c1
Add 0.24.6 release notes
2015-04-20 01:40:21 -03:00
Mikhail Korobov
1794a893f4
Merge pull request #1172 from bagratte/docs
...
minor corrections in documentation.
2015-04-19 21:41:29 +05:00
Julia Medina
ffc60910aa
Remove djangoitem since we moved it to scrapy/scrapy-djangoitem
2015-04-19 13:09:25 -03:00
bagratte
1312bcd007
minor corrections in documentation.
2015-04-19 18:58:15 +04:00
bagratte
beea9267a1
minor corrections in documentation.
2015-04-18 19:48:25 +04:00
Mikhail Korobov
bb4c8c33cc
Merge pull request #1171 from rajathkumarmp/edit-doc
...
Added link to ipython in Tutorial.
2015-04-18 14:13:10 +05:00
rajathkumarmp
02629b5f7b
Added link to ipython in doc.
2015-04-18 13:00:34 +05:30
Pablo Hoffman
757309efea
Merge pull request #1169 from bagratte/docs
...
add some minor stylistic and grammar corrections to tutorial.rst.
2015-04-17 19:52:11 -03:00
José Ricardo
dffc4549d7
Fix small typo in the docs
2015-04-17 23:37:12 +05:00
bagratte
8d339da4e5
add some minor stylistic and grammar corrections to tutorial.rst.
2015-04-17 20:55:02 +04:00
Mikhail Korobov
ad587ea792
rename CrawlerRunner.spiders to CrawlerRunner.spider_loader
2015-04-17 01:54:26 +05:00
Mikhail Korobov
fee5565709
don't expose deprecated crawler.spiders attribute in telnet console
2015-04-17 01:25:48 +05:00
Mikhail Korobov
403e7c7c70
rename scrapy.spidermanager.SpiderManager to scrapy.spiderloader.SpiderLoader
2015-04-16 20:07:53 +05:00
Elias Dorneles
a1f3b3c7f0
images.rst -> media-pipeline.rst
2015-04-15 14:26:08 -03:00
Elias Dorneles
fd1c688ad6
addressing issues from review at #1150
2015-04-15 14:21:48 -03:00
Pablo Hoffman
fb85bd4b10
Merge pull request #1132 from sibiryakov/dns-options2
...
[MRG+1] Dns cache size and timeout options
2015-04-15 14:08:19 -03:00
Pablo Hoffman
20c8d712de
Merge pull request #1164 from scrapy/deploy-docs-2
...
deployment docs tweaks
2015-04-15 13:57:51 -03:00
