Daniel Graña 64ba6e7982 Dump stacks for all running threads and fix engine status dumped by StackTraceDump extension

2011-12-15 17:05:45 -02:00

..

commands

added support for using '-' in scrapy crawl -o, to dump items to standard output

2011-10-23 03:06:59 -02:00

contrib

Dump stacks for all running threads and fix engine status dumped by StackTraceDump extension

2011-12-15 17:05:45 -02:00

contrib_exp

moved module scrapy.core.downloader.responsetypes to scrapy.responsetypes

2011-08-07 02:49:57 -03:00

core

Avoid _disconnectedDeferred AttributeError exception in Twisted>=11.1.0

2011-12-12 18:11:43 +00:00

http

added support for ajax crawleable urls

2011-10-28 16:33:12 -02:00

selector

avoid stripping trailing spaces on lxml-based selectors. closes #270

2010-10-27 21:39:28 -02:00

settings

fixed PickeItemExporter bug, added unittest, and added pickle to suported feed exports formats

2011-10-25 02:36:51 -02:00

templates

removed unused import from 'crawl' spider template

2011-07-20 01:04:16 -03:00

tests

ignore meta-refresh redirects embedded in <script> tags. related to issue 18

2011-11-14 16:54:13 -02:00

utils

ignore meta-refresh redirects embedded in <script> tags. related to issue 18

2011-11-14 16:54:13 -02:00

xlib

Correctly handle query parameters on s3:// urls

2011-08-08 13:23:45 -03:00

__init__.py

bumped version to 0.15

2011-11-17 14:44:34 -02:00

cmdline.py

new mechanism to override settings in scrapy commands before the Crawler object is available

2011-09-15 13:27:01 -03:00

command.py

new mechanism to override settings in scrapy commands before the Crawler object is available

2011-09-15 13:27:01 -03:00

conf.py

Removed hacky SCRAPY_SETTINGS_DISABLED environment variable

2010-09-22 16:08:18 -03:00

crawler.py

added from_crawler class method to SpiderManager

2011-08-16 11:16:35 -03:00

dupefilter.py

pass close reason to close() method of new DupeFilter

2011-08-24 11:26:35 -03:00

exceptions.py

replaced DeprecationWarning by a new ScrapyDeprecationWarning category, since the default DeprecationWarning is silenced on Python 2.7+

2011-08-08 10:39:53 -03:00

extension.py

Made ExtensionManager a subclass of MiddlewareManager

2010-08-22 05:33:08 -03:00

interfaces.py

removed class method from_settings from ISpiderManager interface

2011-08-26 09:24:01 -03:00

item.py

Changed default representation of items to pretty-printed dicts. This improves

2011-06-03 01:13:01 -03:00

link.py

added fragment attribute ot Link object

2011-07-27 12:28:36 -03:00

linkextractor.py

Some Link extractor improvements:

2011-05-18 12:32:34 -03:00

log.py

replaced DeprecationWarning by a new ScrapyDeprecationWarning category, since the default DeprecationWarning is silenced on Python 2.7+

2011-08-08 10:39:53 -03:00

logformatter.py

improved download errors propagation to the spiders, and removed no longer needed code to simplify

2011-07-13 14:10:05 -03:00

mail.py

removed (somewhat hacky) MAIL_DEBUG setting

2010-08-22 22:42:00 -03:00

middleware.py

Another step towards singleton removal: deprecated crawler singleton import (from scrapy.project import crawler) by a new class method that extensions can implement to receive the crawler

2011-08-08 11:42:44 -03:00

mime.types

moved module scrapy.core.downloader.responsetypes to scrapy.responsetypes

2011-08-07 02:49:57 -03:00

project.py

Another step towards singleton removal: deprecated crawler singleton import (from scrapy.project import crawler) by a new class method that extensions can implement to receive the crawler

2011-08-08 11:42:44 -03:00

resolver.py

removed wrong blocking api usage (socket.gethostbyname()) from downloader when using CONCURRENT_REQUESTS_PER_IP

2011-08-03 23:55:59 -03:00

responsetypes.py

moved module scrapy.core.downloader.responsetypes to scrapy.responsetypes

2011-08-07 02:49:57 -03:00

shell.py

fixed "No free spider slots" bug when calling fetch() from scrapy shell

2011-11-14 20:03:43 -02:00

signals.py

Merged item passed and item scraped concepts, as they have often proved

2011-06-03 01:13:00 -03:00

spider.py

Added support for unifying access to per-spider settings. Refs #245

2010-09-22 16:09:13 -03:00

spidermanager.py

improve handling of KeyError exception when creating spiders in spider manager. closes issue 49

2011-11-14 17:00:25 -02:00

squeue.py

Some changes to persistent scheduler after some initial usage feedback:

2011-09-23 13:03:07 -03:00

stats.py

Splitted stats collector classes from stats collection facility (#204)

2010-08-22 01:24:07 -03:00

statscol.py

call stats collector engine_stopped() after the engine is closed (to make sure all data from extensions has been collected), and added that method to documented api

2011-06-06 03:12:40 -03:00

telnet.py

Some telnet console changes:

2011-08-08 15:01:08 -03:00

webservice.py

Another step towards singleton removal: deprecated crawler singleton import (from scrapy.project import crawler) by a new class method that extensions can implement to receive the crawler

2011-08-08 11:42:44 -03:00