BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-24 05:45:31 +00:00
Code Releases Wiki Activity
3,515 Commits 39 Branches 119 Tags
Commit Graph

696 Commits

Author SHA1 Message Date
Stefan
8ed2d0cda1 improved changes to allow retrieval of alternate links in sitemaps, see #360 2013-09-07 12:56:30 +02:00
Rocio Aramberri
d227d530f6 Added COMPRESSION_ENABLED setting to enable or disable the HttpCompressionMiddleware 
Added COMPRESSION_ENABLE setting to docs

Added COMPRESSION_ENABLED setting to default settings
 2013-08-01 11:31:28 -03:00
Dan
1ca31244b0 Fixed ordering of super argument call. 2013-07-16 14:50:10 -04:00
Dan
e12b689c4f Updated documentation of spider arguments to include required super call. 2013-07-16 14:26:53 -04:00
Mikhail Korobov
1a1c93fafe tiny FormRequest doc fix 2013-07-15 15:47:34 +06:00
Mikhail Korobov
ac2fadf3ab DownloaderMiddleware.process_response docs fix 
"returns an exception" -> "raises an exception"
 2013-07-08 19:41:58 +06:00
Mikhail Korobov
39e5da5f66 improve docs for DownloaderMiddleware.process_response 2013-07-08 19:17:29 +06:00
Pablo Hoffman
0f4b70f582 remove no deprecated request_scheduled signal 
It will be replaced by more accurate scheduler signals (proposal will
come soon)
 2013-06-27 11:23:24 -03:00
nramirezuy
bef8ade956 removed request_received and added request_scheduled 2013-06-26 16:45:46 -03:00
Pablo Hoffman
819b2776dd Merge pull request #326 from berendiwema/master 
Include example of how to stop the reactor from script
 2013-06-25 13:30:07 -07:00
nramirezuy
83b2774354 remove wrong default httpcache 2013-06-25 17:01:29 -03:00
Berend Iwema
aec314db09 added a bit more documentation on how to close the reactor when running scrapy from a script 2013-06-25 16:08:22 +02:00
Pablo Hoffman
bbde1d0e0b Merge pull request #275 from stav/doc 
doc: Response.replace() cannot take meta argument
 2013-06-24 11:09:28 -07:00
Capi Etheriel
50fa46d183 Document CrawlSpider.parse_start_urls method 2013-06-09 04:03:20 -03:00
Daniel Graña
b4fca90bba merge 0.16.5 release notes 2013-05-30 18:49:00 -03:00
cacovsky
8007762890 Add FAQ entry referencing Request.meta usage 2013-05-27 13:02:17 -03:00
Pablo Hoffman
845c64b89d add benchmarking to 0.18 release notes 2013-05-17 10:38:42 -03:00
Pablo Hoffman
ca12886acb update copyright notes 2013-05-16 15:05:52 -03:00
Pablo Hoffman
8e49fed918 minor improvements to benchmarking doc 2013-05-16 13:23:13 -03:00
Pablo Hoffman
76087e336a add scrapy bench command for benchmarking, with documentation 2013-05-16 13:15:25 -03:00
Pablo Hoffman
66311db23e mention crawlera in best practices, as a way to deal with bans 2013-05-04 18:20:23 -03:00
Pablo Hoffman
9361c89573 remove scrapyd doc, as it was moved to its own repo 2013-04-27 04:15:42 -03:00
Pablo Hoffman
d02da2f31f ported code to use queuelib 2013-04-23 17:48:09 -03:00
Pablo Hoffman
7a1536f76e Merge pull request #290 from nramirezuy/item-copy 
added copy method to item
 2013-04-19 09:27:44 -07:00
Nicolás Ramírez
6df274bba5 added copy method to item 2013-04-19 13:23:53 -03:00
Mikhail Korobov
b245d592aa Update faq.rst 
spider.DOWNLOAD_DELAY is deprecated
 2013-04-18 02:42:15 +06:00
Juan M Uys
4de3aa4932 Update overview.rst 2013-04-08 14:13:15 +02:00
Pablo Hoffman
96c2332e0e fix inaccurate downloader middleware documentation. refs #280 2013-04-02 11:35:32 -03:00
Steven Almeroth
70179c7c0c doc: remove trailing spaces 2013-03-21 13:57:39 -06:00
Steven Almeroth
0d7747d353 doc: Response.replace() cannot take meta argument 
>>> response.replace(meta={'foo':1})
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/srv/scrapy/scrapy-fork/scrapy/scrapy/http/response/text.py", line 45, in replace
    return Response.replace(self, *args, **kwargs)
  File "/srv/scrapy/scrapy-fork/scrapy/scrapy/http/response/__init__.py", line 77, in replace
    return cls(*args, **kwargs)
  File "/srv/scrapy/scrapy-fork/scrapy/scrapy/http/response/text.py", line 22, in __init__
    super(TextResponse, self).__init__(*args, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'meta'
 2013-03-21 13:49:55 -06:00
Pablo Hoffman
2a5c7ed4da make Crawler.start() return a deferred that is fired when the crawl is finished 2013-03-20 14:48:59 -03:00
Pablo Hoffman
b347c14b5f update engine status output on telnet console documentation 2013-03-18 19:12:12 -03:00
Shane Evans
5c2a82f1f7 fix typo 2013-03-17 19:34:55 +00:00
Pablo Hoffman
bb20907254 minor updated to faq 2013-03-14 16:43:00 -03:00
Pablo Hoffman
098ccff862 added FAQ about error: "cannot import name crawler" 2013-03-14 12:57:59 -03:00
Pablo Hoffman
8391b36251 minor updates to contributing doc 2013-03-13 03:24:25 -03:00
Pablo Hoffman
51c301b3a2 added link to python binary libs, for windows installation 2013-03-13 03:18:33 -03:00
Pablo Hoffman
8e72730792 Merge pull request #261 from stav/allowed_domains 
allow spider allowed_domains to be set/tuple, #259
 2013-03-12 20:44:51 -07:00
Steven Almeroth
650eda68da doc: add comment about commit history cleanliness 2013-03-10 18:51:04 -06:00
Pablo Hoffman
eeb69d2f70 added #260 to release notes 2013-03-08 11:59:38 -02:00
Jordi Llonch
5b118ff4ab added documentation (experimental feature) 2013-03-06 06:36:23 +11:00
Pablo Hoffman
3c8eef99cb docs/contributing: added note explaining what Scrapy contrib is 2013-03-04 01:35:17 -02:00
Steven Almeroth
f62b6660d4 doc: fix typo in spider middleware 2013-03-02 19:46:31 -06:00
Pablo Hoffman
7400ceb1ed added 502 to RETRY_HTTP_CODES 2013-02-22 19:12:59 -02:00
Pablo Hoffman
a038f46859 doc: fixed rst title 2013-02-14 11:11:17 -02:00
Pablo Hoffman
22edc44c6c doc: remove links to diveintopython.org, which is no longer available. closes #246 2013-02-14 11:09:40 -02:00
Pablo Hoffman
1ff8b4f831 updated release notes with previous commit 2013-02-12 00:59:25 -02:00
Daniel Graña
5db45b3825 remove scrapyd, it was migrated to its own repository 2013-02-06 05:24:07 +00:00
whodatninja
8e3b5baac5 Fix typo labeling attrs type bool instead of list 2013-02-05 15:10:41 -05:00
Daniel Graña
3cf7f4975b Add 0.16.4 to release notes 
Conflicts:
	docs/news.rst
 2013-01-23 11:29:38 -02:00