BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 12:03:40 +00:00
Code Releases Wiki Activity
4,368 Commits 39 Branches 119 Tags
Commit Graph

4368 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mikhail Korobov
6fb7093154 add utf8 encoding header to spider templates 2014-06-26 00:30:27 +06:00
Mikhail Korobov
df4ba41c82 Merge pull request #688 from stillzhl/master 
add encoding utf-8 to the first line
 2014-06-26 00:26:12 +06:00
Daniel Graña
436c1c8980 address latest comments 2014-06-25 15:12:58 -03:00
Daniel Graña
2ad8db6ae6 Merge pull request #761 from dangra/lxmlextractor 
Promote LxmlLinkExtractor as LxmlExtractor
 2014-06-25 15:07:02 -03:00
Daniel Graña
a9ecef5662 promote LxmlLinkExtractor as default in docs 2014-06-25 14:34:30 -03:00
Mikhail Korobov
ef81127b99 Merge pull request #760 from dangra/546-remaining 
recognize jl extension as jsonlines exporter and update docs
 2014-06-25 23:31:48 +06:00
Daniel Graña
5b2faf61c3 recognize jl extension as jsonlines exporter and update docs 2014-06-25 13:55:15 -03:00
Daniel Graña
f98bf2c521 Merge pull request #737 from Curita/settings-cleanup 
Settings api cleanup
 2014-06-25 13:01:50 -03:00
Daniel Graña
f224ac1328 Restore compatibility with Settings.overrides while still deprecating it 2014-06-25 13:01:12 -03:00
Daniel Graña
f72b1597b4 Merge pull request #718 from dangra/twisted-14.0.0 
[WIP] does it work with Twisted 14.0.0?
 2014-06-25 09:50:31 -03:00
Daniel Graña
d48158427a hold a reference to backwards compatible _contextFactory 2014-06-25 04:22:18 -03:00
Daniel Graña
bfe02b019e Revert "limit Twisted support to pre-14.0.0 while #718 is fixed" 
This reverts commit 65f69e16095799e05de79a3f8f449f7b9edb27aa.
 2014-06-25 04:22:18 -03:00
Daniel Graña
559af943dc Add bumpversion config 2014-06-25 03:19:04 -03:00
Daniel Graña
1b32ece918 Merge branch 'settings-cleanup' of https://github.com/Curita/scrapy into Curita-settings-cleanup 2014-06-25 02:55:20 -03:00
Daniel Graña
4300334a23 Upload sdist and wheel packages to pypi using travis-ci deploys 2014-06-25 02:53:12 -03:00
Daniel Graña
499438ec44 indent parsed-literal as part of ordered list 2014-06-24 10:32:56 -03:00
Pablo Hoffman
ee5c3209af doc: use |version| substitution in ubuntu packages 2014-06-24 09:52:51 -03:00
Paul Tremberth
90e69141d3 Add doc on LxmlLinkExtractor class 2014-06-23 22:35:56 +02:00
Paul Tremberth
d34a57a10e Add LxmlLinkExtractor class similar to SgmlLinkExtractor (#528) 2014-06-23 22:35:56 +02:00
Daniel Graña
05ffca2781 Merge pull request #626 from dangra/500-leveldb-cache 
[WIP] Add a LevelDB cache backend
 2014-06-23 15:30:13 -03:00
Daniel Graña
a90fd81c97 add leveldb cache backend docs 2014-06-23 14:37:29 -03:00
Daniel Graña
9d57ecfc35 Merge pull request #752 from tpeng/slightly-improve-contract-tests 
print spider name even it has no contract tests when -v is specified
 2014-06-18 03:15:21 -03:00
Daniel Graña
7449b25bed Merge pull request #745 from crlane/cl/offsite-middleware-bugfix 
Cl/offsite middleware bugfix
 2014-06-18 03:01:21 -03:00
tpeng
5fb22ba4e7 print spider name even it has no contract tests when -v is specified 2014-06-17 13:39:10 +02:00
Pablo Hoffman
7fd8a1a387 minor change to request_scheduled signal doc 2014-06-14 20:51:16 -03:00
Pablo Hoffman
313ff26a79 Merge pull request #746 from Digenis/doc 
Document signal "request_scheduled"
 2014-06-14 20:49:43 -03:00
Nikolaos-Digenis Karagiannis
5f79940295 Document signal "request_scheduled" 2014-06-14 11:08:18 +03:00
Cameron Lane
a069d36652 [#744] Test for allowed domains including NoneTypes 2014-06-12 11:43:03 -04:00
Cameron Lane
eefa878f98 [#744] Ensure domain is not None before building regex 2014-06-12 11:42:14 -04:00
Pablo Hoffman
ad11fc01cb crate release notes for 0.24 and #699 to it 2014-06-10 18:46:53 -03:00
Julia Medina
6730e3c9f7 Fix settings usage across tests 2014-06-10 11:26:50 -03:00
Julia Medina
43ae69e436 Fix settings usage in runspider and crawl commands 
Explicit conversion of FEED_EXPORTERS and FEED_EXPORTERS_BASE values into
dictionaries, and cast of the keys to a list for py3 support.
 2014-06-10 11:26:50 -03:00
Julia Medina
c81b087de1 Change how settings are overriden in ScrapyCommands 2014-06-10 11:26:50 -03:00
Julia Medina
080a28f3e9 Change command's default_settings population in cmdline.py 2014-06-10 11:26:50 -03:00
Julia Medina
c496468e2a get_project_settings now returns a Settings instance 2014-06-10 11:26:50 -03:00
Julia Medina
b397f5e935 Update get_crawler method in utils/test.py with new Settings interface 
This early adaptation of the new Settings interface is needed for running tests
 2014-06-10 11:26:50 -03:00
Julia Medina
ed033854e2 setmodule helper method on Settings class 2014-06-10 11:26:50 -03:00
Julia Medina
bdca06240c Fix settings repr on the logs of the shell and tutorial docs topics 2014-06-10 11:26:50 -03:00
Julia Medina
8a3a6236d9 Settings topic updated 2014-06-10 11:26:50 -03:00
Julia Medina
00e6470ed0 Fix and extend the documentation of the new Settings api 2014-06-10 11:26:38 -03:00
Julia Medina
b4b17049e9 Settings and SettingsAtribute tests 2014-06-10 11:21:44 -03:00
Julia Medina
22572834fb Deprecate CrawlerSettings, as its functionality is replicable by Settings class 2014-06-10 11:19:15 -03:00
Julia Medina
95ba68683e New set and setdict method using SettingsAttribute in Settings 2014-06-10 11:19:15 -03:00
Julia Medina
384efce0a6 Settings priorities dictionary 2014-06-10 11:12:44 -03:00
Julia Medina
1276b12c25 New SettingsAttribute class 2014-06-09 19:08:08 -03:00
Daniel Graña
752787e634 Add a LevelDB cache backend 2014-06-04 16:09:07 -03:00
Pablo Hoffman
9a4ee20991 Merge pull request #742 from dangra/sort-scrapy-list 
sort spiders in "scrapy list" cmd.
 2014-06-03 17:34:04 -03:00
Daniel Graña
f6d48213f6 sort spiders in "scrapy list" cmd. closes #736 2014-06-03 15:36:58 -03:00
Daniel Graña
1c9effd7b4 Avoid IPython warning. thanks @bryant1410. closes #623 2014-06-03 15:11:20 -03:00
Daniel Graña
9830efb946 Merge pull request #697 from allait/master 
Add a note about reporting security issues
 2014-06-03 11:13:13 -03:00