BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-25 15:03:56 +00:00
Code Releases Wiki Activity
5,341 Commits 39 Branches 119 Tags
Commit Graph

1183 Commits

Author SHA1 Message Date
Raul Gallegos
a06a5f00f4 adding configurable encoding for httpproxy authentication 2016-01-21 09:35:04 -05:00
Elias Dorneles
d4c4ca8062 fix version number to appear new feature 2016-01-21 09:42:15 -02:00
Capi Etheriel
659715ecd9 implements FormRequest.from_response CSS support 2016-01-21 01:05:20 -02:00
Daniel Graña
b1e44436bc Merge pull request #1642 from orangain/doc-metarefresh 
[MRG+1] DOC: Update MetaRefreshMiddlware's setting variables
 2016-01-19 13:33:50 -03:00
carlosp420
e15f361b05 fixed typo You -> you 2016-01-19 11:12:43 -05:00
Ralph Gutkowski
79147a61a7 Update stats.rst 2016-01-15 19:25:56 +01:00
Ralph Gutkowski
bb38400db5 Update Stats Collection documentation 
`pages_crawled` value doesn't exist. Replace it with `downloader/response_count` in order to avoid confusion.
 2016-01-15 19:00:58 +01:00
Valdir Stumm Jr
2abc9bc901 Update deprecated examples 
* update the scrapy.org example to deal with the new layout.
* replaced slashdot.org by reddit.com, because it seems that slashdot is blocking requests.
 2016-01-06 10:29:45 -02:00
Daniel Graña
1b435b2887 Add 1.0.4 release notes 2015-12-30 15:43:54 -03:00
orangain
bcce8d3d80 DOC: Update MetaRefreshMiddlware's setting variables 
* `REDIRECT_MAX_METAREFRESH_DELAY` has been deprecated and was
  renamed to `METAREFRESH_MAXDELAY`.
* Merge duplicate documents about `METAREFRESH_MAXDELAY` appeared both
  in the settings page and the downloader-middlewares page.
 2015-12-17 15:25:02 +09:00
orangain
4be4ef038e DOC: Add captions to toctrees which appear in sidebar 2015-12-12 16:21:00 +09:00
Rolando Espinoza
f8ae99d18f DOC Removed pywin32 from install instructions as it's already declared as dependency. 2015-12-05 09:48:17 -04:00
palego
2c251000d3 custom project templates 
allow override of TEMPLATES_DIR for startproject
copy full TEMPLATES_DIR/project tree
doc update
 2015-12-01 03:07:27 +01:00
Pablo Hoffman
eb4daa34a2 Merge pull request #1544 from rolando/conda-docs 
Added installation notes about using Conda for Windows.
 2015-11-26 14:11:28 -03:00
Rolando Espinoza
cc8ddb68b0 Added installation notes about using Conda for Windows and other OSes. 2015-11-26 13:08:56 -04:00
Julia Medina
48539af163 Merge pull request #1601 from mvj3/minor-documentation-fixes 
Fixed minor grammar issues.
 2015-11-16 14:58:07 -03:00
David Chen
0025d5a943 Fixed minor grammar issues. 2015-11-16 07:30:17 +08:00
Jakob de Maeyer
e66f649894 Bring back _BASE settings 2015-11-11 17:39:56 +01:00
Julia Medina
72eeead6db Merge pull request #1524 from Digenis/relocate_telnet_console 
[MRG+1] Relocate telnetconsole to extensions/
 2015-10-30 12:10:38 -03:00
Jakob de Maeyer
bb6dee611c Move Settings documentation to docstrings 2015-10-27 12:43:33 +01:00
Jakob de Maeyer
26586ef5a6 Deprecate _BASE settings, unify _BASE backwards-compatibility 2015-10-27 12:43:23 +01:00
Jakob de Maeyer
a769a1ef78 Introduce BaseSettings with full dictionary interface 2015-10-27 12:38:52 +01:00
Valdir Stumm Jr
d577c4702d fixed a typo in the documentation. 2015-10-26 00:00:20 -02:00
Christian Pedersen
cc76fb3dff Version 1 now exists 2015-10-14 14:07:34 +02:00
Mikhail Korobov
1b6d60c251 DOC fix docs after GH-1289. 2015-10-09 01:26:09 +05:00
hoatle
2869cf8dde fix another invalid xpath error 2015-10-07 16:03:43 +07:00
Hoat Le
4e66955411 fix ValueError: Invalid XPath: //div/[id="not-exists"]/text() on selectors.rst 
>>> response.xpath('//div/[id="not-exists"]/text()').extract_first() is None
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/http/response/text.py", line 109, in xpath
    return self.selector.xpath(query)
  File "/home/vagrant/.virtualenvs/scrapy/lib/python2.7/site-packages/scrapy/selector/unified.py", line 100, in xpath
    raise ValueError(msg if six.PY3 else msg.encode("unicode_escape"))
ValueError: Invalid XPath: //div/[id="not-exists"]/text()
 2015-10-07 15:43:02 +07:00
Νικόλαος-Διγενής Καραγιάννης
d523c75f5e Relocate telnetconsole to extensions/ 2015-10-05 21:16:04 +03:00
smirecki
8379bea7ed Typos corrections 
I've made a few small corrections, some spelling changes and typo fixes.
I've tried to respect regional spelling differences and avoided proposing hyphenating compound words.

 Please enter the commit message for your changes. Lines starting
 2015-10-02 23:48:27 -04:00
Marius Gedminas
0620e76433 Fix list formatting 2015-09-29 03:33:30 +05:00
Marius Gedminas
eaad10facf Typo 2015-09-28 14:44:15 +05:00
Daniel Graña
3c596dcf46 Merge pull request #1467 from dacjames/master 
add support for a nested loaders
 2015-09-16 22:03:19 -03:00
hy
14f7f22555 fix typos in downloader-middleware.rst and exceptions.rst, middlware -> middleware 2015-09-16 16:59:23 +08:00
Daniel Collins
036109e7de updte nested loader documentation 2015-09-15 23:49:35 -07:00
Mikhail Korobov
600f27c39b Merge pull request #1350 from scrapy/installation-docs-update 
Updates in Installation and Contributing sections in the docs
 2015-09-09 17:54:35 +05:00
Julia Medina
eb7b9d506f Add note to ubuntu install section about debian compatibility 2015-09-08 18:49:44 -03:00
Daniel Collins
88c92cb68b provide documentation for nested loaders 2015-09-04 13:15:48 -07:00
Daniel Graña
dd473145f2 Merge pull request #1473 from ArturGaspar/master 
[MRG+1] Support for returning deferreds in middlewares
 2015-09-02 13:04:26 -03:00
Robert Weindl
833efbfcbd Update tutorial.rst 
Add missing "scrapy." prefix.
 2015-09-02 20:35:23 +05:00
Artur Gaspar
9ce9a293a6 Always check robots.txt before making another request in RobotsTxtMiddleware. 2015-09-02 10:23:24 -03:00
Artur Gaspar
ca83a0b028 Support for returning deferreds in downloader middleware methods. 2015-09-01 13:22:43 -03:00
Daniel Graña
b4e382c867 Merge pull request #1318 from yarikoptic/fix-docs-explicit-rtd-theme-path 
[MRG+1] DOC(ENH): specify path to rtd theme explicitly
 2015-08-31 10:42:02 -03:00
David Tagatac
08162a15d8 minor: scrapy.Spider docs grammar 2015-08-27 17:37:16 -04:00
Mikhail Korobov
9616d91e4a Merge pull request #1444 from cyberplant/bpython_support 
[MRG +1] bpython support
 2015-08-27 21:28:05 +05:00
Rick
aa88fd797a Small grammatical change 2015-08-26 15:01:50 -03:00
Mikhail Korobov
cfae62f9cc Merge pull request #1441 from aivarsk/fix-common-practices 
Make common practices sample code match the comments
 2015-08-23 17:36:09 +05:00
Jakob de Maeyer
d164398a27 Fix RedirectMiddleware not honouring meta handle_httpstatus keys 2015-08-21 13:22:42 +02:00
nyov
509cc8d41e Add support for bpython console. 
Adds support for configuration of shells from scrapy.cfg
and SCRAPY_PYTHON_SHELL.

config snippet:

cat <<EOF >> ~/.scrapy.cfg
[settings]
# shell can be one of ipython, bpython or python;
# to be tried as the interactive python console
# (in above order, unless set here).
shell = python
EOF

(closes #270, #1100, #1301)
 2015-08-21 01:12:58 +01:00
Aivars Kalvāns
b8b1e8e544 Make common practices sample code match the comments 2015-08-19 16:54:10 +03:00
Julia Medina
d3c3408cd4 Replace alternative OSX install workaround with virtualenv 2015-08-19 03:19:27 -03:00