Konstantin Lopuhin
1d5ab67183
pass test_getPageHead on py3
2016-01-14 12:04:26 +03:00
Konstantin Lopuhin
73ff87c1dc
decode body from utf-8, as scrapy stores body as bytes, and twisted has already converted to unicode
2016-01-14 12:03:08 +03:00
Konstantin Lopuhin
9f2be23a39
webclient tests, py3: fix setUp, pass test_getPage
2016-01-14 11:42:23 +03:00
Konstantin Lopuhin
f3889b0bce
py3 compat: encode delimiter, method and path in ScrapyHTTPPageGetter
2016-01-14 11:41:49 +03:00
Elias Dorneles
57f99fc34e
Merge pull request #1644 from yarikoptic/master
...
[MRG+1] BF: robustify _monkeypatches check for twisted - str() name first (Closes #1634 )
2016-01-13 08:59:48 -02:00
Mikhail Korobov
f01fd07642
Merge pull request #1661 from redapple/py3-iterators
...
[MRG+1] PY3: port utils/iterators
2016-01-13 10:03:02 +05:00
Paul Tremberth
d7d4ef67a6
Changes following comments
2016-01-12 11:08:49 +01:00
Paul Tremberth
d4872940db
PY3: port utils/iterators
2016-01-07 10:51:27 +01:00
Elias Dorneles
95e8ff8ba1
Merge pull request #1660 from stummjr/patch-4
...
[MRG+1] Update deprecated examples
2016-01-06 10:43:08 -02:00
Valdir Stumm Jr
2abc9bc901
Update deprecated examples
...
* update the scrapy.org example to deal with the new layout.
* replaced slashdot.org by reddit.com, because it seems that slashdot is blocking requests.
2016-01-06 10:29:45 -02:00
Daniel Graña
1b435b2887
Add 1.0.4 release notes
2015-12-30 15:43:54 -03:00
Yaroslav Halchenko
4f49aab7c0
BF: robustify _monkeypatches check for twisted - str() name first ( Closes : #1634 )
...
In my case, while running datalad tests using nose, scrapy was failing
since v was None
2015-12-18 16:16:05 -05:00
Mikhail Korobov
a5db7f862c
Merge pull request #1639 from seales/master
...
Spelling fixes
2015-12-14 23:33:14 +05:00
Mikhail Korobov
1bc243a876
Merge pull request #1638 from orangain/toc-captions
...
[MRG+1] DOC: Add captions to toctrees which appear in sidebar
2015-12-14 23:32:18 +05:00
seales
719b1353a7
Spelling fixes
2015-12-13 19:39:48 -08:00
orangain
4be4ef038e
DOC: Add captions to toctrees which appear in sidebar
2015-12-12 16:21:00 +09:00
Daniel Graña
b5285c3f38
Merge pull request #1628 from rolando/conda-docs
...
[MRG] Removed pywin32 from conda install instructions as it's already declared as dependency.
2015-12-05 14:45:17 -03:00
Rolando Espinoza
f8ae99d18f
DOC Removed pywin32 from install instructions as it's already declared as dependency.
2015-12-05 09:48:17 -04:00
Elias Dorneles
7d187735ff
Merge pull request #1359 from nyov/nyov/drop-optional_features
...
[MRG+1] drop deprecated "optional_features" set
2015-12-04 14:08:44 -02:00
Daniel Graña
17aba44f16
Merge pull request #1565 from starrify/fix-1564
...
[MRG+1] fixed: Issue #1564 (Incorrectly picked URL in `scrapy.linkextractors.regex.RegexLinkExtractor` when there is a `<base>` tag. )
2015-12-04 10:31:13 -03:00
Daniel Graña
dc65026395
Merge pull request #1597 from Digenis/various_formreq_fixes
...
[MRG+1] Various FormRequest tests+fixes
2015-12-04 09:55:12 -03:00
Daniel Graña
3881eaff45
Merge pull request #1575 from palego/startproject-templates-override
...
[MRG+1] Startproject templates override
2015-12-04 09:54:48 -03:00
Daniel Graña
a97f28fbf0
Merge pull request #1598 from Digenis/disable_dupefilter_in_shell
...
[MRG+1] Disable dupefilter in shell to prevent hanging.
2015-12-04 09:51:56 -03:00
Daniel Graña
21f638b139
Merge pull request #1622 from sibiryakov/twisted-1550-fix
...
[MRG+1] Ignoring xlib/tx folder, depending on Twisted version.
2015-12-04 09:40:35 -03:00
Mikhail Korobov
015cd78940
Merge pull request #1618 from sibiryakov/engine-heartbeats
...
Nextcall heartbeats in engine (event loop halting)
2015-12-04 16:28:45 +05:00
Alexander Sibiryakov
c4d29ecaef
Ignoring xlib/tx folder, depending on Twisted version.
2015-12-02 17:08:21 +01:00
Alexander Sibiryakov
5750ce92c0
nextcall repetitive calls (heartbeats).
2015-12-02 11:39:47 +01:00
palego
2c251000d3
custom project templates
...
allow override of TEMPLATES_DIR for startproject
copy full TEMPLATES_DIR/project tree
doc update
2015-12-01 03:07:27 +01:00
Pablo Hoffman
eb4daa34a2
Merge pull request #1544 from rolando/conda-docs
...
Added installation notes about using Conda for Windows.
2015-11-26 14:11:28 -03:00
Rolando Espinoza
cc8ddb68b0
Added installation notes about using Conda for Windows and other OSes.
2015-11-26 13:08:56 -04:00
Pengyu CHEN
751155eade
removed: Unused code
2015-11-19 00:38:05 +08:00
Julia Medina
48539af163
Merge pull request #1601 from mvj3/minor-documentation-fixes
...
Fixed minor grammar issues.
2015-11-16 14:58:07 -03:00
David Chen
0025d5a943
Fixed minor grammar issues.
2015-11-16 07:30:17 +08:00
Νικόλαος-Διγενής Καραγιάννης
ebfdb9bb03
readable xpath with exslt
2015-11-14 23:24:07 +02:00
Νικόλαος-Διγενής Καραγιάννης
4f98be60be
FormRequest: fix unicode xpath exception
2015-11-13 18:56:05 +02:00
Νικόλαος-Διγενής Καραγιάννης
395ef805eb
FormRequest: test unicode xpath expr & exception
2015-11-13 18:54:02 +02:00
Νικόλαος-Διγενής Καραγιάννης
2d25eab0df
FormRequest: <input>'s default type must be text
2015-11-13 18:05:07 +02:00
Νικόλαος-Διγενής Καραγιάννης
7a438a51b7
FormRequest: test default <input> type (is text)
2015-11-13 18:03:54 +02:00
Νικόλαος-Διγενής Καραγιάννης
650acad2b7
FormRequest: fix case-insensitive type attributes
2015-11-13 17:57:46 +02:00
Νικόλαος-Διγενής Καραγιάννης
881bf19e68
FormRequest: test case-insensitive type attribute
2015-11-13 17:57:25 +02:00
Julia Medina
54216d7afe
Merge pull request #1586 from jdemaeyer/fix/backwards-compatible-per-key-priorities
...
[MRG+1] Backwards compatible per key priorities
2015-11-11 14:08:50 -03:00
Jakob de Maeyer
4f364764ae
Simplify BaseSettings.__get__(), .getpriority()
2015-11-11 17:40:42 +01:00
Jakob de Maeyer
44f6ada034
Overwrite, not update, dictionary-like settings
2015-11-11 17:40:42 +01:00
Jakob de Maeyer
52ecee6a62
Replace BaseSettings._getcomposite() with public .getwithbase() method
2015-11-11 17:39:57 +01:00
Jakob de Maeyer
b6a023ce98
Add backwards compatibility for build_component_list
2015-11-11 17:39:56 +01:00
Jakob de Maeyer
e66f649894
Bring back _BASE settings
2015-11-11 17:39:56 +01:00
Mikhail Korobov
9548691fdd
Merge pull request #1563 from starrify/master
...
[MRG+1] fixed: Issue #1562 (Incorrectly picked URL in `scrapy.http.FormReques t.from_response` when there is a `<base>` tag)
2015-11-10 21:33:24 +05:00
Pengyu CHEN
94486bb294
added: Test case for the fix.
2015-11-02 23:00:42 +08:00
Pengyu CHEN
e19bf4aecc
added: Test case for the fix
2015-11-02 22:52:41 +08:00
Julia Medina
57f87b95d4
Merge pull request #1528 from scrapy/create-crawler
...
public Crawler.create_crawler method
2015-10-30 12:36:35 -03:00
