Mikhail Korobov
e1ceaf3b5f
require w3lib 1.17+
2017-02-15 00:32:44 +05:00
Paul Tremberth
1295c17a26
Bump parsel requirement to at least parsel v1.1
2017-02-02 17:47:22 +01:00
Paul Tremberth
b3406677b9
Update classifiers in setup.py
2017-01-09 14:40:02 +01:00
Paul Tremberth
78d34cc44c
Merge pull request #2454 from crasker/fix/low_parsel_version_number
...
[MRG+1] Fix/low parsel version number
2016-12-19 17:11:17 +01:00
zhouyc
2b3abdb700
update parsel version which will cause attribute error
2016-12-19 16:42:31 +08:00
nyov
c8cf1a303d
Bump Twisted dependency to 13.1.0 (released June 2013)
2016-12-01 22:02:11 +00:00
Ashish Kulkarni
bb3b806467
Use w3lib.url.canonicalize_url() from w3lib 1.15.0
...
Also remove code/imports which are now unused due to this change.
fixes #2157
2016-08-16 17:42:16 +05:30
Paul Tremberth
c8bff23751
Bump w3lib version dependency in setup.py
2016-04-29 10:29:37 +02:00
nyov
3787fec460
Remove duplicate code now handled by newer w3lib
...
see f3029a6a10
2016-03-30 14:58:03 +00:00
Mikhail Korobov
17d3bec699
declare Python 3 support in setup.py
2016-03-01 16:34:13 +05:00
Elias Dorneles
26ebccd37a
upgrade parsel and use its function to instantiate root for finding form
2015-08-11 14:09:34 -03:00
Elias Dorneles
2fe6d128f5
upgrade parsel and using promoted root attribute
2015-08-11 14:09:16 -03:00
Elias Dorneles
17d7347a36
update minimal parsel version, add deprecated classes for csstranslator modules, fix test
2015-08-11 14:09:00 -03:00
Elias Dorneles
ce21884a97
migrating scrapy Selector to use Parsel
2015-08-11 14:08:43 -03:00
Mikhail Korobov
75d3d03382
add service_identity to scrapy install_requires
2015-08-11 20:14:22 +05:00
Daniel Graña
776064aee1
PY3 port signals
2015-07-25 18:04:55 +02:00
Elias Dorneles
3d3633f3d2
favoring web scraping over screen scraping in the descriptions
2015-04-25 11:20:20 -03:00
Pablo Hoffman
5846d6154c
emphasize web crawling over screen scraping on scrapy description. closes #586
2015-03-18 00:24:16 -03:00
Daniel Graña
0254f58763
Merge branch 'moderm-setuppy'
2014-08-08 18:09:43 -03:00
Daniel Graña
029c51acef
There is a trove classifier for Scrapy framework!
...
Added by https://bitbucket.org/pypa/pypi/issue/179
2014-08-05 14:43:13 -03:00
Paul Tremberth
480cfa1994
Update w3lib requirement to 1.8.0
2014-08-04 16:01:28 +02:00
Daniel Graña
fcd34b6561
set zip_safe=False
2014-08-01 16:26:08 -03:00
Daniel Graña
1fc4e59cf4
do not ship tests package
2014-08-01 16:23:37 -03:00
Daniel Graña
a8f45dc6dd
Modernize setup.py
2014-08-01 15:43:33 -03:00
Daniel Graña
bfe02b019e
Revert "limit Twisted support to pre-14.0.0 while #718 is fixed"
...
This reverts commit 65f69e16095799e05de79a3f8f449f7b9edb27aa.
2014-06-25 04:22:18 -03:00
Daniel Graña
65f69e1609
limit Twisted support to pre-14.0.0 while #718 is fixed
2014-06-02 13:27:20 -03:00
alexanderlukanin13
ab6496647c
Added "six>=1.5.2" to requirements
2014-01-15 13:05:00 +06:00
Daniel Graña
cfe588103c
include_package_data is required to build wheels from published sources
2013-11-28 16:25:57 -02:00
Daniel Graña
2df8156431
Drop Python 2.6 support
2013-10-29 13:44:00 -02:00
alexanderlukanin13
192c323a0c
wrong variable name
2013-10-18 11:30:48 +06:00
Daniel Graña
757e262267
update requirement for cssselect >= 0.9
2013-10-17 09:33:15 -02:00
Capi Etheriel
bc17e9d412
Adds HtmlCSSSelector and XmlCSSSelector classes, cssselect as optional dependency.
...
Ported .get() from _Element and .text_content() from HTMLMixin
Add CSS selectors to scrapy shell
Documenting CSS Selectors: Constructing selectors
Documenting CSS Selectors: Using Selectors
Make CSS Selectors a default feature.
Adds XPath powers to CSS Selectors and some syntactic sugar.
Removes methods copied over from lxml.html.HtmlMixin.
Updating docs to use new CSS Selector super powers.
Documenting CSS Selectors: Regular Expressions
Moving section after Nesting section, since it mentions it.
Documenting CSS Selectors: Nesting Selectors
Fix XPath specificity in lxml.selector.CSSSelectorMixin.text
Cleaning up unused stuff from cssel.py
Changing the behavior of lxml.selector.CSSSelectorMixin.text.
Concatenating all of the descendant text nodes is more useful
than returning it in pieces (there's xpath() if you need that).
Documenting CSS Selectors: CSS Selector objects
Documenting CSS Selectors: CSSSelectorList objects
Documenting CSS Selectors: HtmlCSSSelector objects
Documenting CSS Selectors: XmlCSSSelector objects
Fixing some documentations typos and errors
Enforcing the 80-char width lines
Tidying up CSS selectors and CSSSelectorMixin objects
Adding some missing references in documentation.
Fixing lxml.selector.CSSSelectorList.text
2013-10-10 18:23:15 -02:00
Pablo Hoffman
a9c3519897
updated required twisted version to 10.0
2013-10-01 14:07:38 -03:00
Pablo Hoffman
d02da2f31f
ported code to use queuelib
2013-04-23 17:48:09 -03:00
Daniel Graña
910effd145
get scrapy version from package data
2013-02-06 11:44:26 -02:00
Daniel Graña
5db45b3825
remove scrapyd, it was migrated to its own repository
2013-02-06 05:24:07 +00:00
Pablo Hoffman
b9a96147ed
setup.py: use README.rst to populate long_description
2012-11-25 22:22:33 -02:00
Daniel Graña
d74a067227
require w3lib 1.2 or greater
2012-05-15 17:11:51 -03:00
Pablo Hoffman
4f28ffcb2c
removed no longer needed dependency on simplejson
2012-04-10 16:01:36 -03:00
Pablo Hoffman
6e8edbd72e
switched default selectors backend to lxml
2012-04-10 15:52:14 -03:00
Pablo Hoffman
b6ae266546
Removed (very old and possibly broken) backwards compatibility support for Twisted 2.5
2012-03-15 00:28:24 -03:00
Pablo Hoffman
e521da2e2f
Dropped support for Python 2.5. See: http://blog.scrapy.org/scrapy-dropping-support-for-python-25
2012-03-01 08:18:12 -02:00
Pablo Hoffman
de3a3b68dc
bumped required w3lib version to 1.1, after refactoring encoding detection to use the new w3lib.encoding module
2012-02-29 07:44:22 -02:00
Pablo Hoffman
b6220b8e95
use git describe for building version from git, and removed support for building version from hg
2012-01-02 13:05:26 -02:00
Pablo Hoffman
1e43afeaea
added support for generating version from git revision, and use it in extras/makedeb.py
2011-09-09 03:03:46 -03:00
Pablo Hoffman
804c0279ec
setup.py: only add lxml requirement if libxml2 is not available
2011-07-13 13:04:42 -03:00
Pablo Hoffman
7f97259ba7
added w3lib to requirements, in installation guide
2011-05-01 11:14:57 -03:00
Pablo Hoffman
b10f4fae35
Moved several functions from scrapy.utils.{http,markup,multipart,response,url} (and their tests) to a new library called 'w3lib'. Scrapy will now depend on w3lib.
2011-04-18 22:37:19 -03:00
Pablo Hoffman
97a790d659
improved mechanism for tagging version with hg revision
2010-12-13 10:21:39 -02:00
Pablo Hoffman
7c712eeda1
Removed scrapy.xlib.simplejson module. Scrapy now requires simplejson if running on Python 2.5. Closes #289
2010-11-16 03:11:12 -02:00
