Mikhail Korobov
936266a61a
Merge pull request #934 from Dineshs91/zsh-support
...
[MRG+1] Added zsh completion for Scrapy command-line tool and updated bash completion
2015-07-31 03:18:19 +05:00
Daniel Graña
47b1b748c2
Generated version as pep440 and dpkg compatible
2015-06-16 00:16:09 +00:00
agusc
140f58251b
removed SUFFIX from scrapy name package
2015-06-15 17:02:09 -03:00
Elias Dorneles
3d3633f3d2
favoring web scraping over screen scraping in the descriptions
2015-04-25 11:20:20 -03:00
dinesh
e2b4fabbf5
Fix zsh completion path
2015-04-03 08:34:16 +05:30
dinesh
9d0ea5a396
Added zsh completion for the Scrapy command-line tool
2015-04-03 08:18:47 +05:30
Pablo Hoffman
5846d6154c
emphasize web crawling over screen scraping on scrapy description. closes #586
2015-03-18 00:24:16 -03:00
Daniel Graña
3b64b2449e
update other places where w3lib version is mentioned
2014-08-04 15:56:41 -03:00
Daniel Graña
e17866aaab
there is not support for python3 yet
2014-07-01 17:12:01 +00:00
Daniel Graña
e58c23e4a9
Update python compatible version set to debian packages
2014-07-01 12:12:31 -03:00
alexanderlukanin13
ab6496647c
Added "six>=1.5.2" to requirements
2014-01-15 13:05:00 +06:00
Daniel Graña
2df8156431
Drop Python 2.6 support
2013-10-29 13:44:00 -02:00
Daniel Graña
bd79b6e1d3
debian package requires python-cssselect
2013-10-24 16:47:46 -02:00
Pablo Hoffman
ca12886acb
update copyright notes
2013-05-16 15:05:52 -03:00
Pablo Hoffman
d02da2f31f
ported code to use queuelib
2013-04-23 17:48:09 -03:00
Daniel Graña
5db45b3825
remove scrapyd, it was migrated to its own repository
2013-02-06 05:24:07 +00:00
Daniel Graña
d74a067227
require w3lib 1.2 or greater
2012-05-15 17:11:51 -03:00
Pablo Hoffman
abcac4fcbd
updated maintainer to scrapinghub
2012-05-02 03:25:35 -03:00
Pablo Hoffman
78185921c2
renamed and improved README to provide a more helpful github landing page
2012-04-28 23:30:00 -03:00
Pablo Hoffman
6e8edbd72e
switched default selectors backend to lxml
2012-04-10 15:52:14 -03:00
Pablo Hoffman
9006227358
bumped required python-w3lib version in debian/control
2012-03-05 20:25:38 -02:00
Pablo Hoffman
85e2b493b4
make scrapyd debian package dependent on the same (or higher) version of scrapy package
2012-01-13 10:55:20 -02:00
Pablo Hoffman
8a45dd121b
scrapyd: fixed issue with ubuntu package: /var/lib/scrapyd/items dir not being created by default
2012-01-12 17:17:50 -02:00
Pablo Hoffman
76af0cdd44
updated documentation and code to use -s instead of --set option
2011-09-01 14:35:37 -03:00
Pablo Hoffman
718428c0ab
debian/control: added python-setuptools to Recommends, because it's need by 'scrapy deploy' command
2011-05-01 11:00:02 -03:00
Pablo Hoffman
bf73002428
removed googledir example, replaced by dirbot project on github. updated docs accordingly
2011-04-28 02:28:39 -03:00
Pablo Hoffman
ebcbb9f453
debian: added python-w3lib package to dependencies
2011-04-19 00:55:08 -03:00
Pablo Hoffman
b10f4fae35
Moved several functions from scrapy.utils.{http,markup,multipart,response,url} (and their tests) to a new library called 'w3lib'. Scrapy will now depend on w3lib.
2011-04-18 22:37:19 -03:00
Pablo Hoffman
60f6a9b054
moved scrapyd python module to scrapy debian package. left scrapyd package only for installing service (upstart script) and scrapy user
2011-03-23 15:45:40 -03:00
Pablo Hoffman
b1c89508f5
fixed wrong changes commited in previous changeset
2011-01-28 16:22:39 -02:00
Pablo Hoffman
4361150494
added missing scrapyd/default_scrapyd.conf file to MANIFEST.in
2011-01-28 16:21:00 -02:00
Pablo Hoffman
7dc521c56e
make scrapyd package depend on specific scrapy version
2011-01-11 17:03:51 -02:00
Pablo Hoffman
ac8166251e
make scrapy*-0.12 packages conflict with scrapy*-0.11 packages
2011-01-04 13:56:28 -02:00
Pablo Hoffman
2897061b98
Make scrapy conflict with previous versionsof the debian package
2010-11-17 17:30:22 -02:00
Pablo Hoffman
3926ca45b3
debian/control: Added scrapy/scrapyd to Provides
2010-11-17 17:08:21 -02:00
Pablo Hoffman
a034d078c8
Changed Debian packaging to use the scrapy version in the package name, so we can have multiple Scrapy versions in the same apt repo
...
--HG--
rename : debian/scrapy.1 => debian/scrapy-files/scrapy.1
rename : debian/000-default => debian/scrapyd-files/000-default
rename : debian/scrapyd.upstart => debian/scrapyd.scrapyd.upstart
rename : debian/scrapy.1 => extras/scrapy.1
2010-11-17 17:03:00 -02:00
Pablo Hoffman
1d5c56089c
Automated merge with http://hg.scrapy.org/scrapy-0.10
2010-10-27 14:42:47 -02:00
Pablo Hoffman
c3e5b4bb03
changed pid file name to scrapyd
2010-10-27 14:42:22 -02:00
Pablo Hoffman
2bba87f69f
Automated merge with http://hg.scrapy.org/scrapy-0.10
2010-10-27 14:20:21 -02:00
Pablo Hoffman
f47b9f608c
simplified lockfile used by scrapyd (/var/run/scrapyd.pid instead of /var/run/scrapyd/scrapyd.pid). closes #271
2010-10-27 14:18:40 -02:00
Pablo Hoffman
d15a97ff61
Updated Scrapy version in debian/changelog
2010-09-28 16:45:05 -03:00
Pablo Hoffman
08952562e9
Make Ubuntu package conflict with standard Scrapy Debian/Ubuntu package
2010-09-17 16:25:02 -03:00
Pablo Hoffman
9158e9d682
Some changes to Scrapyd to support multiple configuration files, to make it easier to deploy Scrapyd applications. Also documented 'egg_runner' and 'application' options
...
--HG--
rename : debian/scrapyd.cfg => debian/000-default
rename : scrapyd/default_scrapyd.cfg => scrapyd/default_scrapyd.conf
2010-09-07 09:17:25 -03:00
Pablo Hoffman
37e9c5d78e
Added new Scrapy service with support for:
...
* multiple projects
* uploading scrapy projects as Python eggs
* scheduling spiders using a JSON API
Documentation is added along with the code.
Closes #218 .
--HG--
rename : debian/scrapy-service.default => debian/scrapyd.default
rename : debian/scrapy-service.dirs => debian/scrapyd.dirs
rename : debian/scrapy-service.install => debian/scrapyd.install
rename : debian/scrapy-service.lintian-overrides => debian/scrapyd.lintian-overrides
rename : debian/scrapy-service.postinst => debian/scrapyd.postinst
rename : debian/scrapy-service.postrm => debian/scrapyd.postrm
rename : debian/scrapy-service.upstart => debian/scrapyd.upstart
rename : extras/scrapy.tac => extras/scrapyd.tac
2010-09-03 15:54:42 -03:00
Pablo Hoffman
e14cc2c12a
Moved scrapy-ws script to extras/ and fixed broken methods due to changes in web service API
...
--HG--
rename : bin/scrapy-ws.py => extras/scrapy-ws.py
2010-08-27 00:33:08 -03:00
Pablo Hoffman
8e83f527b3
Removed scrapy-sqs script, as it has been superseded by the new scrapy 'queue' command
2010-08-23 22:04:49 -03:00
Pablo Hoffman
0d9e75c684
Added bash completion for the Scrapy command-line tool. Closes #210
2010-08-21 03:23:45 -03:00
Pablo Hoffman
34554da201
Deprecated scrapy-ctl.py command in favour of simpler "scrapy" command. Closes #199 . Also updated documenation accordingly and added convenient scrapy.bat script for running from Windows.
...
--HG--
rename : debian/scrapy-ctl.1 => debian/scrapy.1
rename : docs/topics/scrapy-ctl.rst => docs/topics/cmdline.rst
2010-08-18 19:48:32 -03:00
Pablo Hoffman
c5fd113c09
minor fix to path name
2010-07-31 16:02:59 -03:00
Pablo Hoffman
f6eac8e348
Splitted single Debian package into two packages ( closes #187 ):
...
- scrapy: which provides only the library and scrapy-ctl command
- scrapy-service: which provides the service, upstart script, system user, etc
This allows a clean install of just the library for those which are not
interested in the Scrapy service.
--HG--
rename : debian/scrapy.dirs => debian/scrapy-service.dirs
rename : debian/scrapy.install => debian/scrapy-service.install
rename : debian/scrapy.postinst => debian/scrapy-service.postinst
rename : debian/scrapy.postrm => debian/scrapy-service.postrm
rename : debian/scrapy.upstart => debian/scrapy-service.upstart
rename : debian/conf/service_conf.py => debian/service_conf.py
2010-07-31 15:50:12 -03:00
