Pablo Hoffman
abcac4fcbd
updated maintainer to scrapinghub
2012-05-02 03:25:35 -03:00
Pablo Hoffman
78185921c2
renamed and improved README to provide a more helpful github landing page
2012-04-28 23:30:00 -03:00
Pablo Hoffman
6e8edbd72e
switched default selectors backend to lxml
2012-04-10 15:52:14 -03:00
Pablo Hoffman
9006227358
bumped required python-w3lib version in debian/control
2012-03-05 20:25:38 -02:00
Pablo Hoffman
85e2b493b4
make scrapyd debian package dependent on the same (or higher) version of scrapy package
2012-01-13 10:55:20 -02:00
Pablo Hoffman
8a45dd121b
scrapyd: fixed issue with ubuntu package: /var/lib/scrapyd/items dir not being created by default
2012-01-12 17:17:50 -02:00
Pablo Hoffman
76af0cdd44
updated documentation and code to use -s instead of --set option
2011-09-01 14:35:37 -03:00
Pablo Hoffman
718428c0ab
debian/control: added python-setuptools to Recommends, because it's need by 'scrapy deploy' command
2011-05-01 11:00:02 -03:00
Pablo Hoffman
bf73002428
removed googledir example, replaced by dirbot project on github. updated docs accordingly
2011-04-28 02:28:39 -03:00
Pablo Hoffman
ebcbb9f453
debian: added python-w3lib package to dependencies
2011-04-19 00:55:08 -03:00
Pablo Hoffman
b10f4fae35
Moved several functions from scrapy.utils.{http,markup,multipart,response,url} (and their tests) to a new library called 'w3lib'. Scrapy will now depend on w3lib.
2011-04-18 22:37:19 -03:00
Pablo Hoffman
60f6a9b054
moved scrapyd python module to scrapy debian package. left scrapyd package only for installing service (upstart script) and scrapy user
2011-03-23 15:45:40 -03:00
Pablo Hoffman
b1c89508f5
fixed wrong changes commited in previous changeset
2011-01-28 16:22:39 -02:00
Pablo Hoffman
4361150494
added missing scrapyd/default_scrapyd.conf file to MANIFEST.in
2011-01-28 16:21:00 -02:00
Pablo Hoffman
7dc521c56e
make scrapyd package depend on specific scrapy version
2011-01-11 17:03:51 -02:00
Pablo Hoffman
ac8166251e
make scrapy*-0.12 packages conflict with scrapy*-0.11 packages
2011-01-04 13:56:28 -02:00
Pablo Hoffman
2897061b98
Make scrapy conflict with previous versionsof the debian package
2010-11-17 17:30:22 -02:00
Pablo Hoffman
3926ca45b3
debian/control: Added scrapy/scrapyd to Provides
2010-11-17 17:08:21 -02:00
Pablo Hoffman
a034d078c8
Changed Debian packaging to use the scrapy version in the package name, so we can have multiple Scrapy versions in the same apt repo
...
--HG--
rename : debian/scrapy.1 => debian/scrapy-files/scrapy.1
rename : debian/000-default => debian/scrapyd-files/000-default
rename : debian/scrapyd.upstart => debian/scrapyd.scrapyd.upstart
rename : debian/scrapy.1 => extras/scrapy.1
2010-11-17 17:03:00 -02:00
Pablo Hoffman
1d5c56089c
Automated merge with http://hg.scrapy.org/scrapy-0.10
2010-10-27 14:42:47 -02:00
Pablo Hoffman
c3e5b4bb03
changed pid file name to scrapyd
2010-10-27 14:42:22 -02:00
Pablo Hoffman
2bba87f69f
Automated merge with http://hg.scrapy.org/scrapy-0.10
2010-10-27 14:20:21 -02:00
Pablo Hoffman
f47b9f608c
simplified lockfile used by scrapyd (/var/run/scrapyd.pid instead of /var/run/scrapyd/scrapyd.pid). closes #271
2010-10-27 14:18:40 -02:00
Pablo Hoffman
d15a97ff61
Updated Scrapy version in debian/changelog
2010-09-28 16:45:05 -03:00
Pablo Hoffman
08952562e9
Make Ubuntu package conflict with standard Scrapy Debian/Ubuntu package
2010-09-17 16:25:02 -03:00
Pablo Hoffman
9158e9d682
Some changes to Scrapyd to support multiple configuration files, to make it easier to deploy Scrapyd applications. Also documented 'egg_runner' and 'application' options
...
--HG--
rename : debian/scrapyd.cfg => debian/000-default
rename : scrapyd/default_scrapyd.cfg => scrapyd/default_scrapyd.conf
2010-09-07 09:17:25 -03:00
Pablo Hoffman
37e9c5d78e
Added new Scrapy service with support for:
...
* multiple projects
* uploading scrapy projects as Python eggs
* scheduling spiders using a JSON API
Documentation is added along with the code.
Closes #218 .
--HG--
rename : debian/scrapy-service.default => debian/scrapyd.default
rename : debian/scrapy-service.dirs => debian/scrapyd.dirs
rename : debian/scrapy-service.install => debian/scrapyd.install
rename : debian/scrapy-service.lintian-overrides => debian/scrapyd.lintian-overrides
rename : debian/scrapy-service.postinst => debian/scrapyd.postinst
rename : debian/scrapy-service.postrm => debian/scrapyd.postrm
rename : debian/scrapy-service.upstart => debian/scrapyd.upstart
rename : extras/scrapy.tac => extras/scrapyd.tac
2010-09-03 15:54:42 -03:00
Pablo Hoffman
e14cc2c12a
Moved scrapy-ws script to extras/ and fixed broken methods due to changes in web service API
...
--HG--
rename : bin/scrapy-ws.py => extras/scrapy-ws.py
2010-08-27 00:33:08 -03:00
Pablo Hoffman
8e83f527b3
Removed scrapy-sqs script, as it has been superseded by the new scrapy 'queue' command
2010-08-23 22:04:49 -03:00
Pablo Hoffman
0d9e75c684
Added bash completion for the Scrapy command-line tool. Closes #210
2010-08-21 03:23:45 -03:00
Pablo Hoffman
34554da201
Deprecated scrapy-ctl.py command in favour of simpler "scrapy" command. Closes #199 . Also updated documenation accordingly and added convenient scrapy.bat script for running from Windows.
...
--HG--
rename : debian/scrapy-ctl.1 => debian/scrapy.1
rename : docs/topics/scrapy-ctl.rst => docs/topics/cmdline.rst
2010-08-18 19:48:32 -03:00
Pablo Hoffman
c5fd113c09
minor fix to path name
2010-07-31 16:02:59 -03:00
Pablo Hoffman
f6eac8e348
Splitted single Debian package into two packages ( closes #187 ):
...
- scrapy: which provides only the library and scrapy-ctl command
- scrapy-service: which provides the service, upstart script, system user, etc
This allows a clean install of just the library for those which are not
interested in the Scrapy service.
--HG--
rename : debian/scrapy.dirs => debian/scrapy-service.dirs
rename : debian/scrapy.install => debian/scrapy-service.install
rename : debian/scrapy.postinst => debian/scrapy-service.postinst
rename : debian/scrapy.postrm => debian/scrapy-service.postrm
rename : debian/scrapy.upstart => debian/scrapy-service.upstart
rename : debian/conf/service_conf.py => debian/service_conf.py
2010-07-31 15:50:12 -03:00
Pablo Hoffman
ede1df4b4f
updated copyright year, and indentation space
2010-06-14 07:16:51 -03:00
Pablo Hoffman
247fc26598
moved scrapy.tac to extras/
...
--HG--
rename : bin/scrapy.tac => extras/scrapy.tac
2010-06-13 23:09:08 -03:00
Pablo Hoffman
09182efaff
added scrapy-sqs.py to deployed scripts
2010-06-13 19:17:17 -03:00
Pablo Hoffman
37f71a9957
upstart script: exec twistd and use pidfile
2010-06-13 18:59:52 -03:00
Pablo Hoffman
ea8b5ddfd5
debian package: fix dh_auto_build confusing with Makefile, added scrapy-ws.py to deployed scripts
2010-06-11 12:48:35 -03:00
Pablo Hoffman
d13b50a234
Added sources and Makefile for building Debian package
2010-06-11 01:18:16 -03:00
