BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-26 17:04:07 +00:00
Code Releases Wiki Activity
1,027 Commits 39 Branches 119 Tags
Commit Graph

1027 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
olveyra
86c7f37d6a moved options nocache and nopipeline from decobot to scrapy 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40127
 2008-07-27 16:11:23 +00:00
olveyra
b59f62dc91 Added function convert_entity from decobot.utils.text_extraction 
(to complete and fix revision 124)

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40126
 2008-07-27 14:54:11 +00:00
olveyra
1e2ddb47a3 remove wrong import from decobot and added unquote_html 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40125
 2008-07-27 02:49:20 +00:00
olveyra
137ec64318 minor adjustments and some fixes, readded scrapy-admin.py with 
execution permission

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40124
 2008-07-27 02:23:14 +00:00
olveyra
b851ec5f10 deleted scrapy-admin to commit again with execution access 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40123
 2008-07-27 01:38:53 +00:00
samus_
fbb5860f49 re-enabling replays with the new mechanism 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40122
 2008-07-26 15:51:39 +00:00
olveyra
bc00f8cce2 re-reverted commit 119 back again to 118. 
The code removed is confusing.

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40121
 2008-07-25 22:42:57 +00:00
Pablo Hoffman
67ee6bff2e restored code removed in r118. there's nothing wrong with that code 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40120
 2008-07-25 21:28:49 +00:00
olveyra
c97c495d16 removed code that generates confusing and mistaken import error 
message when the import error raises inside scrapy_settings.

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40119
 2008-07-25 19:40:43 +00:00
samus_
0bb68f34f6 forgot to import sys module :P 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40118
 2008-07-25 19:26:46 +00:00
olveyra
8e18ecd5ce first version of scrapy-admin 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40117
 2008-07-25 19:11:01 +00:00
samus_
bd7c80ed0a adding new replay method (beta) 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40116
 2008-07-25 18:50:45 +00:00
Matias Aguirre
22e3d3a02e Add save_on_top to admin sections 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40115
 2008-07-25 17:36:34 +00:00
Matias Aguirre
e421206573 Add publish field 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40114
 2008-07-25 16:59:27 +00:00
Matias Aguirre
9f8ce8f116 Add bool icon for public field in admin 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40113
 2008-07-25 16:59:00 +00:00
Matias Aguirre
baee62cf44 Change positions urls in download and article applications, this are like admin urls now and the views are decorated to force a staff user check before 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40112
 2008-07-25 15:38:11 +00:00
Matias Aguirre
fd4aab5f33 Force all textareas to be 30 rows height 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40111
 2008-07-25 15:35:11 +00:00
Matias Aguirre
80a99dde83 Adding missing template sections 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40110
 2008-07-25 15:23:59 +00:00
Matias Aguirre
4c193f47ed Adding if clause that will display blog entries box only if there are any entry to list 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40109
 2008-07-25 15:19:16 +00:00
Matias Aguirre
56e110af78 Override admin base template with a custom one that has some JS to resize textaeras, useful when editing an article 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40108
 2008-07-25 15:17:43 +00:00
Matias Aguirre
ee8d9a594a Adding nav and footer links statically instead of use the removed links application 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40107
 2008-07-25 15:16:37 +00:00
Matias Aguirre
202e08a9d3 Removing flatpages from applications list, this isn't used 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40106
 2008-07-25 15:11:56 +00:00
Matias Aguirre
d4337610d9 Removing link application, was too much overkill to the functionality needed 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40105
 2008-07-25 15:10:41 +00:00
Pablo Hoffman
9dd144f487 removed parent attribute from Response class 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40104
 2008-07-25 13:42:40 +00:00
olveyra
f3f34d0816 - added basic statistics (needs to be improved) 
- node remotion when connection lost event

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40103
 2008-07-24 15:18:29 +00:00
samus_
e86ee64727 small improvements to the cache 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40102
 2008-07-24 01:52:28 +00:00
olveyra
8b8e3e7532 fixed a bug in which pending list remains stalled because not proper 
clean out of loading list

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40101
 2008-07-23 19:23:47 +00:00
olveyra
0dffb68ecf not update nodes after scheduling, to avoid enter in a loop 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%40100
 2008-07-23 19:13:39 +00:00
olveyra
b5b79042ab - added worker to master notifications. 
- deleted statistics code. will change approach

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4099
 2008-07-23 17:15:16 +00:00
olveyra
980369ba60 changed default webservice port to connect on to 8060 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4098
 2008-07-23 12:32:38 +00:00
olveyra
3a06868c21 deleted date folder from log path 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4097
 2008-07-23 12:21:05 +00:00
Pablo Hoffman
7e32fe10f2 do not hide svn errors 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4096
 2008-07-23 03:13:26 +00:00
Pablo Hoffman
8a0fdaa89b logfiles are now appendable instead of truncable 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4095
 2008-07-22 23:13:08 +00:00
samus_
a7301cc9a2 implemented __getslice__ for the XPathSelectorList, still broken when using step-slices 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4094
 2008-07-22 18:38:02 +00:00
olveyra
a27b085c04 fix 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4093
 2008-07-22 17:56:04 +00:00
samus_
5e2db86cde fixed pickle 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4092
 2008-07-22 17:53:37 +00:00
olveyra
7f62faea5f Added cluster statistics report 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4091
 2008-07-22 17:21:55 +00:00
samus_
85a4514603 reverted wrong XPathSelector commit 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4090
 2008-07-22 16:48:10 +00:00
samus_
e8b5a07a15 cPickle as pickle 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4089
 2008-07-22 16:44:30 +00:00
samus_
30202c54a4 ís_cached fix2 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4088
 2008-07-22 16:27:52 +00:00
samus_
e7fbaa50d8 reverted wrong XPathSelector commit 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4087
 2008-07-22 16:12:42 +00:00
samus_
c9851d8ec9 ís_cached fix 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4086
 2008-07-22 16:11:57 +00:00
samus_
5778ecf15d small fix for string representation of XPathSelector when the node is not an xml object 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4085
 2008-07-22 14:19:41 +00:00
samus_
97b2e7df55 added pickled_meta to avoid using eval 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4084
 2008-07-22 11:51:53 +00:00
samus_
76a9d2da10 migrated all sha to hashlib 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4083
 2008-07-22 11:33:05 +00:00
olveyra
ce905ce78f - code fixes 
- fix when rescheduling with a new priority
- added freeslots to status_as_dict
- remove a domain from loading list only when reported as running in
some node.

--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4082
 2008-07-21 19:14:37 +00:00
samus_
9c345016b8 added INFO message 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4081
 2008-07-21 14:07:57 +00:00
samus_
d2121141a3 implemented CACHE2_EXPIRATION_SECS and migrated sha to hashlib 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4080
 2008-07-21 13:23:23 +00:00
Pablo Hoffman
b45d87d0fe removed duplicated code 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4079
 2008-07-18 01:51:49 +00:00
olveyra
cdb3b02510 fix 
--HG--
extra : convert_revision : svn%3Ab85faa78-f9eb-468e-a121-7cced6da292c%4078
 2008-07-17 19:20:10 +00:00