BackTrackZ/scrapy
BackTrackZ/scrapy
1
0
Fork 0
mirror of https://github.com/scrapy/scrapy.git synced 2025-02-23 13:44:25 +00:00
Code Releases Wiki Activity
5,834 Commits 39 Branches 119 Tags
Commit Graph

5834 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Elias Dorneles
3ac3ac4d92 docs: update data flow description and image (fixes: #2278) 
This fixes the explanation to use Requests instead of URLs,
which is what actually happens, and is also consistent with the
new tutorial, which already explains how URLs become Request objects.

I've also changed the "loop", jumping from 9 to step 2.
 2016-09-28 16:38:45 -03:00
Paul Tremberth
d867268976 Merge pull request #2284 from redapple/release-notes-1.1.3-master 
Update release notes with 1.1.3 changes
 2016-09-23 16:25:52 +02:00
Paul Tremberth
80c1e5dc25 Set release date, fix typo and add tutorial improvement issue number 2016-09-23 16:11:50 +02:00
Paul Tremberth
a0f87d2f45 Update release notes for upcoming 1.1.3 bugfix release 2016-09-23 16:11:50 +02:00
Elias Dorneles
c2493c9452 Merge pull request #2282 from pawelmhm/docs-2230 
[docs] document that process_item can return Deferred
 2016-09-23 09:05:34 -03:00
Pawel Miech
b2bfd1e5c5 [docs] document that process_item can return Deferred 2016-09-23 10:36:03 +02:00
Paul Tremberth
a975a50558 Merge pull request #2252 from eliasdorneles/tutorial-upgrades 
[MRG+2] Tutorial: rewrite tutorial seeking to improve learning path
 2016-09-22 16:39:21 +02:00
Elias Dorneles
24bb91528a Merge pull request #2229 from ahlinc/fix_shell_completion 
[MRG+1] Fix completion in `scrapy shell` for new imports
 2016-09-22 11:15:30 -03:00
Elias Dorneles
f4a2208916 addressing review comments and other minor editing 2016-09-22 11:04:45 -03:00
Paul Tremberth
2e08a9b412 Merge pull request #2271 from redapple/mailsender-lists 
[MRG+1] Add note on "to" and "cc" as lists for sending emails
 2016-09-22 12:00:14 +02:00
Elias Dorneles
d636e5baa8 better description for start_requests expected return value 2016-09-21 18:54:12 -03:00
Elias Dorneles
32017a76f8 recommend learn python the hard way for beginners 2016-09-21 11:06:36 -03:00
Elias Dorneles
38266cc949 recommend Dive into Python and Python tutorial instead of LPTHW for non-beginners 2016-09-21 11:02:24 -03:00
Elias Dorneles
c126c59361 address more review comments 2016-09-20 18:19:25 -03:00
Elias Dorneles
a876ea5bd2 minor grammar fix 2016-09-20 15:10:49 -03:00
Elias Dorneles
bc41fdf20e address review comments, add debug log to initial spider 2016-09-20 15:04:08 -03:00
Elias Dorneles
a19af5b164 Merge pull request #2273 from redapple/version-stability 
[MRG+1] Remove mention of odd-numbered versions for development releases
 2016-09-20 14:15:52 -03:00
Paul Tremberth
40293551b2 Remove mention of odd-numbered versions for development releases 
Fixes GH-1317
 2016-09-20 18:15:45 +02:00
Elias Dorneles
125b691102 more reviewing and editing, minor restructure, syntax fixes 2016-09-20 12:47:03 -03:00
Paul Tremberth
e59d79bf37 Add note on "to" and "cc" as lists for sending emails 
Fixes GH-2244
 2016-09-20 17:18:49 +02:00
Elias Dorneles
8975371a57 Merge branch 'master' into tutorial-upgrades 2016-09-20 09:45:05 -03:00
Elias Dorneles
f4f93c5c26 fix tox docs build, adjust title 2016-09-20 09:19:59 -03:00
Elias Dorneles
3fd947b30d Merge pull request #2269 from redapple/unserializable-warning 
Log warning when request cannot be serialized (instead of error)
 2016-09-20 09:00:58 -03:00
Paul Tremberth
a135dbaf19 Log warning when request cannot be serialized (instead of error) 
Fixes GH-2035
 2016-09-20 12:47:33 +02:00
Valdir Stumm Junior
fee07835f2 Completing the data extraction section 2016-09-19 19:19:44 -03:00
Elias Dorneles
2a409d1d95 [wip] changing introduction to scraping with selectors 2016-09-19 17:13:04 -03:00
Daniel Graña
eb49b459c1 Merge pull request #2212 from redapple/debian-jessie-baseline 
Add Debian Jessie test env
 2016-09-19 15:17:45 -03:00
Paul Tremberth
41cd9f401f Merge pull request #2243 from pawelmhm/image-pipeline-2198 
[MRG+1] [image & file pipeline] loading setting for user classes
 2016-09-19 18:43:52 +02:00
Elias Dorneles
063315258e Merge pull request #2202 from scrapy/doc-arch-overview2 
[MRG+1] DOC move Data Flow below the picture; add links to components
 2016-09-19 08:11:18 -03:00
Mikhail Korobov
490f6e08f3 Merge pull request #2239 from redapple/streamlogger-flush 
[MRG+1] Add flush() method to StreamLogger
 2016-09-19 14:44:45 +06:00
Mikhail Korobov
5657f6b8ef Merge pull request #2258 from redapple/feed-export-started 
[MRG+1] Feed exporter: start exporting only on first item
 2016-09-19 14:40:30 +06:00
Mikhail Korobov
552368727a Merge pull request #2225 from Tethik/parse_command_rules_fix 
[MRG+1] Two small fixes for when using the parse command and the '-r' flag (rules).
 2016-09-19 14:39:09 +06:00
Joakim Uddholm
8c38dde4e8 Moved parse command tests to its own file. Added some checks to check for logged errors. 2016-09-19 05:33:05 +02:00
Joakim Uddholm
88cf86f5f2 Merge pull request #1 from redapple/tethik_parse_command_rules_fix 
Add tests for crawl command non-default cases
 2016-09-19 00:51:36 +02:00
Paul Tremberth
48f6a065b8 Flush StreamLogger handlers 2016-09-17 15:25:45 +02:00
Paul Tremberth
27f88ad9cb Merge pull request #2260 from waynelovely/tutorial-fix-20160917-1 
Fix a dict key in the tutorial
 2016-09-17 14:19:21 +02:00
Wayne Lovely
cc8497abb1 Fix a dict key in the tutorial 2016-09-17 11:09:28 +00:00
Mikhail Korobov
992b2517b0 Merge pull request #2248 from redapple/scrapy-shell-import-scrapy 
[MRG+1] Make scrapy available in shell without explicit import statement
 2016-09-17 06:10:06 +06:00
Mikhail Korobov
91fcafde5e Merge pull request #2257 from scrapy/mention-stackoverflow 
Mentions stackoverflow as support channel (fixes #2255)
 2016-09-17 06:06:53 +06:00
Paul Tremberth
03ab077249 Feed exporter: start exporting only on first item 
Fixes GH-872
 2016-09-17 01:36:56 +02:00
Valdir Stumm Junior
233b98d642 include section describing spider arguments 2016-09-16 18:08:10 -03:00
Elias Dorneles
31545a9f84 tutorial: updating extracting data section to introduce CSS and XPath equally 2016-09-16 17:13:24 -03:00
Elias Dorneles
147e75602d update after review comments (thanks @stummjr) 2016-09-16 16:47:24 -03:00
Elias Dorneles
31260cf02f mentions stackoverflow as help channel (fixes #2255) 2016-09-16 16:05:36 -03:00
Elias Dorneles
de1a6ac677 Merge pull request #2249 from scrapy/fix-overview-spider 
[MRG+1] docs: update overview spider code to use toscrape.com and minor changes
 2016-09-16 16:00:23 -03:00
Elias Dorneles
21de617c77 mention that spiders need to subclass scrapy.Spider 2016-09-16 15:55:14 -03:00
Elias Dorneles
b2a5cddbb0 tutorial: update section about following links, expand examples 
adding an AuthorSpider to demonstrate further a different crawling
arrangement.
 2016-09-16 15:49:49 -03:00
Valdir Stumm Junior
0cd9dfcc85 small fixes on tutorial 2016-09-16 15:21:49 -03:00
Valdir Stumm Junior
0da497cf7a updates on the first section (our first spider) 2016-09-16 11:55:23 -03:00
Elias Dorneles
c508f40689 use harcoded URLs, remove item reference on second spider 2016-09-15 18:05:09 -03:00