mirror of
https://github.com/scrapy/scrapy.git
synced 2025-02-25 01:43:50 +00:00
Update docs.
modified: docs/topics/media-pipeline.rst
This commit is contained in:
parent
537083524e
commit
6988e9cd4b
@ -77,30 +77,6 @@ PIL.
|
|||||||
.. _Python Imaging Library: http://www.pythonware.com/products/pil/
|
.. _Python Imaging Library: http://www.pythonware.com/products/pil/
|
||||||
|
|
||||||
|
|
||||||
Usage example
|
|
||||||
=============
|
|
||||||
|
|
||||||
In order to use a media pipeline first, :ref:`enable it
|
|
||||||
<topics-media-pipeline-enabling>`.
|
|
||||||
|
|
||||||
Then, if a spider returns a dict with the URLs key ('file_urls' or
|
|
||||||
'image_urls', for the Files or Images Pipeline respectively), the pipeline will
|
|
||||||
put the results under respective key ('files' or images').
|
|
||||||
|
|
||||||
If you prefer to use :class:`~.Item`, then define a custom item with the
|
|
||||||
necessary fields, like in this example for Images Pipeline::
|
|
||||||
|
|
||||||
import scrapy
|
|
||||||
|
|
||||||
class MyItem(scrapy.Item):
|
|
||||||
|
|
||||||
# ... other item fields ...
|
|
||||||
image_urls = scrapy.Field()
|
|
||||||
images = scrapy.Field()
|
|
||||||
|
|
||||||
If you need something more complex and want to override the custom pipeline
|
|
||||||
behaviour, see :ref:`topics-media-pipeline-override`.
|
|
||||||
|
|
||||||
.. _topics-media-pipeline-enabling:
|
.. _topics-media-pipeline-enabling:
|
||||||
|
|
||||||
Enabling your Media Pipeline
|
Enabling your Media Pipeline
|
||||||
@ -171,6 +147,51 @@ Where:
|
|||||||
used). For more info see :ref:`topics-images-thumbnails`.
|
used). For more info see :ref:`topics-images-thumbnails`.
|
||||||
|
|
||||||
|
|
||||||
|
Usage example
|
||||||
|
=============
|
||||||
|
|
||||||
|
.. setting:: FILES_URLS_FIELD
|
||||||
|
.. setting:: FILES_RESULT_FIELD
|
||||||
|
.. setting:: IMAGES_URLS_FIELD
|
||||||
|
.. setting:: IMAGES_RESULT_FIELD
|
||||||
|
|
||||||
|
In order to use a media pipeline first, :ref:`enable it
|
||||||
|
<topics-media-pipeline-enabling>`.
|
||||||
|
|
||||||
|
Then, if a spider returns a dict with the URLs key (``file_urls`` or
|
||||||
|
``image_urls``, for the Files or Images Pipeline respectively), the pipeline will
|
||||||
|
put the results under respective key (``files`` or ``images``).
|
||||||
|
|
||||||
|
If you prefer to use :class:`~.Item`, then define a custom item with the
|
||||||
|
necessary fields, like in this example for Images Pipeline::
|
||||||
|
|
||||||
|
import scrapy
|
||||||
|
|
||||||
|
class MyItem(scrapy.Item):
|
||||||
|
|
||||||
|
# ... other item fields ...
|
||||||
|
image_urls = scrapy.Field()
|
||||||
|
images = scrapy.Field()
|
||||||
|
|
||||||
|
If you want to use another field name for the URLs key or for the results key,
|
||||||
|
it is also possible to override it.
|
||||||
|
|
||||||
|
For the Files Pipeline, set :setting:`FILES_URLS_FIELD` and/or
|
||||||
|
:setting:`FILES_RESULT_FIELD` settings::
|
||||||
|
|
||||||
|
FILES_URLS_FIELD = 'field_name_for_your_files_urls'
|
||||||
|
FILES_RESULT_FIELD = 'field_name_for_your_processed_files'
|
||||||
|
|
||||||
|
For the Images Pipeline, set :setting:`IMAGES_URLS_FIELD` and/or
|
||||||
|
:setting:`IMAGES_RESULT_FIELD` settings::
|
||||||
|
|
||||||
|
IMAGES_URLS_FIELD = 'field_name_for_your_images_urls'
|
||||||
|
IMAGES_RESULT_FIELD = 'field_name_for_your_processed_images'
|
||||||
|
|
||||||
|
If you need something more complex and want to override the custom pipeline
|
||||||
|
behaviour, see :ref:`topics-media-pipeline-override`.
|
||||||
|
|
||||||
|
|
||||||
Additional features
|
Additional features
|
||||||
===================
|
===================
|
||||||
|
|
||||||
@ -185,12 +206,14 @@ adjust this retention delay use the :setting:`FILES_EXPIRES` setting (or
|
|||||||
:setting:`IMAGES_EXPIRES`, in case of Images Pipeline), which
|
:setting:`IMAGES_EXPIRES`, in case of Images Pipeline), which
|
||||||
specifies the delay in number of days::
|
specifies the delay in number of days::
|
||||||
|
|
||||||
# 90 days of delay for files expiration
|
# 120 days of delay for files expiration
|
||||||
FILES_EXPIRES = 90
|
FILES_EXPIRES = 120
|
||||||
|
|
||||||
# 30 days of delay for images expiration
|
# 30 days of delay for images expiration
|
||||||
IMAGES_EXPIRES = 30
|
IMAGES_EXPIRES = 30
|
||||||
|
|
||||||
|
The default value for both settings is 90 days.
|
||||||
|
|
||||||
.. _topics-images-thumbnails:
|
.. _topics-images-thumbnails:
|
||||||
|
|
||||||
Thumbnail generation for images
|
Thumbnail generation for images
|
||||||
@ -249,7 +272,13 @@ For example::
|
|||||||
IMAGES_MIN_HEIGHT = 110
|
IMAGES_MIN_HEIGHT = 110
|
||||||
IMAGES_MIN_WIDTH = 110
|
IMAGES_MIN_WIDTH = 110
|
||||||
|
|
||||||
Note: these size constraints don't affect thumbnail generation at all.
|
.. note::
|
||||||
|
The size constraints don't affect thumbnail generation at all.
|
||||||
|
|
||||||
|
It is possible to set just one size constraint or both. When setting both of
|
||||||
|
them, only images that satisfy both minimum sizes will be saved. For the
|
||||||
|
above example, images of sizes (105 x 105) or (105 x 200) or (200 x 105) will
|
||||||
|
all be dropped because at least one dimension is shorter than the constraint.
|
||||||
|
|
||||||
By default, there are no size constraints, so all images are processed.
|
By default, there are no size constraints, so all images are processed.
|
||||||
|
|
||||||
|
Loading…
x
Reference in New Issue
Block a user