ApifyScheduler

A Scrapy scheduler that uses the Apify RequestQueue to manage requests.

This scheduler requires the asyncio Twisted reactor to be installed.

Index

Methods

• __init__
• close
• enqueue_request
• has_pending_requests
• next_request
• open

Methods

__init__

• __init__(): None

• Returns None

close

• close(reason): None

• Close the scheduler.

  Shut down the event loop and its thread gracefully.

  ---

  Parameters

  • reason: str

    The reason for closing the spider.

  Returns None

enqueue_request

• enqueue_request(request): bool

• Add a request to the scheduler.

  This could be called from either from a spider or a downloader middleware (e.g. redirect, retry, ...).

  ---

  Parameters

  • request: Request

    The request to add to the scheduler.

  Returns bool

has_pending_requests

• has_pending_requests(): bool

• Check if the scheduler has any pending requests.

  ---

  Returns bool

next_request

• next_request(): Request | None

• Fetch the next request from the scheduler.

  ---

  Returns Request | None

open

• open(spider): Deferred[None] | None

• Open the scheduler.

  ---

  Parameters

  • spider: Spider

    The spider that the scheduler is associated with.

  Returns Deferred[None] | None