Skip to content

Scrapier doesn’t publish a fixed per-key request quota. Capacity is shared, so the API tells you when to slow down, and your client should listen.

  • 429 means you’re being throttled. Wait for the number of seconds in the Retry-After header, then retry.
  • 503 means Scrapier is at capacity right now. It usually carries Retry-After: 30. Wait, then retry.
  • If there’s no Retry-After, use exponential backoff with jitter: wait about 1 s, then 2 s, then 4 s and so on, each multiplied by a random factor, up to a minute.

Both are free: failed calls never cost credits. The Errors page has a ready-made retry snippet.

Run no more than 3 requests in parallel per key. If you scrape a long list, work through it with a small pool rather than firing everything at once. You’ll see fewer 429 and 503 responses, and the whole job usually finishes sooner.

Scraping a live website takes time. Some endpoints take several minutes.

  • Set your HTTP client’s timeout to at least 210 seconds: curl --max-time 210, AbortSignal.timeout(210_000) in JavaScript, timeout=210 in Python requests.
  • Automation tools often default to much shorter timeouts. Raise them in n8n, Make or Zapier where you can.
  • A timeout on our side returns 502 or 504. Both are free and safe to retry.
  • 5xx and 429: retry with backoff. You’re only charged when a call succeeds.
  • 400, 401, 403, 404: don’t retry. Fix the request or key first.
  • 402 with a balance field: don’t retry. Top up. A 402 without balance came from the target site; treat it like a 5xx.
  • GET requests are safe to repeat. Keep in mind that every successful repeat is billed again, so don’t re-run calls that already succeeded.