Errors
Any non-2xx response is an error, and errors are never charged.
The rule
Section titled “The rule”- Branch on the HTTP status code, not on the body.
- Read the message from
error, or fromdetailif there is noerror. - Retry only
429,503and other5xxresponses, with backoff.
Don’t expect "success": false on every error. Scrapier returns three body shapes:
| Shape | Returned by | Example statuses |
|---|---|---|
{"success": false, "error": "…"} |
The scraper for that endpoint | 400, 404, 500, 503 (capacity), 504, and some target-site errors |
{"error": "…"} (402 adds balance and required) |
The Scrapier gateway | 401, 402, 502, 503 (credit check) |
{"detail": "…"} |
Authentication and throttling | 403, 429 |
At a glance
Section titled “At a glance”| Status | Meaning | Retry? | Charged? |
|---|---|---|---|
400 | Bad or missing input. | No — fix the params | No |
401 / 403 | Missing, invalid or revoked API key. | No — fix the key | No |
402 | Out of credits (body has balance). | No — top up | No |
404 | Unknown endpoint slug. | No — check the slug | No |
429 / 503 | Busy or throttled. | Yes — after Retry-After | No |
5xx | Target site or upstream failure, or timeout. | Yes — with backoff | No |
All statuses
Section titled “All statuses”| Status | Who returns it | Body shape | Retry? | Charged? |
|---|---|---|---|---|
| 400 | Scraper | success + error |
No, fix the input | No |
| 401 | Gateway | error |
No, add the header | No |
| 402 | Gateway (billing) | error, balance, required |
No, top up | No |
| 403 | Authentication | detail |
No, fix the key | No |
| 404 | Scraper | success + error |
No, check the slug | No |
| 429 | Throttling | detail |
Yes, after Retry-After |
No |
| 500 | Scraper | success + error |
Yes, with backoff | No |
| 502 | Gateway | error |
Yes, with backoff | No |
| 503 | Scraper (capacity) or gateway (credit check) | success + error, or error |
Yes, after Retry-After or backoff |
No |
| 504 | Scraper | success + error |
Yes, with backoff | No |
4xx: fix the request
Section titled “4xx: fix the request”Bad or missing input, for example a missing required parameter or a value the scraper can’t use. The message says what’s wrong and varies by endpoint.
{ "success": false, "error": "…" }The X-API-Key header is missing.
{ "error": "Missing HTTP_X_API_KEY header." }Your balance is below the endpoint’s price (1 to 4 credits). Nothing ran and nothing was charged. Top up in Billing; retrying won’t help.
{ "error": "Insufficient credits. Buy more to scrape more.", "balance": 1, "required": 2 }The key is unknown, invalid or revoked. There’s no success field, and the message is in detail.
{ "detail": "Authentication service unavailable or invalid API key." }{ "detail": "API key has been revoked." }The endpoint slug doesn’t exist. Check it against the API reference or GET /endpoints, and keep the trailing slash.
{ "success": false, "error": "Unknown v2 endpoint: <slug>" }Too many requests right now. Wait for the number of seconds in the Retry-After header, then retry. See Rate limits & reliability.
{ "detail": "Request was throttled. Expected available in N seconds." }Something failed on the target site or on our side, or the request took too long. These are free, and safe to retry with exponential backoff.
An unexpected error inside the scraper.
{ "success": false, "error": "Internal server error: …" }The gateway couldn’t get a usable answer in time, or the upstream response was broken. The body has an error message, for example:
{ "error": "Main API timed out." }Busy. There are two cases:
- At capacity: too many scrapes are running. The response carries
Retry-After: 30. Wait that long and retry. - Credit check unavailable: we couldn’t verify your balance, for example
{"error": "Could not verify credits."}. Retry with backoff.
{ "success": false, "error": "…" }The scraper hit its time limit for this endpoint.
{ "success": false, "error": "Scraper exceeded …s timeout" }Some endpoints take several minutes. Set your client timeout to at least 210 seconds so your client doesn’t give up before Scrapier does.
Handling errors in code
Section titled “Handling errors in code”async function scrape(slug, params, { retries = 4 } = {}) { const url = `https://api.scrapier.io/api/v2/${slug}/?${new URLSearchParams(params)}`; for (let attempt = 0; ; attempt++) { const res = await fetch(url, { headers: { "X-API-Key": process.env.SCRAPIER_API_KEY }, signal: AbortSignal.timeout(210_000), }); const body = await res.json().catch(() => ({})); if (res.ok) return body.data;
const message = body.error ?? body.detail ?? `HTTP ${res.status}`; const billing402 = res.status === 402 && "balance" in body; const retryable = res.status === 429 || res.status >= 500 || (res.status === 402 && !billing402); if (!retryable || attempt >= retries) throw new Error(`${res.status}: ${message}`);
const retryAfter = Number(res.headers.get("Retry-After")); const waitMs = retryAfter > 0 ? retryAfter * 1000 : Math.min(60_000, 2 ** attempt * 1000) * (0.5 + Math.random()); await new Promise((r) => setTimeout(r, waitMs)); }}
const data = await scrape("instagram-profile", { usernames: "mrbeast" });import os, random, timeimport requests
def scrape(slug, params, retries=4): url = f"https://api.scrapier.io/api/v2/{slug}/" headers = {"X-API-Key": os.environ["SCRAPIER_API_KEY"]} for attempt in range(retries + 1): resp = requests.get(url, headers=headers, params=params, timeout=210) try: body = resp.json() except ValueError: body = {} if resp.ok: return body["data"]
message = body.get("error") or body.get("detail") or f"HTTP {resp.status_code}" billing_402 = resp.status_code == 402 and "balance" in body retryable = resp.status_code == 429 or resp.status_code >= 500 or (resp.status_code == 402 and not billing_402) if not retryable or attempt == retries: raise RuntimeError(f"{resp.status_code}: {message}")
retry_after = resp.headers.get("Retry-After") wait = int(retry_after) if retry_after and retry_after.isdigit() else min(60, 2 ** attempt) * random.uniform(0.5, 1.5) time.sleep(wait)
data = scrape("instagram-profile", {"usernames": "mrbeast"})A 200 with an inner success
Section titled “A 200 with an inner success”A few endpoints include their own success or error fields inside data. Those belong to the scraped data. The outer success is what counts: an HTTP 200 with outer "success": true is a successful, billed call. The endpoint page points this out where it applies.