Skip to content

Any non-2xx response is an error, and errors are never charged.

  1. Branch on the HTTP status code, not on the body.
  2. Read the message from error, or from detail if there is no error.
  3. Retry only 429, 503 and other 5xx responses, with backoff.

Don’t expect "success": false on every error. Scrapier returns three body shapes:

Shape Returned by Example statuses
{"success": false, "error": "…"} The scraper for that endpoint 400, 404, 500, 503 (capacity), 504, and some target-site errors
{"error": "…"} (402 adds balance and required) The Scrapier gateway 401, 402, 502, 503 (credit check)
{"detail": "…"} Authentication and throttling 403, 429
StatusMeaningRetry?Charged?
400Bad or missing input.No — fix the paramsNo
401 / 403Missing, invalid or revoked API key.No — fix the keyNo
402Out of credits (body has balance).No — top upNo
404Unknown endpoint slug.No — check the slugNo
429 / 503Busy or throttled.Yes — after Retry-AfterNo
5xxTarget site or upstream failure, or timeout.Yes — with backoffNo
Status Who returns it Body shape Retry? Charged?
400 Scraper success + error No, fix the input No
401 Gateway error No, add the header No
402 Gateway (billing) error, balance, required No, top up No
403 Authentication detail No, fix the key No
404 Scraper success + error No, check the slug No
429 Throttling detail Yes, after Retry-After No
500 Scraper success + error Yes, with backoff No
502 Gateway error Yes, with backoff No
503 Scraper (capacity) or gateway (credit check) success + error, or error Yes, after Retry-After or backoff No
504 Scraper success + error Yes, with backoff No

Bad or missing input, for example a missing required parameter or a value the scraper can’t use. The message says what’s wrong and varies by endpoint.

{ "success": false, "error": "…" }

The X-API-Key header is missing.

{ "error": "Missing HTTP_X_API_KEY header." }

Your balance is below the endpoint’s price (1 to 4 credits). Nothing ran and nothing was charged. Top up in Billing; retrying won’t help.

{ "error": "Insufficient credits. Buy more to scrape more.", "balance": 1, "required": 2 }

The key is unknown, invalid or revoked. There’s no success field, and the message is in detail.

{ "detail": "Authentication service unavailable or invalid API key." }
{ "detail": "API key has been revoked." }

The endpoint slug doesn’t exist. Check it against the API reference or GET /endpoints, and keep the trailing slash.

{ "success": false, "error": "Unknown v2 endpoint: <slug>" }

Too many requests right now. Wait for the number of seconds in the Retry-After header, then retry. See Rate limits & reliability.

{ "detail": "Request was throttled. Expected available in N seconds." }

Something failed on the target site or on our side, or the request took too long. These are free, and safe to retry with exponential backoff.

An unexpected error inside the scraper.

{ "success": false, "error": "Internal server error: …" }

The gateway couldn’t get a usable answer in time, or the upstream response was broken. The body has an error message, for example:

{ "error": "Main API timed out." }

Busy. There are two cases:

  • At capacity: too many scrapes are running. The response carries Retry-After: 30. Wait that long and retry.
  • Credit check unavailable: we couldn’t verify your balance, for example {"error": "Could not verify credits."}. Retry with backoff.
{ "success": false, "error": "…" }

The scraper hit its time limit for this endpoint.

{ "success": false, "error": "Scraper exceeded …s timeout" }

Some endpoints take several minutes. Set your client timeout to at least 210 seconds so your client doesn’t give up before Scrapier does.

async function scrape(slug, params, { retries = 4 } = {}) {
const url = `https://api.scrapier.io/api/v2/${slug}/?${new URLSearchParams(params)}`;
for (let attempt = 0; ; attempt++) {
const res = await fetch(url, {
headers: { "X-API-Key": process.env.SCRAPIER_API_KEY },
signal: AbortSignal.timeout(210_000),
});
const body = await res.json().catch(() => ({}));
if (res.ok) return body.data;
const message = body.error ?? body.detail ?? `HTTP ${res.status}`;
const billing402 = res.status === 402 && "balance" in body;
const retryable = res.status === 429 || res.status >= 500 || (res.status === 402 && !billing402);
if (!retryable || attempt >= retries) throw new Error(`${res.status}: ${message}`);
const retryAfter = Number(res.headers.get("Retry-After"));
const waitMs = retryAfter > 0 ? retryAfter * 1000 : Math.min(60_000, 2 ** attempt * 1000) * (0.5 + Math.random());
await new Promise((r) => setTimeout(r, waitMs));
}
}
const data = await scrape("instagram-profile", { usernames: "mrbeast" });

A few endpoints include their own success or error fields inside data. Those belong to the scraped data. The outer success is what counts: an HTTP 200 with outer "success": true is a successful, billed call. The endpoint page points this out where it applies.