Pagination
The one rule
Section titled “The one rule”Every response has data.next_cursor. To get the next page, send it back unchanged as the cursor parameter. When next_cursor is null, you’ve reached the last page.
{ "success": true, "data": { "posts": ["…"], "next_cursor": "eyJpIjoiaW5zdGFncmFtIiwicCI6Mn0" } }- Treat the cursor as opaque. Don’t decode, edit or build it yourself.
- Required inputs are ignored when you send a cursor. Your original inputs are stored in it. Parameters marked locked in cursor in the reference are also reused from the first call.
- Session cookies are the exception. Endpoints that need your own login cookie (for example
session_idon some Instagram endpoints) need it on every page. The endpoint page tells you which. - Each page is one billed call. Cap how many pages you fetch.
- Don’t count on a cursor staying valid for a long time. Use it soon after you get it.
A loop
Section titled “A loop”# Page 1curl -G "https://api.scrapier.io/api/v2/google-maps/" \ -H "X-API-Key: YOUR_API_KEY" \ --data-urlencode "queries=coffee shops" \ --max-time 210
# Next page: send only the cursor from the previous responsecurl -G "https://api.scrapier.io/api/v2/google-maps/" \ -H "X-API-Key: YOUR_API_KEY" \ --data-urlencode "cursor=PASTE_NEXT_CURSOR_HERE" \ --max-time 210const url = "https://api.scrapier.io/api/v2/google-maps/";let params = { queries: "coffee shops" };const places = [];
for (let page = 0; page < 10; page++) { // each page is one billed call const res = await fetch(`${url}?${new URLSearchParams(params)}`, { headers: { "X-API-Key": "YOUR_API_KEY" }, signal: AbortSignal.timeout(210_000), }); const body = await res.json(); if (!res.ok) throw new Error(body.error ?? body.detail ?? `HTTP ${res.status}`); places.push(...(body.data.places ?? [])); if (!body.data.next_cursor) break; // null = last page params = { cursor: body.data.next_cursor };}import requests
URL = "https://api.scrapier.io/api/v2/google-maps/"HEADERS = {"X-API-Key": "YOUR_API_KEY"}params = {"queries": "coffee shops"}places = []
for _ in range(10): # each page is one billed call resp = requests.get(URL, headers=HEADERS, params=params, timeout=210) resp.raise_for_status() data = resp.json()["data"] places.extend(data.get("places") or []) if not data.get("next_cursor"): # None = last page break params = {"cursor": data["next_cursor"]}The result list’s key changes per endpoint (places, posts, products, reviews…). Each endpoint page names it.
Session cookies on every page
Section titled “Session cookies on every page”For endpoints that need your own session cookie, add it next to the cursor:
params = {"cursor": data["next_cursor"], "session_id": "YOUR_SESSION_ID"}Single-lookup endpoints
Section titled “Single-lookup endpoints”Some endpoints return everything in one call, such as a single profile, post or transcript. Their next_cursor is always null, and the reference marks them Single lookup.