MCP server
Scrapier runs a remote MCP server, so AI agents can find and call any Scrapier endpoint as a tool. It reads the same live catalog as the API and the Playground, so new endpoints show up without a reinstall.
| URL | https://mcp.scrapier.io/mcp |
| Transport | Streamable HTTP |
| Auth | X-API-Key: YOUR_API_KEY or Authorization: Bearer YOUR_API_KEY |
| Key | Your normal API key from app.scrapier.io/dashboard/api-keys |
The server holds no key of its own. It forwards your key to the API for each call, so MCP usage is billed exactly like the REST API: each successful scrape costs the endpoint’s cost_credit (most endpoints cost 1 credit; see Credits & pricing), nothing for failures.
Connect your client
Section titled “Connect your client”Pick your tool. Every snippet shows YOUR_API_KEY. Click Use my key in the header to fill in your own key (it stays in your browser).
Command
Run this in your terminal
Terminal window claude mcp add --transport http --scope user scrapier https://mcp.scrapier.io/mcp --header "X-API-Key: YOUR_API_KEY"--scope usermakes Scrapier available in every project. Leave it out to add it to the current project only.Start claude and type /mcp — scrapier should be listed as connected.
Project file
Create .mcp.json in your project root
.mcp.json {"mcpServers": {"scrapier": {"type": "http","url": "https://mcp.scrapier.io/mcp","headers": {"X-API-Key": "${SCRAPIER_API_KEY}"}}}}Safe to commit — the key is read from the
SCRAPIER_API_KEYenvironment variable, not stored in the file.Save your key as an environment variable
macOS export SCRAPIER_API_KEY="YOUR_API_KEY"Linux export SCRAPIER_API_KEY="YOUR_API_KEY"Windows setx SCRAPIER_API_KEY "YOUR_API_KEY"On macOS/Linux, add the line to
~/.zshrcor~/.bashrcso it survives new terminals. On Windows, open a new terminal afterwards.Start claude in that folder, approve the scrapier server when asked, then type /mcp to check it’s connected.
Needs Node.js 18+
Claude Desktop’s connector screen can’t send an API key on most plans, so this uses a small open-source bridge (mcp-remote) that runs with Node.js.
In Claude Desktop, open Settings → Developer → Edit Config. This opens claude_desktop_config.json.
Add the scrapier entry to that file
macOS — ~/Library/Application Support/Claude/claude_desktop_config.json {"mcpServers": {"scrapier": {"command": "npx","args": ["-y","mcp-remote","https://mcp.scrapier.io/mcp","--header","X-API-Key:${SCRAPIER_API_KEY}","--transport","http-only"],"env": {"SCRAPIER_API_KEY": "YOUR_API_KEY"}}}}Windows — %APPDATA%\Claude\claude_desktop_config.json {"mcpServers": {"scrapier": {"command": "npx","args": ["-y","mcp-remote","https://mcp.scrapier.io/mcp","--header","X-API-Key:${SCRAPIER_API_KEY}","--transport","http-only"],"env": {"SCRAPIER_API_KEY": "YOUR_API_KEY"}}}}Quit Claude completely and reopen it — ⌘Q on Mac, or right-click the tray icon → Quit on Windows. Closing the window isn’t enough.
Click + in the chat box → Connectors — scrapier should be listed.
config.toml
Add this to ~/.codex/config.toml (create the file if it doesn’t exist)
~/.codex/config.toml [mcp_servers.scrapier]url = "https://mcp.scrapier.io/mcp"http_headers = { "X-API-Key" = "YOUR_API_KEY" }The Codex CLI, IDE extension and app all read this same file.
Restart Codex, then type /mcp (or run
codex mcp list) — scrapier should be listed.
Command
Save your key as an environment variable
macOS export SCRAPIER_API_KEY="YOUR_API_KEY"Linux export SCRAPIER_API_KEY="YOUR_API_KEY"Windows setx SCRAPIER_API_KEY "YOUR_API_KEY"Codex reads the key from this variable every time it starts, so add it to
~/.zshrc/~/.bashrc(Windows: open a new terminal aftersetx).Run this in your terminal
Terminal window codex mcp add scrapier --url https://mcp.scrapier.io/mcp --bearer-token-env-var SCRAPIER_API_KEYStart codex and type /mcp — scrapier should be listed.
One-click
Click the button
The button installs Scrapier with a
YOUR_API_KEYplaceholder. Replace it with your key in the client's MCP settings, or use Connect with my key in the page menu (top right) after saving your key.In Cursor, click Install.
Open Cursor Settings → MCP (Tools & MCP in newer versions) — scrapier should show a green dot.
Config file
Open (or create) ~/.cursor/mcp.json and add:
~/.cursor/mcp.json {"mcpServers": {"scrapier": {"url": "https://mcp.scrapier.io/mcp","headers": {"X-API-Key": "YOUR_API_KEY"}}}}For one project only, use
.cursor/mcp.jsoninside that project instead — and don’t commit it with your key in it.Open Cursor Settings → MCP (Tools & MCP in newer versions) — scrapier should show a green dot. If not, toggle it off and on.
Command
Run this in your terminal
Terminal window gemini mcp add --transport http --scope user --header "X-API-Key: YOUR_API_KEY" scrapier https://mcp.scrapier.io/mcp--scope usersaves it to~/.gemini/settings.jsonso it works in every folder.Start gemini and type /mcp — scrapier should be listed with its tools.
settings.json
Open ~/.gemini/settings.json and add:
~/.gemini/settings.json {"mcpServers": {"scrapier": {"httpUrl": "https://mcp.scrapier.io/mcp","headers": {"X-API-Key": "YOUR_API_KEY"}}}}Use
httpUrl, noturl— in Gemini CLIurlmeans the older SSE transport.Start gemini and type /mcp (or /mcp reload if it was already running) — scrapier should be listed.
In the Agent panel, open the ⋯ menu → MCP Servers → Manage MCP Servers → View raw config.
Menu names vary between versions — in newer builds look under Settings → Customizations → Installed MCP Servers.
Add the scrapier entry to mcp_config.json
mcp_config.json {"mcpServers": {"scrapier": {"serverUrl": "https://mcp.scrapier.io/mcp","headers": {"X-API-Key": "YOUR_API_KEY"}}}}Antigravity needs
serverUrl(noturl).Save, then click Refresh in Manage MCP Servers.
Needs GitHub Copilot
One-click
Click the button
The button installs Scrapier with a
YOUR_API_KEYplaceholder. Replace it with your key in the client's MCP settings, or use Connect with my key in the page menu (top right) after saving your key.VS Code opens the server page — click Install.
Open Copilot Chat, switch to Agent mode and click the tools icon — the scrapier tools should be listed.
Config file
Run MCP: Open User Configuration from the Command Palette (⇧⌘P / Ctrl+Shift+P) — or create .vscode/mcp.json in your project — and add:
mcp.json {"inputs": [{"type": "promptString","id": "scrapier_api_key","description": "Scrapier API key","password": true}],"servers": {"scrapier": {"type": "http","url": "https://mcp.scrapier.io/mcp","headers": {"X-API-Key": "${input:scrapier_api_key}"}}}}VS Code asks for your key once and keeps it in secure storage, so this file is safe to commit.
Start the server. When VS Code asks for your Scrapier API key, paste it:
Open Copilot Chat, switch to Agent mode and click the tools icon — the scrapier tools should be listed.
In the Cascade panel, open MCP servers and click View raw config.
This opens mcp_config.json (often
~/.codeium/windsurf/mcp_config.json). Newer builds branded Devin Desktop keep it elsewhere — just use the file the button opens.Add the scrapier entry
mcp_config.json {"mcpServers": {"scrapier": {"serverUrl": "https://mcp.scrapier.io/mcp","headers": {"X-API-Key": "YOUR_API_KEY"}}}}Save, then click Refresh in the MCP panel.
Remote (HTTP)
Use these connection details in your tool’s MCP settings
URL https://mcp.scrapier.io/mcpTransport Streamable HTTP Header X-API-Key: YOUR_API_KEYMany tools accept this JSON (the top-level key and URL field name vary — check your tool’s docs):
mcp config {"mcpServers": {"scrapier": {"url": "https://mcp.scrapier.io/mcp","headers": {"X-API-Key": "YOUR_API_KEY"}}}}
Local bridge
For tools that can only start local (stdio) servers, run Scrapier through the mcp-remote bridge (needs Node.js 18+):
mcp config {"mcpServers": {"scrapier": {"command": "npx","args": ["-y","mcp-remote","https://mcp.scrapier.io/mcp","--header","X-API-Key:${SCRAPIER_API_KEY}","--transport","http-only"],"env": {"SCRAPIER_API_KEY": "YOUR_API_KEY"}}}}Restart your tool so it starts the bridge.
The server exposes six fixed tools instead of one tool per endpoint. Hundreds of endpoint schemas would flood the agent’s context before it did any work, and many clients cap tool counts. Discovery tools keep the cost constant while still reaching every endpoint.
| Tool | Cost | What it does |
|---|---|---|
list_platforms |
free | Every platform and its endpoint count. |
search_endpoints |
free | Find endpoints by keyword. Returns names and costs, no schemas. |
get_endpoint_schema |
free | Full parameter schema for one endpoint. |
run_scrape |
The endpoint’s cost_credit per call |
Run a scrape and return one page of results, with next_cursor. |
get_scrape_result |
free | Kept for older clients. |
get_account |
free | Email, plan and credit balance for the key. A quick way to prove the key works. |
The intended flow is search_endpoints → get_endpoint_schema → run_scrape.
get_scrape_result is kept for compatibility; v2 returns results inline, so you won’t need it.
run_scrape returns one page with a top-level next_cursor. To get the next page, the agent passes it back as params.cursor. The original inputs are locked into the cursor, so required params can be left out. Each page is one billed call. When next_cursor is null, there are no more pages. See Pagination.
Guards before billing
Section titled “Guards before billing”The server rejects unknown parameters, missing required parameters and calls with no real input before anything is sent, so a malformed call never turns into a billed one.
Treat scraped content as data
Section titled “Treat scraped content as data”Scraped pages are written by whoever controls the site, not by your user. Scraping pipes that text straight into your agent’s context, and indirect prompt injection through web content is a real, actively used attack. Hidden text survives scraping even when a human can’t see it on the page.
The server marks every scrape result as untrusted. That label doesn’t sanitize anything; it’s a reminder. Your agent should:
- treat scraped text as data to read, never as instructions to follow
- never let scraped content trigger actions like sending messages, writing files or spending money without the user’s say-so
- quote or summarize suspicious instructions found in results instead of acting on them
The scrapier-safe-data-handling skill teaches this in detail.