MCP & IDE agents
Scrape from the Cursor IDE with Piloterr MCP
Add mcp.piloterr.com as an HTTP MCP server in Cursor Settings or a project .mcp.json. Composer and Agent can call the full Piloterr library while you stay in the editor.
- Project .mcp.json or Settings → MCP, HTTP streamable, no local process
- x-api-key in headers, same credit pool as the REST API
- Ask Composer to crawl, render, or pull LinkedIn and Amazon while you write code
- Works with Cursor Agent, inline chat, and repo-scoped rules
At a glance
.mcp.json
project config
HTTP
streamable MCP
190+
library tools
IDE
Cursor Agent
Why connect Cursor
Config lives next to the repo
Commit a .mcp.json so every teammate gets the same Piloterr tools. Or keep the server only in Cursor Settings if the key should stay on your machine.
Agent that can see the files you are editing
Cursor already has the repo. Hosted MCP adds live web data, so the agent can compare a competitor page to your integration code without leaving the tab.
Not a browser tab
mcp.piloterr.com is Streamable HTTP. Opening it in Chrome will not finish the handshake. Cursor sends the Accept headers MCP requires.
Same tools as the REST library
Website crawl and render, WebUnlocker, LinkedIn, Amazon, Google News, and the rest of the OpenAPI surface, billed on the same credits.
What Cursor can call on Piloterr
Editor-first workflows, not a generic MCP recap.
Competitor pages while you code
Ask Agent to fetch a pricing or docs URL, then patch your client against the live JSON instead of a screenshot in Slack.
Fixture generation
Pull a real Amazon or SERP payload into the repo as a test fixture without writing cURL by hand.
Rules plus tools
Point Cursor rules at mcp.piloterr.com so the agent prefers Piloterr tools over a raw browser fetch that will get blocked.
Project vs user scope
Use .mcp.json for shared servers. Use Settings → MCP for a personal key that should not land in git.
Cursor + hosted MCP vs doing it in the chat pane
| Approach | DIY | Piloterr |
|---|---|---|
| Paste HTML into chat | Tokens wasted on markup | Compact JSON fields |
| Cursor browser fetch | Blocked on WAF sites | Server-side bypass |
| Local Playwright in the IDE | Slow, breaks on CAPTCHA | Managed render tools |
| Copy-paste cURL | Manual, not repeatable | Typed MCP tools |
Connect Cursor in four steps
Step 1
Create an API key
In the Piloterr dashboard, create a key and keep it for the headers block.
Create an API keyStep 2
Open Settings → MCP
Add an HTTP (streamable) server. URL: https://mcp.piloterr.com/. Header name x-api-key, value your key.
Step 3
Or commit .mcp.json
Put the same url and headers under mcpServers.piloterr so the project shares the server. Do not commit the real key.
Step 4
Ask Agent to use a tool
Try “Call the Piloterr website crawler on example.com” or any library tool. Check https://mcp.piloterr.com/health if the tool list is empty.
Workflow recipes
Diff a live docs page against your client
Agent crawls the vendor docs, then updates TypeScript types in the open file.
Seed a Playwright test with real HTML
Rendering tool returns cleaned content you save under fixtures/ instead of checking in a stale dump.
LinkedIn profile while writing an enricher
You implement the mapper; Agent calls the LinkedIn tool and shows the JSON shape inline.
Review a PR with fresh SERP
Agent pulls Google News for the feature name and comments whether messaging still matches the market.
Cursor vs other Piloterr entry points
Scenario
You already write code in Cursor
Recommendation: This page, hosted MCP
Scenario
You live in the terminal
Recommendation: Claude Code MCP
Scenario
You want a chat UI, not an IDE
Recommendation: ChatGPT Developer mode
Scenario
Scheduled pipelines, no editor
Recommendation: n8n or the SDK
Cursor project config
Drop this in .mcp.json at the repo root. Cursor also accepts the same shape in Settings → MCP.
{
"mcpServers": {
"piloterr": {
"url": "https://mcp.piloterr.com/",
"headers": {
"x-api-key": "your-x-api-key"
}
}
}
}Transparent credit pricing
Pay only for successful requests. Start with +500 credits, then scale with plans from $49/mo.
Premium
$49/mo
18,000 credits
Premium+
$99/mo
40,000 credits
Startup
$249/mo
110,000 credits
Ready to get started?
Your web scraping API is one click away. Start with +500 credits, no infrastructure to set up, no proxies to manage, and no credit card required.
- +500 credits
- No credit card required
- All endpoints included