Website Crawler
Website Crawler

Website Crawler

A robust solution for efficiently extracting a wide range of data from web pages.
Public access
Private access
January 21, 2024
2500
ms
798
k requests
254
votes
▶ Try for freeGet it customized
50 free requests, no credit card required
Request accessGet it customized
Access to this API endpoint is limited, submit a request to gain access

Header

x-api-key required

string

This parameter defines the Piloterr private key to use for authentification.

Parameters

query

string

A website url with `http` or `https` protocol

impersonate_version

string

Impersonate a browser version : chrome99, chrome100, chrome101, chrome104, chrome107, chrome110, chrome99_android, edge99, edge101, safari15_3, safari15_5.

allow_redirects

boolean

If set to false, do not follow redirects. false by default.

Sign up now and enjoy 50 free API requests + full access to our premium features and tools.

Introducing our advanced HTML scraping API: a robust solution designed for efficiently extracting data from web pages. Leveraging cutting-edge technology, our API ensures stable requests, even on the most intricate websites. Ideal for businesses and developers needing access to real-time, dependable information. Please always use this API in compliance with prevailing laws and the terms of use of the targeted websites. Emulates Chrome TLS fingerprint, backed by rotating proxies and smart retries. TLS Fingerprinting is a technique that allows you to bypass TLS certificate pinning.

Key Features

Cutting-Edge Technology

Leveraging state-of-the-art technology, our API ensures stable requests. It's tailored to work seamlessly even with the most intricate websites, providing you with accurate data every time.

Built for Businesses and Developers

This is an essential tool for businesses and developers who require access to real-time, dependable information. Whether you're into market research, competitor analysis, or web development, our API will transform the way you gather data.

Safety and Compliance

Always ensure to use this API in compliance with prevailing laws and the terms of use of the targeted websites. We advocate for responsible and ethical scraping practices.

Advanced TLS Fingerprinting

Our API emulates the Chrome TLS fingerprint. What's TLS Fingerprinting? It's a technique that allows you to bypass TLS certificate pinning, enhancing the flexibility and efficiency of your scraping operations. And with the backing of rotating proxies and smart retries, you're ensured uninterrupted service.

Effortless Bypassing Capabilities

Bypass Cloudflare Bot Managment

Navigate through Cloudflare's security seamlessly.

Bypass Akamai

No more stumbling blocks when dealing with Akamai's protective measures.

Bypass PerimeterX

Overcome PerimeterX defenses with ease.

Bypass DataDome

DataDome's security won’t stand in your way anymore.

Step into the future of web scraping with our advanced HTML scraping API. Your data extraction tasks just got a lot easier.

Supported impersonate versions

  • chrome99
  • chrome100
  • chrome101
  • chrome104
  • chrome107
  • chrome110
  • chrome116
  • chrome119
  • chrome120
  • chrome99_android
  • edge99
  • edge101
  • safari15_3
  • safari15_5
  • safari17_0
  • safari17_2_ios
GET
POST
/api/v2/website/crawler
Example Request
curl --location --request GET 'https://piloterr.com/api/v2/website/crawler?query=https://example.com' \
--header 'Content-Type: application/json' \
--header 'x-api-key: <token>' \
Response
<!doctype html>
<html>
<head> <title>Example Domain</title> <meta charset="utf-8" /> <meta http-equiv="Content-type" content="text/html; charset=utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=1" /> <style type="text/css"> body { background-color: #f0f0f2; margin: 0; padding: 0; font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; } div { width: 600px; margin: 5em auto; padding: 2em; background-color: #fdfdff; border-radius: 0.5em; box-shadow: 2px 3px 7px 2px rgba(0, 0, 0, 0.02); } a:link, a:visited { color: #38488f; text-decoration: none; } @media (max-width: 700px) { div { margin: 0 auto; width: auto; } } </style>
</head> <body>
<div> <h1>Example Domain</h1> <p>This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.</p> <p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>
Premium
$49/mo
2,50€ = 1,000 requests
18,000 requests
7 requests per second
Clean & parsed data
Email support
Premium+
$99/mo
2,23€ = 1,000 requests
40,000 requests
10 requests per second
Clean & parsed data
Email support
Startup
$249/mo
2,08€ = 1,000 requests
110,000 requests
15 requests per second
Clean & parsed data
Email & chat support
Startup+
$499/mo
$2.17 = 1,000 requests
230,000 requests
20 requests per second
Clean & parsed data
Chat & Email support
Enterprise+
$999/mo
$1.88 = 1,000 requests
530,000 requests
30 requests per second
Clean & parsed data
Account manager
Custom
+$2,000/mo
Custom CPM price
Custom requests count
Custom requests rate
Clean & parsed data
Account manager
Premium
45€/mo
2,50€ = 1,000 requests
18,000 requests
7 requests per second
Clean & parsed data
Email support
Premium+
89€/mo
2,23€ = 1,000 requests
40,000 requests
10 requests per second
Clean & parsed data
Email support
Startup
229€/mo
2,08€ = 1,000 requests
110,000 requests
15 requests per second
Clean & parsed data
Email & chat support
Startup+
459€/mo
2,00€ = 1,000 requests
230,000 requests
20 requests per second
Clean & parsed data
Chat & Email support
Enterprise+
919€/mo
1,73€ = 1,000 requests
530,000 requests
30 requests per second
Clean & parsed data
Account manager
Custom
$2,000/mo
Custom CPM price
Unlimited requests
Custom requests rate
Clean & parsed data
Account manager

Get 50 free requests when signing-up

Use Piloterr's private proxy pool

Get 10% off when buying annual subscriptions, contact us to get started.
Accepted payment methods:

Get 50 free requests now!

Sign up now and enjoy 50 free API requests + full access to our premium endpoints!

"Piloterr offers a great set of APIs to retrieve companies and profiles data. Endpoints are well documented, responses are fast and there is great support if needed (way better than rapidapi by the way)! A really good job from the Piloterr team! Continue the good work guys!"

Mantiks

Alexandre, CEO Mantiks

"5* tool, reliable and efficient - works perfectly to collect Google data. Bonus, the support is really responsive."

Corentin Robert

Growth

"They know what they're doing and they know how to provide the best tools to helps startups to grow faster. Their solution is a no-brainer for every tech-marketing team. Plus, the best support I have ever seen in the SaaS industry..."

Polar Analytics

Paul-François, Lead Growth

Frequently asked questions

Everything you need to know about our products & services.

Are Piloterr solutions secured and GDPR compliant?

We have implemented industry-standard security measures to ensure the safety of your data. Our platform uses SSL encryption to protect your API requests. We're also fully aligned with GDPR and CCPA compliance standards.

How frequently are new APIs added, can I suggest new ones?

We are always adding new APIs to our platform based on customer demand and emerging technologies. If you have a specific API you would like us to add, please contact our support team and let us know.

Can I upgrade, downgrade and cancel anytime?

Active plans can be upgraded, downgraded and cancelled anytime. Changes are immediate. When changing plan, the remaining unused requests of your current plan are added from your next invoice amount.

What is your cancellation policy?

We understand that things change. You can cancel your plan at any time and we’ll refund you the difference that you already paid (annual subscriptions only and under the terms described here).

Do my unused credits rollover to the next month?

Your monthly subscription restarts with your monthly allowance of successful search credits on the first day of your billing cycle's subscription.

Do you offer technical support for users?

Yes! We are always happy to help users to be able to use our API! You can chat us and we'll try and get back to you within a single business day.

How requests API calls are calculated?

Only successful requests are counted towards your monthly searches. Cached, errored and failed searches are not.

Can I request custom API endpoints?

Of course! We are always expanding our code and building new APIs for different search engines. You can create a feature request on our Roadmap. We aren't able to build APIs for every websites for a variety of reasons but we do always make additions where they are needed and able to be done.