View all API endpoints

Website Crawler

HTML scraping API : A robust solution designed for efficiently extracting data from web pages. 🕷
Website
August 23, 2023
2500
ms
798
k requests
58
votes
▶ Try for free
Documentation API
Get it customized
No credit card required
README
API Reference
Related Endpoints

Introducing our advanced HTML scraping API: a robust solution designed for efficiently extracting data from web pages. Leveraging cutting-edge technology, our API ensures stable requests, even on the most intricate websites. Ideal for businesses and developers needing access to real-time, dependable information. Please always use this API in compliance with prevailing laws and the terms of use of the targeted websites. Emulates Chrome TLS fingerprint, backed by rotating proxies and smart retries. TLS Fingerprinting is a technique that allows you to bypass TLS certificate pinning. 💫

Key Features

Cutting-Edge Technology

Leveraging state-of-the-art technology, our API ensures stable requests. It's tailored to work seamlessly even with the most intricate websites, providing you with accurate data every time.

Built for Businesses and Developers

This is an essential tool for businesses and developers who require access to real-time, dependable information. Whether you're into market research, competitor analysis, or web development, our API will transform the way you gather data.

Safety and Compliance

Always ensure to use this API in compliance with prevailing laws and the terms of use of the targeted websites. We advocate for responsible and ethical scraping practices.

Advanced TLS Fingerprinting

Our API emulates the Chrome TLS fingerprint. What's TLS Fingerprinting? It's a technique that allows you to bypass TLS certificate pinning, enhancing the flexibility and efficiency of your scraping operations. And with the backing of rotating proxies and smart retries, you're ensured uninterrupted service.

Effortless Bypassing Capabilities

Bypass Cloudflare

Navigate through Cloudflare's security seamlessly.

Bypass Akamai

No more stumbling blocks when dealing with Akamai's protective measures.

Bypass PerimeterX

Overcome PerimeterX defenses with ease.

Bypass DataDome

DataDome's security won’t stand in your way anymore.

Step into the future of web scraping with our advanced HTML scraping API. Your data extraction tasks just got a lot easier. 💫

Content:
H2
H3

Header

x-api-key required

string

Parameter defines the Piloterr private key to use.

Parameters

query

string

A website url with `http` or `https` protocol

impersonate_version

string

Impersonate a browser version : chrome99, chrome100, chrome101, chrome104, chrome107, chrome110, chrome99_android, edge99, edge101, safari15_3, safari15_5.

allow_redirects

boolean

If set to false, do not follow redirects. false by default.

GET
POST
/api/v2/website/crawler
Example Request
curl --location --request GET 'https://piloterr.com/api/v2/website/crawler?query=https://example.com' \
--header 'Content-Type: application/json' \
--header 'x-api-key: <token>' \
Response
<!doctype html>
<html>
<head> <title>Example Domain</title> <meta charset="utf-8" /> <meta http-equiv="Content-type" content="text/html; charset=utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=1" /> <style type="text/css"> body { background-color: #f0f0f2; margin: 0; padding: 0; font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif; } div { width: 600px; margin: 5em auto; padding: 2em; background-color: #fdfdff; border-radius: 0.5em; box-shadow: 2px 3px 7px 2px rgba(0, 0, 0, 0.02); } a:link, a:visited { color: #38488f; text-decoration: none; } @media (max-width: 700px) { div { margin: 0 auto; width: auto; } } </style>
</head> <body>
<div> <h1>Example Domain</h1> <p>This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.</p> <p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>
Sign up now and enjoy 100 free API requests + full access to our premium features and tools.
Sign-up

Email & Phone Extractor

GET
POST
Website
Email and phone extractor allows you to scrape any website for contact information. That's right, any website!
2036
ms
55
k requests

Website Technology

GET
POST
Website
Discover the technology behind any website. Leverage this information for lead generation, market analysis, competitor..
2078
ms
25
k requests
Other API endpoints that might be of interest for you:

StockX Product

GET
POST
StockX
Retrieve detailed product information from StockX, including images, colorway, recent sales, and purchase data.
1020
ms
511
k requests

Company Database

GET
POST
Database
Enhance your data with our cutting-edge enrichment API, updated daily, to provide in-depth insights about businesses.
800
ms
582
k requests

LinkedIn Company Info

GET
POST
Linkedin
LinkedIn Company Scraper API is your gateway to vital business insights!
1023
ms
820
k requests

Company Email Pattern

GET
POST
Database
This API allows you to retrieve the email pattern used by a specific company. 📨
800
ms
42
k requests

StockX Search

GET
POST
StockX
Search for authentic products seamlessly on StockX, the leading online marketplace for sneakers and fashion.
2800
ms
521
k requests

Leboncoin Search

GET
POST
Leboncoin
Enables users to search for ads on Leboncoin, the leading classified ads platform in France.
2847
ms
287
k requests

LinkedIn Job Info

GET
POST
Linkedin
Unlock the power of LinkedIn job data. Gateway to in-depth job insights and seamless integrations.
1021
ms
214
k requests

Threads User Info

GET
POST
Threads
Easily retrieve user information from Threads.net using their ID or username.
1542
ms
250
k requests

Instagram User Info

GET
POST
Instagram
Unlock the potential of Instagram data with our API. Efficiently extract user insights.
1256
ms
145
k requests

Threads User Threads

GET
POST
Threads
Retrieve the latest thread posts created by a user, ensuring you're always updated.
2541
ms
145
k requests

Linkedin Profile

GET
POST
Linkedin
Extract the public information of a LinkedIn profile swiftly and efficiently with our LinkedIn Profile Info API.
4206
ms
68
k requests

LinkedIn Job Search

GET
POST
Linkedin
Experience unmatched job data extraction capabilities with our LinkedIn Jobs API.
1520
ms
214
k requests

Google Search

GET
POST
Google
Unearth the Power of Google Search - One API Call at a Time. Stay Ahead, Stay Informed.
2104
ms
541
k requests

Wellfound Company Info

GET
POST
Wellfound
Unlock insights on demand: dive deep into company profiles with Wellfound
2420
ms
17
k requests

Crunchbase Company Info

GET
POST
Crunchbase
Transform URLs into insightful company information sourced from Crunchbase.
2548
ms
165
k requests

Instagram Post Info

GET
POST
Instagram
Effortlessly Instagram scrape posts, track competitors, and discover market trends.
1423
ms
561
k requests

StockX Product 360

GET
POST
StockX
Explore 360-degree image API, designed to elevate user experiences by providing a full panoramic view.
2301
ms
54
k requests

Crunchbase Fundings Round

GET
POST
Crunchbase
Discover the financial journey of companies on Crunchbase.
2478
ms
48
k requests

LinkedIn Job Count

GET
POST
Linkedin
Real-time count for Linkedin Job, segmented by industry, region, and role, aiding in understanding employment trends.
2100
ms
147
k requests

Take a look at our blog posts

Interviews, tips, guides, industry best practices and news.
What is Residential Proxy ?
Proxy
10
min read

What is Residential Proxy ?

A residential proxy is a type of proxy server that routes internet traffic via an intermediary...
Read post
Web Scraping for Lead Generation
Scraping
8
min read

Web Scraping for Lead Generation

Gathering contact information from websites in order to generate leads for businesses is known...
Read post
Free Proxies for Web Scraping
Proxy
9
min read

Free Proxies for Web Scraping

Free Proxies are the web servers that act as intermediaries between computers and the internet...
Read post
View all posts
By clicking “Accept”, you agree to the storing of cookies to enhance site navigation and analyze site usage. View our Privacy Policy for more information.
PreferencesAccept