Definition updated on November 2023

What is robots.txt?

Search engine spiders are given specific pages or sections of a website to not index via a file called Robots.txt. The majority of significant search engines, including Google, Bing, and Yahoo, are aware of and abide by Robots.txt instructions.

Showing 0 of 100
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
No results found.
There are no results with this criteria. Try changing your search.