Skip to content

Topic

robots.txt

A plain-text file served from a site's root that tells crawlers which paths they may fetch and where the site's sitemaps live.

Current clusters