# For all robots User-agent: * # Block access to specific groups of pages Disallow: /order-confirmation Disallow: /login Disallow: /wholesale Request-rate: 1/10 # maximum rate is one page every 10 seconds Crawl-delay: 10 # 10 seconds between page requests Visit-time: 0400-0845 # only visit between 04:00 and 08:45 UTC # Allow search crawlers to discover the sitemap # Sitemap: /sitemap.xml