Robots.txt Specification

User-agent Rules

General Rules

• User-agent: *
  • Allow: /

Disallowed Areas

• Disallow: /admin/
• Disallow: /private/

Search Engine Rules

• User-agent: Googlebot

  • Allow: /

• User-agent: Bingbot

  • Allow: /

• User-agent: Slurp

  • Allow: /

AI/Search Assistant Rules

• User-agent: GPTBot

  • Allow: /

• User-agent: OAI-SearchBot

  • Allow: /

• User-agent: ChatGPT-User

  • Allow: /

• User-agent: ClaudeBot

  • Allow: /

• User-agent: PerplexityBot

  • Allow: /

Sitemap

• Sitemap: https://trashlab.com/sitemap.xml