# beSirius — robots.txt # Search engine crawlers User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: Twitterbot Allow: / User-agent: facebookexternalhit Allow: / User-agent: LinkedInBot Allow: / # AI / LLM crawlers — explicitly allowed for GEO visibility User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: Claude-SearchBot Allow: / User-agent: Google-Extended Allow: / User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / User-agent: Bytespider Allow: / User-agent: CCBot Allow: / User-agent: Meta-ExternalAgent Allow: / # Default policy User-agent: * Allow: / # Block tracking-parameter URLs (avoid duplicate-content penalties from GSC) Disallow: /*?utm_ Disallow: /*&utm_ Disallow: /*?_hsenc Disallow: /*&_hsenc Disallow: /*?_hsmi Disallow: /*&_hsmi Disallow: /*?hsCtaTracking Disallow: /*?yclid Disallow: /*?gclid Disallow: /*?fbclid Disallow: /*?mc_eid Disallow: /*?mc_cid # Block legacy Webflow pagination query params Disallow: /*?*_page= Disallow: /*&*_page= # Block HubSpot session tracking params Disallow: /*?__hstc Disallow: /*&__hstc Disallow: /*?__hssc Disallow: /*?__hsfp # Block legacy `ref=` referral params Disallow: /*?ref= Disallow: /*&ref= # Block leaked draft from old CMS Disallow: /draft---do-not-publish-or-delete-everyday-ai-for-sustainability-professionals-besirius-copy Sitemap: https://besirius.io/sitemap.xml