#                    ↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖              ↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖↖
#                  ↖↘↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗↙↖        ↖↙↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↘↖
#                  ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↘↖      ↖↘↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                  ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙      ↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                  ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙      ↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                  ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙      ↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                  ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙      ↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                  ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙      ↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                  ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙      ↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                  ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙      ↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                  ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙      ↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                  ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↓      ↓↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                  ↖↘↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑→↖    ↖→↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↘↖
#                    ↓↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑←  ←↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↓
#                     ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                      ←→↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↘↘↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←
#                       ↖→↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙  ↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑→↖
#                         ↓↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗←    ←↗↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↓
#                         ↖↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑→↖      ↖→↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↙↖
#                           ←↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑→↖        ↖↘↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑↑←
#                            ↖→↑↑↑↑↑↑↑↑↑↑↑↑↑↑↓            ↙↑↑↑↑↑↑↑↑↑↑↑↑↑↑↗↖
#                             ↖↓↑↑↑↑↑↑↑↑↑↑↑↗↙↖            ↖↙↗↑↑↑↑↑↑↑↑↑↑↑↘↖
#                              ↖↓↑↑↑↑↑↑↑↑↑↗←                ←↗↑↑↑↑↑↑↑↑↑↓↖
#                                ↖↑↑↑↑↑↑↑→                    ↘↑↑↑↑↑↑↑←
#                                 ↖→↑↑↑↑↓↖                    ↖↓↑↑↑↑↗↖
#                                  ↖↘↑↑↓↖                      ↖↙↑↑→↖
#                                   ↖↖↖                          ↖↖↖
#
# Design & Engineering Studio
# https://webrenew.com

# Allow all crawlers - we're generous!
User-agent: *
Allow: /

# Sitemap
Sitemap: https://webrenew.com/sitemap.xml

# LLMs.txt for AI assistants
# https://webrenew.com/llms.txt

# Crawl-delay (be nice to our server)
Crawl-delay: 1

# Specific crawler optimizations
User-agent: Googlebot
Allow: /
Crawl-delay: 0

User-agent: Bingbot
Allow: /
Crawl-delay: 0

User-agent: GPTBot
Allow: /

User-agent: ChatGPT-User
Allow: /

User-agent: CCBot
Allow: /

User-agent: Claude-Web
Allow: /

User-agent: PerplexityBot
Allow: /

# Archive bots welcome
User-agent: ia_archiver
Allow: /

# Social media bots
User-agent: Twitterbot
Allow: /

User-agent: facebookexternalhit
Allow: /

User-agent: LinkedInBot
Allow: /

User-agent: Slackbot
Allow: /

# Developer tools
User-agent: Screaming Frog SEO Spider
Allow: /

# Don't index debug files (if any)
Disallow: /debug/
Disallow: /*.log$
Disallow: /do-not-commit-this/

# Private proposal links — shared directly with clients
Disallow: /proposal/

# Dashboard is private
Disallow: /dashboard/

# Allow everything else!
# Thanks for visiting 🤖