User-agent: * Allow: / # Disallow auth-only and dynamic utility routes from being crawled Disallow: /dashboard Disallow: /login Disallow: /register Disallow: /forgot-password