# All robots will spider the domain User-agent: * Disallow: User-agent: * Disallow: /private/ User-agent: * Disallow: /cms/ User-agent: * Disallow: /inc/