# robots.txt - helps control searchbots and webspiders User-agent: * Disallow: /cgi-bin Disallow: /mt Disallow: /crm