# robots.txt file User-agent: * Disallow: manage.pl User-agent: Microsoft URL Control Disallow: / User-agent: NPBot Disallow: / User-agent: psbot/0.1 Disallow: / User-agent: ia_archiver Disallow: /