User-agent: * # match psu spider and create rules Disallow: /aspnet_client/ Disallow: /calendar/ Disallow: /css_parts/ Disallow: /IMAGES/ Disallow: /js/ Disallow: /searches/ Disallow: /styles/