GET /some/url.txt HTTP/1.1 x-cc-id: ccc04-01 Host: www.delorie.com:81 User-Agent: CCBot/1.0 (+http://www.commoncrawl.org/bot.html) Accept: text/html,application/xhtml+xml,text/xml;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Connection: close Cache-Control: no-cache Pragma: no-cache