GET /some/url.html HTTP/1.1 User-Agent: CCBot/2.0 (https://commoncrawl.org/faq/) Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 If-Modified-Since: Mon, 23 Sep 2019 17:28:40 CEST Accept-Encoding: br,gzip Host: www.delorie.com:81 Connection: Keep-Alive