grabbing HTTP headers

Sometimes it's useful to inspect the HTTP response headers from a Webserver (for example, to know if the Webserver is running Apache, IIS, or something else). Most Web browsers have some sort of 'Page Info' feature which will display the response headers. But it's often more convenient to do it from the command line.

The text-only browser lynx has a nice feature for this. Typing the following command will dump the HTTP response headers to the screen without displaying the content of the www.example.com homepage:

lynx -dump -head http://www.example.com/

wget can also do this. wget ordinarily downloads the Web content to a local file without displaying response headers. The following will show the headers and discard the content (-S displays the headers, and -O diverts the output here to /dev/null):

wget -S -O /dev/null http://www.example.com/

The curl utility can do much the same thing (note that this is a lower-case o to specify the output destination, and there a bare hyphen after the -D, indicating that the headers should be written to stdout):

curl -D - -o /dev/null http://www.example.com/

netcat offers a fourth way of getting the headers by allowing you to hurl a custom HTTP request at port 80 on the Webserver:

printf "HEAD / HTTP/1.0\n\n" | nc www.example.com 80

The previous example would only work for HTTP. For HTTPS, you can do a similar trick using the s_client mode for the openssl utility (this example uses an HTTP v1.1 request, which requires a host request header):

printf "HEAD / HTTP/1.1\nhost: www.example.com\n\n" \
| openssl s_client -ign_eof -connect www.example.com:443

The -ign_eof keeps the connection open so that the s_client will see the printf output: this also requires manually closing the connection (Control-C should do it). Additionally, you may get certificate verification errors from openssl. If so, try specifying your system's certificate authority bundle (which contains the public keys of a list of trusted certificate authorities, and which may be in a different location that this example):

printf "HEAD / HTTP/1.1\nhost: www.example.com\n\n" \
| openssl s_client -ign_eof \
-CAfile /etc/pki/tls/certs/ca-bundle.crt \
-connect www.example.com:443