I've argued that there's a need for an HTTP-specific mechanism for signing HTTP responses. So let's try and design one. (Usually at this point, I would start coding, but with security-related stuff, I think it's better to have more discussion up front.)
Before drilling down into syntax, I would like to work out the design at a more abstract level. Let's suppose that the mechanism will take the form of a new Signature header. Here is my current thinking as to the steps involved in constructing a Signature header:
- Choose which security token to use and create one or more identifiers for it.
- Choose the suite of cryptographic algorithms to use.
- Choose which request and response headers to sign.
- Compute the cryptographic hashes of the request and response entity bodies if they are present. Base64-encode those hashes.
- Create a Signature header template containing the information from steps 1 to 4; this differs from the final Signature header only in that it has a blank string at the point where the final Signature header will have the base64-encoded signatures value.
- Canonicalize the request headers.
- Combine the response headers with the Signature header template and canonicalize them.
- Create a string that encodes the HTTP method, the request URI, the canonicalized request headers from step 6, the response status, and the canonicalized response headers from step 7.
- Compute the cryptographic hash of the string created in the previous step.
- Sign the cryptographic hash created in the previous step. Base64-encode this to create the signature value.
- Create the final Signature header by inserting the base64-encoded signature value from the previous step into the Signature header template from step 5.
Let's flesh this out with a bit of q&a.
- What kinds of security token can be used?
At least X.509 certificates should be supported. But there should be the potential to support other kinds of token.
- How are security tokens identified?
It depends on the type. For X.509, it would make sense to have a URI that allowed the client to fetch the certificate. It would also be desirable to have an identifier that uniquely identifies the certificate, so that the client can tell whether it already has the certificate without having to go fetch it. As far as I can tell, in the X.509 case, people mostly use the SHA-1 hash of the DER encoding of the certificate for this.
- How does the server know what kind of signature (if any) the client wants?
The client can provide a Want-Signature header in the request. (It makes more sense to me to call this Want-Signature than Accept-Signature, just as RFC 3230 uses Want-Digest, since any client should be able to accept any signature.)
- What if the response uses a transfer encoding?
It's the entity body that's hashed in step 4. Thus, the sender computes the hash of the entity body before applying the transfer encoding.
- What if the response uses a content encoding?
It's the entity body that's hashed. Thus, the sender computes the hash of the entity body after applying the content encoding. From 7.2 of RFC 2616:
entity-body := Content-Encoding( Content-Type( data ) )
- What if the response contains a partial entity body (and a Content-Range header)?
The hash covers the full entity-body (that is. the hash is computed by the recipient after all the partial entity bodies have been combined into the full entity-body). In the terminology of RFC 3230, it covers the instance (where there is a relevant instance), rather than the entity. Thus the hash identifies the same thing as a strong entity tag (which in my view makes the terminology of RFC 3230 rather unfortunate).
- Why and how are headers canonicalized?
As I understand the HTTP spec, proxies are allowed to change various semantically insignificant syntactic details of the headers before they pass them on. For example, section 2.2 of RFC 2616 says any linear white space may be replaced with a single SP. Section 4.2 says explicitly proxies must not change the relative order of field values with the same header field name, but it seems to imply that proxies are allowed to change syntactic details of the header fields that are not syntactically significant (such as the relative order of headers with different field names). It is not clear to me to what extent section 13.5.2 overrides this. Perhaps some HTTP wizard can help me out here. In any case, there needs to be just enough canonicalization to ensure that the canonicalization of the headers sent by the origin server will be the same as the canonicalization of the headers as seen by the client, no matter what HTTP/1.1 conforming proxies (or commonly-deployed non-conforming proxies) might be on the path between the origin server and the client. I would note that the Amazon REST Authentication scheme does quite extensive canonicalization.
- Can there be multiple signatures?
Yes. In the normal HTTP style, the Signature header should support a comma-separated list of signatures. The order of this list would be significant. There should be a way for each signature in the list to specify which of the previous signatures in the list are included in what it signs. There's a semantic difference between two independent signatures, and a later signature endorsing an earlier signature.
- How about streaming?
Tricky. The fundamental problem is that HTTP 1.1 isn't very good at enabling the interleaved delivery of data and metadata. This is one of the things that Roy Fielding's Waka is supposed to fix. I don't think it's a good idea to put a lot of complexity into the design of a specific header to fix a generic HTTP problem. The fact that the hash of the entity body is computed in a separate, independent preliminary step makes it a bit easier for the server to precompute the hash when the content is static. Also the signature header could be put in a trailer field when using a chunked transfer-coding. However, although this helps the server, it screws the client's ability to stream, because the client needs to know the hash algorithm up front. And, of course, it also requires all the proxies between the client and the server to support trailer fields properly.