Friday, February 6, 2009

content-negotiation for humans

Simon Willison asked for opinions on how to deliver JSON content properly while also assisting browser-driven exploration and debugging.

Here is a short, simple example of how to use content-negotiation to achieve this.

This is a complete, working webmachine resource with trivial content:

-export([init/1, to_json/2, content_types_provided/2]).


init([]) -> {ok, x}.

to_json(_,X) -> {"{\"key\": \"value\"}\n", X}.

content_types_provided(_,X) ->
{[{"application/json", to_json},{"text/plain", to_json}], X}.

A typical request/response, with a tiny bit of header noise trimmed:

$ curl -v http://localhost:8000/js/simonw
> GET /js/simonw HTTP/1.1
> Accept: */*
< HTTP/1.1 200 OK
< Vary: Accept
< Server: MochiWeb/1.1 WebMachine/0.20 (There was kicking.)
< Content-Type: application/json
< Content-Length: 17
{"key": "value"}

This is what you generally want. The JSON was delivered with the proper content-type and so on. Note the presence of the "Vary" header.

However, if the same request is made with a typical browser's Accept header, like FireFox's, the result will be different:

> GET /js/simonw HTTP/1.1
> Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
< HTTP/1.1 200 OK
< Vary: Accept
< Server: MochiWeb/1.1 WebMachine/0.20 (There was kicking.)
< Content-Type: text/plain
< Content-Length: 17
{"key": "value"}

This time we got the same JSON content, but in text/plain so it will display nicely in a browser window if requested directly. A Web application loading this content would probably set the Accept header in XHR requests to "Accept: application/json" and get the first response.

This is a straightforward use of content-negotiation that serves a useful purpose and (by using Accept properly and providing the Vary response header) still works well with intermediaries and the rest of the mechanics of the Web.


Subbu Allamaraju said...

How does the Accept header in the second example (with text/xml, application/xml, text/html and so on) imply that the UA prefers text/plain? Since there is no q-value of zero for other types not specified in the Accept header, I would expect the server to return application/json as usual.

A second problem with this model is that it would break any code that is expecting to process application/json.

Justin Sheehy said...

That Accept header says that the client prefers text/plain to application/json, though it prefers (for example) text/html to either and text/xml to that.

The Accept header above, which I copied from firefox2, gives the following q-values:

text/xml (1)

application/xml (1)

application/xhtml+xml (1)

image/png (1)

text/html (0.9)

text/plain (0.8)

everything else (0.5)

(section 14.1 of rfc 2616 defines the header)

This browser has clearly expressed a preference for text/plain over application/json in cases where the server can supply both.

This model won't break any code expecting to process application/json for two reasons. First, any such code should express that expectation in its Accept header. That is basic HTTP good behavior. Second, in the case where the code did not express any preference between available types (or when there is a tie) the server is permitted to choose. In this case, since it was mentioned first in content_typed_provided, webmachine would provide application/json.

Mathias Picker said...

Does webmachine somehow supports language negotiation? I only found media-type negotiation, but maybe (hopefully) I didn't look in the right place...

Justin Sheehy said...


Currently natural language is the one type of HTTP negotiation that is not supported by Webmachine, for historical reasons.

(we support encoding, charset, and media type negotiation by the means described in this post)

Natural language negotiation may be added in an upcoming version. If you have a pressing need for it, drop me an email message and we'll discuss.

Ben Godfrey said...

Ah excellent, I was about to try and work out how to do this when I found your post. Thanks!