Versioning REST Web Services

Managing changes to APIs is hard. That is no surprise to anyone who has ever maintained an API of any sort. Web services, being a special case of API, are susceptible to many of the difficulties around versioning as other types of APIs. For HTTP based REST style web services the combination of resources and content negotiation can be used to mitigate most of the issues surrounding API versioning.

Let’s assume you have a REST/HTTP web service that has some account resources. Say you can make a request like this:

===>
GET /accounts/3 HTTP/1.1
Accept: application/vnd.mycompany.myapp+xml
<===
HTTP/1.1 200 OK
Content-Type: application/vnd.mycompany.myapp+xml

<account>
  <name>Inigo Montoya</name>
</account>

First, you probably noticed that my example uses a vendor MIME media type to describe the representation. Using a more generic MIME media type like application/xml is much more common, at least in my experience. Using generic media types is perfectly legal but a bit silly. You are not really asking for any old XML document, but rather an XML document that has a quite specific schema. Aside from my idealistic rantings, using a specific media type has some strong practical benefits which are at the core of this post.

Backwards compatible changes

Often changes will need to be made to expose new behavior of the system that do not negatively impact correctly implemented clients. Say, for example, you want to start tracking email address for accounts. If the application/vnd.mycompany.myapp+xml format documentation is clear that elements that are not recognized should be ignored you can simply add a email element to the account representation.

<account>
  <name>Inigo Montoya</name>
  <email-address>mailto:prepare-to-die@youkilledmyfather.example</email-address>
</account>

Any client that was created before the addition of the email element will simply ignore it’s presence. Problem solved.

Incompatible changes

Unfortunately, not all changes can be implemented in a way that is backwards compatible. For example, a couple of months after adding email to accounts the sales team sign a deal for 1 bazillion dollars. But the new customer demands that each account be allowed to have more than one email address. After thinking for a while, you decide that the best way to expose that is by changing the account representation as follows.

<account>
  <name>Inigo Montoya</name>
  <email-addresses>
    <email-address priority='1'>mailto:prepare-to-die@youkilledmyfather.example</email-address>
    <email-address priority='2'>mailto:vengeance@youkilledmyfather.example</email-address>
  <email-address>
</account>

That, of course, will break any clients that are expecting the old format – so pretty much all of them. This is a place where we can bring content negotiation to bear. You can simply define a new media type – say application/vnd.mycompany.myapp-v2+xml – and associate new multi-email format with it. Clients can then request whichever format they want. Older clients don’t know the new media type so they get served the older single email format.

===>
GET /accounts/3 HTTP/1.1
Accept: application/vnd.mycompany.myapp+xml
<===
HTTP/1.1 200 OK
Content-Type: application/vnd.mycompany.myapp+xml

<account>
  <name>Inigo Montoya</name>
  <email-address>mailto:prepare-to-die@youkilledmyfather.example</email-address>
</account>

Newer clients do know the new media type so they can have access to the new functionality.

===>
GET /accounts/3 HTTP/1.1
Accept: application/vnd.mycompany.myapp-v2+xml
<===
HTTP/1.1 200 OK
Content-Type: application/vnd.mycompany.myapp-v2+xml

<account>
  <name>Inigo Montoya</name>
  <email-addresses>
    <email-address priority='1'>mailto:prepare-to-die@youkilledmyfather.example</email-address>
    <email-address priority='2'>mailto:vengeance@youkilledmyfather.example</email-address>
  <email-address>
</account>

Everyone gets what they need. Easy as pie.

Alternate approaches

The most commonly proposed approach for versioning REST/HTTP web service interfaces today seems to be to mutilate the URIs by inserting a version. For example,

http://foo.example/api/v1/accounts/3

I really hate this approach as it implies that an account in one version of the API is really a different resource than the account in a different version of the API.

It also forces clients into a nasty choice, either support multiple versions of the API simultaneously or break one of the core constrains of REST. For example, say a client exists for the v1 API that saves references (URIs that include the version indicator) to accounts. Some time later the client is updated to support the new version of the API. In this situation the The client can support both versions of the API simultaneously because all the previously stored URIs point at the old version of the API or it has to mung all the URIs it has stored to the point at the new API. Munging all the URIS breaks the HATEOAS constraint of REST and supporting multiple versions of the API is a maintenance nightmare.

Conclusion

Changes to REST/HTTP web service interfaces come it three basic flavors, changes to the properties associated with a type of resource, additions of new types of resources and deprecation of obsolete types of resources. If you are following the HATEOAS constraint of REST the approach described here can be used to safely handle all three scenarios.

This approach does lead to media types being created, but media types are cheap so we can – and should – have as many as we need. Used properly, content negotiation can be used to solve the problems related to versioning a REST/HTTP web service interface.


If you’re interested in REST/HTTP service versioning be sure not to miss the rest of the series.

69 comments on “Versioning REST Web Services

  1. -

    Avoid Versioning – Please…

    Peter Williams made some good posts suggesting content-negotiation as a means to advertise and ask for version support. This is in contrast to using version identifiers in URIs as is commonly done. This is a neat idea. Peter’s post prompted some debat…

  2. -

    You hit the nail on the head, Peter. This is the right way to do it(tm), and the right way to use content negotiation. An incompatible (must-understand) change produces a new document type. Old and new clients are supported simultaneously based on their different Accept headers.

    Just make sure that every client always supplies an accept header from the very beginning of your project, otherwise you are stuck when you do need to make that incompatible change.

    In the long term you should still be aiming to minimise backwards-incompatible document-type change, and build widely-understood document types. Whatever problem domain you are in, you are sure to go through more than a couple of legacy content types before you settle on something that will stand the test of time.

  3. -

    Hi Peter,

    I really like this your approach.
    I used another approach to the same problem in my current application, by adding an “X-Restful-Interface-Version” header to the HTTP headers.
    But your solution makes much more sense.

    Regards, Bas

  4. -

    I created REST web service.I retrive the xml data using get method by servlet.how to send the data using post method by servlet.

  5. -

    Fundamental to the solution is the version support negotiation using the http:accept header. This will work fine for the GET operation.
    But how does this address the ‘create’ scenario? A newer clients may want to create a new resource, but happends to interact with an server only supporting older resource representations.
    Since the client is the resource provider in this case, the negotiation is not a part of the request…

  6. - Post author

    Well, if the client is following the HATEOAS principle it will never attempt to create a resource that is not supported by the server. If you are using the links generated by the server to determine next steps, you can never end up making requests against unsupported resources.

    Formats should not be a huge problem either. Media type includes not only a specific format but a particular set of semantics around that format. Those semantics includes the format of requests from the client. For example, part of the HTML semantics are that forms are, by default, submitted as application/x-www-form-urlencoded, therefore a (non-read-only) client should not request the text/html media type unless it is prepared to make POST request using that format.

    A custom media type should have similar semantics regarding the formats of POST and PUT requests. If some incompatible change is needed in the request body formats then a new media type would be required to support such a change.

  7. -

    […] one of our RESTful web services to support versioning of the resources. I came across a post, Versioning REST Web Services by Peter Williams that describes a way to use Accept headers and content negotiation as a way of […]

  8. -

    Straight caching through all layers (cdn for ex.) does not work in your approach. The url solution is cachable.

    Cheers

  9. -

    I like the approach, but wouldn’t it be cleaner to add a ‘version’ (or something similar) parameter to the Content-Type, so you don’t end up with an effectively unique Content-Type for each new version? ‘Accept: application/vnd.mycompany.app+xml; version=1.0’ would be just as expressive, but cleaner.

  10. -

    Sounds reasonable, but you should probably include a “Vary” header in the response to indicate to caches that you’re switching on the “Accept” request header.

  11. -

    “Managing changes to APIs is hard. That is no surprise to anyone who has ever maintained an API of any sort.”

    …and yet there are surprisingly few people discussing these issues. Thanks for doing so.

  12. -

    I’m pretty new to REST, but I thought that one URI always had to return the same data? So if I specify I accept any content-type (not sure if that’s possible – but if not, substitute ‘any’ by ‘all’), which version should be returned? The first version ever?

    And even if you do that, it still means that with content negotiation you get different representations for the same URI – I was under the impression one of the principal goals behind REST was to have only one representation for one URI.

    Comment #6 does have a point about caching, which is one of the reasons I thought one URI == one representation.

    (Please note I’m not criticizing you post – it’s extremely interesting (and very well-written) and seems like a great solution to the versioning problem, but I’m confused as to whether it fits the REST goals – please enlighten me.

    Also, am I correct to understand from comment #5 that specifying a mimetype (like you would with a POST) is also possible with a PUT?

  13. -

    […] Versioning REST Web Services Managing changes to APIs is hard. That is no surprise to anyone who has ever maintained an API of any sort. Web services, being a special case of API, are susceptible to many of the difficulties around versioning as other types of APIs. For HTTP based REST style web services the combination of resources and content negotiation can be used to mitigate most of the issues surrounding API versioning. (tags: rubyonrails restful versioning) […]

  14. - Post author

    LudoA, A URI names a single resource which may have many representations. Say you had chart of last years earning by month. This chart might have a URI http://foo.example/charts/42. Serving the PNG, JPG and Flash versions of that chart from the same URL fits perfectly in to the REST model. Each of those formats represents the same thing, the earning chart. The various formats are just different representations of the same information. To support this multiple representation approach we have content negotiation. It allows the requesting agent to tell the server in what format it would prefer the information. And it may specify more than one, each with it’s own precedence.

    Specifying Accept as */* is vile kludge if requesting agent is going to actually interpret the representations it is getting. But to answer your question, if the requesting agent claims it can accept any content type the server should serve which ever representation it prefers. That might the most comprehensive version, or the one that is fastest to generate, or whatever. The requesting agent has said “I don’t care what sort of representation you give me”, so the server is free to do whatever it wants.

    Comment #6 is either wrong or CDNs are a lot dumber than I would have expected. Martin Atkins (comment #8) is right, in real life the responses should set the Vary header to include Accept so that caches can do the right thing — I left it out to reduce the complexity of the examples. If that header is set correctly it is fairly straight forward to implement caching that works correctly with content negotiation, whether it is used for versioning or for some other purpose. The HTTP spec describes the caching algorithm for handling HTTP traffic in some detail.

    And yes, you can specify the content type of the body of a HTTP PUT.

  15. -

    Thanks a lot for the very clear(!) explanations – it’s often hard to explain a very simple thing in simple terms, but you succeeded completely :-)

    It makes some sense that a URI refers to resource and not to a representation — as I originally thought it did –, given the ‘R’ in URI ;-)

    I wasn’t aware of the Vary header – good to know though. The people who designed HTTP really thought this through, apparently.

  16. -

    Hi,

    I like the approach you describe here, but the majority of changes I’ve seen to web services have been of a non-versionable variety: often they are made necessary by sub-surface application changes that make the old forms impossible to serve. In your example, perhaps the application no longer wishes to expose e-mail addresses at all, or the service provider decides to start supporting a new authentication scheme, both for security reasons? In neither of these cases would it be advisable to offer anyone the old representation.

    I definitely share your distaste for sticking version numbers in the URL. The first place I saw this antipattern was in the Del.icio.us API back in 2004. They’re still hosting it from the old domain, still promising to “deprecate early versions aggressively”, and still at version 1, where they will stay for as long as the service remains online.

    We had this debate when we were designing Digg’s API, and decided early that version numbers made no sense because they were only defensible for the most inconsequential of changes. The old Del.icio.us is no longer available, you can’t rev back to last season’s TV schedule, stores don’t honor outdated coupons, and web services start and stay at version “now”.

  17. -

    I like the idea, seems fundamentally cleaner than putting version info in the request URL, for the reasons stated above. Some questions/issues:

    1) Seems the service is now locked into supporting multiple representations of the same object. Doesn’t this get difficult to manage over time?

    2) This quote seems overly simplified: “Well, if the client is following the HATEOAS principle it will never attempt to create a resource that is not supported by the server.” I think the question was valid (post #4 above), though I don’t have a very good answer, either, other than to just say always keep the server newer than the client…

    Thoughts?

  18. - Post author

    Michael Case,

    This approach assumes that the application wants to support multiple versions of it’s API simultaneously. As Michal Migurski pointed out you might decide that is not acceptable. In my experience truly non-versionable API changes are actually fairly rare, but deciding that you will not invest the effort to support multiple versions is quite reasonable. Assuming, of course you are willing to accept that some changes you might want to make to your API will break many, or all, of the clients.

    As for point 2 I don’t think that is oversimplified. If the client is only using URL it has extracted from the server responses and it is following the documented semantics of the server responses, it should not be a problem. For example, if the semantics of the API say you can update an item by PUTing a modified version of the representation to the URI found at ./link[rel='edit']/@href of the original representation, the only way that can go wrong is because of a bug.

  19. -

    Very well said.

    The only thing I’d add is that new versions should be rare; the real value of REST (and the rest, ahem, of the Internet architecture) comes from stable, well-understood formats, not localised ones that change every week.

  20. -

    I can’t disagree more with your assertion that creating mime types is a cheap process.

    On the contrary, great care should be given to deciding on a format you’ll publish as a mime type.

    In your example, having defined an xmll language to support open-ended extensibility on all nodes would’ve nicely solved the problem of versioning your schema.

    Processing a media type doesn’t imply that such media type will be accepted whenever it respects the spec. Validation can and should happen, and the server responsible for the resource is quite in its right to refuse a representation that doesn’t comply to some rules that have not been respected (but are not in the specification of your mime type).

    Typical example is an html document containing a form. When posting, any value will be acceptable as per app/urlform. It is however perfectly valid to reject such request when username is empty.

    Designing for serendipity is exactly why you want to design very few media types and very rarely, so that business processes and requirements do not impact the media types but impact the processing of representations.

    As you correctly point out, whenever such a validation error occurs, the server should be indicating with the returned represetnation what the error is and how to correct it.

    This is the reason why xhtml, xslt and others are keeping the same mime type over versions. Complexity of handling changes in the shape of a document format should be at the document processing level, not at the protocol level. By using conneg and new media types, you effectively shift the complexity of format shape without solving the problem of modifiable behavior, and you increase the cost of maintaining the versions of your media types.

  21. - Post author

    Sebastien Lambla,

    Highly extensible formats do not solve API versioning problems. We might all switch to N3 tomorrow but that does not mean that our systems would be able to communicate effectively. Extensible formats are one thing — and a good thing, too — but conneg is just as much about application semantics as syntax.

    I don’t think that having “few media types” constitutes designing for serendipity. We could just all use ‘application/octet-stream’ but that is hardly very useful for getting anything interesting actually done. You could go too far, for sure, but I think that we have thus far erred on the side of too few media types.

    Agreeing on set of application semantics in combination with a syntax is hard work. Any time you achieve that it is worthy of a MIME media type of it’s very own.

  22. -

    Hello Peter,

    This is an interesting idea. I mostly like the concept, except that I don’t find it to be at all discoverable. Is there an HTTP request I can make that will tell me what content-types a resource supports? I looked up HEAD and it doesn’t seem to do that.
    Your recommendation also conflicts with this http://www.w3.org/2001/tag/doc/alternatives-discovery.html, which seems as well reasoned out as your argument, but with improved discoverability.

    Mark

  23. -

    […] Peter Williams – Versioning REST Web Services (tags: rest versioning api) This was written by andy. Posted on Thursday, July 9, 2009, at 1:36 am. Filed under Uncategorized. Bookmark the permalink. Follow comments here with the RSS feed. Post a comment or leave a trackback. […]

  24. -

    I’m interested in how this works in practice, in particular any breaking change to any contract related to “application/vnd.mycompany.myapp” is going to cause us to release a new version and I’m wondering whether this results in a lot of new versions?

  25. - Post author

    Colin Jack, It works very well for facilitating incomparable changes to an API. However, each such incompatible change adds long term maintenance burdens to the server implementation. Before making any incompatible change to an API the benefits of the change should be weighed against the cost of maintaining that change.

    The approach described here results in a very low burden on the client side combined less of a burden on the service side than most versioning approaches, but it is not free. It is usually better for everyone to avoid introducing incompatible changes whenever possible.

  26. -

    Hi Peter,
    Nice Article. i am developing a REST API for my company and coming up with a version management solution is high on the agenda (with few more items)..
    for the moment I use content negotiation (Accept headers) to defiine the media type the client requets. for e.g. we support the below
    1. text/xml
    2. application/json
    3. application/atom+xml
    4. application/rss+xml
    also we envisage that the consumer of these services can be any device (just not a browser).. can be a Google MAP, rss or atom reader or even a hand held devise. so in addition to using headers we used . extensions also. e.g. …faults.xml, faults.rss likewise.

    So if we use a vendor MILE media type, we would have to extend the above to some thing like below:
    1. text/v2-xml
    2. application/v2-json
    3. application/atom-v2+xml
    4. application/rss-v2+xml
    ?
    Also I have seen that Google Data API uses two mechanism
    1) version header
    2) a query param based method (nor a /v2 type that would break a resource hierarchy ) – i.e. xyz.com/faults/423434?v=2

    in order to support most of the devises (that can process headers or simple devises) what is your advise pls?

  27. - Post author

    Sajindra,

    Certainly either approach is workable but they both have downsides.

    Using a custom version header is effectively not much different from using a custom MIME type. The server will do content negotiation on a non-standard header, which is perfectly legal. The downsides are your application framework probably does not support this out of the box and it is going to be opaque to all the intermediates. However if the server includes the version header in the Varies header everything should work fine.

    Adding the version identifier in the query string is exactly the same as inserting it somewhere in the path part of the URI. The primary downside of this approach is the pain it causes which upgrading existing clients. You cannot simply update the client to support the next version of the format and assume that all stored URLs are usable. Rather the client will have to support the previous format for all stored URLs and the new format for all new URLs. If the client does not store URLs this issue is, obviously, moot. But in API development you are often not in control of, or even privy to, clients and their development.

  28. -

    […] Peter Williams – Versioning REST Web Services barelyenough.org/blog/2008/05/versioning-rest-web-services – view page – cached Managing changes to APIs is hard. That is no surprise to anyone who has ever maintained an API of any sort. Web services, being a special case of API, are susceptible to many of the difficulties around versioning as other types of APIs. For HTTP based REST style web services the combination of resources and content negotiation can be used to mitigate most of the issues surrounding API versioning. — From the page […]

  29. -

    […] a message with no out-of-band-knowledge. Extending this model further, there are attempts to attach version identifiers and schema references to media types. The end goal of these attempts is to let arbitrary clients […]

  30. -

    What about Asbjørn’s suggestion of separating version from the mime-type?

    Accept: application/vnd.mycompany.app+xml; version=1.0

  31. -

    The ONE issue I see with this is the JScript library/language construct called ‘JSONRequest’. Since it has no ability to modify headers (except gzip/identity), it cannot ever use headers to indicate microformat, do authorization, or anything else. I suppose is possible to put all that in the URL.

    The best model of API security that I’ve seen is Amazon’s. If all of it’s security model could be put in the URL, I suppose then ‘JSONRequest’, with the microformat in the URL also, then would be able to do custom microformats across scenarios then.

    I’d like to see a flow chart of handling versioning versus format with all the possibliities: headers, versioning, format, URL based values.

  32. -

    Nice Post. It would be great if all the changes could be applied in series that is backwards compatible. However changes on a server require more maintenance. I think more people should put more thought to this as well. Thanks again.

  33. -

    Peter, what about using the “level” extension in the Accept/Content-Type instead of including it in the mime-type name?

    So instead of:

    Accept: application/vnd.mycompany.myapp-v2+xml

    You use:

    Accept: application/vnd.mycompany.myapp+xml; level=2

    Documentation on the “level” extension is a bit hard to find, but I did find references to it in RFC 2616 for the Accept header, and the Apache docs and it seems to be specified for exactly this purpose.

  34. - Post author

    Dan, i talked a little about that here and here.

    Over all l think the level parameter is a great way to describe backward compatible versions, but i think it is lacking for describing incompatible changes.

  35. -

    I have weighed in on this subject with an opinion of my own, but my response is much longer than what is warranted for this comment section. I have written up my opinion and published it on my blog at http://www.notmessenger.com/rest/3-tenets-for-implementing-a-rest-api/ This is not an attempt to get traffic to my blog or a spam message, but merely an attempt to add my opinion to the conversation. Thank you for time.

  36. -

    Should a RESTful web service API be versioned?…

    No, I do not believe it is necessary to “version” a REST api. It is certainly possible to build RESTful apis and manage breaking changes without ever including version numbers in the URI.    By far the most important thing to ensure is that clients a…

  37. -

    […] impact of this versioning of the data. There are some discussions about this by Peter Williams, Versioning Rest Web Services, and Ganesh Prasad asks, Does Rest Need […]

Comments are closed.