Managing changes to APIs is hard. That is no surprise to anyone who has ever maintained an API of any sort. Web services, being a special case of API, are susceptible to many of the difficulties around versioning as other types of APIs. For HTTP based REST style web services the combination of resources and content negotiation can be used to mitigate most of the issues surrounding API versioning.
Let’s assume you have a REST/HTTP web service that has some account resources. Say you can make a request like this:
===>
GET /accounts/3 HTTP/1.1
Accept: application/vnd.mycompany.myapp+xml
<===
HTTP/1.1 200 OK
Content-Type: application/vnd.mycompany.myapp+xml
<account>
<name>Inigo Montoya</name>
</account>First, you probably noticed that my example uses a vendor MIME media type to describe the representation. Using a more generic MIME media type like application/xml is much more common, at least in my experience. Using generic media types is perfectly legal but a bit silly. You are not really asking for any old XML document, but rather an XML document that has a quite specific schema. Aside from my idealistic rantings, using a specific media type has some strong practical benefits which are at the core of this post.
Backwards compatible changes
Often changes will need to be made to expose new behavior of the system that do not negatively impact correctly implemented clients. Say, for example, you want to start tracking email address for accounts. If the application/vnd.mycompany.myapp+xml format documentation is clear that elements that are not recognized should be ignored you can simply add a email element to the account representation.
<account>
<name>Inigo Montoya</name>
<email-address>mailto:prepare-to-die@youkilledmyfather.example</email-address>
</account>Any client that was created before the addition of the email element will simply ignore it’s presence. Problem solved.
Incompatible changes
Unfortunately, not all changes can be implemented in a way that is backwards compatible. For example, a couple of months after adding email to accounts the sales team sign a deal for 1 bazillion dollars. But the new customer demands that each account be allowed to have more than one email address. After thinking for a while, you decide that the best way to expose that is by changing the account representation as follows.
<account>
<name>Inigo Montoya</name>
<email-addresses>
<email-address priority='1'>mailto:prepare-to-die@youkilledmyfather.example</email-address>
<email-address priority='2'>mailto:vengeance@youkilledmyfather.example</email-address>
<email-address>
</account>That, of course, will break any clients that are expecting the old format – so pretty much all of them. This is a place where we can bring content negotiation to bare. You can simply define a new media type – say application/vnd.mycompany.myapp-v2+xml – and associate new multi-email format with it. Clients can then request whichever format they want. Older clients don’t know the new media type so they get served the older single email format.
===>
GET /accounts/3 HTTP/1.1
Accept: application/vnd.mycompany.myapp+xml
<===
HTTP/1.1 200 OK
Content-Type: application/vnd.mycompany.myapp+xml
<account>
<name>Inigo Montoya</name>
<email-address>mailto:prepare-to-die@youkilledmyfather.example</email-address>
</account>Newer clients do know the new media type so they can have access to the new functionality.
===>
GET /accounts/3 HTTP/1.1
Accept: application/vnd.mycompany.myapp-v2+xml
<===
HTTP/1.1 200 OK
Content-Type: application/vnd.mycompany.myapp-v2+xml
<account>
<name>Inigo Montoya</name>
<email-addresses>
<email-address priority='1'>mailto:prepare-to-die@youkilledmyfather.example</email-address>
<email-address priority='2'>mailto:vengeance@youkilledmyfather.example</email-address>
<email-address>
</account>Everyone gets what they need. Easy as pie.
Alternate approaches
The most commonly proposed approach for versioning REST/HTTP web service interfaces today seems to be to mutilate the URIs by inserting a version. For example,
http://foo.example/api/v1/accounts/3I really hate this approach as it implies that an account in one version of the API is really a different resource than the account in a different version of the API.
It also forces clients into a nasty choice, either support multiple versions of the API simultaneously or break one of the core constrains of REST. For example, say a client exists for the v1 API that saves references (URIs that include the version indicator) to accounts. Some time later the client is updated to support the new version of the API. In this situation the The client can support both versions of the API simultaneously because all the previously stored URIs point at the old version of the API or it has to mung all the URIs it has stored to the point at the new API. Munging all the URIS breaks the HATEOAS constraint of REST and supporting multiple versions of the API is a maintenance nightmare.
Conclusion
Changes to REST/HTTP web service interfaces come it three basic flavors, changes to the properties associated with a type of resource, additions of new types of resources and deprecation of obsolete types of resources. If you are following the HATEOAS constraint of REST the approach described here can be used to safely handle all three scenarios.
This approach does lead to media types being created, but media types are cheap so we can – and should – have as many as we need. Used properly, content negotiation can be used to solve the problems related to versioning a REST/HTTP web service interface.
Related Posts
If you’re interested in REST/HTTP service versioning be sure not to miss the rest of the series.
Comments 33
You hit the nail on the head, Peter. This is the right way to do it(tm), and the right way to use content negotiation. An incompatible (must-understand) change produces a new document type. Old and new clients are supported simultaneously based on their different Accept headers.
Just make sure that every client always supplies an accept header from the very beginning of your project, otherwise you are stuck when you do need to make that incompatible change.
In the long term you should still be aiming to minimise backwards-incompatible document-type change, and build widely-understood document types. Whatever problem domain you are in, you are sure to go through more than a couple of legacy content types before you settle on something that will stand the test of time.
Posted 23 May 2008 at 6:38 am ¶Hi Peter,
I really like this your approach.
I used another approach to the same problem in my current application, by adding an “X-Restful-Interface-Version” header to the HTTP headers.
But your solution makes much more sense.
Regards, Bas
Posted 17 Jun 2008 at 8:24 am ¶I created REST web service.I retrive the xml data using get method by servlet.how to send the data using post method by servlet.
Posted 18 Aug 2008 at 10:07 pm ¶Fundamental to the solution is the version support negotiation using the http:accept header. This will work fine for the GET operation.
Posted 16 Sep 2008 at 3:38 pm ¶But how does this address the ‘create’ scenario? A newer clients may want to create a new resource, but happends to interact with an server only supporting older resource representations.
Since the client is the resource provider in this case, the negotiation is not a part of the request…
Well, if the client is following the HATEOAS principle it will never attempt to create a resource that is not supported by the server. If you are using the links generated by the server to determine next steps, you can never end up making requests against unsupported resources.
Formats should not be a huge problem either. Media type includes not only a specific format but a particular set of semantics around that format. Those semantics includes the format of requests from the client. For example, part of the HTML semantics are that forms are, by default, submitted as
application/x-www-form-urlencoded, therefore a (non-read-only) client should not request thetext/htmlmedia type unless it is prepared to make POST request using that format.A custom media type should have similar semantics regarding the formats of POST and PUT requests. If some incompatible change is needed in the request body formats then a new media type would be required to support such a change.
Posted 17 Sep 2008 at 9:46 am ¶Straight caching through all layers (cdn for ex.) does not work in your approach. The url solution is cachable.
Cheers
Posted 13 Oct 2008 at 5:38 am ¶I like the approach, but wouldn’t it be cleaner to add a ‘version’ (or something similar) parameter to the Content-Type, so you don’t end up with an effectively unique Content-Type for each new version? ‘Accept: application/vnd.mycompany.app+xml; version=1.0′ would be just as expressive, but cleaner.
Posted 13 Oct 2008 at 7:51 am ¶Sounds reasonable, but you should probably include a “Vary” header in the response to indicate to caches that you’re switching on the “Accept” request header.
Posted 13 Oct 2008 at 9:31 am ¶“Managing changes to APIs is hard. That is no surprise to anyone who has ever maintained an API of any sort.”
…and yet there are surprisingly few people discussing these issues. Thanks for doing so.
Posted 13 Oct 2008 at 10:21 am ¶I’m pretty new to REST, but I thought that one URI always had to return the same data? So if I specify I accept any content-type (not sure if that’s possible – but if not, substitute ‘any’ by ‘all’), which version should be returned? The first version ever?
And even if you do that, it still means that with content negotiation you get different representations for the same URI – I was under the impression one of the principal goals behind REST was to have only one representation for one URI.
Comment #6 does have a point about caching, which is one of the reasons I thought one URI == one representation.
(Please note I’m not criticizing you post – it’s extremely interesting (and very well-written) and seems like a great solution to the versioning problem, but I’m confused as to whether it fits the REST goals – please enlighten me.
Also, am I correct to understand from comment #5 that specifying a mimetype (like you would with a POST) is also possible with a PUT?
Posted 14 Oct 2008 at 3:39 am ¶LudoA, A URI names a single resource which may have many representations. Say you had chart of last years earning by month. This chart might have a URI
http://foo.example/charts/42. Serving the PNG, JPG and Flash versions of that chart from the same URL fits perfectly in to the REST model. Each of those formats represents the same thing, the earning chart. The various formats are just different representations of the same information. To support this multiple representation approach we have content negotiation. It allows the requesting agent to tell the server in what format it would prefer the information. And it may specify more than one, each with it’s own precedence.Specifying Accept as
*/*is vile kludge if requesting agent is going to actually interpret the representations it is getting. But to answer your question, if the requesting agent claims it can accept any content type the server should serve which ever representation it prefers. That might the most comprehensive version, or the one that is fastest to generate, or whatever. The requesting agent has said “I don’t care what sort of representation you give me”, so the server is free to do whatever it wants.Comment #6 is either wrong or CDNs are a lot dumber than I would have expected. Martin Atkins (comment #8) is right, in real life the responses should set the Vary header to include Accept so that caches can do the right thing — I left it out to reduce the complexity of the examples. If that header is set correctly it is fairly straight forward to implement caching that works correctly with content negotiation, whether it is used for versioning or for some other purpose. The HTTP spec describes the caching algorithm for handling HTTP traffic in some detail.
And yes, you can specify the content type of the body of a HTTP PUT.
Posted 14 Oct 2008 at 9:28 am ¶Thanks a lot for the very clear(!) explanations – it’s often hard to explain a very simple thing in simple terms, but you succeeded completely :-)
It makes some sense that a URI refers to resource and not to a representation — as I originally thought it did –, given the ‘R’ in URI ;-)
I wasn’t aware of the Vary header – good to know though. The people who designed HTTP really thought this through, apparently.
Posted 15 Oct 2008 at 2:13 am ¶Hi,
I like the approach you describe here, but the majority of changes I’ve seen to web services have been of a non-versionable variety: often they are made necessary by sub-surface application changes that make the old forms impossible to serve. In your example, perhaps the application no longer wishes to expose e-mail addresses at all, or the service provider decides to start supporting a new authentication scheme, both for security reasons? In neither of these cases would it be advisable to offer anyone the old representation.
I definitely share your distaste for sticking version numbers in the URL. The first place I saw this antipattern was in the Del.icio.us API back in 2004. They’re still hosting it from the old domain, still promising to “deprecate early versions aggressively”, and still at version 1, where they will stay for as long as the service remains online.
We had this debate when we were designing Digg’s API, and decided early that version numbers made no sense because they were only defensible for the most inconsequential of changes. The old Del.icio.us is no longer available, you can’t rev back to last season’s TV schedule, stores don’t honor outdated coupons, and web services start and stay at version “now”.
Posted 15 Oct 2008 at 10:47 pm ¶I like the idea, seems fundamentally cleaner than putting version info in the request URL, for the reasons stated above. Some questions/issues:
1) Seems the service is now locked into supporting multiple representations of the same object. Doesn’t this get difficult to manage over time?
2) This quote seems overly simplified: “Well, if the client is following the HATEOAS principle it will never attempt to create a resource that is not supported by the server.” I think the question was valid (post #4 above), though I don’t have a very good answer, either, other than to just say always keep the server newer than the client…
Thoughts?
Posted 16 Oct 2008 at 4:28 pm ¶Michael Case,
This approach assumes that the application wants to support multiple versions of it’s API simultaneously. As Michal Migurski pointed out you might decide that is not acceptable. In my experience truly non-versionable API changes are actually fairly rare, but deciding that you will not invest the effort to support multiple versions is quite reasonable. Assuming, of course you are willing to accept that some changes you might want to make to your API will break many, or all, of the clients.
As for point 2 I don’t think that is oversimplified. If the client is only using URL it has extracted from the server responses and it is following the documented semantics of the server responses, it should not be a problem. For example, if the semantics of the API say you can update an item by PUTing a modified version of the representation to the URI found at
Posted 16 Oct 2008 at 5:41 pm ¶./link[rel='edit']/@hrefof the original representation, the only way that can go wrong is because of a bug.Very well said.
The only thing I’d add is that new versions should be rare; the real value of REST (and the rest, ahem, of the Internet architecture) comes from stable, well-understood formats, not localised ones that change every week.
Posted 03 Dec 2008 at 12:04 am ¶I can’t disagree more with your assertion that creating mime types is a cheap process.
On the contrary, great care should be given to deciding on a format you’ll publish as a mime type.
In your example, having defined an xmll language to support open-ended extensibility on all nodes would’ve nicely solved the problem of versioning your schema.
Processing a media type doesn’t imply that such media type will be accepted whenever it respects the spec. Validation can and should happen, and the server responsible for the resource is quite in its right to refuse a representation that doesn’t comply to some rules that have not been respected (but are not in the specification of your mime type).
Typical example is an html document containing a form. When posting, any value will be acceptable as per app/urlform. It is however perfectly valid to reject such request when username is empty.
Designing for serendipity is exactly why you want to design very few media types and very rarely, so that business processes and requirements do not impact the media types but impact the processing of representations.
As you correctly point out, whenever such a validation error occurs, the server should be indicating with the returned represetnation what the error is and how to correct it.
This is the reason why xhtml, xslt and others are keeping the same mime type over versions. Complexity of handling changes in the shape of a document format should be at the document processing level, not at the protocol level. By using conneg and new media types, you effectively shift the complexity of format shape without solving the problem of modifiable behavior, and you increase the cost of maintaining the versions of your media types.
Posted 04 Dec 2008 at 11:23 am ¶Sebastien Lambla,
Highly extensible formats do not solve API versioning problems. We might all switch to N3 tomorrow but that does not mean that our systems would be able to communicate effectively. Extensible formats are one thing — and a good thing, too — but conneg is just as much about application semantics as syntax.
I don’t think that having “few media types” constitutes designing for serendipity. We could just all use ‘application/octet-stream’ but that is hardly very useful for getting anything interesting actually done. You could go too far, for sure, but I think that we have thus far erred on the side of too few media types.
Agreeing on set of application semantics in combination with a syntax is hard work. Any time you achieve that it is worthy of a MIME media type of it’s very own.
Posted 09 Dec 2008 at 11:42 pm ¶Hello Peter,
This is an interesting idea. I mostly like the concept, except that I don’t find it to be at all discoverable. Is there an HTTP request I can make that will tell me what content-types a resource supports? I looked up HEAD and it doesn’t seem to do that.
Your recommendation also conflicts with this http://www.w3.org/2001/tag/doc/alternatives-discovery.html, which seems as well reasoned out as your argument, but with improved discoverability.
Mark
Posted 04 May 2009 at 7:27 pm ¶I’m interested in how this works in practice, in particular any breaking change to any contract related to “application/vnd.mycompany.myapp” is going to cause us to release a new version and I’m wondering whether this results in a lot of new versions?
Posted 20 Jul 2009 at 4:11 am ¶Colin Jack, It works very well for facilitating incomparable changes to an API. However, each such incompatible change adds long term maintenance burdens to the server implementation. Before making any incompatible change to an API the benefits of the change should be weighed against the cost of maintaining that change.
The approach described here results in a very low burden on the client side combined less of a burden on the service side than most versioning approaches, but it is not free. It is usually better for everyone to avoid introducing incompatible changes whenever possible.
Posted 20 Jul 2009 at 12:09 pm ¶Hi Peter,
Nice Article. i am developing a REST API for my company and coming up with a version management solution is high on the agenda (with few more items)..
for the moment I use content negotiation (Accept headers) to defiine the media type the client requets. for e.g. we support the below
1. text/xml
2. application/json
3. application/atom+xml
4. application/rss+xml
also we envisage that the consumer of these services can be any device (just not a browser).. can be a Google MAP, rss or atom reader or even a hand held devise. so in addition to using headers we used . extensions also. e.g. …faults.xml, faults.rss likewise.
So if we use a vendor MILE media type, we would have to extend the above to some thing like below:
1. text/v2-xml
2. application/v2-json
3. application/atom-v2+xml
4. application/rss-v2+xml
?
Also I have seen that Google Data API uses two mechanism
1) version header
2) a query param based method (nor a /v2 type that would break a resource hierarchy ) – i.e. xyz.com/faults/423434?v=2
in order to support most of the devises (that can process headers or simple devises) what is your advise pls?
Posted 27 Aug 2009 at 5:04 am ¶Sajindra,
Certainly either approach is workable but they both have downsides.
Using a custom version header is effectively not much different from using a custom MIME type. The server will do content negotiation on a non-standard header, which is perfectly legal. The downsides are your application framework probably does not support this out of the box and it is going to be opaque to all the intermediates. However if the server includes the version header in the Varies header everything should work fine.
Adding the version identifier in the query string is exactly the same as inserting it somewhere in the path part of the URI. The primary downside of this approach is the pain it causes which upgrading existing clients. You cannot simply update the client to support the next version of the format and assume that all stored URLs are usable. Rather the client will have to support the previous format for all stored URLs and the new format for all new URLs. If the client does not store URLs this issue is, obviously, moot. But in API development you are often not in control of, or even privy to, clients and their development.
Posted 27 Aug 2009 at 12:19 pm ¶What about Asbjørn’s suggestion of separating version from the mime-type?
Accept: application/vnd.mycompany.app+xml; version=1.0
Posted 01 Oct 2010 at 11:46 am ¶The ONE issue I see with this is the JScript library/language construct called ‘JSONRequest’. Since it has no ability to modify headers (except gzip/identity), it cannot ever use headers to indicate microformat, do authorization, or anything else. I suppose is possible to put all that in the URL.
The best model of API security that I’ve seen is Amazon’s. If all of it’s security model could be put in the URL, I suppose then ‘JSONRequest’, with the microformat in the URL also, then would be able to do custom microformats across scenarios then.
I’d like to see a flow chart of handling versioning versus format with all the possibliities: headers, versioning, format, URL based values.
Posted 04 Jan 2011 at 8:18 pm ¶Nice Post. It would be great if all the changes could be applied in series that is backwards compatible. However changes on a server require more maintenance. I think more people should put more thought to this as well. Thanks again.
Posted 06 Jan 2011 at 6:46 am ¶Peter, what about using the “level” extension in the Accept/Content-Type instead of including it in the mime-type name?
So instead of:
Accept: application/vnd.mycompany.myapp-v2+xml
You use:
Accept: application/vnd.mycompany.myapp+xml; level=2
Documentation on the “level” extension is a bit hard to find, but I did find references to it in RFC 2616 for the Accept header, and the Apache docs and it seems to be specified for exactly this purpose.
Posted 20 Jan 2011 at 10:20 am ¶Dan, i talked a little about that here and here.
Over all l think the level parameter is a great way to describe backward compatible versions, but i think it is lacking for describing incompatible changes.
Posted 28 Jan 2011 at 10:37 am ¶I have weighed in on this subject with an opinion of my own, but my response is much longer than what is warranted for this comment section. I have written up my opinion and published it on my blog at http://www.notmessenger.com/rest/3-tenets-for-implementing-a-rest-api/ This is not an attempt to get traffic to my blog or a spam message, but merely an attempt to add my opinion to the conversation. Thank you for time.
Posted 16 Mar 2011 at 8:51 pm ¶How does this work with read/write APIs? If I add an attribute as a non-breaking change and so don’t change the media type, old clients will continue to PUT and POST data missing the new attribute. I can fix this on the server by setting the attribute to a default value, or leaving the current value of the attribute unchanged, but does it cause problems with caches, which may replace their cached version with the partial update? Isn’t this effectively the same problem as doing partial updates with PUT?
Posted 08 Dec 2011 at 7:34 am ¶The important characteristic of PUTs is that they must be idempotent, not that they do not allow partial updates. It is very common for the body of PUT request to not include every property of the resource. For example, last updated timestamp fields.
I this case the put would be idempotent. The older client neither knows nor cares about this new property and so it does not really care how it’s values are handled. Either of the ways you mentioned to handle the missing property value would be ok.
Caches will take care of themselves. Caches don’t use the body of the PUT. They merely invalidate the previously cached representations for that resource. That means the next GET request for that resource will go to the server and it’s response will be cached. Assuming the new property is always returned the caches will keep up to date.
Posted 08 Dec 2011 at 8:36 am ¶You talk about using the ACCEPT header – does the same apply to the Content-type of a POST/PUT? Tha API consists of both input AND output.
Posted 22 Dec 2011 at 7:08 pm ¶Michael,
Posted 27 Dec 2011 at 9:46 am ¶It absolutely applies to the content type of POST and PUT requests. Just as the accept header field of GET requests allows the server to give the client the particular representation it requires, the content-type header field of POST and PUT requests gives the server the information it needs to correctly interpret the data set by the client.
Trackbacks & Pingbacks 22
[...] my previous post on this subject I described an approach to versioning the API of a REST/HTTP web service. This approach has [...]
[...] Peter Williams – Versioning REST Web Services (tags: restful versioning webservices architecture) [...]
[...] RESTful Web Services 18 05 2008 Peter Lacey posts here and here on how to version RESTful Web Services using custom MIME media types and I find this very [...]
[...] Dubray takes issue with my approach of using content negotiation to manage service versioning in HTTP. I actually hesitate to respond to Mr. Dubray because the overall tone of his piece is [...]
[...] poring over versioning issues on RESTful web services i stumbled upon Peter Williams post proposing the use of vendor MIME media types to get the versioning under control. I really like [...]
Avoid Versioning – Please…
Peter Williams made some good posts suggesting content-negotiation as a means to advertise and ask for version support. This is in contrast to using version identifiers in URIs as is commonly done. This is a neat idea. Peter’s post prompted some debat…
[...] one of our RESTful web services to support versioning of the resources. I came across a post, Versioning REST Web Services by Peter Williams that describes a way to use Accept headers and content negotiation as a way of [...]
[...] I’m planning on a bigger post about exactly what our JSON document means, and our mime-types, and everything. For now, a good explaination of the reasoning behind our mime-types can be found over on Peter’s blog. [...]
[...] Versioning REST Web Services Managing changes to APIs is hard. That is no surprise to anyone who has ever maintained an API of any sort. Web services, being a special case of API, are susceptible to many of the difficulties around versioning as other types of APIs. For HTTP based REST style web services the combination of resources and content negotiation can be used to mitigate most of the issues surrounding API versioning. (tags: rubyonrails restful versioning) [...]
[...] Peter Williams – Versioning REST Web Services (tags: webdev xml rest) [...]
[...] Peter Williams – Versioning REST Web Services [...]
[...] Versioning REST Web Services [...]
[...] Peter Williams – Versioning REST Web Services (tags: rest versioning api) This was written by andy. Posted on Thursday, July 9, 2009, at 1:36 am. Filed under Uncategorized. Bookmark the permalink. Follow comments here with the RSS feed. Post a comment or leave a trackback. [...]
[...] just brushed the surface of this topic. For more, Peter Williams has an excellent discussion of it here, here, and here. (disclaimer &emdash; Peter is a former coworker and personal friend. This section [...]
[...] Peter Williams – Versioning REST Web Services barelyenough.org/blog/2008/05/versioning-rest-web-services – view page – cached Managing changes to APIs is hard. That is no surprise to anyone who has ever maintained an API of any sort. Web services, being a special case of API, are susceptible to many of the difficulties around versioning as other types of APIs. For HTTP based REST style web services the combination of resources and content negotiation can be used to mitigate most of the issues surrounding API versioning. — From the page [...]
[...] This post was Twitted by dnene [...]
[...] a message with no out-of-band-knowledge. Extending this model further, there are attempts to attach version identifiers and schema references to media types. The end goal of these attempts is to let arbitrary clients [...]
Should a RESTful web service API be versioned?…
No, I do not believe it is necessary to “version” a REST api. It is certainly possible to build RESTful apis and manage breaking changes without ever including version numbers in the URI. By far the most important thing to ensure is that clients a…
[...] impact of this versioning of the data. There are some discussions about this by Peter Williams, Versioning Rest Web Services, and Ganesh Prasad asks, Does Rest Need [...]
[...] VERSIONING REST WEB SERVICES [...]
[...] Peter Williams – Versioning REST Web Services (tags: rest api http) [...]
[...] Peter Williams – Versioning REST Web Services – an old article but a nice alternative for handling API versioning [...]
Post a Comment