REST/HTTP Service Versioning (Response to Jean-Jacques Dubray)

Jean-Jacques Dubray takes issue with my approach of using content negotiation to manage service versioning in HTTP. I actually hesitate to respond to Mr. Dubray because the overall tone of his piece is rather off putting. On the other hand, he raises a couple of interesting questions which I have been really looking for and excuse to talk about. So I will give it a go.

Handling obsolescent service providers

Mr. Dubray asks how we deal with version skew between the client and server.

Backwards compatibility is when the consumer comes in with a “newer” request version than the service provider can provide. This is common when a consumer uses different providers for the same type of service. So ultimately, you need to provide some room to define the version of both the consumer and the version of the service provider that it is targeting. Your mechanism only supports “one version”.

Not true, the versioning mechanism I describe easily handles multiple versions. First, lets be clear, a service provider cannot provide capabilities that where not conceived of until after it was written. So Mr. Dubray must be interested in is the ability of a single consumer to successfully communicate with multiple versions of the service provider. I agree with him that this is an absolutely vital feature of any versioning mechanism.

Fortunately, content negotiation deals with this issue quite handily. I left this out of the original post for simplicities sake but it well worth talking about. HTTP allows user agents – or service consumers, if you prefer – to specify more than one acceptable response format. For example, the following is a perfectly legal HTTP conversation.

===>
GET /accounts/42
Accept: application/vnd.myapp-v2+xml, application/vnd.myapp-v1+xml;q=0.8

<===
200 OK
Content-Type: application/vnd.myapp-v1+xml

<account>
  <name>Inigo Montoya</name>
</account>

The Accept header field in the request indicates that the consumer can operate using either version 1 or 2 of the API but it prefers version 2. Accept headers can include any number of MIME media types along with preference indicators (the q=number part). This allows consumers to inform the server of all acceptable dialects of the API with which it can work. In the example, the server obviously did not support version 2 of the API and therefore responded using version 1.

Resource deprecation

Further along Mr. Dubray asks this question,

Another flaw of your versioning strategy is that URIs are by default part of the versioning strategy. I have often pointed out that “Query-by-examples” are encoded by members of the REST community (MORCs) in a URI syntax, for instance:

/customer/{id}/PurchaseOrders ...

Peter, how do you express that a particular QBE belongs to one version and not to the other?

I don’t. The set of purchase orders associated with a particular customer is not version specific. The customer has agreed to purchase the same things regardless of which version of the service you are talking to.

Perhaps the question Mr. Dubray is really trying to ask is, what happens if you want to deprecate such resource?

(One reason to do so might be that the purchase order collections become too big to reasonably render in a single response. There are other, better ways to solve that particular problem but it is a nice concrete use case for resource deprecation.)

Resource deprecations is easily handled in REST using media types to handle versioning. First some ground rules, user agents should never be constructing such a URI. Doing so should be a gross violation of the HATEOAS constraint of REST. Rather they would be extracting that URI from the representation of the customer provided by the server. In such a case, an HTTP conversation getting the purchase orders for a customer might look like this.

===>
GET /customer/42
Accept: application/vnd.myapp-v1+xml
<===
200 OK
Content-Type: application/vnd.myapp-v1+xml

<customer>
  <purchase-orders href="http://service.example/customer/42/purchase-orders"/>
</customer>


===>
GET /customer/42/purchase-orders
Accept: application/vnd.myapp-v1+xml
<===
200 OK    
Content-Type: application/vnd.myapp-v1+xml

<purchase-orders>
  ...
</purchase-orders>

At version 2 of the API we deprecate the all-purchase-orders-for-customer resource – removing all references to it in the customer representations – and replace it with a purchases-order-by-month-by-customer resource. A similar HTTP conversation with a client capable of handling version 2 of the API would look like this.

===>
GET /customer/42
Accept: application/vnd.myapp-v2+xml
<===
200 OK
Content-Type: application/vnd.myapp-v2+xml

<customer>
  <purchase-orders-by-month href-template="http://service.example/customer/42/purchase-orders?in_month={xmlschema-gYearMonth}"/>
</customer>


===>
GET /customer/42/purchase-orders?in_month=2008-05
Accept: application/vnd.myapp-v2+xml
<===
200 OK    
Content-Type: application/vnd.myapp-v2+xml

<purchase-orders>
  ...
</purchase-orders>

Notice that in version 2 of the API the all-purchase-orders-for-customer resource is no longer exposed in any way. As a human you might guess that it still exists, and indeed it would need to in order to handle requests to version 1 of the API. However, a version 2 consumer will never make a request to that resource because it is not mentioned in the version 2 representations. Indeed, any requests for the all-purchase-orders-for-customer by a version 2 consumer would be met with a 406 Not Acceptable response because it is not part of the version 2 API.

Wrap up

Toward the end Mr. Dubray gets into full rant mode with these bits,

You will soon start realizing that resources do have a state that is independent of the “application” since by definition a resource can participate in multiple “applications”. This is the essence of SOA, i.e. the essence of reuse.

Resources certainly may participate in multiple “applications”. There is nothing in the REST principles that prevent that. I don’t really claim to be an SOA expert. I just make systems work using REST principles. So far I have not found a problem reusing my resources in multiple applications. In fact, REST seems to excel at that very thing.

At least, some of the MORCs Member Of the REST Community could have the courtesy to acknowledge that they are indeed building a programming model on top of REST, that this programming model needs clear semantics and that these semantics are not intrinsically part of REST (nor always RESTful).

I, for one, will readily acknowledge that we have built, and are continuing to build, programming models on top of REST. REST is merely a set of principles, articulated as constraints, that facilitate the creation of useful network based architectures. I would be very surprised if many in the REST community would disagree with me. These programming models do, for the most part, adhere to the REST principles.

Building REST/HTTP web services is certainly not fully understood yet. That does not make it special, hardly any sort of system design or architecture is fully understood. However, REST seems, to me at least, to be a better fit for today’s applications and technologies than any of the alternatives.


If you’re interested in REST/HTTP service versioning be sure not to miss the rest of the series.

15 comments on “REST/HTTP Service Versioning (Response to Jean-Jacques Dubray)

  1. -

    Peter:

    thanks for responding in such detail to my post. My tone is reflecting of the discussions I have with Stefan Tilkof who 6 months ago thought it was a good idea to encode the version in the URL. I am also constantly being reminded that “I don’t understand” REST.

    I would like to reconsider the first section of your response, what I meant by “one version” is that you can express only one version number (you need two one for forwards compatibility and the other one for backwards compatibility): either the consumer request version or the provider, your mechanism does not seem to support both sides. Of course I understand that once you pick a side, your mechanism supports any number of versions.

    what happens if you want to deprecate such resource

    that question seems odd, you don’t deprecate a “resource”, you deprecate a service that enables you to interact with a particular resource. This is huge problem in information system construction. Some “resource”, such as “customer” have a life span well beyond the the life of a particular “system” or “application”.

    The question I have in general to the REST camp, is now that we have proof that Media Types are the way to go and that Teo said that yes for resource-to-resource inter-actions require some agreement at the media type level. Don’t you think that you have now reinvent WSDL one-for-one? I URI are not used as identifiers (they are used as QBEs), media types need to be shared and agreed, and media types need to contain actions. How different is that from WSDL? Aside from the fact that you call a message a “Resource Representation”, I frankly don’t see any difference.

    Overall, I have no doubt that you can successfully construct information systems with HTTP. The question is: what is the trade-off? what do you gain, what do you loose? (objectively). Once we agree on that we can all return to our favorite programming model, because in the end one is not superior to the other.

  2. - Post author

    what I meant by “one version” is that you can express only oneversion number (you need two one for forwards compatibility and the other one for backwards compatibility): either the consumer request version or the provider, your mechanism does not seem to support both sides.

    Are you suggesting that version indicators should have multiple parts, a la the standard {major}.{minor} (or {incompatible_change}.{compatible_change}) approach for versioning libraries? If that is what you mean I certainly would not be opposeto doing that I am not sure if it is really necessary though.

    If you don’t mean that then I don’t understand the point you are making. (Maybe an example would help?)

    you don’t deprecate a “resource”, you deprecate a service that enables you to interact with a particular resource.

    I suppose it depends on how you define resource. Does a resource with no way to access it really exist? I suppose it might in some sort of conceptual sense but for all practical purposes it does not. For example, in my example of a collection becoming too large to render in a single go you might argue that the purchase orders collections still exist but if you cannot make requests about them it hardly seems to matter. When I say resource, I mean a conceptual entity that can be accessed via HTTP at a particular URI.

    now that we have proof that Media Types are the way to go

    I don’t think there is consensus regarding this point in the REST community yet. I hope that my success with this approach can bereplicated by others, but I think the jury is still out at this point.

    Don’t you think that you have now reinvent WSDL one-for-one?

    No. WSDL is hopelessly complex (at least for my brain). A REST programming model will certainly expose the same level of computational capabilities as SOAP+WSDL does. But that is very much like saying that all general purpose languages are Turing strength. Just because you can express solutions to business problems in a particular language or architecture does not mean it is the best choice.

    The question is: what is the trade-off? what do you gain, what do you loose? (objectively).

    For me the big benefits of REST/HTTP are easy exploration, good conceptual chunking and the ability to use more pre-built tools.

    Easy exploration because everything you can do next is always expressed right in the last response you got from the server. Good conceptual chunking because dividing the system into a set of object with a uniform interface and varying representations means that you can learn a REST API iteratively. And pre-built tools, because REST/HTTP is standardized and REST/HTTP requests vary significantly less than SOAP request you can use tools like curl to play with the system.

    As for downsides, I think REST (particularly the HTTP variety) pushes more work into the clients. This can lead to some duplication of effort if clients don’t shard code. On the other hand pushing the work to the leaves of system is great for scalability.

  3. -

    Peter:

    we are not on the same page with the respect the versions that you need (I think). Let me explain, because this is important.

    Yes, in general you will need Major and Minor -of course. What I am saying is that the inter-action between two software agents require that you identify the exact version of the requestor/consumer and the major version of the responder/provider.

    I could I have a request v2.x sent to a service v2.y. The requestor MUST announce its version, while it targets a major version of a provider (the goal of forwards compatibility is to let the provider upgrade without ANY change to the existing resource consumers).

    Unless I am mistaken your versioning scheme only supports conveying the version of the service provider/responder. This is actually an “odd” design because it is the requestor that says “I really want to talk to v2.1”. This is counter to “forwards compatibility” requirements.

    The way versioning works is the requestor talks to the latest forward compatible provider (it might not know exactly which minor version it is, this is the whole point of versioning). The requestor clearly announce its version, just in case the service provider needs to do something special (at least validate that this is a valid request for this version).

    What I am arguing is that you don’t have a way to “talk to the latest major version” of something because the URI locks you in, the identifier cannot contain any version information as you rightfully pointed out.

    BTW, I use Dave Orchard’s version of F/B compatibility: http://www.xml.com/pub/a/2004/10/27/extend.html

    It is reverse compared to the one you are using. Frankly this is a detail, let’s not debate which is which, I don’t care.

    I suppose it depends on how you define resource.

    That would be a great thing to clarify. For me it has definitely the characteristics of becoming persistent and network accessible with a uniform IDENTIFIER. A URI is only and nothing more than an identifier.

  4. - Post author

    Jean-Jacques,

    Thanks for taking the time to explain your point about versions. I think might see what you are after now.

    Unless I am mistaken your versioning scheme only supports conveying the version of the service provider/responder. This is actually an “odd” design because it is the requestor that says “I really want to talk to v2.1”.

    The way versioning works is the requestor talks to the latest forward compatible provider (it might not know exactly which minor version it is, this is the whole point of versioning).

    Good point. I think a good way to deal with is with a media type parameter. For incompatible changes to the API a new media type would be created (as I previously suggested). However, an optional “level” parameter could be added as a way to assert that the server was capable of handling the minimal acceptable level of compatible version changes. For example, if the client need some version that the server is unable to provide you might end up with this

    --->
    GET /accounts/42/hobbies
    Accept: application/vnd.myapp-v1+xml;level=12
    <---
    406 Not Acceptable
    Content-Type: text/plain
    
    This server only supports levels 1-8 of application/vnd.myapp-v1+xml
    

    And if the server could fulfill it you would end up with

    --->
    GET /accounts/42/hobbies
    Accept: application/vnd.myapp-v1+xml;level=12
    <---
    200 OK
    Content-Type:  application/vnd.myapp-v1+xml;level=35
    
    <hobbies>
      <hobby>Fencing</hobby>
    </hobbies>
    

    Any requests without an explicit level parameter would be fulfilled as if the level parameter where present with a value of 1. That means the server responses with whatever level it is capable of.

    Would that meet the requirements you are concerned about?

    As for defining a resource… One nice thing about “resource” is that is it sufficient concrete that everyone immediately has some idea about what it means, and yet sufficient vague as to all it to mean just about anything.

    A resource certainly be identified by more than one URI. However, two URIs only point to the same resource iff requests to both URIs return the same representation. Therefore, the /customer/42/purchase-orders resource is not the same resource as the /customer/42/purchase-orders?in-month=2008-05 resource.

  5. -

    Peter:

    yes, I think this solves the problem of expressing both the exact version of the consumer and the major version of the provider.

    Of course, you can’t really use Media Types for anything else now, but I think this is a fair way of dealing with Versioning in REST.

    The other issue is that REST does not support “action”. I guess you read my posts between Stefan, Teo, Subbu and I. Action do exist. There is nothing fundamentally wrong in embedding an action in a resource representation, but for versioning it actually means that multiple media types belong to the same “unit” of versioning. Your mechanism does not let you create a unit of versioning at the “inter-action” level.

    Of course, now you have to deal with the fact that a “resource” is in general “fragmented” by the usage of URIs as QBEs. So let me take an example to illustrate my point.

    I am a merchant of some sort and I sell a hot set of RESTgets. I started to design my customer resource with only one contact info. But as my customers come back for more RESTgets I realize that I need to support more than one contact info, because they want to ship their RESTgets to their second residence, vacation home…

    So in the beginning I started to with
    /customer/{id}/contact

    and now I have a
    /customer/{id}/contacts
    /customer/{id}/contacts/{number}
    I also repurposed my original contact URI as the “primary contact”, but I did not like it so I also created a:
    /customer/{id}/primaryContact

    How would I know which version is which ? Cool URIs don’t change, I know, but then you are forced to change their “meaning” like above /contact became /primaryContact

    This is a trivial example, but imagine a more complex data structure that evolve fairly rapidly.

    The problem here with REST is that you are using an “identifier” method as a “data access” method. In this case, you have nothing to version.

    So at the end of the day, if you ask me, I really like the decoupling between inter-action interface and resource (i.e. WSDL) that makes versioning a ton easier.

  6. -

    Jean-Jacques,

    You aren’t actually versioning a service or a resource with this content-type setting. You are setting the version of the document type.

    The model here is a heterogeneous one. The assumption is that there might be five different server implementations and twelve client implementations, all with various kinds of jargonization (version forking as well as incrementing) to their protocols and document types. Any and all of these might have been upgraded or left at their original version over the lifetime of the architecture. The Web’s approach introduces a triangle of methods, document types and URLs that each evolve as separately moving parts.

    Document types use must-ignore to encourage a certain fuzzyness of version that supports upgrade and some jargonization of a document type. For example, I could use a standard document type with a bit of extra information inserted for good measure. It will be understood in a limited context, but the rest of the document will still be understood universally. If I really exhaust the evolution path for a particular document type I come up with a new one an rely on content negotiation as per Peter’s approach.

    In a true REST architecture both methods and document types are controlled. There is a limited number of ways that a particular schema of data is allowed to be encoded into, and every way that exists at a given time can be expected to be understood by the document recipient.

    A client issues a GET request with an Accept indicating the understood formats. The server knows how to pack the returned information into at least one of these, and responds. The client processes the document, extracting information according to its needs. Communication is achieved, and with the resource that the client intended. Versioning takes second place to heterogeneous evolution.

    See also my article on REST Rewiring: http://soundadvice.id.au/blog/2008/04/18#rest-rewiring

  7. -

    Ben:

    The Web’s approach introduces a triangle of methods, document types and URLs that each evolve as separately moving parts.

    I don’t think the architecture of the web ever considered “versioning” as a key design requirement. It kind of “works” with web pages. As soon as you live this trivial use case for “connected systems” I can only see issues.

    A client issues a GET request with an Accept indicating the understood formats. The server knows how to pack the returned information into at least one of these, and responds.

    I’d like to point out that in a true versioning strategy, the server should respond based on its own version. So if client 2.1 calls server 2.4, the response should be in 2.4 format. This is why forwards compatibility is so valuable because it prevents the server to have to operate all kinds of minor versions. Imagine a situation where you have 3 major version and 10 minor in each. How many versions are you going to operate/maintain? With forwards compatibility you only operate/maintain 3.

    sorry, I don’t see the light, and again, I am trying. I live in a world where reuse, reliability, security (authorization), predictability… are the norm. I see so many holes in this approach that I don’t see why I should spend any time investigating it. Now if your context does not require so much security, reliability, predictability… and you really need the kind of scalability that REST can give you, great, I am all for it. But please, don’t mix and match these two context, as I mentioned you are creating a losing proposition for everyone involved.

  8. -

    I know this post is a bit old but, Jean Jaque :

    So in the beginning I started to with
    /customer/{id}/contact

    and now I have a
    /customer/{id}/contacts
    /customer/{id}/contacts/{number}
    I also repurposed my original contact URI as the “primary contact”, but I did not like it so I also created a:
    /customer/{id}/primaryContact

    There’s always http redirects…

    ===>
    GET /customer/22/contact
    Accept: application/vnd.myapp-v1+xml
    <===
    301 MOVED PERMANENTLY
    Location: /customer/{id}/primaryContact
    ===>
    GET /customer/22/primaryContact
    Accept: application/vnd.myapp-v1+xml
    <===
    200 OK
    Content-Type: application/vnd.myapp-v1+xml

  9. -

    Is there any recommendation how to respond to a request accepting only version 1, after there has been an incompatible change that makes it impossible to produce a valid version 1 response?

  10. - Post author

    Mattias J, a 406 Not Acceptable is the proper response in that situation. Preferably with a body that describes what media types are supported.

  11. -

    A minor point, your example of an Accept header taking alternate content types and then stating version 2 is its preference is slightly wrong. Without a q factor in there, they all default to q=1… which means the server is free to serve whichever version it likes. The HTP spec does not say what should happen in this case. In my framework I default to the order of the Accept types, but as Microsoft’s web api team blindly state, they don’t have to do this so serving their default is perfectly legal:

    http://aspnet.uservoice.com/forums/147201-web-api/suggestions/2620003-use-order-of-entries-in-the-accept-header-for-entr?tracking_code=519af301292bc3a981a99d51a6231ba5

    I believe they are being short sighted, but the best way to guarentee you get the best possible representation is to use the q factor.

    Hopefully others will raise this with Microsoft as well… I tried here also:

    http://forums.asp.net/t/1771514.aspx/1?Accept+type+ordering

  12. -

    Ah… apologies, your q factor is present… just scrolled off the screen!

  13. -

    Peter, first off, thanks very much for your blog posts on this subject; they’re really helping me think through things. One question that’s not clear to me from your examples… would you use different media types for different resources, or the same one? An example will make this clearer. Say my service exposes Customer resources and Order resources. I can see two possibilities.
    1) the media type “application/vnd.myapp-v1+xml;level=12” can be used for both Customer AND Order resources, and in each case simply means the 12th level of the 1st version of the XML representation for that resource.
    2) there would be distinct media types “application/vnd.myapp.customer-v1+xml;level=12” and “application/vnd.myapp.order-v1+xml;level=12”, each to be used with the corresponding resource.

    It seems to me that approach #1 could work, since the resource is identified by the URI and therefore doesn’t have to be part of the media type. On the other hand, it feels funny for a media type to be so loosely defined that it could be applied to two representations that looking nothing alike.

  14. -

    While pondering the approach discussed above, one more use case came to mind which it nicely handles – experimental (beta) releases of new representation versions. For example, suppose that version 1.12 is the latest supported version for general use, but version 1.13 is available for beta testers. A client specifying “application/vnd.myapp-v1+xml;level=5” gets version 1.5, or nothing (a 406). A client specifying “application/vnd.myapp-v1+xml” (i.e. no “level” parameter) gets version 1.12, because the absence of the “level” parameter means “I accept any 1.x”, and thus the server is free to return any 1.x version it chooses, in this case the most recent generally available version. However, a beta client could specify “application/vnd.myapp-v1+xml;level=13” and get that version specifically. Nice!

Comments are closed.