API Versioning in Preservica 6.2

If you already consume the Preservica API and you do not currently use the versioned API endpoint (but use the non-versioned API endpoint instead), we recommend that you switch to using the versioned API endpoint if your code relies on the namespace returned in the API response.

You can keep using the non-versioned API endpoint if desired but be aware that the namespace might change after each release. If you do not rely on the namespace, you can safely keep using the default non-versioned API endpoint.

See “What do I need to do?” at the end of the article for more details.

In Preservica 6.2, this approach only applies to our Entity API, but in the future we are likely to do something similar for other REST APIs.

Why Version APIs?

The simple argument for why APIs must be versioned is: because they change. Every time we change something in one of our APIs, that potentially violates an assumption that a client was making. Even adding a new endpoint could potentially do that – perhaps the 404 was being used to error check or bounds limit a script. Adding more information to an existing endpoint certainly could, if a client was doing string analysis of the response, or marshalling it strictly to a domain object without the new field.

In addition, even a strictly backward compatible change is still a change of schema, and although our API schemas are implicit (we don't publish the schema for our Entity API as an XSD), we return XML with a schema namespace and when we change what the API returns, we should update that namespace because the previous implicit schema is no longer correct.

As an example, in our Access Token API, in Preservica 6.1 we modified the response to return role information, if you request it. In Preservica 6.0, if I made a request to api/accesstoken/login, I would get a response like:

Now, I can also pass a new parameter to this endpoint, includeUserDetails=true, and I will get:

We couldn't just make the default do that, because this API is not versioned, and it might potentially break client expectations. That means that flexible clients (e.g. scripts that just JSON.parse the response into a dynamic object) don't get the new information. But if an API is versioned, we can add information to the result, and flexible clients get access to that information immediately, whereas those with strict namespace or domain object requirements can request a specific version until they are ready to update their expectations.

How We Do Versioning

There are several possible approaches to API versioning of a web-based API. This somewhat tongue in cheek article has a good summary of the three main options, and why none of them is perfect. We've chosen to put the API version in the URL, because although it makes the URL not strictly RESTful (it doesn't just represent a resource), it's very easy to explain and use.

So for our Entity API, as well as going to e.g. api/entity/information-objects/0f88230e-70e1-4181-b9a5-60c929af96fc to get the current version, you will also be able to go to api/entity/v6.2/information-objects/0f88230e-70e1-4181-b9a5-60c929af96fc – and if you do that, then even if we make changes in Preservica 6.3, your API client will still see the same as it does in 6.2. You can take the time to update your client and test it against the 6.3 API before making a change in your production system to rely on /api/entity/v6.3 instead.

For example, here's what you'll get if you try to get the identifiers on an entity in Preservica v6.2:

• /api/entity/v6.0/structural-objects/8bc63085-824b-4cdd-b6d2-77c64269599a/identifiers will return:

Note the 6.0 namespace, and the lack of ApiId.

• /api/entity/v6.1/structural-objects/8bc63085-824b-4cdd-b6d2-77c64269599a/identifiers will return:

Note the 6.0 namespace, but that ApiId is now available.

• /api/entity/v6.2/structural-objects/8bc63085-824b-4cdd-b6d2-77c64269599a/identifiers will return:

Note the 6.2 namespaces. The non-versioned endpoint in Preservica v6.2 will respond similarly.

If you make a request to an endpoint that didn't exist in a version (for example /api/entity/v6.0/retention-policies), you'll get a HTTP 404.

If you don't specify a version in the URL, you'll always get the most up to date response. We'll continue to make efforts to keep the changes we make be non-breaking, i.e. adding optional parameters, adding endpoints and adding response fields, so clients that don't have strict document requirements can benefit from the extra information without needing changes across a Preservica upgrade – though we may still have to make breaking changes occasionally.

Additionally, if we fix a bug (where the response an API request gives is not as intended), the old version of the API request won't preserve that bug. If we change intended behaviour then we will preserve the previous behaviour in the versioned endpoint. We may also make exceptions for security reasons – for example, the /api/accesstoken/login endpoint mentioned above used to allow you to send parameters in the query string, not the POST body, and accidentally expose your password in log files. Even if that was a versioned endpoint, we would not maintain that vulnerability in an old version once we discovered it.

What about 6.0 and 6.1?

We introduced this type of endpoint in Preservica 6.0. So there are versioned endpoints at api/entity/v6.0. However, in 6.1 we made some small changes to that API without managing the versioning as described here, and (at least at release time) we did not expose /api/entity/v6.1. This means that, in Preservica 6.1, the endpoint /api/entity/v6.0 actually returns the same as the unversioned endpoint, i.e. including the changes made in 6.1.

From Preservica 6.2, these endpoints have been given their proper meaning: if you request /api/entity/v6.0 you will get responses like those generated by Preservica 6.0, and if you request /api/entity/v6.1 you will get responses like those generated by Preservica 6.1.

XIP Schema

The same considerations apply to adding information to our data model, which is made available through the XIP schema. With Preservica 6.0, we released a version of this schema at http://preservica.com/XIP/v6.0. In Preservica 6.1, we made some small additions to the XIP schema to enable API features, but we didn't change the namespace. This was okay, because the additions were small and XIP you're likely to generate will still match both schemas.

In Preservica 6.2, we have added some significant new objects to the XIP schema (characterisation information for representations), This information is included in the XIP for an export, and that XIP wouldn't validate against earlier versions of Preservica. So we have updated the schema and released it as http://preservica.com/XIP/v6.2.

The /v6.2 (and unversioned, in Preservica 6.2) version of the Entity API will return and accept XIP objects in the new namespace, http://preservica.com/XIP/v6.2. In the future it's likely that API changes may also go with XIP schema updates. The versioned endpoints for /v6.0 and /v6.1 generate and expect XIP v6.0.

The ingest pipeline will still accept XIP v6.0, and you should generally be able to submit content to Entity API endpoints in the old namespace, but you should try to make sure that your XIP namespace is appropriate for the API version you are requesting.

What do I need to do?

If you consume our Entity API, you need to make one of two choices:

• Consume the API responses in a namespace-agnostic way, and use the non-versioned endpoint. In this case, we may make changes to the API between versions, e.g. by adding more information to a response, and you should consume it in such a way that you can cope with that. You should also read the Breaking Changes list each release to see if we've made a breaking change to an API you're using.
• Consume the API response in a way that relies on the namespaces (for example using XPath with fixed namespaces). In this case you should use the versioned endpoint that speaks in terms of the namespaces that you expect.

If you can start using the /api/entity/v6.0 endpoints today, that will ensure a smooth passage across the 6.2 release. If not (because you rely on features added to the API in Preservica 6.1), you should prepare to update your API consumers to use /api/entity/v6.1 when we roll out Preservica 6.2 to your environment.

Because of the changes in the XIP schema in Preservica 6.2, there are some other changes you may have to make, although for most of you it will hopefully be seamless:

• If you have a custom XIP transform, you will have to update the transform to target http://preservica.com/XIP/v6.2. This should only be the case for custom XIP-to-CMIS transforms (which are normally not needed in v6) and a couple of niche ingest scenarios.
• If you have external post-processing which consumes a Preservica export, including reading the XIP metadata from inside a PAX, that post-processing will have to be able to work with XIP v6.2. I don't expect any of you to be doing this.

Support for Old Versions

The purpose of a versioned API is to allow you to manage the upgrade gracefully, not to preserve old versions of an API endpoint forever. Eventually, an API may change enough that supporting old versions becomes expensive, and we expect you (at least if you're using our SaaS platform) to maintain your API client code to stay fairly up to date.

We are still working on the details but I would expect us to designate certain releases as Long Term Support. We would then guarantee to keep versioned API endpoints at least as far back as the previous LTS release.

With the release of Preservica 6.2 we will be supporting v6.0 and up.