Configuring Auto-Migration Policies via API Endpoints
John Gabriel
November 29th, 2024
Six months ago, with the release of version 7.3, we added the ability to configure your auto-migration policies via the Next-Generation Interface. In most cases, this page is all you need to tailor the preservation process to your exact needs. However, there may be situations where you want to adjust your migration policies via code, and in this article, we will cover the API endpoints that allow for that.
Migration Config
A full list of API endpoints for migration can be found in the Swagger documentation. Where possible, we encourage developers to use the /settings/migration/config/policy
endpoint over calling the /settings/migration/config/profiles
and /settings/migration/config/rules
endpoints, because the former validates the incoming policy before applying it.
auto-migration
endpoints
The two /settings/migration/config/auto-migration
endpoints, GET and PUT, allow you to view and set whether auto-migration is Off
, IngestOnly
(meaning that the migration policy will be applied only to assets ingested from that point onwards), and All
(meaning that the migration policy will be applied retroactively as well as going forwards). Setting migration policies retroactively can take a while depending on how many assets you have preserved, so it's worth doing an impact report before you do this, which we will cover later in this article.
The response body for the GET endpoint and the request body for the PUT endpoint take exactly the same form:
{
"mode": "IngestOnly"
}
The policy
endpoint
The /settings/migration/config/policy
PUT endpoint allows you to apply a new policy, in the form of a list of profiles, and a list of rules. This process is not incremental; if an existing profile or rule is not referenced in the incoming policy, it will be removed. If retrospective migration is enabled (see above), a migration will start once the policy has been applied.
The process occurs in the following steps:
- The policy is validated
- Profiles that do not exist in the new policy are removed
- New profiles are added, and existing profiles are updated
- Rules that do not exist in the new policy are removed
- New rules are added, and existing rules are updated
- The order of the rules is updated
Policy validation
Before the new policy is applied, we check the following:
- Does every rule reference a profile that exists in the new policy?
- Do the rule orders form a sequential list, starting at 1?
- For all rules that have security descriptor selectors, do the tags in those selectors actually exist?
After that, we individually validate each profile and rule as we would if they were being created or updated individually.
An example policy
The policy below applies a preservation ruleset and an access representation ruleset to all files in a specific folder.
{
"profiles": [
{
"id": "b532965e193dae55160cc8dc7e9e61b8",
"name": "Spreadsheet profile",
"normalizationSettings": {
"ruleSets": [
"90a2c9a9-8805-5405-8a6b-d0c6bdbd7012"
],
"formatOverrides": {}
},
"newRepresentationSettings": {
"representationType": "Access",
"createIfExists": false,
"ruleSets": [
"d0ba9018-1df3-548e-bbc4-6c90c942cff0"
],
"formatOverrides": {}
}
}
],
"rules": [
{
"id": "b7b7e5e26bada1b8a1c2dc61474393e5",
"priority": "1",
"profileId": "b532965e193dae55160cc8dc7e9e61b8",
"selectorSettings": {
"hierarchySelector": {
"entityRef": "c740eae2-2008-49f6-a0df-e4502ba783a2"
}
}
}
]
}
Draft Migration Config
As mentioned above, migrations can take a long time, so it's a good idea to know how much of an impact your migration policies will have. To help with this, we have endpoints for creating draft policies. Most of these are container in the Swagger documentation alongside the policy endpoints above. However, when creating the migration impact reports, we will use endpoints in the Analytics API.
Draft policies contain regular policies in a wrapper alongside a name, like so:
{
"name": "Draft profile 1",
"profile": {
...
}
}
You can create a draft profile using the /settings/migration/config/drafts
POST endpoint. It will return the profile you provided in the body, along with an id
, which you'll use when referring to it later on.
You can start generating an impact assessment for a draft policy by calling the /analytics/datasets/policy.impact/generation
POST endpoint. This endpoint takes a single x-www-form-urlencoded
parameter, with the key of draftPolicyApiId
, and the value of the id returned by the /drafts
endpoint from earlier. This endpoint returns a progress token in the form of an alphanumeric string (e.g. 0273db76866d6eac5dbed4307cafd622
).
Running an impact assessement report may take quite a while, particularly if your policy covers a large proportion of your repository. If you want to cancel the assessment before it is finished, you can call the /analytics/progress/{token}/status
PUT endpoint, replacing token with the progress token, and with the following body:
{
"status": "CANCELLED"
}
You can check the status of an ongoing impact assessment by calling the /analytics/progress/{token}
GET endpoint, using the progress token from earlier. Once the assessment is complete, you can check the results with the /api/analytics/datasets/policy.impact/results
GET endpoint. The response body will look similar to the following.
{
"results" : [ {
"draftPolicyId" : "9b96a8432ac6e1747820401db14d93d4",
"totalAssets" : 6624,
"totalAssetsNotChanged" : 6624,
"totalNewNormalisationAssets" : 0,
"totalRemovedNormalisationsAssets" : 0,
"totalNewRepresentationAssets" : 0,
"totalRemovedRepresentationAssets" : 0,
"totalNewNormalisationsCOs" : 0,
"totalRemovedNormalisationsCOs" : 0
} ]
}
Conclusion
As you may expect, the API endpoints described in this article are the very same ones used by the Next-Generation Interface in the Preservation Policy Editor. This means that you can completely automate the process of managing policies and draft policies to suit your specific requirements. Should you intend to pursue this option, the Swagger documentation is, as ever, a valuable resource. Happy preserving!
More updates from Preservica
Developer Blog - API Updates in Preservica 7.5
This post provides you with a summary of the API updates we have made in Preservica 7.5.
Richard Smith
October 28th, 2024
Auto Re-Characterise Blog
The last few Preservica releases have come with announcements of Automated Digital Preservation (ADP) features starting to appear in our New Generation Interface.
Jack O'Sullivan
October 28th, 2024
Developer Blog - API Updates in Preservica 7.4
This post provides you with a summary of the API updates we have made in Preservica 7.4.
Sam Marshall
October 28th, 2024
Developer Blog - API Updates in Preservica 7.3
This post provides you with a summary of the API updates we have made in Preservica 7.3.
Lukasz Sadowski
June 19th, 2024
Open API library and latest developments on GitHub
Visit the Preservica GitHub page for our extensive API library, sample code, our latest open developments and more.
Protecting the world’s digital memory
The world's cultural, economic, social and political memory is at risk. Preservica's mission is to protect it.