A single process represents one logical action (e.g. ‘ingest this OPEX’, ‘deprecate this CO’, ‘apply these preservation settings’) no matter how many actual internal jobs, background tasks or workflows Preservica might use to achieve that action. And you, as a user, are interested in the outcome of that process, not the internals of how Preservica chooses to execute it.

User view

Process monitoring is available through the monitoring UI, available at /monitor or from the Ingest menu, if it’s enabled for you.

This UI shows you a summary of the processes you have run in the top half, which can be filtered to only show certain types of process. For example if you are running auto-preservation processes and you’d like to see only those, select Automated System Tasks. This list shows the most recent processes first so you can look at what’s happening now easily. The process cards give a summary of the state of the process: a status icon, the number of warning or error messages it’s generated, and the count and size of content associated with it. (Not all processes generate all of these numbers.)

In the lower panel you see messages that come from processes. The message list is filtered by the controls in the panel above, and also by selecting processes in the top panel; in this example I have selected a single Characterise auto-preservation process so I’m only seeing messages from that process. If messages related to a particular entity then there will be a clickable link to view that entity in Explorer. The messages are also shown with the most recent at the top so you can watch a process without having to scroll.

You can also download the messages matching your current filters.

Within Explorer/SDB

In the lower right of the main Preservica interface is the Notification panel. It shows you a summary of running and recently completed (within the last hour) processes, and can be popped out to show more information (similar to the card view in the monitoring app).

In this example I have some ingest processes which have failed, and one which is currently running. Processes will show a progress bar if there is any meaningful progress that can be shown (which depends on the process type).

If your monitor application is enabled then the monitor names in this panel will be links to open the monitoring app with that monitor pre-selected.

API

We expose all the data shown to you in the monitoring UI through the Process Monitor API: https://eu.preservica.com/api/processmonitor/documentation.html.

There are two main endpoints:

• GET /monitors: Return a list of all the monitors. You can filter this list based on process name or category.
• GET /messages: Return a list of messages. You can filter based on the same options that are available to you in the UI. This endpoint is paged because there can be a lot of messages.

There is also an endpoint that returns timeseries data (one snapshot of size and file count per day) for a long running process. This is useful for incremental ingest or auto-preservation actions if you keep using the same process for multiple days, but the way we typically advise you to use the system doesn’t do this so for most of you this endpoint won’t be useful.

Use cases

We currently use processes for:

• Ingest. Any ingest will create a single process per logical ingest, whether that represents one workflow (standard ingest), two (web/single file ingest) or an arbitrary number (incremental OPEX ingest). It’s most useful for the incremental ingest as you don’t know which workflow to look in for messages and entity links. Information is displayed to the user that relates to the file, before any entity has been created and links to entities are added once they’ve been created.
• Background tasks from applying settings or configuration changes. This applies to auto-preservation tasks started by registry updates or by changing the migration settings, and it’s likely to apply to additional background actions triggered from ingest in future. There is no workflow associated with these actions but users might still want to track them.
• Content replacement/deprecation. Pressing the ‘Replace’ button in Explorer will start a workflow, but that’s not the user experience
• Move/link: started by non-workflow interactions in Explorer

Completed processes are archived (and will no longer be visible in the API or UI) 90 days after completion.