Data Purging

Live allows the administrators to configure Storage Purge Rules in order to delete data using time-based rules that doesn't need to be kept longer in the environment.

The administrator can Simulate the purge operation before to Run the purge in the environment.

The Save button will keep the set of purge rules persisted but no purge will be executed automatically. Live only purges data when an administrator clicks on Run purge using the rules displayed in this screen.

System event types to purge

Most of event types starting with __ are system-wide and used internally or by Live or by some plugins in order to track the operations or to fulfill some monitoring dashboards (for example the default System Monitoring dashboard).

The Storage > Stats menu in administration page delivers a way to understand the size and how many events are persisted in the storage providers. That page delivers a toggle button to display the typical system events as following.

It's very important the administrator be aware which event types can be deleted and that decision will depend on each customer installation.

In general, in most of installations the administrators can purge the following event types older than 3 months:

  1. __failed: data not persisted into storage providers due any failure (you might want to keep a backup for them before purging)

  2. __queries: stats about the pipes query execution in the entire system

  3. __metrics: a lot of information about platform metrics in general, including disk storage, CPU and memory, you might to keep them longer if you need wider spans in system monitoring dashboards

  4. __log: output generated by logging frameworks (typically only INFO/WARN/ERROR but it depends on system configuration) and stored additionally as event type, probably the default log files in your installation already have that data persisted, so it's safe to delete them

Below the example of how to configure expressions for collections that can be purged for period older than 3 months:

Last updated