You are here

How to change Elasticsearch data archiving

This article describes how logs are cleaned and how some of the parameters may be customized in Airlock 7.1 and newer.

How log cleanup works

Airlock stores its log files in Elasticsearch. A fresh index is created every day, storing all logs of the corresponding day. As soon as the usage in /var exceeds a certain limit (75%) or when the log files exceed a certain age (365 days), the corresponding indices are deleted to ensure enough capacity is available in /var.

Custom configuration

These limits should suit most system configurations. However, for a high traffic system with small disk drives, there might not be enough space left to store full logs of the current day. Or your company may have a policy that restricts the storage of logs to a shorter period. In this case you need to adjust the settings to your own values as follows:

  • Create the file:# vi /opt/airlock/custom-settings/monitor/airlock-elasticsearch-space-nurse.cfg
  • Add the following settings to the file
    PERCENT_TO_TRIGGER=75
    PERCENT_TO_KEEP=70
    MAX_DAYS_TO_KEEP=365
  • Adjust the settings to your needs
  • Restart the nurse systemctl restart airlock-elasticsearch-space-nurse

Settings explained

PERCENT_TO_TRIGGER=75 If disk usage is higher than the configured percentage, a cleanup will be triggered.
PERCENT_TO_KEEP=70 Cleanup deletes the oldest indices until the disk usage is at the configured percentage.
MAX_DAYS_TO_KEEP=365 Regardless of the trigger levels, all indices older than the configured number of days will be deleted.

        
    

 

Knowledge Base Categories: