Why are MetaDefender Core configuration changes lost after pod restarts in Kubernetes?
This article applies to MetaDefender Core releases deployed on Kubernetes (all versions where MetaDefender Core is deployed as multiple pods)
Issue
In a Kubernetes deployment, configuration changes to MetaDefender Core, such as enabling custom blocklist rules (e.g., blocking OpenSSL Encrypted Files), are not retained after pod restarts. When running multiple replicas:
- Only one pod operates as expected.
- Other pods generate errors such as:
(common.workflow) Workflow block ended with exception, block='GetEngineInfo'
- File type settings in the blocklist configuration may be missing or incomplete in the UI for some pods.
Root Cause
Kubernetes assigns a new pod name every time a pod is terminated and recreated. MetaDefender Core uses this name as part of its instance identity when connecting to the PostgreSQL database. Since some configuration data is stored on a per-instance basis, restarting pods or scaling to multiple replicas without proper configuration causes:
- Loss of configuration across pod restarts
- Inconsistent behavior between pods
- Errors due to missing engine packages in pods that do not share persistent storage
Additionally, if the following directory is not mounted to a shared Persistent Volume Claim (PVC), only the initial pod can access the required engine packages:
/opt/ometascan/core_data/var/lib/ometascan/updates
This leads to runtime errors and failed scans in the affected pods.
Resolution
To ensure configuration persistence and functional multi-replica deployment, take the following actions:
1. Mount the Updates Directory to a Shared PVC
Mount the following directory to a shared Persistent Volume Claim (PVC) across all MetaDefender Core pods:
/opt/ometascan/core_data/var/lib/ometascan/updates
This ensures that all pods can access the same engine package files. Based on the number of engines in your license, make sure the PVC has adequate storage capacity.
2. Set a Fixed Instance Name
In the Helm chart’s values.yaml
file, define a static value for the MD_INSTANCE_NAME
environment variable:
env:
- name: MD_INSTANCE_NAME
value: md-core-instance
This prevents MetaDefender Core from treating each restarted pod as a new instance and ensures configuration is retained.
3. Restart Pods
After applying these changes, restart all md-core
pods to apply the updated settings.
Prevention
To avoid recurrence:
- Always set a consistent value for the
MD_INSTANCE_NAME
environment variable. - Use a shared PVC for the engine updates directory in all deployments with multiple replicas.
If Further Assistance is required, please proceed to log a support case or chatting with our support engineer.