High availability
The bundled postgres, redis and rabbitmq are single-pod deployments. They exist so that a fresh install works out of the box and matches the Docker Compose stack — not for production. Each is a single point of failure whose loss stops the cluster.
MetaDefender Cluster Workers already running continue to serve traffic through a MetaDefender Cluster Control Center outage.
Choosing an approach
In production, prefer a managed service from your cloud provider. Azure, AWS and Google Cloud all offer PostgreSQL-compatible databases with built-in high availability, backups and dynamic scaling — considerably less to operate than a database cluster you run yourself.
If the dependency must live inside the Kubernetes cluster, the options below are tested:
Dependency | On-cluster solution |
|---|---|
PostgreSQL | CloudNativePG operator, 3 instances |
Redis | dandydeveloper/redis-ha (Redis + Sentinel), 3 nodes, HAProxy enabled |
RabbitMQ | RabbitMQ Cluster Operator, 3 nodes |
Namespace and DNS. The Service names below assume the HA components run in the same namespace as the mdcluster release. If they live elsewhere, use the fully qualified form <service>.<namespace>.svc.cluster.local.
RabbitMQ
For production, use the official RabbitMQ Cluster Operator.
The Bitnami community chart also works and is quicker to stand up, but note that it now ships images from the bitnamilegacy repository and is best-effort and unsupported.
Create bitnami_rabbitmq.yaml:
This creates pods md-rabbitmq-ha-0, -1 and -2, reachable through the headless Service md-rabbitmq-ha-headless.
Redis
Create dandy_redis.yaml:
This creates pods md-redis-ha-server-0, -1 and -2, reachable through the md-redis-ha Service.
Front the cluster with a single writable endpoint instead. Either enable the chart's HAProxy (haproxy.enabled=true, giving one endpoint at md-redis-ha-haproxy:6379), or use a managed Redis that presents a single writable address.
PostgreSQL
CloudNativePG runs PostgreSQL as an operator-managed cluster with automatic failover.
Create the credentials Secret
cloudpg_secret.yaml:
Define the cluster
cloudpg_cluster.yaml:
The role needs createdb and superuser because the MetaDefender Cluster Control Center's database initialisation step creates the MetaDefender Cluster databases.
Deploy
Install the operator with cluster-admin privileges — the manifest creates cluster-scoped resources:
Then apply the Secret and the cluster:
CloudNativePG exposes three Services: md-postgres-ha-rw (the current primary, read-write), md-postgres-ha-ro (replicas) and md-postgres-ha-r (any instance). MetaDefender Cluster must use -rw. The endpoint follows the primary automatically on failover.
MetaDefender Cluster File Storage
File Storage is part of the mdcluster chart rather than an external dependency, but it holds submitted files and their metadata, so it needs its own treatment.
Configuration
FILE_STORAGE_MIN_REPLICA and FILE_STORAGE_MAX_REPLICA control how many copies of each file are stored. They are a redundancy factor, not a limit on how many instances the MetaDefender Cluster Control Center connects to.
Setting | Meaning |
|---|---|
| Minimum copies of each file required for a write to succeed — and minimum healthy instances. |
| How many copies of each file to write |
This must hold:
FILE_STORAGE_MIN_REPLICA <= FILE_STORAGE_MAX_REPLICA <= file-storage.replicas
Requirements
Persistence must be enabled. With persistence.enabled: false each replica uses an ephemeral emptyDir and loses its files on restart, which defeats the point of running several.
Each replica needs its own ReadWriteOnce volume. Do not point the replicas at a single shared ReadWriteMany volume — each instance owns its own metadata database and expects exclusive access.
Spread the replicas across nodes. By default nothing prevents all MetaDefender Cluster File Storage pods from landing on the same node, which would make node loss a total outage. Add anti-affinity:
Use requiredDuringScheduling only if you have at least as many eligible nodes as replicas; otherwise the surplus pods stay Pending. Use preferredDuringScheduling for a best-effort spread. For zone-level fault tolerance, combine this with a zone-aware StorageClass — a pod cannot move to another zone if its volume is pinned to the first one.
Point MetaDefender Cluster at the external infrastructure
Add the following to your values file. Passwords must match the RabbitMQ chart auth and the CloudNativePG role Secret exactly — a mismatch produces authentication failures at startup rather than a clear configuration error.
Install as usual: