Container-Based Setup

Before running the setup, please check (System Requirements) to install all required dependencies of MetaDefender Distributed Cluster (MDDC).

Setup order requirement

Please follow the installation order to complete the system setup properly.

OrderServiceNotes
1Redis, RabbitMQ, PostgreSQL and MDDC Identity Service
  • Could be setup in parallel in any order among them.
  • Make sure they are all fully functional and accessible before proceeding to the next setup order #2.
2MDDC Control Center
  • Ensure it's able to connect to those services in #1
  • Make sure it is fully functional and accessible.
3MDDC File Storage
  • Ensure they're able to connect to MDDC Control Center
  • Make sure it is fully functional and accessible.
4MDDC Worker for MDDC API Gateway and MDDC Worker for MetaDefender Core
  • Could be setup in parallel in any order among them.
  • Ensure they're able to connect to MDDC Control Center
  • Make sure they are all fully functional and accessible.

Image name and version

All the images can be found at OPSWAT Docker Hub with the following information:

version is the currently release version.

MDDC Identity Service

Docker image
Copy

MDDC File Storage

Docker image
Copy

MDDC Control Center

Docker image
Copy

MDDC Worker for MDDC API Gateway

Docker image
Copy

MDDC Worker for MetaDefender Core

Docker image
Copy

Environment variables

1. MDDC Identity Service

Environment VariableNecessityDescription
MDDC_IDENTITY_SERVICE_DB_HOSTRequiredProvide the database host for MDDC Identity Service
MDDC_IDENTITY_SERVICE_DB_PORTOptionalProvide the database port for MDDC Identity Service Default: 5432
MDDC_IDENTITY_SERVICE_DB_USERRequiredProvide the database user for MDDC Identity Service
MDDC_IDENTITY_SERVICE_DB_PASSWORDRequiredProvide the database password for MDDC Identity Service
MDDC_USERRequired

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_PASSWORDRequired

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_EMAILRequired

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_APIKEYOptional

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_IDENTITY_SERVICE_CONNECTION_KEYRequiredDefine the connection key in order to register to Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
MDDC_IDENTITY_SERVICE_PORTOptionalDefine the expose port for MDDC Identity Service Default: 8891
LOG_LEVELOptional

Define the log level. Default value: info

Accepted values: info/debug/error/warning

Start MDDC Identity Service container with docker run:

Bash
Copy

2. MDDC File Storage

Environment VariableNecessityDescription
MDDC_FILE_STORAGE_CONNECTION_KEYRequiredDefine the connection key in order to register to MDDC Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
MDDC_FILE_STORAGE_PORTOptionalDefine the expose port for MDDC File Storage. Default is 8890.
MDDC_FILE_STORAGE_HOSTOptionalDefine the MDDC File Storage's host address. If it's not specified, it will get the container's internal IP address.
LOG_LEVELOptional

Define the log level. Default value: info.

Accepted values: info/debug/error/warning.

MDDC_CONTROL_CENTER_HOSTRequiredProvide the MDDC Control Center's host address.
MDDC_CONTROL_CENTER_PORTOptionalProvide the MDDC Control Center's port. Default is 8892.
MDDC_USERRequired

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_PASSWORDRequired

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_APIKEYOptional

Define the information to initiate the administrator account. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.

Persistent storage is located at /opt/opswat/mddc-file-storage. If end-users require data to be retained across container lifecycles, they must mount a volume to this path with 777 permissions to ensure full read/write access for all processes.

Start MDDC File Storage container with docker run.

Bash
Copy

3. MDDC Control Center

Environment VariableNecessityDescription
MDDC_LAKE_DB_HOSTOptionalProvide the database host for Data Lake. In case that the end-user does not have the Data Lake, it's required to provide this variable to automate the database preparation.
MDDC_LAKE_DB_PORTOptionalProvide the database port for Data Lake. Default is 5432.
MDDC_LAKE_DB_USEROptionalProvide the database user for Data Lake. In case that the end-user does not have the Data Lake, it's required to provide this variable to automate the database preparation.
MDDC_LAKE_DB_PASSWORDOptionalProvide the database password for Data Lake. In case that the end-user does not have the Data Lake, it's required to provide this variable to automate the database preparation.
MDDC_WAREHOUSE_DB_HOSTOptionalProvide the database host for Data Warehouse. In case that the end-user does not have the Data Warehouse, it's required to provide this variable to automate the database preparation.
MDDC_WAREHOUSE_DB_PORTOptionalProvide the database port for Data Warehouse. Default is 5432.
MDDC_WAREHOUSE_DB_USEROptionalProvide the database user for Data Warehouse. In case that the end-user does not have the Data Warehouse, it's required to provide this variable to automate the database preparation.
MDDC_WAREHOUSE_DB_PASSWORDOptionalProvide the database password for Data Warehouse. In case that the end-user does not have the Data Warehouse, it's required to provide this variable to automate the database preparation.
MDDC_CACHE_HOSTOptionalProvide the caching host (Redis).
MDDC_CACHE_PORTOptionalProvide the caching port (Redis).
MDDC_CACHE_USEROptionalProvide the caching username (Redis). If the end-user does not provide it, Redis will be added without authentication.
MDDC_CACHE_PASSWORDOptional

Provide the caching password (Redis). If the end-user does not provide it, Redis will be added without authentication.

Do not support double quotes (") and backslash (\) in the password.

MDDC_BROKER_HOSTOptionalProvide the broker host (RabbitMQ).
MDDC_BROKER_PORTOptionalProvide the broker port (RabbitMQ).
MDDC_BROKER_USEROptionalProvide the broker username (RabbitMQ).
MDDC_BROKER_PASSWORDOptionalProvide the broker password (RabbitMQ).
MDDC_CONTROL_CENTER_DB_HOSTRequiredProvide the database host for MDDC Control Center.
MDDC_CONTROL_CENTER_DB_PORTOptionalProvide the database port for MDDC Control Center. Default is 5432.
MDDC_CONTROL_CENTER_DB_USERRequiredProvide the database username for MDDC Control Center.
MDDC_CONTROL_CENTER_DB_PASSWORDRequiredProvide the database password for MDDC Control Center.
MDDC_USERRequired

Provide the administrator account that is defined in MDDC Identity Service. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_PASSWORDRequired

Provide the administrator account that is defined in MDDC Identity Service. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_APIKEYOptional

Provide the administrator account that is defined in MDDC Identity Service. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_IDENTITY_SERVICE_HOSTRequiredProvide the MDDC Identity Service host in order to add it to MDDC Control Center.
MDDC_IDENTITY_SERVICE_PORTOptionalProvide the IMDDC dentity Service port in order to add it to MDDC Control Center. Default is 8891.
MDDC_IDENTITY_SERVICE_CONNECTION_KEYRequiredProvide the MDDC Identity Service connection key in order to add it to MDDC Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
MDDC_CONTROL_CENTER_ENCRYPTION_KEYRequiredDefine the encryption key for communication between MDDC Control Center and the services. Must be 32 characters long and contain only lowercase letters (a–z) and digits (0–9).
MDDC_CERT_PATHOptional

Provide the directory path that contains the certificate and private key in order to enable https Note: when provide this variable, it's supposed to mount this path to /certs/ as volume For example: --volume /your-path:/certs

Note: In cases where SSL fails to enable due to the File Storage service not being ready, the end-user can either restart the MDDC Control Center or manually activate SSL as a workaround.

LOG_LEVELOptional

Define the log level. Default value: info.

Accepted values: info/debug/error/warning.

Start MDDC Control Center container with Docker run.

Bash
Copy

4. MDDC Worker for API Gateway

Environment VariableNecessityDescription
MDDC_WORKER_CONNECTION_KEYRequiredDefine the connection key in order to register to MDDC Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
MDDC_WORKER_PORTOptionalDefine the expose worker's port. Default is 8893.
MDDC_WORKER_HOSTOptionalDefine the worker's host address. If it's not specified, it will get the container's internal IP address.
MDDC_CONTROL_CENTER_HOSTRequiredProvide the MDDC Control Center's host address.
MDDC_CONTROL_CENTER_PORTOptionalProvide the MDDC Control Center's port Default is 8892.
MDDC_USERRequired

Provide the administrator account that is defined in MDDC Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_PASSWORDRequired

Provide the administrator account that is defined in MDDC Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_APIKEYOptional

Provide the administrator account that is defined in MDDC Identity Service. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_API_GATEWAY_PORTOptionalDefine the expose port to scan files via MDDC API Gateway. Default is 8899.
LOG_LEVELOptional

Define the log level. Default value: info.

Accepted values: info/debug/error/warning.

If multiple MDDC API Gateway containers are deployed on the same host, make sure their ports are configured to avoid conflicts.

Start MDDC Worker for MDDC API Gateway container with Docker run.

Bash
Copy

5. MDDC Worker for Core

Environment VariableNecessityDescription
MDDC_WORKER_CONNECTION_KEYRequiredDefine the connection key in order to register to MDDC Control Center. Must be 4 to 64 characters long, using only letters and digits (0–9, a–z, A–Z).
MDDC_WORKER_PORTOptionalDefine the expose worker's port. Default is 8893.
MDDC_WORKER_HOSTOptionalDefine the worker's host address. If it's not specified, it will get the container's internal IP address.
MDDC_CONTROL_CENTER_HOSTRequiredProvide the MDDC Control Center's host address.
MDDC_CONTROL_CENTER_PORTOptionalProvide the MDDC Control Center's port. Default is 8892.
MDDC_USERRequired

Provide the administrator account that is defined in MDDC Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_PASSWORDRequired

Provide the administrator account that is defined in MDDC Identity Service. It can be optional if the end-user provides the MDDC_APIKEY. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
MDDC_APIKEYOptional

Provide the administrator account that is defined in MDDC Identity Service. This account is to automatically do the following tasks:

  • Add Redis to MDDC Control Center if specified.
  • Add RabbitMQ to MDDC Control Center if specified.
  • Add Data Lake to MDDC Control Center if specified.
  • Add Data Warehouse to MDDC Control Center if specified.
  • Add MDDC File Storage to MDDC Control Center if specified.
  • Add MDDC Worker to MDDC Control Center, upload MDDC API Gateway installer to MDDC Control Center, and deploy MDDC API Gateway to MDDC Worker.
  • Add MDDC Worker to MDDC Control Center, upload MetaDefender Core installer to MDDC Control Center, and deploy MetaDefender Core to MDDC Worker.
LOG_LEVELOptional

Define the log level. Default value: info.

Accepted values: info/debug/error/warning.

MDDC_LICENSE_KEYOptionalProvide the license key to activate MetaDefender Core.
MDDC_LICENSE_DESCRIPTIONOptionalDefine description of the license key.

If multiple MetaDefender Core containers are deployed on the same host, make sure their ports and hosts are configured to avoid conflicts.

Start MDDC Worker for MetaDefender Core container with Docker run.

Bash
Copy

Start MetaDefender Distributed Cluster with Docker Compose

  1. Create a local file named docker-compose.yaml and copy the following content to this file:
yaml
Copy
  1. Prepare an environment variable file named .env.example and provide with your own values
  2. Run the application with the command:
YAML
Copy

Known limitation

  • When the host experiences resource limitations or degraded performance, some containers may fail to start properly. In such cases, restarting the container is recommended to restore normal operation.
Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard