Best practices

Although it is possible to install Redis Caching Server, RabbitMQ Message Broker and Postgres Database Server and MetaDefender Distributed Cluster File Storage on the same machine, they should be installed separately on various machines to optimize their performance.

Redis Caching Server

The caching server consumes a large amount of memory while operating; hence, a machine with ample and high-speed memory is best suited to this component.

RabbitMQ Message Broker

The broker is one of keys that powers MetaDefender Distributed Cluster architecture, it ensures tasks are delivered to MetaDefender Core instances in an equitable manner, delivering a "broken" task to a healthy MetaDefender Core instance and spreading tasks to new instances if more MetaDefender Core instances are added to system. For that reason, the broker should be hosted on a separate machine.

Postgres Database Server

MetaDefender Distributed Cluster database is split into three main clusters.

Data Lake stores scan results and other details related to requests such data_id, hashes, etc.

Since Data Lake is shared among MetaDefender Core and API Gateway instances, it should be hosted on a large-volume and high-speed disk. The network is also essential to Data Lake; a high-speed network is necessary.

Data Warehouse, which prepares materials for building executive reports, uses a single connection to Data Lake and collects data periodically.

Since executive reports may be stored for a long period of time for Control Center to access, Data warehouse should be hosted on a large-volume machine.

Control Center-related database, storing all user details, configurations and other settings. Control Center database can be hosted on the same machine as Control Center.

MetaDefender Distributed Cluster File Storage

File Storage is shared among MetaDefender Core and API Gateway instances. The server consumes a large amount of disk to store the submitted files from instances. Since all file-related traffic goes through File Storage, a high speed network is essential. Rocky 9.0 is recommended to host File Storage.

MetaDefender Distributed Cluster API Gateway

Due to differences in Operating System and Nginx support on Windows and Linux, API Gateway should be hosted on a Linux machine running Rocky 9.0 for high throughput of file scan submissions.

MetaDefender Core

One of strong aspects of MetaDefender Distributed Cluster is that it can support a hybrid architecture in which API Gateway and File Storage instances may be hosted on Linux while MetaDefender Core instances can be on Windows. Therefore, based on customer requirements, MetaDefender Core instances can be hosted on Windows or Linux machines.