Scan Priorities & Throughput Tuning

This guide explains how MetaDefender Storage Security (MDSS) controls how fast your storages are scanned, and how you can tune that speed. It covers three settings:

• Scan Priority (Low / Medium / High)

• Prefetch Count (RABBITMQ_DISCOVERY_PREFETCH_COUNT, RABBITMQ_SCANNING_PREFETCH_COUNT, RABBITMQ_REMEDIATIONS_PREFETCH_COUNT)

• Consumer Multiplier (RABBITMQ_CONSUMER_MULTIPLIER)

No prior knowledge of the internals is required — everything is explained with a simple analogy and worked examples.

How scanning works (in one minute)

When MDSS scans a storage, the work moves through an internal pipeline made up of three stages:

1. Discovery — MDSS walks through your storage and finds every file that needs to be scanned.

2. Scanning — each file found is then scanned for threats.

3. Remediation — when a file needs action (for example, a threat is found), MDSS applies the configured action to it, such as moving, deleting, or quarantining the file.

All three stages process files using parallel workers. The settings in this guide simply control how many workers run and how much work each worker holds at a time.

1. Scan Priority (Low / Medium / High)

Every scan you start is given a priority. The priority decides how much processing power that scan receives relative to other scans. Higher-priority scans are automatically assigned more parallel workers, so they finish faster.

Key things to know:

• Low is the default. If you don't choose a priority, the scan runs as Low.

• Priority is set per scan. Two scans on the same storage can run at different priorities at the same time.

• Higher priority does not pause lower-priority scans. All priorities run in parallel — High simply gets many more workers, so it drains its queue much faster and effectively finishes ahead.

2. Prefetch Count

What it is: the number of files a single worker pulls and holds at one time before reporting them as done.

In the moving-team analogy, this is how many boxes each mover carries per trip. Carrying more per trip means fewer empty-handed walks back — the mover keeps working instead of waiting for the next box.

Effect of the value:

• Too low (e.g. 1): each worker finishes one file, then waits to be handed the next. Small idle gaps add up and reduce throughput.

• Balanced (e.g. 10–50): the worker always has the next files ready, so it never sits idle. Recommended for most deployments.

• Too high (e.g. 400): a single worker can hoard a large batch of files while other workers sit idle, and memory usage rises. Rarely beneficial.

3. Consumer Multiplier

What it is: how many workers the service runs in parallel. This is how many movers you put to work at once.

The important detail: the multiplier is a base number that is automatically scaled up for higher priorities. You set one value, and MDSS applies it proportionally:

>

When to increase it: when scans are running slower than you'd like, files are clearly piling up, and your server still has spare CPU and memory. Each extra worker scans another file in parallel.

When to leave it at the default (1): most deployments. Raising it above 3–5 is rarely necessary and increases load on the system. Any value above the maximum (10) is automatically capped.

4. Putting it all together

The total number of files a stage can process at once is:

Remember that the number of workers depends on both the multiplier and the priority (Low ×1, Medium ×3, High ×6).

A step-by-step tuning walkthrough

The best way to understand these settings is to follow a real tuning session. Imagine a customer scanning a large file share. We start with the out-of-the-box defaults and adjust one thing at a time, checking the effect after each change.

Iteration 0 — Start with the defaults

Files the Scanning stage can process at once, per priority:

Iteration 1 — Increase the prefetch count

Each worker is finishing its files and then waiting for the next batch. We give every worker a bigger armful by raising the prefetch count from 10 to 20:

Iteration 2 — Increase the consumer multiplier

We add more workers by raising the multiplier from 1 to 2. This doubles the workers at every priority:

Iteration 3 — Tune the other stages if needed

Scanning is no longer the bottleneck, but the customer notices that after scanning, acting on flagged files (remediation) is lagging behind. Each stage has its own prefetch count, so we raise the remediation stage too:

When to stop

Keep increasing only while the queues are backing up and the server still has spare CPU and memory. The moment CPU approaches 80%, stop — adding more workers to a maxed-out server makes things slower, not faster. At that point the answer is more hardware, not bigger numbers.

5. Quick reference

Default and maximum values

Important notes

• These settings increase parallelism, not raw machine speed. They help only when workers are waiting for work while your CPU and memory still have room to spare. On a server that is already at full CPU, raising these values will not help — add more resources instead.

• Changing the environment variables requires a service restart to take effect. Set them in the service's environment configuration (.env / customer.env).

• Values above the maximum are automatically capped to the limits shown above.