NGINX Integration Module

MetaDefender ICAP Server integrates with NGINX via the upstream module (reverse proxy) and related configuration directives.

The integration is enabled via a certified NGINX Dynamic Module.

Prerequisites

  • You are running NGINX version 1.20.2 or above. The module supports both NGINX Community and NGINX Plus.

  • MetaDefender ICAP Server version 5.1.0 or above is installed and licensed, and relevant NGINX integration settings are enabled.

  • The integration uses a bridge module (called "Ometascan NGINX module") between your NGINX web server and MetaDefender ICAP Server.

    • The module installer can be downloaded on My OPSWAT Portal - Product Downloads > MetaDefender ICAP Server > Modules and Utilities section.

on RHEL or CentOS, httpd_can_network_connect must be enabled by command:

setsebool -P httpd_can_network_connect 1

More details about httpd_can_network_connect, please refer to https://www.nginx.com/blog/using-nginx-plus-with-selinux/#Issue-1:-Proxy-Connection-is-Forbidden

Flow Diagrams

Directives

Forward

ometascan_pass

Syntaxometascan_pass URL;
Default
Contextlocation

Sets the protocol, address and optional URI of the proxied MD ICAP server. “http” or “https” protocols can be specified. The address can be specified as a domain name or IP address, and an optional port:

ometascan_pass http://icap_server:8080;

If an error occurs when sending a sub-request to the MD ICAP Server (timeout, network issue, etc) an error will be sent back to the ICAP client, and the request will be blocked.

When using ometascan_pass, the proxy_request_buffering of the Nginx proxy module will be disabled.

When the sub-request to MD ICAP Server error (Timeout, Network, etc). The error will respond to the client, and the request blocked.

When allowed with sanitized the filename parameter of Content-Disposition header will be renamed corresponding Metadender Core's CDR setting

ometascan_methods

Syntaxometascan_methods <list methods>;
Defaultometascan_methods GET HEAD POST PUT PATCH DELETE;
Contextlocation

This directive specifies HTTP request methods that are considered by ometascan_pass. HTTP request methods not listed will be ignored. The following HTTP methods are allowed: GET, HEAD, POST, PUT, PATCH, and DELETE

ometascan_send_timeout

Syntaxometascan_send_timeout time;
Defaultometascan_send_timeout 60s;
Contextlocation

Sets a timeout for transmitting a request to the proxied server. The timeout is set only between two successive write operations, not for the transmission of the whole request. If the proxied server does not receive anything within this time, the connection is closed.

Time intervals can be specified in milliseconds, seconds, minutes, hours, days and so on. Refer here for more information.

Time intervals can be specified in milliseconds, seconds, minutes, hours, days and so on, using the following suffixes:

ms milliseconds

s seconds

m minutes

h hours

d days

w weeks

M months, 30 days

y years, 365 days

Ref: Configuration file measurement units (nginx.org)

ometascan_connect_timeout

Syntaxometascan_connect_timeout time;
Defaultometascan_connect _timeout 60s;
Contextlocation

Defines a timeout for establishing a connection with a proxied server. Note that this timeout should not exceed 75 seconds.

ometascan_read_timeout

Syntaxometascan_read_timeout time;
Defaultometascan_read__timeout 60s;
Contextlocation

Defines a timeout for reading a response from the proxied server. The timeout is set only between two successive read operations, not for the transmission of the whole response. If the proxied server does not transmit anything within this time, the connection is closed.

ometascan_x_forwarded_for

Supported since version 1.1.0

Syntaxometascan_x_forwarded_for on/off;
Defaultometascan_x_ forwarded_for off;
Contextlocation

ometascan_intercept_errors

Syntaxometascan_intercept_errors on/off;
Defaultometascan_intercept_ errors off;
Contextlocation

Determines whether ometascan blocked responses should be passed to a client or intercepted and redirected to nginx for processing with the error_page directive.

ometascan_allow_bad_request_traffic

  • This will allow bad request traffic go to the protected system
  • Only enable this directive for testing purpose
Syntaxometascan_allow_bad_request_traffic on/off;
Defaultometascan_allow_ bad_request_traffic off;
Contextlocation

The ICAP Server will respond to the 400 Bad Request when receives an invalid HTTP RFC request from the ometascan module. By default, the module will respond to 400 Bad requests from the client. Enabling this option the module will allow requests whose bad requests without scanning

  • Supported since Ometascan NGINX module v1.2.0
  • Must use MD ICAP Server v5.4.0 or newer
  • The ometascan_allow_bad_request_traffic on required the ometascan_pre_cache on

ometascan_set_header

Syntaxometascan_set_header field value;
Default
Contextlocation

Allows redefining or appending fields to the request header passed to the proxied server. The value can contain text, variables, and their combinations. These directives are inherited from the previous configuration level if and only if there are no ometascan_set_header directives defined on the current level. By default, no field is redefined.

If the value of a header field is an empty string then this field will not be passed to a proxied server:

ometascan_set_headerAccept-Encoding "";``

Caching

ometascan_pre_cache

Syntaxometascan_pre_cache on/off;
Defaultometascan_pre__cache off;
Contextlocation

This configuration will not work when AppProtect is used and put on same scope

Turn on/off pre-caching request when sending to MD ICAP Server.

When the cache is enabled the module will make a copy of the request on the NGINX server. And send it to the back-end when received allowed message from MD ICAP Server

ometascan_pre_cache_size

ometascan_pre_cache_size is deprecated since Ometascan NGINX module v1.2.0

Syntaxometascan_pre_cache_size size;
Defaultometascan_pre_cache_size 9223372036854775807;
Contextlocation

Config maximum caching size per request. Sizes can be specified in bytes, kilobytes (suffixes k and K) or megabytes (suffixes m and M), for example, “1024”, “8k”, “1m”.

SSL/TLS

ometascan_ssl_trusted_certificate

Syntaxometascan_ssl_trusted_certificate file;
Default_
Contextlocation

Specifies a file with trusted CA certificates in the PEM format used to verify the certificate of the proxied HTTPS server.

ometascan_ssl_verify

Syntaxometascan_ssl_verify on | off;
Defaultometascan_ssl_ verify off;
Contextlocation

Enables or disables verification of the proxied HTTPS server certificate.

ometascan_ssl_verify_depth

Syntaxometascan_ssl_verify_depth number;
Defaultometascan_ssl_ verify_depth 1;
Contextlocation

Sets the verification depth in the proxied HTTPS server certificates chain.

ometascan_ssl_protocols

Syntaxometascan_ssl_protocols [SSLv3] [TLSv1] [TLSv1.1] [TLSv1.2] [TLSv1.3];
Defaultometascan_ssl_ protocols TLSv1.1 TLSv1.2 TLSv1.3;
Contextlocation

Enables the specified protocols for requests to a proxied HTTPS server.

ometascan_ssl_ciphers

Syntaxometascan_ssl_ciphers ciphers;
Defaultometascan_ssl_ ciphers DEFAULT;
Contextlocation

Specifies the enabled ciphers for requests to a proxied HTTPS server. The ciphers are specified in the format understood by the OpenSSL library.

The full list can be viewed using the “openssl ciphers” command.

ometascan_ssl_session_reuse

Syntaxometascan_ssl_session_reuse on | off;
Defaultometascan_ssl_ session_reuse on;
Contextlocation

Determines whether SSL sessions can be reused when working with the proxied server. If the errors “SSL3_GET_FINISHED:digest check failed” appear in the logs, try disabling session reuse.

Required:

  • The header contains a key Connection with the value keepalive.
  • Enabled persistence connection of the MD ICAP Server (Global settings)

E.g: proxy_set_header Connection 'keepalive';

ometascan_ssl_name

Syntaxometascan_ssl_name name;
Defaultometascan_ssl_ name <ometascan_pass_host>;
Contextlocation

Allows overriding the server name used to verify the certificate of the proxied HTTPS server and to be passed through SNI when establishing a connection with the proxied HTTPS server.

By default, the host part of the ometascan_pass URL is used.

ometascan_ssl_server_name

Syntaxometascan_ssl_server_name on | off;
Defaultometascan_ssl_ server_name off;
Contextlocation

Enables or disables passing of the server name through TLS Server Name Indication extension (SNI, RFC 6066) when establishing a connection with the proxied HTTPS server.

Example Config

Copy
  • client_max_body_size Sets the maximum allowed size of the client request body. If the size in a request exceeds the configured value, the 413 (Request Entity Too Large) error is returned to the client. Please be aware that browsers cannot correctly display this error. Setting size to 0 disables checking of client request body size.
  • ometascan_pass Set URL of the MD ICAP Server
  • ometascan_methods POST PUT; Only scan POST and PUT methods

Integration Steps

Get installation file of ometascan nginx module

Download from My OPSWAT Portal - Downloadable Utilities > NGINX Module Library section

Command Line Install

CentOS

Bash
Copy

Debian

Bash
Copy

Custom NGINX Configuration Directives

This is to load the new NGINX integration module and configure all relevant settings in the NGINX configuration file (e.g. /etc/nginx/nginx.conf).

Example 1:

Copy
  • client_max_body_size Sets the maximum allowed size of the client request body. If the size in a request exceeds the configured value, a 413 error (Request Entity Too Large) is returned to the client. Please be aware that browsers cannot correctly display this error. Setting size to 0 disables checking of client request body size.
  • ometascan_pass Set URL of the MD ICAP Server
  • ometascan_methods POST PUT; (only enables scans for POST and PUT methods)

Example 2 (Load Balancing):

Copy

Configure HTTPS

Generate the certificate

Copy

HTTPS Example Config

Bash
Copy

Add ssl, ssl_certificate and ssl_certificate_key variables. Refer here for more information regarding nginx HTTPS configuration.

Performance Testing Results

Performance test results are performed in a controlled environment and serve only as a reference. The results demonstrate the raw throughput capacity for the MD ICAP Server, outside of any scanning activity performed by MetaDefender Core. Overall solution performance will depend on several factors including available system resources, file content (dataset) and network performance.

Test Environment

ComponentValue
CPU cores8
RAM8 GB
Storage50 GB
CPU limit4000MHz
Squid cachingNo caching
# files in dataset542 (500KB-1MB)
Dataset size (run 10x)378MB
Simultaneous users (threads)100
Total requests542000

Results

MetricClient - BackendClient - Squid - Backend*Client - Squid - MD ICAP - Backend *Client - Nginx - MD ICAP - Backend
Request throughput (req/s)105.991.817.865.6
Data throughput (kB/s)75629655491300046861
Total time (sec)51185905297748260
Type to search, ESC to discard
Type to search, ESC to discard
Type to search, ESC to discard
  Last updated