Troubleshooting

To perform load testing, we recommend using the Gatling tool which you can find on https://github.com/OPSWAT

How to check the logs with Kubectl

• We can check the logs of the different containers running for the MetaDefender Core pod.

  • For init container "check-db" run kubectl logs [-n <namespace> ] <pod-name> -c check-db-ready
  • For app container "md-core" run kubectl logs [-n <namespace> ] <pod-name> (No container needed to be indicated)

• To describe the pod and see all the configurations of it.

  • kubectl describe [ -n <namespace> ] <pod-name>

How to collect logs generating a support package

For generating and exporting the logs we need to run the following commands

# Generate support package, it will log the support package name at the last line (<suppor_package_name>)

kubectl exec <md-core_pod_name> -- bash -c "cd /opt/ometascan/core_data/ & /opt/ometascan/usr/bin/ometascan-collect-support-data.sh"

​

#export support package, replace <namespace>, <md-core_pod_name> & <support_package_name>

kubectl cp <namespace>/<md-core_pod_name>:/opt/ometascan/core_data/<support_package_name> <support_package_name>

Installation Script is failing

• Follow pre-install requirements in Installing MetaDefender Core in K8S
• Message Error "INSTALLATION FAILED: found in Chart.yaml, but missing in charts/ directory: aws-load-balancer-controller".
  • Run helm dependency update

Nginx Ingress Issues

Problems found:

1. Tls is not enabled for ingress
2. Controller is not picking up the changes of the services

TLS not enabled

How to debug

• Check ingress rule deployed and name of the tls secret that contains the certificate and key

Solution Proposed

• Create tls secret if not created and add the secret name matching the tls secret created. See TLS configuration

Ingress Controller not picking up the changes

How to debug

• Check the logs of the ingress controller pod. Search for any error or warning that could point to the root cause

Solution Proposed

• If error is not clear try to find in the ingress controller user guide to review the configuration.

Errors Connecting to Database

Problems found

1. MetaDefender Core cannot connect to database
2. PostgreSQL extenstion is not installed

Cannot connect to database

How to debug

• MetaDefender Core init container is not finishing. Check logs of check-db init container to see if the database if reachable.
• If MetaDefender Core init container is terminated means that the database could be reached with the configuration provided. In this case it is needed to check the MetaDefender Core app container and search for 'connection error'

Solutions Proposed

• Check DB_HOST in mdcore-env configmap
• Check user and password values in mdcore-postgres-cred secret

PostgreSQL extension not installed

How to debug

• Check logs of MetaDefender Core pod and search for éxtension missing' message.

Solutions Proposed

• There are some extensions that are needed to be installed, sometimes they are installed by the admin user that MetaDefender Core use for connecting to the database but other times there are some database services in the CSPs that does not allow this kind of actions and the extensions have to be installed from the CSP console.

Error Activating MetaDefender Core license

Problems found

1. MetaDefender Core pod does not get activated

MetaDefender Core does not get activated

How to debug

• MetaDefender Core pod is restarting as it does not get activated because of the livenessprobe is failing. By default the health check endpoint /readyz is checking if the pod is activated, so if this fails the pod will get restarted.
• It is needed to review the MetaDefender Core pod logs to see what is the specific error, it could be things like
  • Not enough activations left
  • Activation Key expired

Solutions Proposed

1. Each activation key has a number of slots/Core that can be activated and generate a deployment id, if your reach the limit could be because either any of the deployments was not deactivated properly or because you need to get more Core licensed in your activation key

   1. Check the sidecar container activation-manager logs to see if there is any error on deactivation
   2. Contact support if there is any error in the logs, or you need to extend the number of slots/Core

2. In case it is saying that the activation key is expired, please contact Sales to extend your license

MetaDefender Core Engines Errors

Problems found

1. Engines does not get activated

Engines does not get activated

How to debug

• Check, in the MetaDefender Core console, the configuration under Inventory > Modules > Metascan. There would be an error indicating that the engines are failing to be activated or installed.
• Check the logs of the MetaDefender Core pod and search for errors when installing the engines. "Waiting for init message timeout, no response from engine"

Solutions Proposed

• Review the resources allocated to the MetaDefender Core pod. The limit resources settings in K8S must meet the recommended system in the documentation (for more details: link)

IKARUS engine

For IKARUS, --shm-size=1gb should be added when starting the docker.

If you're using K8S, the section below should be also specified in the yaml file:

spec:

  volumes:

  - name: dshm

    emptyDir:

      medium: Memory

  containers:

  - volumeMounts:

      - mountPath: /dev/shm

        name: dshm