Troubleshooting Containers

Kubernetes

How to check the logs with Kubectl

  • In case the md-icapsrv pod does not initialize we need to check the pod logs to see why is failing

    1. The worker node running the md-icapsrv pod doesn’t have enough resources allocated for the pod. (Pending state)

    2. Provide the License Key in file values.yml if not provided the health check will fail as the container cannot be activated

    3. Need to set value ACCEPT_EULA is true if the application can not start with reason accept EULA.

    4. If you have enabled TLS to need to uncomment scheme: HTTPS in health check. The key of the secret need to mapping with the field .certSecretSubPath and .certKeySecretSubPath

kubectl logs <pod_name> -n <namespace>

Start a Bash session

  1. Find the MetaDefenteder ICAP Server pod with the command

kubectl get pods -A

→ pod name: md-icapsrv-77c4654f4b-bkclc

→ namespace: default

  1. Start a Bash session

command: kubectl exec -n <namespace> -it -c <container name> <pod name> bash E.g: kubectl exec -n default -it -c md-icapsrv md-icapsrv-77c4654f4b-bkclc bash
Info

Default container name when you use MetaDefender ICAP Server’s Helm chart is md-icapsrv

Collect support packages

There are two different support packages that can be generated.

  1. Support package from local machine

  2. Support package with script within the pod

Support package from local machine

This is the easiest way to have a first idea of what could be happening in the environment. It will collect

  • Logs from running pods

  • Logs from previous pods

  • Configmap

To run this script from your local machine please see metadefender-k8s/troubleshooting at main · OPSWAT/metadefender-k8s

./sp.sh <NAMESPACE>

Support package with script within the pod

After accessing to the bash of the container. To create a package you must start the script found under

${INSTALL_ROOT}/usr/bin/mdicapsrv-collect-support-data.sh

As the script processes the necessary information, the script generates the support package output.

The package file is a tar.gz archive with the following name:

mdicapsrv--support-<TIMESTAMP>.tar.gz

Where the timestamp was the date when the package was generated.

Example:

mdicapsrv--support-1660647413.tar.gz

The generated package will be placed in the same location as the script that was called.

Warning

If you run container with read-only file system mode or non-root user make sure you have write permission to the working directory when you execute collect support packages script

Copy the support package from the container to outside

Use kubectl cp Kubectl Reference Docs (kubernetes.io)

# Copy /tmp/foo from a remote pod to /tmp/bar locally kubectl cp <some-namespace>/<some-pod>:/tmp/foo /tmp/bar

Example:

kubectl cp default/md-icapsrv-77c4654f4b-bkclc:/mdicapsrv--support-1660648209.tar.gz /home/icap/mdicapsrv--support-1660648209.tar.gz

Docker

Collect logs

You can collect the logs of the container running

docker logs <container_name>

Collect support package

Start a Bash session

  1. Find the CONTAINER ID by command

docker container ls -a

E.g: the container ID = d1769cb64afe


  1. Start a Bash session with the command docker exec -it <container id> bash

docker exec -it d1769cb64afe bash

Support package with script within the pod

To create a package you must start the script found under

${INSTALL_ROOT}/usr/bin/mdicapsrv-collect-support-data.sh

As the script processes the necessary information, the script generates the support package output.

The package file is a tar.gz archive with the following name:

mdicapsrv--support-<TIMESTAMP>.tar.gz

Where the timestamp was the date when the package was generated.

Example:

mdicapsrv--support-1660647413.tar.gz

The generated package will be placed in the same location as the script that was called.

Warning

If you run container with read-only file system mode or non-root user make sure you have write permission to the working directory when you execute collect support packages script

Copy the support package from the container to outside

Docker

Use docker cp | Docker Documentation docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH|-

Example:

docker cp d1769cb64afe:/mdicapsrv--support-1660647413.tar.gz /home/icap