Unable to deploy MetaDefender Core in Kubernetes with containerd engine 2.2.x

Summary

We have identified an issue affecting our application deployments in Kubernetes environments using containerd 2.2.0-2.2.2. This behavior is caused by a known bug in the container runtime and is not related to changes in our application.

Issue Description

A known bug in containerd version 2.2.0-2.2.2 prevents containers from being successfully created in certain Kubernetes environments. As a result, Pods may fail during startup and remain in a non-running state.

The following is an example when deploying to EKS 1.35 (the log is redacted)

Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 82s default-scheduler Successfully assigned default/md-core-7fb8c84585-vk2gt to ip-172-31-5-122.ec2.internal
Normal Created 76s kubelet Container created
Normal Started 75s kubelet Container started
Warning Failed 70s kubelet Error: failed to create containerd container: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount1122317056: openat xxx: path escapes from parent
Warning Failed 60s kubelet Error: failed to create containerd container: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount3050653287: openat xxx: path escapes from parent
Warning Failed 48s kubelet Error: failed to create containerd container: mount callback failed on /var/lib/containerd/tmpmounts/containerd-mount2061916826: openat xxx: path escapes from parent

Reference: https://github.com/containerd/containerd/issues/12683

Affected Environments:

  • ** Kubernetes 1.35 or containerd 2.2.0-2.2.2

Workaround

Until the vendor provides a fix, use one of the following mitigations:

  • Use a Kubernetes node image that does not include containerd 2.2.3 and later

  • Pin node runtime version in cluster provisioning