Managing container vulnerability risks: Tools and best practices

The sooner you can identify vulnerabilities in containers, the better, and this advice on practices and tools can help.

A network of connected virtual container blocks.
NoLimit46 / GettyImages

Containers are quickly becoming the de facto form of compute and workload deployments in the cloud-native ecosystem. The latest Cloud Native Computing Foundation (CNCF) Cloud Native Survey shows that 96% of organizations are either actively using containers and Kubernetes or are evaluating them. Containers have well-known benefits such as portability, consistency and efficiency, but they aren’t without security concerns.

Container security is a complex activity that, much like broader cybersecurity, requires a combination of people, processes and technology, with the first being the most critical. Organizations looking to shift to widespread container use should be upskilling existing staff as well as bringing in others with the necessary skill sets to ensure a secure cloud-native operating model, of which containers are a key component.

If the activities from the largest government bodies and technical authorities are any indication, the focus on software supply chain security is just heating up, requiring a level of rigor and maturity that many organizations haven’t reached yet. By implementing the practices and tools discussed below while also keeping in touch with both industry best practices and guidance, we can collectively get much closer to the desired end state of secure container use.

Containers an intertwined part of cloud security

First, it’s important to understand the role and interactions of containers in cloud environments. The cloud-native ecosystem typically has the four C’s of cloud security: cloud, clusters, containers and code. Each layer builds on the next and insecurities at any layer can impact the layers that follow, such as applications deployed on insecure containers. Vulnerabilities in the cloud, Kubernetes clusters or applications themselves can all cause their own issues, but those are out of scope here.

Container security is no trivial activity, particularly due to the states in which containers exist, such as an image or as a running container, coupled with the layers and code that can be placed within the container. The CNCF’s white paper Cloud-Native Security is a great place to start for a better understanding of cloud-native applications and containers and their life cycles.

Beware the dangers of container portability

While one of the most notable benefits of containers is their portability, this can be a vice as much as a virtue. If vulnerabilities are baked into the container and then distributed, you’ve essentially just shipped vulnerabilities to everyone who uses that image and also potentially put any environments it runs in at risk, given they generally run in multi-tenant architectures. This means the portable and distributed nature of container images being widely available and shared bring them in scope with other concerns such as open-source code and infrastructure as code (IaC), all of which can bring their own vulnerabilities.

Containers are often built by outside developers rather than traditional IT teams and then distributed to the enterprise. This means things such as secure coding practices and container security best practices are a great place to start, but what does the latter mean?

Scan containers for vulnerabilities before putting them into production

Some of the fundamental best practices that have emerged include things such as scanning containers in your continuous integration/continuous deployment (CI/CD) pipelines to prevent vulnerabilities from reaching runtime production environments. Open-source options such as Anchore and Trivvy are available as well as options from industry leaders such as Snyk.

Scan the containers during the pipeline deployment activities as part of the broader push to shift security left. Catching vulnerabilities in containers in the pipeline prevents vulnerabilities from being introduced in production environments and potentially exploited by malicious actors. This is more efficient, drives down risk, and is cheaper than fixing vulnerabilities in production.

Since many containers are created by developers in pursuit of deploying their applications, these tools can be made available to them as well, to empower them to address the issues rather than create a back and forth with a likely understaffed and taxed security team, which becomes a bottleneck to value delivery.

Scanning container images in pipelines isn’t a silver bullet, though. Container images are often stored in repositories and exist in running states once deployed to production. It is key to scan them in both environments. New vulnerabilities emerge regularly, so simply pulling a previously scanned image from a repository and deploying it without a new scan may overlook new vulnerabilities that have been published since the previous scan.

The same concept applies for vulnerabilities running in production, coupled with the reality that due to potentially poor access controls, changes may have been made to the container in a running state. Vulnerabilities in running containers can be identified and tooling can notify the appropriate staff to respond accordingly to investigate and potentially intervene.

Use container image signing

Another key activity when it comes to securing container workloads is image signing. Everyone is familiar with the CIA triad of cybersecurity: confidentiality, integrity and availability. Container image signing is akin to ensuring the integrity of the container image. It gives you assurance that the container image you’re using hasn’t been tampered with and can be trusted. This can be done as part of a DevOps workflow as well as in a registry.

Several options are available when it comes to container image signing. One of the most notable options is Cosign, which supports image signing, verification and storage. It also supports various options such as hardware, key management services (KMS), bring-your-own public-key infrastructure (PKI), and more.

Keyless signing options are beginning to emerge and are championed by innovative teams such as Chainguard. Keyless signing essentially supports the ability to use short-lived keys that only exist long enough for the signing activities to occur and are tied to identities.

Build software bills of material for container images

Containers aren’t immune to software supply chain concerns, and organizations are now looking to use tools to help generate software bills of materials (SBOMs) for their container images. One notable example is Anchore’s Syft tool. Syft lets you create SBOMs for your container images as part of CI/CD workflows and positions organizations to have a much deeper understanding of the software they have running in their container ecosystem and be well positioned to respond should another Log4j type scenario occur.

This level of visibility has traditionally been elusive but organizations are focusing more on software supply chain security, following guidance from the White House and associated federal agencies such as in the Cybersecurity Executive Order (EO). The call for increased focus on secure development practices is only increasing, with organizations such as NIST releasing an updated Secure Software Development Framework (SSDF), which calls for the use of SBOMs in activities such as archiving and protecting software releases.

Building on the need to have SBOMs for container images is the push for attestations, which are being championed by companies such as TestifySec as well as NIST in its Software Supply Chain Security Guidance. NIST calls for attestation to SSDF, which calls for the use of SBOMs. Innovative options exist to further enforce SBOM’s as well, such as Syft, which can support SBOM attestation using the in-toto specification. This attestation approach allows the signer to attest that the SBOM is an accurate representation of the container images contents.

Copyright © 2022 IDG Communications, Inc.

Make your voice heard. Share your experience in CSO's Security Priorities Study.