Kubernetes and Security -- An Overview from RedHat_Inc and Intel Corporation

Feb 2019 darkblue_b - B Hamlin - OSGeo dot org

This all-day seminar, presented at no cost to Open Source contributors and others, featured RedHat_Inc senior speakers on a variety of topics related to Kubernetes and security, both current and roadmap.

The event was presented to an audience of about 100 professionals from a wide variety of backgrounds, commercial, government and FOSS. Primary committers on many projects were also in attendance, also mostly from RedHat_Inc.

The morning started out with the news that "security is a hard problem, and is not solved" ... A slide showing commercial security products in cloud systems showed that there has been a massive expansion of products and participants, and another showed that an increasing number of C-Level executives in technology companies are identifying security as a key-concern in recent surveys. Of course, what is not stated on-stage is the relentless news articles on data-breaches of every kind. Later in the day, a reference was made to a general worry about increasingly sophisticated attack vectors up-to and including "AI" driven activity. Couple these concerns with very popular and cost-effective "cloud" computing services, and the idea that security is important is self-evident.

In the past two years, the Kubernetes system [0], originally from Google engineering and implemented in Golang, has moved to the forefront of container-driven workflows in industry. RedHat_Inc and others had been involved with Kubernetes, and also other container systems, but K8s as it is called for short, has momentum and with it, engineering resources. It is safe to say that it is a hard problem to accomplish well, and there is little to be gained by re-inventing these kinds of plumbing-level compute mechanisms.

Overview of Current Systems - Creating Containers

Three parts to a container system workflow:

• source code that will execute in the container
• the build environment that prepares source code to run
• built containers ready to be copied and executed

Source code systems are converging on the git workflow, where each commit is a signed difference to a source code base, organized by branches, with notable commits marked with tags. The git workflow, pioneered by the Linux kernel developers and Linus Torvalds, has gained immense industry adoption when combined with a web-GUI like github, or similar projects like gitea. A feature of the git workflow is that there is an audit trail, for practical purposes resistant to forgery or modification after commits.

Secondly, the build system that creates the container code is more and more a formal system itself. It is not entirely solved, but practical methods of identifying and obtaining standard FOSS build environments is the industry direction. Specific build environments include: Java, C++/C, node.js, php, Rust and Golang.

Third, a completed container may be signed and available in a repository. Early versions of Docker had only one, hard-wired source of containers, but more recently container registries have become part of the Docker project. Due to converging industry efforts, Docker and K8s enjoy a special relationship, now and going forward in the forseeable future.

Industry or others may choose to run one or more of these processes behind firewalls privately. From a security perspective, role-based editing and access to content, and an internal audit-trail give greater control of the process.

Outside of the firewall, an ecosystem can develop "trusted" sources of components and containers, with varying degrees of formality. The Debian project is an example of a well-established trust source. Other dot-orgs can choose their participation level in the trust chain, with an associated responsibility to monitor the operations and results. In the field attacks, particularly in Windows system but also Linux and others, have shown that inserting attack code into a frequently downloaded, high-profile project is an effective attack vector.

Scale

A feature of modern network infrastructure is the ability to scale on demand. Cloud systems can scale in different ways, including:

• Security
• Capacity
• Availability
• Latency
• Portability
• Performance
• Efficiency

A short description of each characteristic is given by the speaker Chris Van Tuin, a senior technologist at RedHat_Inc. In brief, scaling is not "one size fits all" endeavor and can be architected in specific systems with these and other characteristics in mind. From a security perspective, monitoring or "visibility" can be implemented synergistically with these characteristics to meet specific security goals.

Container Technology Details

A spirited and deeply technical presentation was given by senior RedHat_Inc engineer Dan Walsh [1]. The featured twitter channel was #nobigfatdaemons. A tour of Github / containers [2] related material included skopeo, image, podman, storage, CRI-O, conmon, and buildah. A core concept of the presentation was that containers need not be monoliths, and that building containers should be a flexible process with a choice of toolchains. There is a preference from a security perspective of making containers that can run read-only, with any storage needs specifically built with finite (traceable) bounds. Eliminating the "base-image" concept is worthwhile. A useful idea in the presentation was that the execution of containers can be for different purposes, with different security obligations for each of: building; run to experiment and explore; run in production. A demonstration of alternative runtimes for containers was shown, emphasizing the Docker container definition, but flexibly reducing the privileges required for any given container to run.

Many other topics and technology chains were presented during the course of the day. Without listing exhaustively, it could be said that there are numerous, relevant technology projects over the years from RedHat_Inc and others, but that evolution and market-forces are causing each to re-justify itself with respect to K8s and Docker in this presentation. Intel Corporation was a sponsor of this event, and occasional references to Intel were uniformly positive and without critical or controversial content.

Some threat-models were discussed in light of container orchestration, and specific workflow recommendations were offered, many of which look quite similar to secure network models of past years, but with containers and K8s specific details added. There is no question that container models with networked workflows, can be executed with greater resource efficiency and other desirable properties in a cloud platform execution environment.

Many participants in the audience were not English language native speakers. There is global interest in these architectures. International FOSS organizations like OSGeo can also benefit from a good understanding of emerging standards and practices, while at the same time supporting core FOSS values and independence, on several levels. Although the companies presenting on stage have a large commercial interest in the adoption of these technology chains, in this authors opinion the presentations seemed fair and open, given the realities of this fast-moving topic.

Thanks to RedHat_Inc and Intel Corporation for sponsoring this informative event.

[0] ​https://en.wikipedia.org/wiki/Kubernetes

[1] ​https://people.redhat.com/dwalsh/

[2] ​https://github.com/containers

presentations are available here: ​https://www.brighttalk.com/summit/4516-security-symposium/

event site: ​https://www.redhat.com/en/events/security-symposium-sanjose#

topics list:

• Continuous security with Kubernetes
• Network security for applications
• Securing the network with Istio
• Security concerns in the container runtimes
• Introducing Ansible security automation
• Automating security and compliance for hybrid environments

misc video: ​https://www.youtube.com/watch?v=chT4DEYUmW8 ​https://www.youtube.com/watch?v=PgjfChchfGw&index=9&list=PLaR6Rq6Z4Iqe9xnafdhWdSgD-3qsWTm6K