I’ve read a lot of articles recently that seem to suggest containers kinda suck. And, well, sure. They don’t necessarily make code faster, or more secure, or much easier to debug. They require complex orchestration systems, and you can’t even use them to test your code on a different OS. Gone are the days of evangelist posts about how containers are the answer to all of life’s problems; now the think pieces have titles like, “Why You Don’t Need Docker,” and “I Did This One Weird Trick and Still Hit Kubernetes’s Scaling Limits.”
But this brand of container fatigue is largely a product of how we’re using them. I’ve often seen containers referred to as “lightweight VMs,” yet it’s this focus on containers as a form of virtualization that leads to a lot of the angst. Yes, there are parallels and points of overlap, but containers are most useful when we think of them as a tool for solving a related, but different, set of problems. They are, in fact, exceptionally helpful—as long as you’re not trying to use them to mimic an entire operating system.
If a VM is like a house, then a container is like a tent.
Think of it this way: If a VM is like a house, then a container is like a tent. You can try to fit all your furniture inside, but given the constraints of matter and spacetime, you’re better off only trying to fit the things you need. Tents are lightweight—quick to put up and pull down—but you’ll probably feel the wind shaking the tent poles. Rain won’t get in, but a tent won’t keep out the cold like brick walls and a roof. And with enough effort, you’ll be able to claw your way out, even if the tent flap is stuck shut.
If you’re trying to stuff a tent with a two-bedroom-townhouse-with-attached-garage amount of stuff, then maybe what you really need is a two-bedroom townhouse with an attached garage. Just like we can compile code to statically link all dependencies into a single binary, containers offer a way to statically link all the runtime dependencies your code might have. And just like you wouldn’t want your compiler to include the whole OS, container images shouldn’t be OS replicas packaged up with your application code.
The more binaries and files there are in a container image, especially ones that provide interfaces to the kernel, the more opportunities there are to claw through that tent wall and escape the container. This is especially relevant when running many containers on the same host or VM. Tents, after all, aren’t a particularly secure place to store your valuables. Your valuables in a tent in your living room, however? Pretty secure. Your valuables in a tent in your living room next to someone else’s tent? Well, it really depends on who (or what) is inside that tent waiting for you to pop out for lunch.
From a security perspective, containers are useful because they provide a very limited attack surface (that is, a much smaller number of binaries that can be exploited), along with a way to version everything your program needs to run, in addition to the code itself. If you bloat the container image, you lose this benefit and end up with a less secure, slower approximation of an operating system. Much as you’d likely choose to live in a two-bedroom townhouse over a tent, if what you need is a lightweight operating system, containers aren’t your best option.
No one would argue that tents serve the same purpose as brick-and-mortar houses—nor that they should!
Finally, there’s the whole business of resource isolation. While cgroups are pretty neat as an isolation mechanism, they’re not hardware-level guarantees against noisy neighbors. Because cgroups were a later addition to the kernel, it’s not always possible to ensure they’re taken into account when making system-wide resource management decisions. The bottom line: If you can’t tolerate the noise, perhaps don’t invite people to camp out in your living room.
No one would argue that tents serve the same purpose as brick-and-mortar houses—nor that they should! Containers make terrible VMs. (And, to be fair, VMs are equally poor replacements for containers.) If we insist on judging a tent as a house, or a container as a VM, then it’s always going to fall short. I’d like to propose that we reframe their utility: Instead of lightweight VMs, let’s think of them as heavyweight process wrappers. By cordoning off your process from the OS, wrapping it up in namespaces and permissions, and bundling it together with its runtime dependencies, containers provide a powerful execution primitive. As long as we don’t expect foundations that go down to the bedrock, containers can be incredibly valuable tools. As with any technology, the trick is knowing when to use them.