Once upon a time, I worked at a storied institution that arguably invented the architectural pattern of traffic shaping at scale. To this day, it’s responsible for guiding packets for the majority of captive data centers on the planet. If you use a cell phone in the United States, its hardware is likely directing service requests for your cell carrier.
After many years and much success in global and local traffic management, the company’s customers began to shift from hardware-based load-balancing solutions to software. Open-source solutions like HAProxy, Nginx, and Envoy were not only gaining in popularity, but were quickly becoming the default traffic-shaping primitives the hyperscalers—such as AWS, Azure, and Google Cloud Platform—offered to their customers. Seeing this as an existential threat to continued growth and profitability, the company embarked on a “digital transformation” in an effort to reposition itself, strengthen its product-market fit, and make its internal culture more responsive to the clear and present danger of free and open-source software.
Among the projects I supported was a migration of the company’s internal billing systems to support a nascent subscription model more closely aligned with the expectations of companies native or migrating to the hyperscalers. Eventually, we found ourselves debating whether the new billing system should leverage containers for service delivery.
You don’t need a container migration strategy—you need leadership.
I found this confusing. Why were we debating a technology solution before understanding the data driving its selection? A rigorous design should start with first principles and an understanding of the functional and nonfunctional requirements before a single architectural decision is made. This might include a series of foundational questions, such as how many consumers may be using the service concurrently at any given time and which scale factors, or the parts of the system driving scale, are most critical. The team working on this new billing system had clarity on why the change was necessary, and they’d applied the appropriate rigor to understanding external forces, like their customers’ expectations. But in the container debate, they lacked that same clarity on why this particular design was superior to others.
I see now that the source of contention was not just technical. Rather, it was a problem of changing mindsets and learning to adapt to environmental pressures, a problem I suspect many companies undergoing a digital transformation face. This experience proffered a pearl of wisdom I’d like to share with you: You don’t need a container migration strategy—you need leadership.
To change, or not to change
Change management is the practice of preparing, supporting, and helping individuals, teams, and organizations to handle change. When that change is technical, it also involves the standardization of methods and procedures for the efficient and prompt handling of changes to infrastructure. The aim is to minimize the number of incidents—both organizational and technical—during a transition, as well as their impact. An effective leader will guide and support their team through a transition with focus and sensitivity, but they’ll also recognize when a change is not the right approach because the trade-offs outweigh the benefits.
Reflecting on the container debate I just mentioned, I believe the team wasn’t engaging rigorously enough on the fault domains and failure modes a container solution would impose on the company’s technical and organizational system. How a team prepares itself to manage and mitigate unforeseen failure modes is critically important. Containers provide service isolation and reduce complexity with respect to resource contention and conflicting dependencies, but that doesn’t mean they’ll automatically support your application’s use case or be immediately supportable by your team given its current capabilities and capacity.
All change comes with costs, and it’s a leader’s job to tally them and evaluate their impact. Containers might objectively provide for an accelerated integration and deployment pipeline, but would require learning how to manage such a framework effectively, including hiring for and developing the talent required to build and scale such services. For an organization that has invested in enabling infrastructure that already meets their customers’ needs, that cost may be significant.
Containers contain, leaders lead
Containers are a reasonable solution to the classic “works on my machine” problem. Building an isolated set of service resource dependencies in an easily transportable environment can prevent issues of library incompatibility, misaligned upgrades, and missing artifacts. Furthermore, if services can be decomposed into modular units with known boundaries and interfaces, then perhaps more effort can be directed toward understanding the nature of your business and the constraints of your application.
The obvious question, then, is when to choose one target production deployment, such as bare metal, VMs, or containers, over another. Peter Honeyman, a friend of mine and a professor at the University of Michigan, worked on persistence layer theory (read: databases) while obtaining his PhD in computer science at Princeton. His answer to this question, and a fairly comprehensive summary of his successfully defended thesis, would be: It depends—on a whole host of factors. The story I shared earlier touches on the importance of understanding how your application is intended to behave and how your users might actually use it. Beyond that, tougher questions start to emerge.
First, you need to understand what your team is ready to manage operationally. Containers may give you development velocity and some guarantees about the consistency of your target environment, but they don’t automatically solve issues of security, efficiency, or observability. Second, your team needs to figure out how to construct a continuous integration and deployment environment that supports the desired target velocity, as well as what the network topology of your service fabric or mesh will look like. They also need to learn how to protect the container orchestration surface, plus everything inside the container. These are critical concerns that require rigorous debate to separate technology fads from actual business use cases.
You may disappoint some of your engineers, but you’ll ultimately make better business decisions.
Jeff Lawrence, founder and managing director of Organizational Agility Advisors, notes that leadership is “disappointing your own people at a rate they can absorb.” When evaluating a change, it’s important to understand how the discussion about using a particular technology came up in the first place. It might have resulted from an organizational mandate, a groundswell of engineering interest, or a mix of both. But it needs to be rooted in a well-reasoned set of criteria for improving the state of application delivery, tailored to your specific business use case. Furthermore, it should be anchored in your team’s ability to manage your migration end state, and laser-focused on the needs of your users. True rigor requires strong leadership. You may disappoint some of your engineers, but you’ll ultimately make better business decisions.
Leadership is a superpower
This article is really about you, dear reader, and what you’re capable of. You wouldn’t have read this far if you weren’t curious. You wouldn’t have chosen this field if you weren’t talented. My gift to you now is my belief that you are a leader others will follow—whether you’re currently in a leadership role or not. Consider this your nudge to lean into the challenges and opportunities sound leadership enables.
Containers are just another word for cgroups and a few associated pieces of geekery created by Paul Menage and others in the early 2000s. They’re not what’s important. What’s important is supporting your team through the shifts and inflection points that will ultimately benefit your users, satisfy your stakeholders, and support your business’s growth, while also sustaining the health, inclusivity, and sustainability of your team and its processes. These matter more than the particulars of migrating to containers.
Maybe—maybe!—you do need a container migration strategy. But don’t over-index on that notion. Instead, harness the power that’s gotten you this far to help your team effectively deal with change. Give that power a name and use it judiciously. It’ll pay dividends far beyond any technical investment.