Akamai’s first stroke of luck was a planned—but ultimately scrapped—bonanza for the world’s most popular sport. European soccer championships were scheduled to take place between June 12 and July 12, 2020, in 12 cities across the continent. While the tournament was postponed to roughly the same dates in 2021, its presence on the sporting calendar was expected to test Akamai’s video streaming capacity, and the firm had built in headroom to accommodate it.

“We overbuilt so we’d have enough capacity on top, and we had safety margins of a couple of terabits left and right,” says Christian Kaufmann, vice president of network technology at Akamai.

However, increasing capacity to near-total usage is insufficient for shoring up reliability under extreme stress—and having extra capacity in Europe did nothing for American internet users who were also stuck at home and, among other things, spending more time on streaming services. Netflix, for instance, doubled its expected subscriber growth in the first quarter of 2020, with a reported record of 15.77 million paid subscribers globally. A key challenge Akamai faced was maintaining the reliability buffer on its CDNs while accounting for increased usage.

Akamai builds CDNs as close as possible to the end user and in consultation with internet service providers (ISPs). Minimizing latency matters, so the company offers various services to prioritize it. One service places Akamai servers in the ISP’s own network and evenly distributes them geographically, or places them in a central data center close to the ISP. TCP handshakes are a vital element of smooth data downloading, and millisecond-level latency can have an impact on the amount of data users can download at the expected speed. If you’re visiting a website in the U.S., it doesn’t make a meaningful difference whether an end user in New York receives their data from a center down the road or in Milwaukee, says Kaufmann, “but if you’re talking about an HD or 4K video, it pretty much has to be in the same country, ideally in a nearby metro [area].”

What enabled Akamai to maintain high reliability is that its servers are spread around the world. Traffic that requires reliable, close, low-latency connections is routed to nearby CDNs; general web traffic or large patches for operating systems or game downloads go through more remote servers, called spillover or overflow servers.

“As long as you have enough servers and bandwidth, which we had because we built it ahead [of time], the platform in itself is just [pointing] you in the right direction,” says Kaufmann, adding that the principle of building bases worldwide “scales very well.”

Still, during the early days of the pandemic—as the world, and working practices, first entered a state of flux—the increase in usage ate into the buffer Akamai likes to keep around its CDNs.

Pre-pandemic, Kaufmann says, the solution to depleted buffer capacity was relatively straightforward: build some new servers, and build out existing ones to bolster capacity. But that strategy relies on the vagaries of international trade, and turns a problem of handling, directing, and transferring data into a problem of handling, directing, and transferring physical parts. The spare capacity they’d built out for the soccer tournament could only buy them time—at some point, Akamai would have to rely on physical infrastructure changes.

“We need servers, routers, switches,” says Kaufmann. “We have to put them in data centers, and all of them have challenges.”

The majority of hardware suppliers for the world’s data centers are based in China, where a nationwide quarantine in early 2020 brought everything, from production on factory lines to distribution of key parts, to a halt.

Akamai had some additional hardware capacity—spare servers in warehouses it owned, in countries that weren’t under such restrictions—but they quickly used it up. The company started holding weekly internal supply chain calls to identify when and where parts were being detained. Getting the new servers and their complementary parts remained unpredictable as pandemic restrictions shifted.

When the servers were finally built, installation in data centers posed a new challenge. “In some countries, we were regarded as critical infrastructure and were allowed to go to the data center and build stuff,” says Kaufmann. “In others, we had to wait until the lockdown lifted. You end up with big lists of countries and cities, [what you’re allowed to do where], and you try to manage that, because it doesn’t make sense to ship a server where you can’t install it.”

To reduce inefficiency and keep servers online, Akamai built models to identify countries it believed would shortly enter or leave quarantine, trying to time deliveries to coincide with when they’d be able to access data centers. “It was clear that every country would get hit sooner or later, and most of them would go into lockdown,” he says. “You’re trying to manage that just in time.”