Netflix
13,000+ employees
Kathryn Koehler, director of productivity engineering
Over the past year, we’ve adopted formal product management for our central productivity org. Previously, planning was somewhat ad hoc, based on a bottom-up understanding of our customers’ needs gathered by engineering managers and individual contributors through conversations and support interactions.
With our product managers’ guidance, we now have a comprehensive strategy grounded in top customer needs, industry insights, and our overall mission of evolving our infrastructure, platforms, and tooling to support Netflix’s streaming and studio development teams. We prioritize customer issues based on the RICE framework (reach, impact, confidence, and effort), plan on semester boundaries (Q2/Q3 and Q4/Q1), craft objectives that align with our overall org strategy, and incorporate the work that “keeps the lights on” (KTLO). We budget 50 percent of our teams’ capacity for new initiatives and 50 percent for KTLO and other team-based initiatives. We also work with developers to understand the key results that will indicate success or failure—the impact our work has on customers, not just “we shipped the thing.”
We’re new to objectives and key results (OKRs), but we’ve found them useful for aligning our conversations around goals that have measurable customer impact. We also use Nicole Forsgren’s SPACE framework of developer productivity—which measures satisfaction and well-being, performance, activity, communication and collaboration, and efficiency and flow—to evaluate how our platforms and tooling empower Netflix engineers to do their best work.
We’re in the early days of this new approach to planning, but we’re already seeing benefits: Our work more closely aligns with customers’ needs, we’re prioritizing more effectively based on reach and impact, engineering teams are more focused on fewer things, and we’re better able to coordinate timelines and manage dependencies with partner teams.
Mailchimp
1,200+ employees
Eric Muntz, CTO
Engineering planning at Mailchimp is guided by our technology team’s mission statement: “We give marketers production-ready software designed to help them grow. We succeed through togetherness, momentum, and pragmatism.”
We recognize that our customer could be a small business owner or part of a larger internal department (engineering, marketing, etc.), so we need to work closely and cross-functionally to address diverse requirements. We focus on customer success metrics—such as customer satisfaction scores or percentage of completed tasks—as well as the jobs they need to do, which can range from setting up abandoned cart automations to engaging people who are likely to purchase.
The “momentum” aspect of our team’s mission statement is tricky to get right. We want to build momentum, learn quickly, and iterate on customer feedback without cutting too many corners and creating technical debt. For example, we might need to focus on usability problems we didn’t foresee, or maybe users are happy with a product as is and we don’t need to do the iterations we’d originally planned.
Along with standard project management processes for measuring success—such as velocity tracking, sprint grooming, and replanning—we ask ourselves if our technical roadmaps are increasing business success and confidence. Our chief architect, Joe Uhl, believes the job of infrastructure is to give the business the confidence and conviction to innovate. This is especially important in SaaS and for serving small businesses, so we examine qualitative data from our research teams to understand how our customers absorb the product, what’s intuitive or confusing about it, and what they want next. This enables us to determine whether our technology is actually increasing the team’s confidence in serving our customers.
1,100 employees
KD Bhulani, senior director of engineering
We operationalize our mission of bringing community and belonging to everyone via North Star metrics such as growing our global user bases. These serve as a starting point for the annual planning process, in which each organization prepares a proposal to align with senior leaders on high-level OKRs, strategic investments, hiring plans, and budget. We also do a midyear check-in to measure progress and course-correct if necessary.
While company objectives are generally set annually, engineering teams plan and operate on a quarterly cycle. This allows us to demonstrate success, learn from mistakes, coordinate cross-functional work, and hold each other accountable. As a rule of thumb, OKRs should make a team stretch—if our team is achieving 100 percent of their OKRs every quarter, we’ve likely set the bar too low.
During quarterly planning, each team within the engineering organization prepares a list of the top initiatives that will contribute to their OKRs. These could range from improving the onboarding experience for new users to introducing Reddit in new languages. We also use a continuous ideation process to build a robust project backlog, which is then prioritized via an idea review process. Our leads team reviews new ideas on a weekly basis, estimating them using T-shirt sizes (S, M, L, XL) to allow for ROI assessment. Our ROI formula is: ROI = confidence * (expected benefit / engineering cost).
We also expect teams to decide how many weeks of “visibility” they need to ensure the project is ready for development. For consumer teams that are post-product-market fit, we recommend planning engineering allocation for a maximum of four to six weeks at a time. We optimize for agility and flexibility by running two-week sprints.
Lastly, we frequently consider how things might change. Our planning evolves as we discover new information, new problems arise, or assumptions are proven wrong—or more right than we expected.
Asana
1,000+ employees
Kate Reading, platform engineering area lead
Before we begin any project, we seek input from internal and external stakeholders to understand their needs. From there, we propose features based on a scale of “critical” to “nice to have.” We address any questions regarding product vision and engineering feasibility to ensure projects align with our objectives, time requirements, and resourcing options.
Our top priority is team optimization. Before assigning teams to a project, we consider the scope of the solution and the right mix of team members to work on it. We factor in tenure and experience, opportunities for growth, and bandwidth to ensure the team has a healthy balance of projects, including working on tech debt. We also establish a buffer for rest and recovery, making sure team members have an “offsprint” day in the sprint and several offsprint weeks throughout the year.
From there, we build the annual roadmap, taking into account dependencies on other teams, ownership, deadlines, and budget. During our annual “Roadmap Week,” we organize meetings with stakeholders. It’s tradition to close them with a visioning exercise where we go around the room and ask, “If we manifest this work and it fails, what goes wrong? If it’s wildly successful, what makes it a success?” This is a great way to surface risks, fears, and aspirations.
The last step before execution is to have each team propose OKRs covering their goals. We use Asana to hold team members accountable, outline timelines, and track progress. This keeps us aligned and offers visibility to everyone involved. Using our own platform also allows us to continuously refine it. We have a project where we collect product ideas and possible improvements. When we build something, we try it out internally and collect feedback to ensure we’re providing customers the right tools to work together effortlessly.
1Password
480 employees
Erik Goodman, engineering manager
Customer input and being a distributed company are the driving forces behind our planning and execution.
We owe many of our best features to customers’ ideas. A recent example is our Events API. Our product engineering team worked with 22 companies in a closed beta to understand their needs, identify valuable data points, and gather feedback on the API’s performance and functionality. One of our most popular features, Travel Mode, which lets users temporarily remove vaults, actually started as a 1Password use case in a customer’s employee handbook.
When promising ideas emerge, whether in-house or from the community, we review them against company OKRs, existing team structures, engineering capacity, and our overarching vision for 1Password—the value we aim to deliver to our users now and in the future. Then we’ll adjust, identify priorities, and help each team plan for the quarter ahead.
As a distributed company, we can’t sit in a room for all-day planning sessions, so we’ve refined our asynchronous processes. First, each feature is assigned an engineering lead (or a project manager if it’s large enough), who drafts a lightweight design document. This document becomes an organization hub for contributors as well as a project changelog, and it sets out release milestones to measure progress and success. Next, the lead drafts actionable issues, which are added to the backlog. Then, product managers and tech leads determine which features to prioritize and assist engineers with sprint planning. Multiple milestones paired with feature flags enable us to release early and frequently.
After each release, we gather customer feedback and adjust future milestones based on that input. Customers remain our guiding light for new features, as well as continuous improvement and versioning.
LaunchDarkly
300 employees
John Kodumal, CTO and cofounder, and Jonathan Nolen, SVP of engineering and product
Our planning process flows from the following core principles:
Avoid handoffs. A formalized handoff process—from product to design to engineering, for example—can be like playing telephone. Instead, we have integrated teams with PMs, developers, and designers working on squads. Autonomous, multiskilled teams can plan and execute in tight feedback loops, which builds trust and improves our ability to deliver software as intended and on time.
Maintain balance with constraints. Rather than shunting KTLO activities like driver upgrades and incidents to a backlog, we place constraints around acceptable limits. This gives us permission to deviate from the product delivery roadmap when necessary to keep the product healthy. For example, we ensure all critical post-implementation review items are closed within 14 days of an incident.
Deliver incrementally. We’re big believers in continuous delivery. Deploying small batches of code at a high frequency minimizes merge conflicts, reduces incident severity, and tightens feedback loops. These benefits have a positive compounding effect, giving users what they want quickly and efficiently.
Own your estimates. Teams work closely to estimate development schedules. We delegate timeline estimates to the squad level, which increases their chance of successfully delivering work on time, since they know their abilities and limitations best.
Do what you say you’ll do. We evaluate teams’ success based on the commitments they’ve defined at the start of a project. When a team misses commitments—especially a few times in a row—it’s a signal that we need to understand the mismatch. Are things not going to plan? Is there more uncertainty than we thought? Each commitment checkpoint is an opportunity to update assumptions and reset target dates.
Taken together, these principles allow us to focus our planning time and effort on achieving outcomes that are crucial to our business’s success and our customers’ happiness.
Netlify
150+ employees
Cassidy Williams, director of developer experience
Netlify’s developer experience engineering organization combines engineering with developer advocacy and documentation. Because we’re focused on developer experience, most of our planning is driven by engineering trends and research. It also involves a lot of cross-team collaboration.
We work with the UX research team to improve customer pain points through code and documentation, including developing plugins, authoring blog posts, and creating online courses. We also work with product teams to build on new ideas. For example, my team helped launch the collaborative deploy previews feature through tutorials, videos, and social posts, as well as trying out new features ahead of launch. Finally, we work with external developers to identify the demos, content, and features that would be most valuable to them. That’s led to building tools like the free, open-source learning platform Jamstack Explorers and launching our team podcast, Remotely Interesting.
Between content rotations, partnerships between engineers and technical writers, and regular “mob programming” sessions (in which we all code together on a project), our plans are usually executed quickly—within a three-month period at most. Because tech moves fast and outside forces dictate much of our roadmap, we have to be ready to pivot.
We have overarching goals that span a longer time period—improving how our customers can scale, for instance—but the plans to hit those goals cover much shorter periods. For example, our team had a goal of improving e-commerce experiences on Netlify in a two- to three-month period. So when Shopify released its expanded Storefront API, we built several framework examples and wrote tutorials, blog posts, and social content for it. Now, our e-commerce customers have a broader suite of resources to use when building their websites. These shorter goal periods keep our team agile, responsive to change, and continually learning.
Notion
150 employees
Michael Manapat, head of engineering
Over the last 18 months, Notion’s engineering team has grown from 12 people to 50. As a result, our planning process has had to evolve quickly.
Our first attempt at planning consisted of assembling a list of engineering ships for the next quarter. But with each iteration of the process, we’ve extended the time horizon we plan for and the strategic coherence of our plans. The beginning of 2021 was the first time we articulated an 18-month product strategy. Using that strategy as context, we plan for six-month cycles, with engineering teams formulating project details and success criteria for themselves.
While we remain engineering-driven, we’ve put greater emphasis on cross-functional planning as the company has grown. Working with marketing, sales, customer success, and other user-facing teams requires more iteration, discussion, and prioritization throughout the planning process. We go through multiple rounds of iteration on each team’s plans until they converge. From an org structure standpoint, we want engineering to be at least 40 percent of the company so there’s bandwidth to support all of our go-to-market efforts.
Product planning at Notion is particularly challenging because our user base runs the gamut from individual consumers to large enterprises. Despite its fast growth, our engineering team is still quite small, so we frequently need to make trade-offs. For example: Do we work on features for our largest users, or focus more on top-of-the-funnel user acquisition? Having a midterm strategy keeps us aligned with the bigger picture. We’ve found we accomplish more as a company when we’re clear-eyed about cutting everything that isn’t essential to our strategy. Clarity brings focus and energy, which drives better execution.
Editor’s note: Michael Manapat is also the former head of applied machine learning at Stripe.
Stellar Development Foundation
96 employees
Karen Chang, VP of engineering
As a nonprofit, our organizational goal of creating equitable access to the global financial system isn’t tied to traditional success metrics. We support the development of the open-source Stellar blockchain, so ideas can come from business drivers, technology innovations, or community requests.
Our planning process is twofold. First, there’s engineering planning, which is driven by an OKR process in which leadership generates the high-level goals reflected in our annual roadmaps. We also have a weekly product and engineering meeting where anyone at the company can bring a development proposal and engineering and product leads discuss prioritization. This allows us to adapt to the changing landscape of opportunities.
Second, there’s the bottom-up planning process. This includes the open-source community, which can propose new Stellar ecosystem protocols and changes to the core blockchain. The process unfolds in public repos in GitHub, where work is visible, roadmaps are public, and anyone can submit issues, comments, or pull requests. Changes are proposed in a public email group, codified into a draft in a Git repository for protocol requests, and refined through community feedback. The Core Advancement Proposal core team then either rejects or accepts the draft proposal based on how it serves the goals of the Stellar network and aligns with the values of the Stellar protocol, which include security, reliability, low cost to run at scale, and decentralization. If accepted, the features are implemented and decentralized validator nodes on the network make a final vote to adopt. After validators vote in favor, the proposal is adopted and the protocol evolves.
Once we’ve solidified our plans, engineering teams self-organize to map features into execution plans. Sometimes there’s fuzziness around what constitutes platform versus product work, but we negotiate assignments based on skills, availability, and interest to ensure we hand the right projects to the right people.