A summary of “Evidence Guided” by Itamar Gilad — Part 1 (Intro and goals)

The book kicks off with a story about Itamar, who’s consulting CEOs and founders on a regular basis. He asks them a big question: “How do you know these are the right things?”, which lies at the heart of evidence-guided Product Management.

In the book, Itamar talks about two kinds of companies: those that rely on evidence and those that just go with opinions and do “plan-and-execute” in their product development practices. These differences show up in how they plan, execute, and what they achieve. Both types can succeed, but companies that follow an evidence-guided approach tend to do so more consistently. They’re not guessing and taking blind bets based on opinions — they’re making informed decisions and reducing uncertainty based on evidence.

Itamar shares two stories to explain this further. First, there’s Google+.
It was made to react to Facebook’s success, without much proof that it was the right move. It followed a classic process: come up with an idea, plan it out, and make it happen. But Google+ failed big time, costing a lot of time and money. Why? Because our brains aren’t good at dealing with uncertainty. We often rely on opinions even when the evidence says otherwise and fall prey to our human biases such as sunk-cost fallacy, confirmation bias, risk aversion, and many more. Additionally, Itamar points out that research already showed that most ideas fail and most features only create marginal impact, which is why gathering evidence is so crucial.
Then there’s Gmail. Itamar and his team took a different approach. They tested ideas and hypotheses instead of just going with their gut. They tried things step by step, like testing the tabbed inbox feature first with a small group of Google employees. This method led to success, as we can all see with the organized Gmail inbox we use today. If you are interested in more details, you can listen to the following podcast with Lenny Rachitsky.

At the core of Itamar’s argument is the GIST model. He developed a simple framework at Google, with four parts: Goals, Ideas, Steps, and Tasks. Following GIST means constantly checking in, trying new things, and learning from what works. It’s like having a map for product development, where every step is guided by evidence, not just guesswork.

The summary comes in parts:

• Part 1 — Intro and Goals (this post)
• Part 2 — Ideas and Steps
• Part 3 — Tasks and the evidence-guided-company

Goals

Itamar argues that traditional planning sessions, fixated on quarterly or yearly outputs, consume excessive time and overlook outcomes. By prioritizing plan execution, they lose sight of the bigger picture. Conversely, evidence-guided companies take a different approach. They define goals as outcomes — desired changes — and adjust their actions based on new information they learn along the way. This fosters agility, simplifies execution, and provides clearer indicators of progress and success.

Goal setting with OKRs

Itamar delves into the realm of goal setting, employing the popular OKR (Objectives and Key Results) framework as a guide. A well-constructed OKR comprises three elements: an Objective, which paints an inspirational picture of the desired outcome; Key Results, offering measurable targets for success; and Context, providing the why behind the goal’s importance. The book adopts the standard practice of annual company-level OKRs and quarterly team-level OKRs, emphasizing the importance of revisiting goals regularly for refinement and alignment.

Moving beyond OKRs, Itamar explores various models for understanding company and customer behavior (flywheels, funnels,…), ultimately championing the value exchange model. This simple model hinges on value creation and capture — a reciprocal process wherein providing value leads to capturing value, perpetuating a growth cycle.
To quantify this exchange, Itamar suggests two metrics: the North Star Metric (NSM) and the Top Business Metric. The NSM encapsulates the core value delivered to customers and the market, while the Top Business Metric measures the value captured by the organization.
For example, popular North Star metrics are messages sent per month, videos watched per week, or weekly active teams. To find your own, Itamar proposes to ask yourself “Why are customers using my product?”. Conversely, examples of frequent top business metrics are revenue, ARR, monthly or weekly active users, or profit margins.

Both metrics can be integrated into the OKR framework, ensuring a balanced focus on value creation and capture. This dual approach ensures products are useful and sustainable — a crucial balance for long-term success.

Mission statements and their connection to the NSM and top business KPIs.

After introducing the value exchange model, Itamar underscores its alignment with a company’s overarching mission. For instance, Google’s mission to “Organize the world’s information and make it universally accessible and useful” is a guiding beacon. However, such missions are often criticized for their abstract nature, lacking actionable guidance. Here, the North Star Metric (NSM) steps in, offering a more tangible and immediate metric that reflects the vision. While the mission may be perpetual, the NSM provides a narrower timeframe for progress, making it more attainable and actionable within the company’s journey.

Metric trees and goals that don’t fit the framework

Itamar emphasizes that the North Star Metric (NSM) and the Top Business Metric (TBM) are not standalone indicators; they work together and often intersect. While the NSM provides tangible direction, it lacks the granularity for direct team influence. To address this, Itamar advocates breaking down both metrics into individual components. Using a hypothetical scenario involving Documents, he illustrates how these sub-metrics can serve as localized goals for teams to own and drive independently, or in collaboration with others. This breakdown identifies the most impactful components and facilitates actionable insights and metric ownership. Furthermore, Itamar suggests dissecting the TBM to uncover commonalities with the NSM, fostering alignment and synergy between the two overarching metrics.

Itamar recognizes that not all aspects of a company’s operations neatly fit into the North Star Metric (NSM) or Top Business Metric (TBM) framework. For example, areas like code health or development infrastructure stability may fall outside these metrics’ scope. To address this, Itamar proposes that teams and organizational units establish additional goals to encompass these aspects, ensuring a comprehensive approach to improvement.

Alignment

Alignment is crucial for success, especially in large companies. The metric tree lays the groundwork by defining success clearly for everyone. But alignment goes beyond this — and according to Itamar requires a three-step approach:

1. Top-down goals (annually): These are influenced by strategy, values, threats, and opportunities. They’re like the big picture, with objectives and key results steering the ship. An interesting idea here is that top-down goals can also come without key results defined by upper levels, leaving room for lower levels in the organization to shape this part themselves. It also removes pressure from upper levels in cases where insufficient information is available to them to develop proper key results.
2. Bottom-up goals (quarterly): Teams get involved here, adapting and aligning their goals with the bigger picture. It’s about ensuring everyone’s moving in the same direction, with the flexibility to adjust as needed. These goals are an opportunity for leaders to reassess and refine objectives. If a goal no longer fits, updating or removing it is okay. The book suggests that smaller companies might only need two levels of goals (company and team), while larger companies might require additional layers. Teams can adapt company-level key results (KRs) for their goals or align them slightly differently. For instance, if the company aims to increase document usage, teams might focus on increasing user numbers or average documents per user. If metrics are uncertain, teams can set research-based goals to clarify. It’s better to spend time validating metrics than pursuing uncertain objectives. Targets can be set using existing data or educated guesses, adjusted as new information emerges. Itamar emphasizes the importance of starting with fewer KRs and objectives to avoid overload.
3. Finalizing goals (quarterly): Teams and middle managers collaborate to fine-tune objectives, ensuring they’re realistic and ambitious. This phase highlights overlaps, dependencies, and potential conflicts. The main task of management here is to provide feedback, to connect goals between teams, and to come up with mutual agreement.

This process fosters clarity and autonomy. It aligns the organization strategically while empowering teams to execute tactically, fostering a culture of clarity and purpose.

Main takeaways

1. Evidence vs. Opinion: companies that rely on evidence-based decision-making with those that rely on opinions in product development.
2. The GIST Model: Goals, Ideas, Steps, and Tasks — as a framework for evidence-guided product management. This model emphasizes constantly checking in, trying new things, and learning from what works, guiding every step of the product development process with evidence.
3. Goal Setting with OKRs: goal setting using the OKR (Objectives and Key Results) framework, focusing on defining outcomes rather than outputs. Showing how OKRs can foster agility, simplify execution, and provide clearer indicators of progress and success, advocating for regular revisiting and refinement of goals.
4. Value Exchange Model: value creation and capture are reciprocal processes driving growth. Metrics like the North Star Metric (NSM) and the Top Business Metric (TBM) will be used to quantify this exchange and integrate them into the OKR framework.
5. Alignment: is crucial for success, especially in large companies. The book outlines a three-step approach to alignment, involving top-down goals, bottom-up goals, and finalizing goals quarterly. It is a rather common and simple approach, yet my experience tells me it is hard to execute well.

Part 2

For Ideas and Steps continue with Part 2!