Roadmap Retrospectives: Turning Insights into Action

At the conclusion of a roadmap timebox, analyzing data is not just a reflection on what worked and what didn’t — it’s an opportunity to prepare for the next roadmap with actionable insights and strategies. Effective roadmap retrospectives empower teams to learn from past experiences and improve their planning, execution, and delivery in the upcoming roadmap timebox.

Here’s how to conduct a comprehensive roadmap analysis and extract meaningful takeaways.

Gathering the right data is the foundation of any successful analysis. At the end of the roadmap, focus on capturing three main aspects: content, velocity, and over-budget Epics. Let’s explore these one by one.

Content

When it comes to content, understanding both its size and the changes in size throughout the roadmap is crucial. Why? Because assessing scope creep helps set realistic expectations for future planning.

Content Size at the Start

Focus on the overall size of the content committed at the beginning of the roadmap, not the specific details. This distinction is essential because content evolves as the roadmap progresses, influenced by changing priorities, customer feedback, and new requirements (“responding to change over following a plan”). The committed content size serves as the baseline for evaluating changes.

Tracking Changes in Content Size:
Inspect how the size of the committed content increased or decreased over time. Tools like Jira offer two key charts for this purpose:

• Release burndown:
  Use this chart to capture the initial commitment size and track additional content added during the roadmap. For example, if scope typically increases by 30%, this insight allows you to either include a buffer or set management’s expectations that 30% of the original content might need to be de-scoped.

• Version report:
  This chart complements the release burndown by providing trends in unestimated content. A percentage of unestimated content below 20% indicates high confidence in the total content size. The version report also reveals increases or decreases in content size, often reflecting mitigation plans that de-scope content mid-roadmap to maintain feasibility.

By examining the initial roadmap size (when unestimated content falls below 20%) and comparing it to the final size, you can calculate the total scope change — typically an increase.

Velocity

To make realistic commitments, understanding the overall roadmap velocity of your teams is essential. Jira’s Velocity Chart is the go-to tool for this data:

1. Calculate Roadmap Velocity:
   Summarize the velocity across the number of sprints in the roadmap. For instance, in a quarterly roadmap with 2-week sprints, use the velocity of six sprints to determine the roadmap’s total velocity. This metric reflects your teams’ delivery capacity and ensures commitments are grounded in reality.
2. Validate with Release Burndown:
   The release burndown chart can also provide velocity data, showing total delivery within the roadmap and velocity per sprint. Cross-check this with the velocity chart for consistency.

Over-budget Requirements

Scope creep often arises from requirements exceeding their original estimates. “Budget,” in this context, refers to the initial estimation size of a requirement. Requirements that exceed this budget are considered over-budget. For example, a requirement initially estimated at 30 story points but increasing to 45 story points is 50% over-budget.

Analyzing over-budget requirements provides valuable insights:

1. Identifying Trends:
   Larger requirements often have a higher percentage of over-budget increases. This insight can inform strategies like breaking down large requirements into smaller, more manageable slices.
2. Bucket Requirements:
   Some requirements act as “buckets” for work, such as small enhancements or customer feedback. While these are hard to estimate, past data can guide future allocations. For example, if a “small changes” bucket used 35 story points in the previous roadmap, you can allocate a similar size for the next one.

Example of Roadmap Analysis

We will take here a quarter roadmap for example. The teams are working in 2 weeks sprints.

Roadmap Burndown Analysis

Jira Release Burndown

Let’s consider a quarterly roadmap with 2-week sprints.

Jira Release Burndown Analysis:

• Original content size: 483.5 (383.5 + 43 + 57)
• Total content size (end of roadmap): 862.3
• Delivered content: 611 (70% of total content)
• Scope creep: 416.8 (48% of total content)
• Remaining work: 156.8
• Velocity: Teams’ average velocity increased from 75 to 120 story points per sprint.

While the release burndown provides high-level insights, delving into the version report helps identify the initial scope size.

Using the Jira version report, we can gain a clearer understanding of the initial scope. By identifying the first instance when the percentage of unestimated content dropped below 20%, we see that the content size was approximately 600 story points. With the teams’ delivery rate also around 600 story points, the initial commitment appeared to be realistic. However, only 70% of the content was ultimately completed, largely due to scope creep.

Scope creep doesn’t always have to be viewed negatively. Often, as we progress, we learn and make decisions to incorporate additional content based on those learnings. The key is ensuring that this added content is prioritized over less critical items, which should be de-scoped from the plan to maintain focus and feasibility.

To dive deeper into the origins of scope creep, we need to analyze the over-budget requirements data.

Requirements Over-Budget Analysis

In our example roadmap, we have 23 Epics. Each line represents an Epic, and compares the original story points to the accumulated story points, followed by the percentage over-budget and the current status.

• EP-1001: 30 vs 72, 140%, In Progress.
• EP-1002: 26 vs 73, 180%, In Progress.
• EP-1003: 56 vs 11, -80.36%, In Progress.
• EP-1004: 31 vs 39, 25.81%, Dev Done.
• EP-1005: 26 vs 0, -100%, Discovery.
• EP-1006: 36 vs 37, 2.78%, In Progress.
• EP-1007: 55 vs 57, 3.64%, In Progress.
• EP-1008: 30 vs 21, -30%, Dev Done.
• EP-1009: 40 vs 39, -2.5%, GA.
• EP-1010: 17 vs 24, 41.18%, In Progress.
• EP-1011: 20 vs 13, -35%, In Progress.
• EP-1012: 30 vs 32, 6.67%, Rollout.
• EP-1013: 18 vs 22, 22.22%, GA.
• EP-1014: 80 vs 147, 83.75%, In Progress.
• EP-1015: 20 vs 19, -5%, GA.
• EP-1016: 30 vs 44, 46.67%, GA.
• EP-1017: 30 vs 18, -40%, GA.
• EP-1019: 35 vs 63, 80%, In Progress.
• EP-1020: 75 vs 136, 81.33%, In Progress.
• EP-1021: 30 vs 57.5, 91.67%, GA.
• EP-1022: 10 vs 14, 40%, GA.
• EP-1023: 72 vs 82, 13.89%, In Progress.

Key Insights from the Epics data:

• The overall scope creep accounts for approximately 48.34% of the total story points in the roadmap.
• Six Epics exceeded their original story points by over 20%, significantly contributing to the scope creep.
  Notable examples include EP-1001 (140% over-budget), EP-1002 (180% over-budget), and EP-1020 (81.33% over-budget).

The release burndown analysis reveals that the total scope creep amounts to 416.8 story points (excluding Sprint 20, which was part of the initial planning phase). Over-budget Epics contribute 324.5 story points to this total, while under-budget Epics reduce it by 83.3 story points. This indicates that approximately 60% of the scope creep originates from over-budget Epics, which alone represent around 40% of the original roadmap commitment.

Given this analysis, we might consider adjusting our commitments for the next roadmap by factoring in the 40% scope creep caused by over-budget Epics. This would involve committing to a smaller amount of content than the teams’ total velocity allows, leaving room for potential scope adjustments. Additionally, we could adopt a stricter approach to over-budget Epics, capping increases at 20% to maintain better control over scope creep.

Example Conclusion

These insights highlight the critical importance of analyzing roadmap data. Such analysis provides actionable takeaways that can improve the management of future roadmaps and set clear expectations with stakeholders. This example demonstrates how data-driven decisions can translate into impactful action items, such as:

• Limiting over-budget Epics to a 20% increase.
• Adjusting roadmap commitments to 80% of team velocity, leaving room for scope creep.
• Committing based on past velocity metrics.

By integrating these strategies, teams can plan more effectively, enhance execution, and foster a more predictable delivery process.

Conclusion

This guide equips teams to analyze roadmaps comprehensively, paving the way for better outcomes in future roadmaps. By tracking key metrics like velocity, scope creep, and over-budget requirements, teams can set realistic expectations and improve roadmap management. Start applying these practices today to drive impactful results.