Lean software development, reporting and visualising lead and cycle time

In the previous articles I have discussed lead and cycle time, and proposed that it is possible to use these metrics to run experiments and measure the actual improvement. Now let’s see how these metrics can be reported and visualised.

Seeing team’s lead and cycle time profile

Figure 1, X axis shows during which business day work has arrived and colors represent how long work has taken, Y axis shows number of stories

This bar chart shows the team’s user stories lead time and cycle time for the past year. This team follows a two week (10 business days) Sprint pattern. As we can see, the majority of the work gets delivered towards the end of the Sprint between days 8 and 10. We can see that there are user stories that drift across Sprints as they don’t get finished in a single two week Sprint.

To explain this graph to your stakeholders you are better off using quartiles. With quartiles it is possible to explain the above graph by saying ~25% of the work gets delivered by day 5, ~50% of the work gets delivered by day 8, ~75% of the work by day 10 and the last ~25% of the work gets delivered by day 21.

Seeing team’s improvements

One of the things that can be difficult to measure is a team’s performance improvements. However, now that we have the above graph (figure 1) it becomes much easier. Graph below shows the same team that has figured out how to reduce cycle time by 1 day on average.

Figure 2, Cycle time was reduced by 1 day average

Let’s take a moment and unpack what has happened. To reduce cycle time by 1 day average some stories will be delivered 1 day quicker, some by 2 days and some not at all, this gives 1 day average. This means not everything needs to be done 1 day quicker.

If the team has reduced cycle time by 1 day average this means there is space for more work. Originally the team has delivered 505 user stories in 1 year, after 1 day cycle time reduction the team will deliver ~668 user stories in 1 year.

Figure 3, superimposed figure 1 (faded) with figure 2 to make 1 day cycle time changes more obvious

Figure 3 shows before and after superimposed. Sprint work is no longer delivered between day 8 and 10 (figure 1) but between day 6 and 9.

Testing team’s improvements for noise

It is possible to get ahead of yourself and see improvements when there are none. If a team happens to deliver more stories three Sprints in a row does not mean they have improved. It might mean that the team was “regressing to the mean”. Take a look at this visualisation:

Figure 4, X axis shows day when work was delivered, Y axis shows number of stories delivered on that day

Black line represents the actual delivery with faded black representing variance around each day. Red and blue lines are randomly sampled from existing data. This means some stories are randomly chosen from the available collection of stories (that were used to create the black line) and then blue and red lines are drawn.

Important thing to understand that it is exactly the same data, it all comes from figure 1. If you look at the blue or red line, it might make the team feel that they are delivering work earlier in the Sprint and being more stable. This is not the case, it is a random result. Here is how actual improvement would look like:

Figure 5, X axis shows day when work was delivered, Y axis shows number of stories delivered on that day

If you refer back to the figure 2, we know that more work gets delivered and it gets delivered earlier. Black line represents original data (figure 1), blue and red now show an actual improvement (figure 2).

You can verify that this change is genuine and not random noise by using a median test. For figure 4 p-value for lead time was greater than 0.05, for figure 5, p-value was less than 0.05. If you are interested in the above visuals and how they are generated check out my “Exploring lead and cycle time with analytics and statistics” notebook.

Figure 4 and 5 graphs are not the best visuals to show change to the stakeholders, figure 3 seems to be more appropriate in my opinion.