Monitor team practices for better project outcomes

Written By: Chris McInerney

Aug 9, 2021 in  Blog

For any project, we use metrics to assess progress towards an outcome.

These are often an expression of recent performance (“how did we do”) or forward-looking as a scope measurement (“how much is left”). These measurements are important; however, they are not adequate for capturing team performance – or, more importantly – team potential.

If our project was to drive to a nearby city, the average speed and distance travelled would be indicators to assess when we may arrive at our planned destination. They are lagging indicators because they are measurable only after time has elapsed and there is performance to measure.

Continuing with our driving analogy, driving conditions, traffic, fuel efficiency and the amount of fuel in your tank would be considered leading indicators. They affect the ability to perform in the future. The weather and traffic dictate the maximum speed we can safely travel. Fuel efficiency and the amount in the tank determine the need for a gas station stop that will slow our progress. In practice, both lagging and leading indicators are needed to understand our journey; however, we often only focus on the speedometer right in front of us.
In software projects, metrics such as “completed story points” or “defects resolved” are lagging indicators that tell how far the team has come. Arguably, they are a proxy for future accomplishment, but they do not anticipate the ability of a team to respond to scope change or collaborate to resolve challenging defects.

The leading indicators in a software project are the team norms—or Practices used every day. They facilitate the work environment and promote effective activities and relationships. Like the driving example, observing team Practices in software projects provides insight into team capability and resilience that impacts future performance—similar to how weather, traffic, and the amount of fuel in one’s tank are indicators of when you may reach your destination.

What are the Practices in software projects?

From over three decades of software development experience, Intelliware has identified more than 30 Practices that help drive successful projects.  These Practices underpin 12 Outcomes that we aim to achieve on projects, which are, in hand, based on the following three core Values:

1. High Quality Deliverable

2. High Performance Team

3. Effective Process

The Values are high-level objectives that can’t be measured directly.  Safety may be a value that guides how you operate your car, but it’s the day-to-day practices, such as following speed limits and wearing a seatbelt, that indicate how safe a driver you are.

The Outcomes serve as more specific objectives within the Values and are indicators of a team’s ability to achieve the Values. However, as noted before, Outcomes are lagging indicators, so we can only observe success once the work is completed.  We rely on the Practices to measure the potential for a team to succeed with the Outcomes and Values.  The Practices are not the objective; we want teams to achieve the Outcomes and the Values ultimately—the Practices serve as metrics of potential.

A valuable aspect of the Practices is that many of them support more than one Outcome.  For example, the Outcome Build the Right Thing requires several Practices, such as having an Engaged Product Owner and regular Backlog Refinement meetings to keep the Backlog up to date.  But these Practices are not exclusive to Build the Right Thing—they also apply to other Outcomes.  Figure 1 illustrates the relationship between Values, Outcomes and Practices.

onship Between Values, Outcomes & Practices
Figure 1: Relationship Between Values, Outcomes & Practices

The relationship between Values, Outcomes and Practices is similar to managing requirements on an Agile project.  Outcomes are like Epics—Epics are too large and undefined for a team to develop within a Sprint, so they break them up into Stories, which are small enough to be definable at a level of detail that a developer can work with and fit several or more into a Sprint.  But even Stories are too big for a team to take on directly, so at implementation time, Tasking is used to split Stories into small chunks of work that the team can self-organize around to get to the eventual output—the completed Story.  And, similar to how Practices are not unique to Outcomes, there are often Tasks that apply to more than one Story, such as Developer Acceptance Testing (DAT).

Let’s consider some project scenarios, which may sound familiar, that show how Practices are related to Outcomes and Values and can be used to diagnose problems with achieving an Outcome:

1. Several Stories are awaiting QA testing with only three days left in the Sprint

There could be two underlying causes of this problem:

a) Developers are delivering the majority of their completed Stories towards the end of the Sprint. The team may not be implementing the Parallel Testing Practice by delivering a steady stream of Stories to QA during the Sprint to be tested in parallel with development. The quality of the completed Stories, specifically the Build It Right and Minimize Risks Outcomes (and the High Quality Deliverable Value), could be affected by QA finding issues well after a Story is developed and the implementation is still fresh on the developer’s minds.

b) QA analysts are taking a long time to test the Stories.  Developers may not be doing enough testing, effectively passing the burden of logic testing downstream to QA.  More Automated Testing, specifically Unit Testing, may be needed to support the Build it Right and Minimize Risks Outcomes which are part of the High Quality DeliverableValue and the Highly Automated Outcome that’s part of having an Effective Process.

2) Several Stories failed QA in the current Sprint and need to be moved to the next Sprint

Development complete Stories bleeding into the next Sprint because they failed QA is a significant problem.  When this happens, the team may have to talk to their Product Owner about removing Stories from their commitment for the next Sprint to leave capacity to fix the failed Stories from the previous Sprint.  The Product Owner won’t be happy, eroding their trust in the team’s ability to commit and deliver reliably.  If the team decides not to have this conversation with the Product Owner—it’s their call—then adding more scope to the Sprint jeopardizes their ability to maintain a Sustainable Pace and Strong Team Morale.

The main problem is that there wasn’t sufficient time at the end of the Sprint to test and fix all of the Stories that were complete by the developers. Four possible causes of this problem include:

a) Developers delivered most of their Stories to QA towards the end of the Sprint

b) QA analysts took too long to test

c) QA found issues requiring significant effort for the developers to fix

d) The team is over-committing on their Sprints

The first two causes, developers delivering Stories at the end of the Sprint and QA testing taking too long, were discussed in the previous example.

The third cause, significant issues found during QA, could also be the developers not doing enough testing.  There may also be other problems involving work completed before development. For example, issues found during QA related to missed requirements could be due to insufficient Backlog Refinement, which affects the Build the Right Thing Outcome.

The fourth cause, the team is over-committing, points to a Sprint Planning and Delivery Practice problem which can affect many Outcomes, including Building The Right Thing and Team Collaboration.  The team may be using a Velocity that is too high. Or, the team may be committing to Stories with significant testing risks, and QA is not speaking up during Sprint Planning to identify those risks and encourage the team to consider ways to adjust the Sprint commitment to increase the chances of success.  QA not speaking up could point to a Trust and Safety problem.

3) The Product Owner missed the Sprint Review and Demo meeting

A Product Owner missing a Sprint Review could indicate a problem with the Engaged Product Owner Practice, which could affect multiple Outcomes, including:

a) Build The Right Thing: When the Product Owner doesn’t attend the Sprint Review, they miss first-hand feedback from project stakeholders.

b) Incremental Development: Product Owners play a significant role in ensuring that the increments of development, the Stories, are up to date and accurately represent the business’s vision for the product.

4) There are knowledge silos within the team

Knowledge silos indicate the team is not implementing the Collective Code Ownership Practice, which can hinder the Collaboration and Team is Self Organizing Outcomes and the High Performance Team Value.

5) The developers complain about technical debt

Too much technical debt can be the result of many problems, two common ones include:

a) The team is not doing enough Refactoring, a quality problem that affects the Build The Right Thing and Build It Right Outcomes and the High Quality Deliverable Value

b) The team is continually having to cut corners to complete their Sprints on time. Cutting corners could be a Sprint Planning and Delivery problem that can affect multiple Outcomes.  The team may be overcommitting on their Sprints, or new Stories are being added during the Sprints.  

Inspecting and adapting the Practices

How do you determine the health of your team and their ability to deliver on the Outcomes?  By continually inspecting how the team is doing with the Practices and adapting them where necessary to make sure they work for the team.  For example, some questions relevant to the Practices that support the Incremental Development Outcome include:

  • Are Backlog Refinement meetings occurring regularly so that the Backlog is ready in time for Sprint Planning?
  • Is there an Engaged Product Owner actively involved with defining the scope and priority for the Stories?
  • Is there a Milestone Plan in place to guide the overall planning process?
  • Are the Stories in the Backlog of high quality?
  • As part of Sprint Planning and Delivery, are the Sprint Planning meetings resulting in clear, achievable Sprint commitments for the team?
  • Is the team Estimating the Stories in the Backlog, which helps the Product Owner decide priorities and scope for the upcoming Sprint?
  • Do some Stories include Refactoring tasks or incorporate some effort for Refactoring?

Inspecting specific Practices is also helpful.  For example, the qualities of effective Stories include:

  • Acceptance criteria are clear and testable. There is no confusion as to the intent of a Story.
  • Backlog Refinement meetings occur regularly, and high-priority Stories are well understood. No team should show up at Sprint Planning to find Stories at the top of the Backlog they haven’t seen before.
  • Dependencies between Stories are investigated, identified and documented before the Sprint Planning meeting (if not, the Stories should not be selected for the Sprint)
  • Stories are generally small to medium in size, ideally no larger than 2 to 3 points
  • Technical risks associated with a Story have been addressed previously, ideally via a Spike completed as part of a previous Sprint

Recalling the driving analogy we used earlier, it is important to consider several leading indicators, such as driving conditions, when determining how long it will take to reach your destination.  Similarly, team Practices are the leading indicators that create conditions under which a project can excel.  Reserve time to discuss and develop the Practices and then observe them in action.  A high-performing team takes time to develop.  Keeping an eye on the Practices and finding ways to improve them will improve the quality and predictability of project Outcomes.  Your team will be much happier (and more productive) for it!    

Need to transform the way you plan, build, and support digital solutions?

Talk to our delivery managers today.