Why do projects get slower over time? Part 3

This is part three of a three part series on why projects slow down over time.

How many times have you heard - or thought to yourself - that things used to be much better before? We used to be able to release quickly but now there is so much overhead and things move at a snail's pace.

You're not crazy - things do slow down over time. There are many causes but I'm going to delve into three of the main causes over the next few articles.

• Predictability
• Complexity
• Scaling

The curse of success

When you are starting on a new product there are a lot of challenges but coordination isn't one of them. A small team can keep everyone aligned easily. But as you grow cracks start forming - and this slows down the team.

1. Alignment

The more people on a team the more challenging it is to make sure that everyone knows what is happening and where they should be focusing their attention. In a 2014 study on communication in groups the authors state

"Our main result is a clear negative interaction between group size and communication".

The fast pace of the initial releases begins to slow down under the weight of extra communication and meetings.

This is where startups and scaling teams sometimes shift from the cross-functional approach they had taken to develop the initial product and adopt the "more structured" project way of working. New features are broken out as projects, with limited scope and smaller teams, to make the alignment challenges easier.

By design, each project focuses on a small area. This simplifies the alignment on individual projects but it leads to the need for a centralised project review and approval process to ensure that the overall direction of the product aligns with the company strategy. While this solves the alignment challenge it introduces the Predictability and Complexity challenges that come with IT projects.  

2. Increasing Product Complexity

Dan North has a great phrase for staying efficient: "work on software that fits in my head". His point is that as a system grows in complexity it gets really hard to figure out how everything interacts, which slows down future work. This problem affects product managers, UXers, designers, as well as the developers. Each role within the team becomes less efficient as the scale and complexity of the system grows.

3. Stepping on Toes

As more people work on the same product, even in different project teams, the risk of conflicts and unintentional delays increases. Typically projects work in branches, but these need to be merged for delivery. When two teams have been working on the same part of the system there is a risk that they have broken each others code which leads to re-work, frustration and longer release times.  While the example above relates to developers the same issues can occur with where parallel changes invalidate some UX research, or modifying the design of a component means it no longer fits in with the overall product design.

4. Releasing

Releasing is easy when there is one team working on a product. Once two or more teams are working on it you need coordination because there is only one version of the system in production (usually). As as the number of teams grows the coordination becomes tougher, and teams start getting blocked as they wait in line for other teams to release.

Are teams doomed to slow down and get overtaken by more nimble start-ups?

Fortunately, no. There are ways of overcoming all of the challenges that scaling teams face. The reason why teams don't follow them is because the ways of working are new and there just aren't as many good example use cases to follow. There are thousands of companies working with the project model but a lot less who have successfully scaled product structures. Like in the Betamax versus VHS war the better option doesn't always win - the better marketing does.

Rather than give up on the product structure that brought early success we need to solve the scaling problem with a similar approach to how we tackle big features in product development. How can we break this large problem (our growing product) into more manageable pieces? By finding the boundaries in your product, taking into account the user journey and technical architecture, you can separate your product into smaller sub-products. This "modularisation" of the product means that you can assign smaller teams to each module delivering benefits of reduced complexity and easier alignment. In addition, because the sub-products are independent there is less stepping-on-toes and each can be released independently, removing the queuing problems with projects.

It all sounds so simple - but like anything in product development there is no free lunch. It is very hard to draw perfect boundaries between sub-products because the real world is not as neat as we would like. This means that there will need to be cross team collaboration, even just if it is in the case of shared design systems, libraries and components. Also, breaking up a system into sub-products is very technically challenging. The recent drive towards microfrontends and microservices has been to enable this modularisation of systems and overcome the negative scaling effects of teams. If you are working with a legacy monolithic application it can take years to effectively slice it up - both due to the technical challenge but also due to the learning curve as the team get up to speed on the new architecture.

It can seem overwhelming

When we build a product the vision in our head can seem overwhelming as well. But we know that breaking the product into smaller increments allows us to to make smaller bets and learn as we go. The same is true for implementing the architectural changes to enable scaling over time. There is even a pattern, the strangler pattern, that outlines how to extract small pieces of functionality from a system until the new features cover the full surface area of the old system and the monolith can be removed.

The modularised architecture solves a number of problems but you still have to solve the alignment challenge across teams. In a project world the senior management review every proposed feature and make the decisions on which features (projects) should go ahead. We want our teams to own their sub-products so this level of control stifles their autonomy and therefore the innovation on the product. Another way of delivering the necessary alignment is required.

The alignment approach used is often referred to as Mission Command, sharing the goals without explicitly stating how to achieve the goals. The benefit here is that the teams closer to the customer can identify better ways of achieving the business goals and can react quicker to changing situations. By sharing the business objectives of the company along with the metrics that will be used to determine success management can set the boundaries for the teams while still encouraging their creativity and innovation.

There is still a risk that teams may work against each other - hitting one goal might hurt another. This is where product management is required to coordinate across teams to ensure that such risks are minimised. So while the product structure reduces the overhead in alignment it doesn't remove it completely.

Next Steps

Its easy to talk about the steps required to achieve alignment or modularising an application but it is a lot harder to actually do it. The changes required impact nearly everyone in the organisation from the teams stepping up to take more ownership of their products and getting up to speed with the new architecture, to the managers who need to become more comfortable delegation control, to the finance and governance teams who deliver the oversight of the product teams. Rather than hope that people will be able to adapt and hit the ground running it makes sense to support and provide training because you run the risk of replicating the mistakes that other companies have already made.

If you want to learn more about how to build the necessary alignment and execution skills and learn about the mistakes that others have already overcome check out the UXDX conference where we have stages dedicated to each of these topics. The Vision stage focuses on the business, customer, process and team alignment whereas the Execution stage focuses on the skills teams need to deliver systems at scale.

Let me know your thoughts on how to solve the scaling challenge. There is no silver bullet so I'm always keen to hear about new and exciting ways that teams are innovating.