From Average to Exceptional: The Journey of One Development Team to Increase Productivity and Business Outcomes

From Average to Exceptional: The Journey of One Development Team to Increase Productivity and Business Outcome. Photo by Tobias Mrzyk on Unsplash

It might not be a surprise to you that engineers have always been in high demand in our industry. Highly skilled software developers are scarce. This is a reflection of the challenges of the past, present and future of software engineering colliding. Every year, more efficient, more user-friendly, more ingenious software is needed to run our worldwide economy. As a result, software is becoming more complex as well. And as threatening as ChatGPT may seem, it will most certainly not replace the innovation, collaboration, creativity, and the think-on-your-toes problem-solving of most human programmers for the next decade. In contrast, I believe that we need even more highly skilled (human!) software engineers to keep up with the speed of innovation and technology.

However, our industry has some big challenges to face. Only 16.2% of all IT projects are deemed successful as reported by the latest report by The Standish Group. Every year, we lose more than 300 billion dollars because of developer inefficiencies. Based on research, a software developer works, on average, seventeen hours a week on maintenance tasks. The latest State of DevOps report (2022) showed a drop in high-performing teams and an increase in low-performing teams compared to 2021. The researchers hypothesized that this could be due to less effective communication because of the pandemic but time will tell if that is indeed the root cause. 

Even if organizations have adopted Agile or DevOps, the biggest problem of all remains: More than 50 percent of all the projects or features developed by software developers aren’t delivering the expected business value. Rather than pandemic-related inefficiencies, I suspect something else is at play here that is holding us back. It is the ultimate soft skill that if we could master, could change everything. I am talking about behavior change. In my years of working with software development teams to achieve their goals through the use of Lean OKRs, I kept circling back to behavior changes again and again. I am sure you’re all aware of that saying: “Insanity is doing the same thing over and over and expecting different results”? Well, I postulate that behavior change is the remedy to revive our projects and set teams up to face the demanding future of lightening-speed innovation.

New behavior = new results 

Moving the Needle with Lean OKRs

But how to do this? How can we build these high-performing teams? More specifically, how can we change the behaviors of teams? Because, if we want to see better and new results, we must create new and better habits and behaviors. For example, we know, based on decades of research and Agile software development, which principles and practices work and won't work. It fascinates me that we, as an industry, are so drastically underperforming compared to other engineering disciplines like civil, or construction. Why aren’t teams applying these principles and practices? What are we missing? This question has become central in my research over the last few years and I must admit, I’m still learning. 

I’ve worked with hundreds of software teams throughout my career. I’ve worked in both low-performing and high-performing environments and held various roles within these teams from software engineer team member, to mentor and manager. So from experience, I know what it’s like to work as a part of these high-performing teams and how to lead them. If you’ve ever worked in such environments, you’ll truly never want to work anywhere else. When I was in leadership roles, I borrowed several techniques from the field of behavioral science. By changing things up, by shifting our focus from reactionary to that of learning, we saw notable, trackable results and were able to seemingly pivot on a dime.

I can still remember my first day so clearly. I was working for a big financial company and I was recently assigned to a new team as the lead engineer. We were a cross-functional team with four engineers and three business analysts. The team worked with Scrum (forced by Agile coaches), had a huge backlog of work, and hadn’t released any software for the past half year to a production environment, let alone gotten any feedback from real users. 

Many team members had a passive working attitude and I was totally taken aback (although I guess I shouldn’t have been based on their results) Coming in late, leaving early. Working from paycheck to paycheck without any strong motivation to deliver their best work. They came to work, put on their headphones, and only woke up to get some coffee, smoke, or grab lunch. If you ever find yourself walking through an open space full of software developers with headphones on, prepare yourself! It has to be one of the leading indicators of low-performing teams I’ve come across. 

The days started out OK I thought, with daily stand-up meetings with some directional report-outs. However, I quickly noticed that when somebody spoke, the rest was staring at their toes as if their shoelaces had become the most fascinating invention they’d seen. The demos and retrospective meetings were similar in nature. The demos were PowerPoint presentations designed for stakeholders of what the team was going to do over the next half year. The retrospective was a meeting facilitated by the assigned Agile coach, in which it appeared that the team members had had an arts-and-crafts session and summarized the same problems but in different formats and layouts of colorful sticky notes. Sometimes it acted as a therapy session, with tears and all.  During my early observations, there were never any changes made to their way of working. Stakeholders weren’t involved, so couldn’t complain too much. The business analyst had offered the team’s services to the company’s large call center, but barely got involved in talking about any software development work. 

It wouldn’t have taken a professional to see immediately that something was wrong. Yet, apparently, I was the only one. When I asked my team members how things were going, their response was something like: “Just fine, it’s almost the weekend”.

How could this group be transformed from a bunch of individual contributors to a high-performing team? This is where those techniques from behavioral science made their grand entrance. Back then, I researched behavioral science based on my hypothesis that this soft skill had to be our secret weapon because these people had impressive hard skills, competencies and years of experience that in principle should make a rock-star team, but instead were belly-flopping on an almost impressive scale! In recent years, this soft skill was featured extensively in my book called “Lean OKR’s” which takes the OKR goalsetting framework and applies both Lean thinking and behavior changes to see remarkable results. It is a practice I have been applying to software development teams for years that has cracked the code, so to speak.

With that said, there are four steps involved in changing a team to adopt new engineering principles and practices. I prefer to blend some of Kotter’s eight steps of behavior change with some of my own.  

1. Awareness & Urgency

2. Set a goal

3. Change Behavior

4. Behavior Change Support

There’s that one quote: “Change happens when the pain of staying the same is greater than the pain of change”. This is how change happens, isn’t it? Somewhere along the way, we realize that something isn’t working, in short, we’re unhappy. From there, we imagine where we want to be. We set a goal. Not an overly ambitious one, not to start out with, because at this stage we need to build confidence, too. I’ve learned that stepping out of a comfort zone is a fine balance. Not enough of a challenge and old behaviors quickly rear their ugly head, too much of a challenge and there is uncertainty and resistance. There is a sweet spot where behavior change is introduced, grows, is fostered and eventually becomes a new normal.

Steps three and four can be treated as experiments and therefore are non-linear. Any good experiment is always circular anyway. I tend to mix and match behaviorial changes and supporting those changes as I feel it lends itself more easily to a learning mindset.

Awareness & Urgency

Let’s go back to our story of this team and talk about that critical first step that they took, what I believe to be a fundamental first step in any change process: creating a sense of urgency. If you don’t know you have a problem, you will not change. If you don’t feel pain, you won’t take any action. This was also true with the new team I was assigned to. 

One of my favorite tools to create awareness for change is to play games. Besides instant human behavior change (you’ll know what I’m talking about if you’ve ever played Monopoly with your family), it’s a non-threatening environment (for most people then). In many scenarios, I like to play the Agile Fluency Game (AFG) created by James Shore and Arlo Belshee. It’s a super realistic European style board game that simulates two and a half years on a software development team. I play it with board rooms, executives and software teams. The total play time takes 90 minutes, plus some additional time for the introduction and a short debrief, so not a big time investment overall.

My first step was to get the whole team to have lunch (which was a challenge with conflicting agendas and such). Next, during lunch, were people seem jolly and optimistic,  I would do the ask: “It’s so nice to sit together and have some fun with you all, would you be open to playing a new board game with me on a Friday afternoon over some drinks and snacks?” Since some people are (still today) a bit allergic to anything with the word ‘Agile’ in it, I prefer to call the game the Software Simulation Game. Luckily the team loved playing board games, so it was easy to convince them to play that same week.

That Friday afternoon in November I introduced the game. Since you need an officially trained facilitator to play the game, I suggested facilitating the game myself. Within two hours, the team learned more about concepts like User Stories, TDD and Continuous Integration, Technical Debt, and the importance of experimentation than ever before. It was a true eye-opener for them.  

Immediately after the game, they became super enthusiastic about trying some of the elements from the game in real life. If you find this interesting, feel free to pull me aside later today to see how you can become an Agile Fluency Game Facilitator, I just so happen to be a licensed trainer. So far, so good my marketing pitch! Anyway, my mission to create awareness and urgency for this team to change was a succes. The success rate on average is about 80%, which (not to brag) is my score after playing the game with over 50 teams now. Yet another reason I love the Agile Fluency Game; it is good for my self-confidence too, because I see in real time that I am helping others.

Metrics

All games aside, another tool to create awareness is the (correct) use of metrics. These days there is so much hype around DORA metrics, Engineering Performance Metrics, and of course OKRs, that it is difficult to understand what helps and what doesn’t. Many mix up the use of these tools and techniques (which is a topic for another time).

The week after this very successful game night, I suggested running an Agile Fluency diagnostic to see where the team was at in terms of Agile Fluency, a model created by James Shore and Diana Larsen, that helps measure a team’s capabilities and behaviors.  Of course, there are many Agile or DevOps self-assessments out there, but I tend to prefer this one since it is measuring behavior and capabilities, instead of checking the boxes on some technical tools you need to have installed (yes, I’ll say it, using Jenkins does not mean your team is practicing CI). Other tools use binary questionnaires to measure if a team performs certain best practices, which for me, isn’t the point. For example, you can say yes to “do you perform code reviews as teams?”, but that doesn’t help a team understand why that practice is important. In contrast, the Agile Fluency diagnostic lets you examine if you have the capability to deliver code without bugs (or only a few). There are many practices to achieve this level of quality,  ‘Code Reviews’ being one of them (though not a very effective one in my humble opinion).

Going back to the story of our team, running the diagnostic made the team realize where some of the big improvements could be made and where challenges of the team were hidden. One of the biggest challenges was the realization of not getting fast enough feedback from real users. Another one was the existence and accumulation of technical debt, which caused changes in the system to become slower and slower every day as well as more complicated to perform. After creating awareness with both the Game and metrics, their eyes were opened, the urgency was created and the team was highly motivated to do something about it.

Setting goals

So we were now fired up, and our next move was to take action. As a team, we started to write down a goal for the months ahead: “Improve the speed of getting feedback from real customers”. We also defined some specific behaviors the team needed to perform to get there and agreed that we will discuss weekly if we actually performed these behaviors. Why? For accountability, of course! Some of these behaviors included “showing a product demo to at least five real customers per week” and “pair programming for two hours each day”. Although the team’s main concern was that they have never done this before, I comforted them that I had the experience to carry them through. I’ve found that having senior members with this experience is very helpful but not necessary. Another concern was time. How much can we spend on trying this stuff out? I said we could start with an hour a day and take it from there. The whole team found this a reasonable approach and we agreed to trying it out that same week.

Providing this kind of support is critical in behavior change. If you need to perform a certain behavior, you need to be able to have support, guidance, encouragement, and feedback from your environment to continue the pursuit. In this case, a manager or lead allows for time to experiment or on-the-job training. To no surprise, these are again some behavior change techniques that have been proven to work by science.

The behavior change

From the next day onwards, each morning, we came together and planned work for that day with the whole team together. We decided on how to split work between pairs, including the business analysts. As you can imagine, our team meetings looked drastically different to when I had first started with the team. They were engaged, accountable, and had fresh ideas.

During the pairing session with the engineers, I was able to teach them how I prefer to program: TDD, write tests, and build confidence in refactoring. After a few days, engineers loved to pair up with each other. Every day, they were more eager to try out some of the programming concepts and practices I had taught them. They wanted to do more of that (yes, software engineering done well can be incredibly addictive). 

The week after, during the Monday morning check-in routine, one engineer said “I’ve learned so much that last week, this is amazing”. A business analyst said: “I was skeptical about this approach, but together we had so many great ideas to reach out to customers, I would have never come up with this on my own.”

We decided to extend the pairing time to two hours each day. Starting small and then expanding is another behavior change technique. 🙂

As a result, three weeks later we even tried mob programming on some small tasks with the whole team. With mob programming, you sit with the whole team behind a single monitor (or screen). Only one person is allowed to type. Is a variation of pair programming which is very effective for knowledge sharing and moving the needle. It won’t surprise you that this technique is based on some behavior science techniques as well.

After about two months, I could hardly recognize the team from my first encounter. They came in on time, full of energy. We collaborated with each other the whole day as a real team during mob sessions. No more headphones, but rather great conversations with each other. Discussing how to run our next experiment in our production environment and how to measure it. Business analysts were closely involved with the engineers because they finally could try out things that normally would take months. New ideas on how to improve both the system and how we worked came from both engineers and analysts alike and were tried out every week. We were able to deploy software to production after every code check-in, resulting in multiple deployments per day. Even in this harsh corporate environment.

But above all, people in the company started to notice our effectiveness. Real users collaborated with business analysts and engineers, resulting in a system that was saving everybody in the call center a ton of time. 

The story of this particular team is still ongoing. They now work on software systems that bring the highest impact for the company. Some team members left and joined other teams, helping them to change as well.  Sometimes the team members have regrouped and built other great products for the company.

What started as an average corporate software development team, resulted in an exceptional, high-performing team. Was this a unicorn? Not at all! I’ve repeated this journey many times throughout my career.  Every time I learn and improve too. Applying behavior change techniques to help engineering teams become high-performing has become my passion. I’ve studied and tried many techniques. OKRs, metrics, training and mentoring. However, without the support of some bright leaders, this wouldn’t have been possible. Just as the team of this story, I myself have needed strong senior leaders in place that have had my back and supported me, too.

The brilliance of this “ultimate soft skill” is that you will see virtually immediately when productivity drops. When teams have direct access to technical mentors that have studied and practiced behavior change techniques, they can be brought back to the Steps: awareness & urgency, the goal, the behavior change to get there and the support in learning and experimenting. This story isn’t a description of a utopian ideal, it is the “meat and potatoes” of how to go from an average, maybe even below average team, to an exceptional team. It is an investment in time, money, energy, but most importantly in people, that is so well spent and where the return is more than just the bottom line. It is people that been activated to seek out ambitious goals and make measurable changes.

DevOps new?

In this blog, I look at the rich history behind DevOps, which goes back over 200 years.

Actually, it’s two centuries (and arguably two millennia) old. 

Welcome to a new series of blog posts on lean software development. This is a topic very close to my heart. Over the years working as a software developer, architect, consultant and more recently coach, I’ve seen just how much waste there is in our profession. Wasteful software, wasteful development, wasteful processes. And I passionately believe that by drastically reducing that waste we can drive a minor revolution in our industry.

And if you think revolution sounds a bit over-the-top, consider these figures: when properly implemented, a DevOps culture can reduce time-to-market by 400-500% and deliver software within minutes instead of weeks. Turbo-boosting your digital transformation journey. In this first post I trace the origins of lean software development (LSD) from late 1800s right through to ‘DevOps’ today.

You don’t just ‘do’ DevOps

Many of you will be familiar with the 3 DevOps principles (a.k.a. The Three Ways), and some of you probably also know that they’re based on the principles of LSD? In which case, unfortunately, you know a good deal more about the topic than many organisations that claim to be ‘doing’ DevOps. All-too-often, software development teams assume that having a few experts working in the infrastructure domain, or doing their own deployments to production, means they’re doing DevOps. This is, to put it mildly, a misunderstanding.

DevOps is much more than having some ‘operation’ expertise in-house. It’s about having the culture of team cooperation and technical excellence needed to be able to focus on speed and waste elimination. So what exactly are the fundamental principles on which DevOps is built, and where do they originally come from?

The 20th Century: from thinking big to thinking lean

In 1910, legendary car maker, Henry Ford, took manufacturing to another level by championing and refining the assembly line technique of mass production. In fact, his operations were such a step-change in manufacturing efficiency that many historians argue that his methods were a key determining factor in the Allies’ victory in World War II.

Then between 1949 and 1975, the Toyota Production System was developed by Taichii Ohno and Shigeo Shingo, including Lean Manufacturing or ‘Just-in-time’. The principles and behaviors behind TPS are captured in ‘The Toyota Way’. But because a production system can’t be directly applied to product development, Toyota later created Lean Product Development (LPD), based on the same principles.

LPD has been shown to deliver a 10-fold increase in innovation1 and 400-500% increase in the introduction of new products1. There’s no empirical evidence yet confirming that Lean Software Development delivers the same high-efficiency percentage gains, but they likely fall in the same range.

But it was only in 1990 that the terms ‘Lean Thinking’ and ‘Lean’ were introduced when, following an in-depth study of TPS, Womack et al. wrote The Machine that Changed the World. The book became a bestseller and industries across the globe came to recognize the significance of lean production systems.

The start: lean manufacturing

In 1799, the American Eli Whitney (also, incidentally, inventor of the cotton gin, a key breakthrough in the USA’s Industrial Revolution with the tragic side-effect of galvanizing the slave-based cotton industry) adopted and popularized the principle of ‘interchangeable parts’ (though he didn’t invent them: archaeologists have found interchangeable weapon parts from the Punic War waged between Rome and Carthage from 264 to146 BC). As a result, Whitney could provide the US War Dept. with 10,000 muskets for the incredibly low price of $13.40.

Over the years, modifications in the production system by people such as Frederick W. Taylor, Frank and Lillian Gilbreth, and others developed the principle of eliminating waste in manufacturing. And eliminating waste is, of course, one of the core principles of Lean.

7 principles, 3 Ways, 1 foundation

In their 2007 book, Implementing Lean Software Development: From Concept to Cash Mary & Tom Poppendieck2 did a wonderful job of translating TDS to the software development domain, heralding the birth of lean software development. Which brings us back full circle to the ‘Three Ways’3. Because these three principles are based closely on the 7 LSD principles:

• Eliminate waste

• Amplify learning

• Decide as late as possible

• Speed: deliver as fast as possible.

• People: respect for people. One of the key pillars of Lean.

• Build integrity in

• See the whole: refers to systems thinking.

A quick look reveals that ‘the three ways’, from which all DevOps patterns can be derived, map easily onto these 7 LSD principles:

• The First Way: systems thinking can be mapped onto ‘see the whole’ and ‘eliminate waste’ to optimize business value streams.

• The Second Way: amplify feedback loops also maps onto ‘eliminate waste’, as well as ‘decide as late as possible’, ‘deliver as fast as possible’ and ‘build integrity in’.

• The Third Way: culture of continual experimentation and learning clearly maps onto ‘amplify learning’.

The foundation: respect for people

But the foundation underpinning all the principles is ‘respect for people’. Without great teams and great people, you simply can’t implement the others principles. Respect for people also includes fostering a work culture and morale consistent with research into workplace satisfaction and increased motivation, as this will support continuous improvement4.

So next time at the coffee machine someone from another department asks what this new thing ‘DevOps’ is, and your colleague answers it’s all about time-to-market, there’s your chance. To point out that (a) the principles actually go back 200, and arguably even 2000, years and (b) it’s not just about time-to-market. First and foremost, it’s a culture thing. About growing a culture of people- and teams-first. Only then can an organisation hope successfully to implement the principles of lean software development.

References

1. Allen Ward – Lean Product and Process Development Oosterwal, Dantar (January 13, 2010). The Lean Machine. Productivity Press. pp. 237–240. ISBN 978-0814432884.

2. https://www.amazon.com/Implementing-Lean-Software-Development-Addison-Wesley-ebook/dp/B00HNB3VQE/ref=sr_1_2?s=digital-text&ie=UTF8&qid=1492788317&sr=1-2&keywords=lean+software+development

3. http://itrevolution.com/the-three-ways-principles-underpinning-devops/

4. http://less.works/less/principles/continuous-improvement-towards-perfection.html#RespectforPeople