11 rules of thumb to help you measure the success of your #Agile/ #DevOps coach

By : Liat Palace  (Director, Delivery Technology Office Agile/DevOps Coaching Team Lead – Amdocs)  & Shirly Ronen Harel (Co-Founder & Agile / DevOps Coach -WeChange)

While many people use Agile and DevOps coaching services, few of us really know how to measure the success or failure of the coaching process.

What will help determine whether the coaching was a success or failure?

The working environment is complex and includes many parameters that can influence the success of every implementation; in this context, how can we identify the coaching achievements? 

The most important thing is to determine as early as possible whether the coaching is effective, in order to ensure that the efforts are productive.

Here are 11 rules of thumb to help you validate the coaching effectiveness.

1. Clear Expectations & Definition of Success

The first and most important thing a good coach will do is understand his customer’s/ (Or  The project sponsor in some other cases)  perception of success. This means that the coach will ask the project sponsor what he/she considers a success.  “Once we’re done here, or have made progress, what will the change look like?” What will the sponsor consider success? Is it a better delivery rate? Solving communication problems? Successfully addressing quality issues? Better handling of unexpected issues? Is the success story a realistic one? Does the sponsor expect magic? Does he expect aggressive demands on the team? And so on…

The definition of success will affect the implementation vision and goals, along with the ability to achieve success and impact. It will also impact the feedback loops, the people involved, and the implementation journey.

Clearly envisioning success at the beginning will make it easier to identify signs of success and share them.

Here as well, continuous feedback regarding the customer perception of success is required. Things change during the Agile journey, and so does our understanding of success.

Therefore, when asking “are we there yet?”, identifying “there” is key, along with continuously obtaining feedback on the journey.

2. “Language” changes – A wider use of Agile jargon

We start noticing a wider use of Agile jargon, and not in a cynical or offensive way. The use of “DOD”, “READY”, and “working software” becomes more widespread and acceptable as organizational jargon.

Furthermore, the jargon is used as a symbol and tool of the mutual understanding of the actions to be taken. Behind any term there is a wider understanding of what should be done, which is probably different from the “book” or the use of the same method in other organizations.

The use of the terms fits the organization mindset and the process established, and results in fewer misconceptions regarding what, for example, “daily” means, what grooming is for, and so on.

3. “Proud Parent” or “Mr. Miagi” moments – Moments in which the coached team makes the coach proud.

An article in the illustrated Agile blog (8 Ways to Measure Your Impact as an Agile Coach), describes a phenomenon familiar to every coach who delivers value. These are the proud moments when your coached team does something to make you feel proud. These “moments” should occur at least once a week. Keep a log of how often they occur for you. As an example, here is a recent page from my journal, when I captured the experience described above. The impact of an Agile coach should FIRST be measured by the growth in confidence of the coaching environment, and how soon they are able to “stand on their own.”

4. Problem solving – The coach as the Agile “Help Desk”

We start noticing that people seek the coach to help them solve problems, and actively seek his advice in the area of Agile implementation.

This means that the process and the way we work to enable better, faster and higher quality delivery is sufficiently important for us to seek guidance for improvement, and the person to ask is the coach.

A more advanced sign that we are gaining value from the Agile coaching is that not only the coach serves as the source of advice, but also other people who are not managers can give good advice and are perceived as sources of Agile knowledge.

5. Better delivery

The most obvious sign of a successful Agile implementation is a better delivery. Now, what does this mean? Does it mean 100% sprint delivery? Does it mean double or triple the cycle time of user stories? … Nope. It means small improvement steps in team or group delivery.

Real improvement will also include a better understanding and implementation of the DONE concept, which includes quality activities and other lifecycle activities. It means that the team has improved in its ability to balance multi-disciplinary activities. It means that improving tools which affect speed and quality becomes an effort that is sufficiently important to undertake.  It generally also means that communication, problems solving, and waste reduction activities are improving as well.

One of the best indications of good deliveries is a decrease in Escaping bugs. Escaping bugs are customer reported bugs. When they decrease, it means that we have achieved our desired outcome without sacrificing quality.

But this is not the only sign, and probably not the first one. Sometimes we may initially see a period of degradation in delivery, since the team is learning new skills that demand attention and effort, at the expense of other activities.

6. The process itself is a topic of conversation

We notice that just as technical, testing or HR issues are part of the day to day concerns, so is the process. The process becomes part of daily project discussions, management discussions, and team discussions. Improving the way, we work is an ongoing, almost natural issue that is communicated by employees and managers across the organization and throughout the different activities.

7.  Unofficial frequent communication

While at the beginning of the Agile implementation the majority of communication and updates are done during the official meetings, as we progress with the Agile/DevOps implementation, we expect to see much more direct and unofficial communication. People understand that they do not need to wait for an official meeting in order to communicate or solve a problem, and they can approach each other unofficially. Once they are connected to the same vision and understand the project priorities, this creates communication and collaboration that will shorten the problem resolution time.

8. Continuous improvements

Teams that have successfully integrated this mindset are constantly reflecting on what is going on and trying to improve.

You see people standing by the coffee machine, saying things like “we must take this to the next retrospective and investigate what happened here”.

Every process/behavior should be reevaluated all the time, as the conditions are always changing, which means that we are never Done!

This is a live process of changing, evolving and responding; inspecting and adapting and learning from failures.

We need to expect improvements at least once a month, both at the team and project level.

9. TTM of new ideas

New ideas are generated faster, tested faster with experimentation and accepted and implemented faster. This is a good indication of a continuous improvement mindset that allows openness of communication and ideas. We notice that problems are discussed and action items are assigned and actually executed. Small steps are applied instead of big project solutions.

10.  Culture 

High performance organizations differ from low performance organizations in the area of culture. (State of DevOps report 2017)

As the DevOps/ Agile implementation matures, it should be reflected in the organization DNA:

–          Continuous experimentation

–          Treating failure as a learning opportunity

–          Promoting learning

–          Empowerment

–          Employee satisfaction

This is much more than “doing” Agile/DevOps; it is “being” Agile/DevOps.

11. Employee engagement

This should increase, with employees feeling that they are challenged more, learn more, and collaborate more.

How can we make sure that we continue to succeed after coaching phase out?

We usually see implementations fail when one of the conditions changes and the team does not know how to adapt the processes to the new state.

Growing self-repair mechanisms – the ability to change processes, understand the impact and do better all the time – are key differentiators.

Too many times we have seen implementation kits that list all the items to be deployed, but totally ignore the project goals and definition of success.

Let’s be clear: Delivery success should come first and the Agile/DevOps implementation should work towards achieving it, changing and being tailored to suit the project needs.

Evaluate the work that was done; see what was accomplished so far and what was not. Learn from our journey, and from those of others as well.
Prioritize your goals, provide your coach with feedback and update your targets accordingly.

With these rules of thumb, regardless of where you currently stand on the Agile or DevOps path, you are poised to embark on a path to success.