From Scrum to Kanban, How we increased the delivery and fluidify our organization

Story of an Agile change in Catalog Feature Team

The Catalog Feature Team is a mainly frontend team, composed of a Product Owner, a proxy Product Owner, 4 frontend developers, an UI designer, a QA tester, and me the technical project manager / scrum master.
We work on the evolution, and the run of our website on the product pages (detail page and list page), the navigation (menu, header, footer, etc). In few words, all the pages except the checkout.
The initial state was : we worked with Scrum framework, with sprints of 2 weeks. We have the classicals rituals : daily, sprint planning, a kind of retrospective. The demo is missing at the initial state. And all the feature teams build a macro planning/estimation for 3 months. It’s the quarterly planning called RPD.

Scrum and Kanban: same problem, two philosophies

Scrum and Kanban are two iterative systems based on flows. Originally, Scrum was created for software development and Kanban for lean manufacturing.
The Scrum philosophy highlight the team work by the iterative evolution of the product. Each iteration have a fixed duration, for example 2 weeks. The goal is to deliver new features at the end of an iteration.
Kanban is a little different, we don’t work by iteration but by priority. The goal is to limit the WIP, and deliver as soon as possible the more priority ready topic.
Both are Agile methods; the main differences are:

Note that these items are theoretical rules of Scrum and Kanban and need to be adapted to practical cases.
On our side and for the transition of the team, these rules helped us for the choice of the target method. By comparing pain points and theorical rules, we saw that Kanban could be a good solution to help the team to resolve some identified problems targeted during our sprint retrospectives.

Why this transition?

This transition started in our team with the ambition of challenging, managing and fixing pain points.
By convention, the team was organized around sprints, and we said that the team worked in Scrum. But, in fact, we have often seen that the main rules of Scrum were not respected.

Commitment fail

About the commitment fail, during more than a year the sprints have rarely been finished. It’s a high distance with the Scrum rule, because, even if we can fail for a sprint, it shouldn’t be the standard.
In our case, the fact was about a third of the User Stories were finished (deployed in production) at the beginning of the sprint, about a third was implemented and waiting for tests, and the last third was moved to the next sprint for several reasons:
– Scope change,
– Bad estimation,
– Topics not ready,
– etc
To answer this pain point, it was decided to work on the Definition Of Ready (DOR). We organized a team workshop and produced this DOR, as well as a User Story(US) template.
The result was that the US were clearer. Nevertheless, we didn’t have enough ready Stories to create our sprints.

Unsuitable rhythm

Another Scrum rule we couldn’t follow was the rhythm and cadences. As we saw just before, for many reasons, we moved topics to next sprints.
On the one hand, it was due to the internal context, and external factors, and in other hand, due to the delivery process. Basically, the technical flow (from development to production delivery) is continuous and when a User Story is completed, it is delivered in production.
The delivery pace was different from the team flow. For the delivery, except the weekend, and the freeze period, there is a release in production daily. It means that as soon as a topic is tested and validated, it can be live next day. In opposition to the concept of sprints, where we plan a package of topics for 2 weeks.
About the rhythm, in the team we saw that we have a real difficulty to estimate our Stories. One of the identified causes can be that the team is a « end to end » team and the development velocity is very different from the functional testing velocity.
For this reason, at the end of the sprints, we had a part of the Stories developed, but not tested, and so reported to the next sprint.

Iteration not respected

Iterations – more often named “Sprint” – were not respected, because in many cases the Sprint content changed.
Keep in mind that our team works with many business units, because we are the front view for many of them.
In this context, during a Sprint we had several events:
– New topics,
– Not relevant topics,
– Incoming high priority topics,
– Legal topics,
– Scope Changes, or priority changes,
– Etc.
In fact, even if we created a sprint with a good prioritization, and with good inputs, we noticed that the content was often reviewed few days after the Sprint schedule.

Team frustration

The last axis, and this one is not based on the Scrum rules, it is a fact.
The team mood could be increased, and there was a kind of rising frustration in the team.
There were four topics:
– The team must commit to deadlines and planning with missing inputs
– The team must schedule sprints over 3 months with too little information
– The team must build sprints with not ready subjects
– Team lacks understanding of business needs

THE CHALLENGE : Test a new Agile method according to the working environment while increasing the mood of the team

How did we do it?

To give some context around this decision, I want to point the fact that the Sprint Retrospectives are an Agile Ritual that works fine in the team. And the team is always aware and participative during this meeting.
During 3 sessions we noticed that a major part of the actions concerns the pain points of the Scrum method, and it was decided, during a Sprint Retrospective to study Kanban. We highlighted the pros and cons and decided to test the approach.

The positive points

The first step of the transition was to study the positive points.

The Kanban flow is the mirror of the technical flow.

• Technically the delivery flow is a continuous flow, and Kanban rules say that we have to take tasks one by one
• The goal of Kanban is to deliver as fast as possible an open topic. And the delivery flow is continuous. That means that a developed and tested topic can be delivered a few days later.
• With this method, we think that we can deliver faster.

Align with Business workflows.

• Often, the business needs to change the priorities, add hot topics, or postpone topics. Kanban rules allow this working flow.
• Where Scrum says that a sprint can’t be modified, and the team commits on the Stories that compose the Sprint, Kanban allows a continuous prioritization of the backlog.
• And because we start User Stories one by one, it’s possible to add topics, or change the priorities daily.

Team mood increasing

• With Kanban we will have cleaner board. Before the transition, due to the velocity’s differences between development, tests, and UI, there are a lot of tickets in the board, and it became difficult to read.
• We will have more ready topics. Indeed, the backlog can be prioritized daily, and just with this, the top of the backlog will always contain « Ready to Dev » topics.
• Opposite to Scrum, the Kanban allow to help all the team person, and the roles and responsibilities are more flexibles. By being included in some functional or test phases, the developers can better understand the business needs, and the team will have ownership of the topics.

The negative points

We don’t want to only highlight the positive points. For that reason, we studied what our negative points would be.

This method was new in the team, and we knew that we would have to work step by step.

• First initialize Kanban with “theorical” concept
• Then, test the approach, and adopt it in the team
• And then, adapt Kanban to the team, and to the company context

Manage expectations and deadlines with lower predictability.

• We don’t want to put deadlines on each project, or each Stories, because it doesn’t make sense, but sometimes it’s mandatory. For example, legal or strategics topics, or topics that are shared with other teams still on a scrum planification.
• We don’t see in the theorical Kanban rules the way to do it, but we think that we can use Jira field to put the deadline and manage it.

We will no longer be aligned with other teams

• About key dates, like The quarterly planning rythm, Sprints start, etc. as if we were in another timeline.
• It can be harder to be in phase for transversal projects for example.

A new board, and a wonderful Backlog

The 2 firsts steps in term of changes were to work on the Backlog, and the Board.
This work was done with the Product Owner and the Scrum Master of the team. We start with a big clean of the backlog which contained about 400 tickets, some of them were not updated for 3 years.
The goal was to identify irrelevant topics, and what were the topics that needed to be relaunch, and for these ones, the topics that needed new inputs to start.
We spent approximately 2 days, divided in 4 sessions to fully clean the backlog.
We kept less than 100 relevant tickets, and half of them needed to be analyzed functionally, or needed additional information from the business teams.

And the other step was to build a new board, and a new flow for the team. Two main goals for this part: have a cleaner board and have a flow that matches with Kanban method.

The new Board contains

2 columns for the preparation
– Refinement: in this column we will put the tickets that need a functional information (business definition / rules, mock-ups, etc)
– Study: the tickets that are currently in technical design
4 columns for the work / development phase
– Ready to dev
– In progress
– Testing (dev and functional)
– To Deploy (step just before the release in production)
2 « swim lanes »
– Blocked (the goal is to group the opened topics that need to be restart later, we will see the usage in the part “new team flow”).
– Expedite: used when we have a blocking topic, production issue, etc.

No big changes for the rituals

Some Agile meetings were kept, some others are changed :
– Daily meeting (09:30am – daily)
– No change on this meeting, timeboxed to 15min
– Refinement (mon. 03:30pm – weekly)
– Each week’s goal is to present the new topics, check the inputs and, if possible, estimate them. If they are ready, we can put them in the backlog, and add the priority.
– We keep the estimation in story point, that is a relative unit, it represents an effort or a difficulty, not a duration.
– And the prioritization process take 3 criterias: the legal needs, the performances issues, and the user/customer value.
– Backlog review (wed. 02:00pm – every 2 weeks)
– It’s a new meeting that replaces the Sprint planning. The goal is to check the « Ready to dev » topics and give a priority to all these topics.
– Important: it’s the « official session for prioritize », but in fact it’s possible to adjust the priorities daily.
– Feature demo (on demand)
– For the concerned topics, we make a demo to the business team, and stakeholders
– We changed a bit this ritual. Before the transition, we made a big demo at the end of the sprint, now we make short demo on a dedicated topic/feature, with only concerned persons. It’s more efficient for the business teams.
– Retrospective (wed. – monthly)
No change on this meeting, the retrospective is based on the past month.

A new team flow

The goal in Kanban is to have a pull flow. A topic needs to go in production ASAP.
We define a simple rule for developers to optimize the « go to production » flow, when they finish a ticket:
– Ask to the team if someone need help (dev, QA, etc.),
– If not, have a look to the defects on open topics,
– If no defect, have a look to the blocked swim lane,
– If no blocked topics (or if nothing can be unlocked), you can take a new topic.
With this flow, we try to keep the board clean, we try to deliver faster in production, and we try to have less blocked topics.
Indeed, when a developer become available to work on another topic, he/she tries to help the team to deliver in production the current open topics.

NOTHING IS FROZEN AND IT’S IN CONSTANT EVOLUTION
The 2 firsts steps are done, initialization and adaptation, but we continue to work on improving the method.

Feedback after 5 months?

We started Kanban 5 months ago.
We don’t have factual performance metrics because the team composition has changed, with new arrivals, etc. It’s a bit difficult to get metrics on production delivery in this context, and we’ve had some big projects divided differently than usual.
But, based on team feeling, and ascertainment:
– We deliver faster in production
– We have less « abandoned » tickets in the board
– The backlog is cleaner:
– Minus 30% tickets
– And large decrease of irrelevant topics
– The team mood is better
We compared with a survey some items before and after Kanban:
– Fluidity of delivery
– The board readability
– The team mood
– Etc.

The question “Would you like to come back to Scrum in the current context ?” was asked to the team, and we are unanimous, the answer is no.

THE CHANGE IS A GREAT SUCCESS AT TEAM LEVEL
The next steps and next improvement will be first to find a better process for the demos, because it’s not smooth today. And the second one is to find a way to be aligned with the other teams that are still in Scrum.