I have often found myself asking, "How would I improve this process?" I recently found myself asking this question about Scrum and Kanban and I wanted to discover if one could be measurably better than the other. First, I had to do some discovery to make sure I knew how each of these project management frameworks differ.

If you know how Scrum works, you are well aware that it attempts to be Agile by always iteratively improving itself over sprints. Over a sprint, which is usually and incorrectly defined to be a set amount of time, a team goes through a few meetings to improve their productivity. Some of these meetings are to estimate how much work is required for the team to finish a project, some are to reevaluate how their current process of work is going, and some are to go over the current or upcoming sprint. Throughout a sprint, the team members are assigned tasks that are in the sprint scope and work to deliver all tasks that were assigned to the current sprint. Scrum also labels a task size by points. Points are usually not supposed to be tied directly to a set amount of time, but that's how we will be treating task size in this simulation.

Kanban works with less focus on meetings and more focus on the task process. With Kanban, meetings are held to fill a single backlog of tasks. This single backlog is the only place where all team members come to get assigned a task. When a team member is finished with their current task, they come back to the backlog and assign themselves a new task.

I realized here that I could do a simulation using multiple working threads and a few rules to measure whether Scrum or Kanban has a higher output of work. However, I had to make a few assumptions for both Scrum and Kanban.

Assumptions

1. Both Scrum and Kanban meeting durations were ignored. All "meetings" in this simulation happened at a computer's speed to fill in each sprint or backlog for both Scrum and Kanban.
2. Typically in Scrum, tasks are assigned to a team member before the sprint begins and the team member must complete all of their assigned tasks before the sprint can be completed. For purposes of this simulation, all team members ignored assigned tasks and worked to complete all tasks in a sprint together. On past teams of mine, this has been the most common implementation.
3. Instead of having a sprint be a set time, I determined that a sprint was complete when all tasks inside of a sprint were complete and allowed all workers to move ahead to the next sprint. I found that if I didn't implement in this way, the optimal time to complete all work would have been a simple calculation: completedTime >= SUM(totalWork) * sprintTime / sprintSize This is because we would always be waiting for all workers to finish the work for every sprint. The reason the calculation contains a >= is because every sprint has the chance of not being completed in the sprint's allotted time.

Simulating Scrum

In the simulation of the Scrum strategy, I used Python to give me access to 5 threads which represented team members. Each team member had different but static speeds at which they worked on tasks. There was 1 Principal Engineer working at 1x the rate, 2 Senior Engineers working 1.25x slower, 1 Mid-level Engineer working 1.5x slower, and 1 Junior Engineer working 2x slower. I had a long list of tasks that the workers needed to complete. The tasks were simply positive numbers that represented how long the thread would fall asleep before asking for another task. Because this simulated Scrum, I created a queue in which a team member could come and grab a new task when they completed a prior task. After each sprint was complete, the whole team could move to the next sprint (or queue) of work.

Simulating Kanban

Similar to the simulation of Scrum, when I simulated the Kanban strategy, I used Python to give me access to 5 threads which represented the same team members as in the Scrum simulation (1 Principal Engineer, 2 Senior Engineers, 1 Mid-level Engineer, and 1 Junior Engineer). To make sure comparisons were always equal, I always used the same list of tasks for this team to complete. This team, however, did not need to worry about separate queues. This team only had one queue of work to pull from until all work was completed.

Findings

I created 4 different sets of test task data. Each of these tasks were set to 1, 2, 3, or 4 seconds of work. Here are the results:

Let's look at this first set of data. The first run was a little misleading. What I realized was that both Scrum and Kanban ran at the same speed because there was only one sprint of work for Scrum. This means that both worked off of only one queue. Once I got to the larger sets of data, Scrum began to use more queues which created larger waiting times for other workers to complete before starting the next queue. However, in Kanban, there was only ever one waiting time, which was at the end when all tasks were completed. I was quite surprised when I saw that Scrum ran on average 1.20x slower than Kanban. I wanted to push these tests further.

I wanted to try this experiment again and see if I could speed it up. This time, I made all tasks be the size of only 1, 2, 3, or 4 milliseconds. This meant that I now needed to shrink the size of the sprints and also increase the size of the sets without having the tests run for hours. Here are the results:

Conclusion

My initial hypothesis was that Scrum would measure 5% to 10% slower than Kanban. However, I was shocked to see that Scrum was at least 20% to 37% slower than Kanban. I knew from my experience that Kanban had an edge on output, but I would never have guessed it was this big of a difference. If you would like to check my work or even just take a look at how I put this together, take a look at my test repo.

One thing to note before everyone who reads this article now thinks that Kanban is always and will always be better than Scrum: we do have to take into account what Scrum offers. I now firmly believe that Kanban may give an edge on output. However, Scrum does give the project manager the ability to predict just how long a project might take and track the progress to the final goal. Kanban might be able to learn some things from Scrum in that regard, but now I have measured the result and found that Scrum has things to learn from the methodologies of Kanban.