Queueing Theory for Silicon Valley

Is it possible to improve the productivity of the organization without inflicting emotional stress on your employees? Managers of many Silicon Valley corporations go through a lot of pain and drama every year (twice a year in some places) to stack rank their employees, fit them on a bell curve (that supposedly doesn’t exist), do performance improvement plans, layoff the so-called “bottom 5%” of organization, etc.

Today’s post will discuss insights from queueing theory and its implications on software development. Ironically, its usage is more rigorous and prevalent in technology (think networks/routers or code in general) rather than organizational processes. Tech executives who have hundreds of developers in their organization and yet wonder why it is hard to get things done, might find useful nuggets in this post. Let’s dive in…

Table of Contents

• Introduction

• Counterintuitive Math

• The Oblivious Executive

• The Bottleneck at the Top of the Bottle

• Effective CyberSecurity Decentralization

  • Local Efficiency

  • Cycle Time

  • Cross-Train

  • Global Effectiveness

• Appendix: Book Recommendations

Introduction

In 1909, a groundbreaking paper by mathematician Agner Krarup Erlang at the Copenhagen Telephone Company laid the foundation for queueing theory. As telephones surged in popularity, network managers faced the complex challenge of determining the optimal number of phone lines for a given population. This issue was far from simple, as phone calls occurred randomly and also varied in length.

Erlang pioneered the use of statistics to address this challenge, successfully estimating the likelihood of call blockages at various levels of capacity utilization. Today, queueing theory offers valuable insights for software/product developers, who encounter similar challenges with unpredictable work arrival times and task durations.

Queues are the one of the biggest underlying causes of many economic problems in product development - and yet, they remain absolutely invisible and so unmeasured at many software organizations.

Inventory piling up in front of a particular machine on the assembly line in a factory can be easy to visualize. Not so in software development, right? People are easy to blame, but these systemic reasons remain invisible.

Counterintuitive Math

Some of the insights from queueing theory can be surprising/counterintuitive to us. A popular example comes from John D Cook:

How can this be? Why was the wait so long with one teller? There’s not much slack in the system. Customers arrive approximately every 10.3 minutes, while each service takes about 10 minutes.

If you are curious to learn more, I recommend that you read the linked blogpost above and understand more about the Poisson process and exponential distribution assumptions that underpin these calculations - they not only simplify the calculations, but surprisingly also are realistic for many situations.

The Oblivious Executive

Why is it that successful corporations, as they get bigger & bigger, become slower and slower over time? Innovation drops or disappears, quality and security takes a beating, it gets frustratingly hard to get anything done, there is always too much coordination headwind and so on.

Reorganization after reorganization, transformation & change management initiatives, centralization and decentralization yo-yos, change of leaders, mergers and acquisitions to drive up revenue, cost cutting & layoffs in high-cost locations as profits erode, creative accounting practices, outsourcing of thinking by hiring "management” consultants, etc. and yet the problems continue to persist year after year.

Many tech. executives find it difficult to get things done (lack of innovation and decline in productivity) but are oblivious to the complex interdependencies between the various stakeholder teams that have to collaborate to get things done. Things can get very messy when a few key teams/functions (like cybersecurity, privacy, compliance, etc.) are centralized for the sake of efficiency and start enforcing their “policies” via manual reviews and approvals.

Most leaders track project status on Powerpoint slides and Gantt charts. They try to manage timelines instead of queues. As a CXO, if your feedback loop is once in a quarter, doing "Quarterly Business Reviews" sitting in a boardroom, you might be better off trying to drive a car by looking at the rear-view mirror. Many executives have no clue about what's going on in their corporation but proceed at act as it their map is the territory. Further, hierarchy-based decision making is slow and ineffective!

Queues can result in longer cycle time, increased variability and risk. It also affects product quality and eventually people’s motivation. For example, as Mario Platt points out, many of us are blind to the negative impact of handoffs:

So much of our pain at work is self-inflicted because of our blind chasing of efficiency (employees have to be busy all day), specialization and centralization decisions coupled with the failure to effectively manage the interactions of the larger organization with those pockets of specialization. It is high time we seriously questioned the collective assumptions and principles that underpin our management systems.

The Bottleneck at the Top of the Bottle

Here is the real-life story of Mike Duggan, the mayor of Detroit who was once the CEO of Detroit Medical Center (DMC)…

While at the medical center, Mike was tasked with solving a multi-million dollar loss. He starts with the Nursing staff to identify opportunities for improvement and removal of roadblocks. First among their ideas was to speed up patient discharge; it sometimes took up to three hours to get an ambulatory patient wheeled out and on their way home. Mike asked where they needed his help and was quickly told the problem was not with the Nurses but with the ‘Transporters’.

So off he went to the Transporters, trying to find the "culprit" causing these delays. The Transporters told him they had higher priorities: moving people into ER and Surgery, then from Surgery to Recovery, and then to their rooms. They were dealing with sick people first.

Obviously, that felt right to Mike. So, he inquired where he could help them out instead and they pointed out that the lack of operable wheelchairs was the issue. A large percentage of them were always in for repairs. He asked who was the responsible culprit for this backlog, and off he went to the Wheelchair Repair Shop, in the bowels of the DMC facility, to correct the "guilty party".

Mike arrived at the Wheelchair Repair Shop – the first time anyone there had seen the CEO – and he heard a familiar story. The problem was not here, it was with someone else. Someone in Purchasing was the hold up, and so off he went to talk to the culprit in Purchasing.

By now, you may guess that Purchasing also claimed they were not the issue. The Suppliers were not sending parts on time because Accounting was not paying the Suppliers in a timely fashion. Mike went off to Accounting, knowing he must be getting close to the root cause and the ultimate culprit at DMC. His revelation came when he asked Accounting and they confirmed that yes, they were not paying Suppliers on time, rather, they had orders to hold up payments as long as possible. When Mike inquired about who the culprit was so he could hold them personally responsible for this colossal waste of time, the Accountant pointed out that he was acting at the request of the CEO – Mike himself!

As you can see, the chain reaction of events went through multiple departments and eventually came back to Mike himself! He realized how his own instructions to the accounting department went around in loops & turned out to be one of reasons that is “causing" the very major problem he was trying to solve.

Of course, this is an abstraction of the multitude of factors that were at play. If this was at a hospital, imagine how complex the interactions and inter-dependencies can be at a large technology corporation with various lines of business, products, centralization/decentralization and outsourcing decisions of the various functions.

Subscribe to "I'm Serious" to read the rest.