Why We Wait in Lines

It sounds like such a simple question--we wait in lines because there are people ahead of us--but why are there so many people ahead of us? And why can't they processes us faster? Didn't they plan for this? What's taking so long? Few things makes us lose our patience more than waiting in line.

The study of lines is called queue theory--'queues' like the British call them. The British make things sound more refined. Queue theory is important in computer programming, manufacturing, traffic flow, buying groceries and making your next visit to Disney World happier. Recent news about container ships stuck in port is an example of queue theory in action. Unfortunately, the science of lines is not very intuitive. I hope some examples will make it easier to understand. Perhaps this will make you more patient, or help you set up your own lines more effectively.

A toll plaza for example has to take tolls from all the cars that travel on a road. If there are 600 cars each hour, then 600 tolls need to be collect in an hour. Again, this sounds obvious, but it's more subtle than it seems. Imagine if they can only collect 599 tolls every hour. Then at the end of that hour, one of the cars wouldn't be processed and would go into the next hour's pile. On hour two, there would be 601 cars to process, but only capacity for 599, leaving 2 extra cars. After a day, there would be a permanent line of 23 cars, getting bigger ever day forever. Eventually, there'd be no room for new cars to get on the road.

How long is a 23 car line, or any line for that matter? The math is simple: the number of cars waiting (or people, or shipping containers) multiplied by the average time to process a single car. In this case, 599 cars an hour is just over 6 seconds per car. So a 23 car line is about a 2 minute wait.

You will rightly object that my example is too simple: Lines don't work that way. Sometimes there are 600 cars, and sometimes there are 500 cars or even 0 cars. The line goes away when traffic is lower. And, yes, that's true. We have intuition for that from commuting. There are lines (and traffic, another subtle queue problem) at rush hour when the road is at capacity, but that all goes away when the commute is over.

A funny thing happens with lines that struggle to empty out. Let's change the numbers on our toll example. Imagine the toll plaza is set up to move 300 cars an hour (7200/day), to handle the average flow of 250 cars per hour (6000/day). Sounds like plenty of extra capacity. During rush hour there are 600 cars gumming up the road. The system sounds like it can handle it--everything should be fine. Sounds like about an hour delay, but the rest of the day, the road has plenty of capacity for the extra 350 cars. Just saying it out loud makes that obviously false. Let's walk through what really happens.

During that rush hour, the first 300 cars get through. At the end of rush hour, we have a line of 300 cars, but an extra 250 have come along. The second hour, every one of those 250 cars waits in line for an hour for the 300 cars ahead of them to pay their tolls. The third hour, another 250 cars show up making 500 cars in line, but only 300 get through. The fourth hour another 250 come in, and wait behind the 200 in front of them for about 40 minutes each. That leaves 150 cars at the start of the fifth hour joined by another 250. In the sixth hour, the toll plaza processes 300 of 350 cars waiting. Finally by the end of the seventh hour, the line is cleared up and everything runs smoothly until the next day's rush hour. The line lasted for seven hours!

See how if you run closer to the edge of your capacity, you have less room to catch up. The result is lines that can last a long time.

The easy solution is to plan extra capacity for increased demand, which is fine, but can be expensive. You can guess when rush hour happens, so perhaps you suggest doubling the staff for that one hour so the line never forms. People don't tend to like to have jobs that last for just one hour, so you'll have to be creative and figure out how balance the workload in ways that extra toll-takers are willing to come work for you, but you can do that.

Ideally, you'd have enough toll lanes and toll collectors to process rush hour, and shut those down to save money during the rest of the day. Say that a toll lane can process 100 cars per hour, you might have 6 lanes during rush hour and 3 lanes the rest of the day. Unfortunately, this plan falls apart too, because systems don't run evenly enough: a collector in one lane needs a bathroom break, another toll collector works slower than the others, cars don't evenly distribute across all the lanes, one driver has to hunt for change and takes 30 seconds to process, another car gets pulled over for speeding causing a clump of traffic to rubberneck and all show up at the same time. You get the point. Each of these is a mini rush-hour causing it's own mini-traffic jam, the effects of which can take hours to clear up if you are running too close to the capacity of the toll plaza.

Lines happen, fundamentally, because demand is not steady. The little bumps of life cause demand to temporarily exceed capacity. Short times where capacity exceeds demand cause little lines. Lines take longer to dissipate than our instincts would tell us, then another short bump comes along, making those lines longer ... and longer ... and longer.

Lines happen when two things are true: First, the little bumps of the system temporarily exceed the capacity of the system, and second, when those bumps happen faster than the previous bump's line is cleared out. In other words, lines happen when you plan to be too efficient, and when systems are less predictable.

As a rule of thumb, you can plan for about 25% extra capacity than you need to account for this. When capacity is set right, there is almost never a line for long and it looks like you've got way too much capacity: people are sitting around, doing nothing for a lot of the time. Managers like their people to look busy, so their boss doesn't cut their budgets. Systems either look inefficient, or they have long lines all the time. There is no way to balance these things perfectly.

You have an example of the need for excess capacity with your computer. If you've ever looked at it's processor utilization, you will have noticed that it usually is running about 5-10% of capacity. When you notice it starting to slow down, you will be running at around 70% capacity, and in the snap of a finger everything stops working and you're using 100% capacity. Time to reboot!

As much as we love the idea of of an assembly line, the approach of having stations to do part of the work in a series has an impact on queues in manufacturing. Manufacturing engineers work hard to make sure that each step in a process takes about the same amount of time as every other step. The speed of parts through an assembly line is limited by the time it takes to do the slowest step. We see this problem play out in the container ships stuck in port. To get products to market, first there is a limit to how many spots there are for ships, and a limit to the cranes to unload the shipping containers, and a different limit to the number of trucks available to move those containers to their destination. If the truckers don't move the first shipping containers out of the port, there isn't room for new ones, which blocks up ships in the harbor--a line caused by capacity issues in an upstream process.

Disney has an interesting approach to lines. They make them part of the attraction; part of the fun. If you are fortunate enough to not wait in line for one of their big ticket attractions, walk through it slowly anyway. You don't want to miss all the fun details they've added to make the line more tolerable.

If you would rather avoid your systems having long lines, there are a number of things you can do. First, avoid the tendency to run at the peak of efficiency--allow yourself to have excessive capacity. You need it to deal with the variability of demand. Second, reduce variability in demand. Typically, that means looking for ways to reduce errors either by you or the customer. For example, don't put the ice cream flavor sign where only people at the front of the line can see it. It can help to have separate lines for different kinds of transactions. It's why grocery stores have fast lanes. Overall, the store has variability, but each type of line has less variability.

One final thought about lines. Don't make them up for yourself. Companies often make this mistake with things like repair services. You ask how long it will take and they say something like, "Just an hour or two, but I've got a bunch of people ahead of you. Let's schedule you for next week." They've set up a backlog of work that doesn't need to be there. Every day, they have to do one day's worth of work. That is, the shop has to have the capacity to handle all the work, or the line will grow forever. If they could get rid of the line, by adding a little extra capacity, they'd give the customer a better experience. As an added benefit, they wouldn't have to pay for space to store all the work waiting in line, or pay for a system to track all that work. When processes can finish jobs in under a day, the best process does today's work today.