In the late 1950s, a young professor at the Case Institute of Technology in Cleveland finished a class on queues by pointing out that one formula kept turning up in every model they had studied. It seemed very general.
After class, a student named Sid Hess came up to him. “How hard would it be to prove it in general?”
“I guess it shouldn’t be too hard,” said the professor, John Little.
Fifty years later, writing about that moment, Little added: “Famous last words.”
“Then you should do it!” Hess had said. Little worked on it during his family’s summers on Nantucket, where, he wrote, “the beach was our babysitter,” and published the proof in 1961. Then he decided that kind of mathematics was not for him and, as he put it, “retired from queuing.” The case has the story.
Three numbers
The formula is L = λW. The average number of items in a system, L, equals the rate at which they arrive, λ, times the average time each one spends inside, W.
Little and his co-author Steve Graves illustrated it with a supermarket that opens at 7 a.m. and closes at 11 p.m. Suppose 120 shoppers come in each hour and the average shopper stays 25 minutes. Then, on average, there are 50 people in the store.
The surprising part, the part Little proved, is how little the law needs. It doesn’t care how shoppers arrive, how long each one takes, how many checkouts are open or who goes first. Over a stretch of time that starts and ends with the store empty, Little showed in 2011, it holds exactly. For a system that keeps running, it holds for the long-run averages as long as the system is steady.
Turn it around
Managers usually want the third number: how long will this take? Rearranged, W = L / λ. Operations textbooks write it as cycle time equals work in process divided by throughput.
Imagine a small accounting firm that, through tax season, has about 60 returns in progress on average and finishes about 15 a week. On average, each return spends four weeks between coming in and going out. A client promised one week can be moved to the front, but then someone else waits longer. Unless the firm takes on fewer returns or finishes more each week, the average doesn’t move.
That is why a growing backlog is so corrosive. At Microsoft’s XIT team in 2004, more than 80 change requests were waiting and six or seven were finished a month. Requests were typically taking at least five months. The team’s fix was to cap the work going into development, which is the subject of WIP limits.
A reality check
Little’s law is also a lie detector for numbers. Bill Lovejoy of the University of Michigan, who used it to size a hospital’s observation unit, told Little: “I have frequently been handed spreadsheets with data that do not add up. Little’s Law provides a reality check.” If a ward reports its average occupancy, its admissions and its average length of stay, the three have to agree.
He also used it the other way, to plan. Observation patients stayed 1.14 days on average, so for any expected flow of patients the law gives the average number in the unit. Divide that by a target occupancy, say 90%, and you have a first estimate of how many beds to build.
What the law cannot do is change anything. It is “not in charge of changing these quantities,” Little wrote. Nor does it say what happens near full capacity. For that, Little turns to basic queuing theory: in an emergency department, as arrivals approach what the staff can handle, the average number of patients and their length of stay “increase, slowly at first and then rapidly.” That is why Mark Harris, who designed staffing systems for emergency departments, suggests staffing 10% to 20% above the bare minimum as a rule of thumb.
Agents in the queue
AI work runs on the same arithmetic. Anthropic’s API sets rate limits in requests per minute. Suppose each request your agent makes takes 30 seconds and you send them at a steady 50 a minute. Over time, about 25 calls will be in flight on average, though short bursts can run higher. To keep 100 in flight on average over time, at 30 seconds each, you would need to send 200 a minute. For its batch interface, Anthropic caps the queue directly, with a maximum number of requests that can be waiting to be processed at the same time.
The queue that bites most people, though, is the one that ends with them. If, week after week, about twenty finished agent tasks sit in your review pile on average and you clear about five a day, each one spends about four days between landing in the pile and being reviewed. The agents can go faster. The average wait stays where it is until the pile gets smaller or you review faster.