LLMs Are Cheap Now. Where Is the Dishwasher?

Using a language model can feel surprisingly inexpensive compared with the value it produces. That does not mean today’s economics will remain unchanged.

When I first published this note, the scale was already enormous: hundreds of millions of weekly users, rapid adoption across business functions, and forecasts of sharply rising electricity demand from data centres.

As that scale reaches the bottom line, the product question changes. It is no longer only, “Can AI do this?” It becomes, “How can we do this efficiently without sacrificing quality?”

I found a useful analogy in a BC Hydro energy-saving recommendation: load the dishwasher first, then run it. Do not spend the energy required for a full cycle on one plate. Wait until the machine can do useful work at an efficient load.

Products that make hundreds of thousands of model calls need an equivalent strategy. That might mean batching compatible work, caching durable answers, routing simple requests to smaller models, or waiting for an efficient processing window when the user does not need an immediate response.

The right implementation depends on the product. The useful question is the same:

Where is the dishwasher in your product stack?