The AI market is a wild ride, my friends. It's like a rollercoaster with a few luxury seats at the front and a bargain-basement section at the back. But the real story isn't just about the price tags; it's about the shifting dynamics of what's considered a commodity and what's not. And let me tell you, it's a fascinating ride.
In the past, inference was the big ticket item, but now it's becoming a commodity, with prices plummeting. Aman Panjwani, an AI engineer, notes that the cost of GPT-4-class model output has dropped from $20 per million tokens in 2022 to a mere $0.40 today. That's a 55x decline in just four years! And when DeepSeek released its R1 reasoning model at $0.55 per million input tokens, it sent shockwaves through the market, causing prices to plummet.
But hold on, there's a twist. While inference is becoming cheaper, the price of frontier models is soaring. OpenAI doubled the price of GPT-5.5, and Google's Gemini Flash 3.5 is three to six times more expensive than its predecessor. And now, with the release of Anthropic Claude Sonnet 5, the trend continues. Even though its per-token price is lower, it uses more tokens to produce the same results, making it quite costly.
So, what's going on here? Well, it's all about the shift in the market. As AI becomes more accessible and affordable, the demand for inference is increasing, driving down prices. But at the same time, the demand for frontier models is also growing, pushing up their prices. And let me tell you, this is where things get interesting.
As AI engineers and businesses, we're faced with a dilemma. On the one hand, we want to take advantage of the cheaper inference models, but on the other hand, we need to ensure that our investments in frontier models are justified. And that's where the real challenge lies.
In my opinion, the market is splitting in two. Commodity inference is heading towards zero, while frontier inference costs are rising. And as AI becomes more integrated into our lives, this dichotomy will only become more pronounced. So, what does this mean for us? Well, it's time to get creative and find new ways to optimize our AI spending. After all, in the world of AI, there's always a bargain to be found, if you know where to look.
