The Gemini family had a distinct niche in document comprehension, with thousand page input documents taking only 300k tokens. Nothing quite like that in OpenAI or Anthropic world, even at more than 10x the token adjusted price. Should we just give up on Google at this point and engineer around the competitors' limits and eat the costs? Totally unnecessary own goal by team Google.
I agree, though. 2.5 Pro is a great model. Very competent, knows a lot, and can process tons of text (and videos, and images, and audio too iirc?). Basically unlimited access to it too via AI Studio. I used it for processing and transforming bucketloads of data, ingesting masses of transcripts and converting them to flashcards, etc. I’ll be sad to see it go. None of the newer, cheaper, but obviously less intelligent benchmaxxed smaller models really seem to hold a candle to it for lots of things.
but google gonna google.
It's not all roses -- I've seen some regressions -- but generally the 3.x Flash models are pretty great for our use cases.
The great thing about LLMs though is it's incredibly easy to diversify and have fallbacks. But of course that means additional costs, mostly centered around engineering efforts to test and integrate them.
Also - and this is bizarre - the token cost of doing that is higher, not lower, at least in Gemini world, and by a large margin. That's very counterintuitive, but a page encoded as image tokens can be smaller than same page as text, and is not meaningfully lossy on documents that are just typed text because the models are well trained on those.
Models you can download and use elsewhere if Google nixes access