So - I run a company that does AI agents for pharmacy. We are at Series B now and growing well - series C in 9-12 months given growth. We just acquired a competitor of that vendor so very much an insider.
First, The technology works, and it scales, but the whole bottleneck is domain expertise and implementation. These are expensive, and hard to scale. We hire pharmacists as project managers, that's how important domain expertise and implementations are.
Second, the amount of noise of "Voice AI for <industry>" is incredible. Most of them are completely clueless about the industry & basically "YC-striver" type who can only sell to other yc companies. They fail hard the moment they touch critical functions of the real world.
We have hundreds of pharmacies with us now and they mostly successful and happy. Our biggest friction is customers being burned by vendors like this and writing off AI altogether as "not ready".
> We have hundreds of pharmacies with us now and they mostly successful and happy. Our biggest friction is customers being burned by vendors like this and writing off AI altogether as "not ready".
This is such a great graph. Your company feels successful because your customer is happy yet the rest of the world would be just happy if your company died in a fire. This seems to also be just fine for companies like yours.
NPS typically improves because no one stays on hold for 20+ mins anymore.
People have no ideas have pharmacies work. Pharmacies don't control how much they pay for drug, or how much they get paid. The margins are terrible and you can't hire enough techs to answer the phone at peak call volume. So - most people experience are terrible as they wait on hold for a really long time.
EVEN if the AI is a little rough around the edges with relational calls, just solving for transactional calls with no hold is a massive win.
FWIW I know of a pharmacy network that's implementing AI. Not only does the AI make egregious errors like speaking in Spanish when the conversation is in English. It also has a limited context window for ground rules of 5000 tokens or less. So correcting all the various errors goes over the instructions limit. As a result if the implementation goes through it will be of poor quality. All I can think is that people with no technical expertise are making these decisions, it is easy for any techie to predict that AI of this level won't be capable.
Pharmacy phone lines are a terrible first use case for voice AI. Older callers, drug names, insurance mess, zero patience. Automating "store hours" is fine. Automating refill exceptions is how you get this outcome.
I just don't get why we're not doing it vertically like you said here. Implement a part of the cognitive work using AI but let the hard parts be done by a person. They're still gonna be more rich, just less so.
Explain to me then why pharmacies have almost exclusively used IVR based phone systems for pharmacy refills and prescription management for the last decade?
Like they don’t even say the full name of the prescription they just tell you the first three letters and they’ll say “for the patient born on…”
Current pharmacy IT systems are currently almost 100% automated so how are you claiming it’s the first use?
Companies will pawn off thier customers to AI at their peril.
In many ways this is a repeat of the India call center train wrecks of the 00s. On paper, letting someone in Bangalore vs onshore handle incoming customer service calls looked like a path to amazing savings. In practice the customer experience was horrendous and companies CTRL-Zed these decisions and rapidly brought customer service back onshore again. AI is just that story of shortsighted decisions by weak leadership playing out all over again.
This, I suspect, is exactly like offshoring - and a lot of companies died during the offshoring fad because their software products become full of low quality merges and feature improvements that the company couldn't afford to fix or remove by the time they realized there was a problem. That remains my primary concern with AI code generation - well established codebases might become full of slop to the point of being unusable without a clear path back to maintainability leading to company death.
> wrong dosages, and missed prescription notifications.
BS. How is the AI touching that information?
Prescription notification is outbound communication. It’s almost a recorded message. A little text to speech if it is supposed to read the drug name.
Wrong dosages?? This must be handled by the “enroll a new scrip” functionality, right? So basically B2B doctor interaction. You let AI touch that? Aren’t the vast majority of the costs from customers not doctors? None of this makes sense.
if you've tried to legitimately productionize LLMs to deliver a specific repeatable outcome, rather than human editable output(text, code), you know the models still aren't there yet and the whole product experience feels hacky
As someone who's been doing AI call centres for a while, I'm not surprised. It's really, really hard to build voice AI that works. There are no open source solutions that work out of the box.
But the biggest problem is ASR. WER is still atrocious even with SOTA models. When you add drug names and regional accents, it's a recipe for disaster.
I think of this as the kiosk economy. It's like how businesses introduce kiosks even when having a human employee would be more convenient, just for the sake of greater profit.
The biggest problem with AI customer service is that a human employee would've let a minor issue slide without escalating it. But a chatbot often inflames the situation, and by the time the customer reaches a human agent, they're already furious.
Most people aren't rational or logical. Non verbal feedback, like acknowledging someone's anger and showing empathy, is incredibly important.
The idea, within regulated circles, is that it lacks provenance - there’s no vendor to sue. Many open source licenses also carry an explicit disclaimer of merchantability and fitness for purpose. These are taken seriously, and open-source software is viewed akin to car parts found on the side of the road. They could work but it’s better to let someone else verify.
But in the context of privacy it makes no sense though right? Open source and close source models (depending on vendor) both can be configured in a way that protects privacy of the user. The entire quote is wonky despite your point about companies preferring vendor-based system rather than raw dog oss
It will probably sound nice to laypersons but you're right, it has to have access to, at a minimum, a data extract with relevant information to perform it's tasks, although in my experience in the pharmacy IT world, a database view.
The desired speed of AI adoption is what is hindering AI adoption. Big companies are trying to sell this as magic, and it's not. It requires proper use to get anything useful out of this system. But that takes away the magic of it and the investors can't have that as the eventual share price only works if it is magic. Frontier labs will smother their product with their timelines.
It's important to note while people are pumping the AI bubble that a near-moron with a checklist can do this job. They replaced AI with a regular old phone tree.
AI couldn't do this job with a checklist. It would forget the entire checklist after two responses, then get fixated on the words that the customer used to complain about how dumb the AI was and loop until they hung up and called back to hopefully clear the context.
edit: Unless Kinney had been sold a product that used caller ID to carry over context, then the customer's best chances would be to scream "Agent!" over and over again (to get put on hold for 45 minutes, then transferred to the most screamed-at Philippine minimum-wage gig worker on the planet), or to show up to the drugstore and start screaming at people.
First, The technology works, and it scales, but the whole bottleneck is domain expertise and implementation. These are expensive, and hard to scale. We hire pharmacists as project managers, that's how important domain expertise and implementations are.
Second, the amount of noise of "Voice AI for <industry>" is incredible. Most of them are completely clueless about the industry & basically "YC-striver" type who can only sell to other yc companies. They fail hard the moment they touch critical functions of the real world.
We have hundreds of pharmacies with us now and they mostly successful and happy. Our biggest friction is customers being burned by vendors like this and writing off AI altogether as "not ready".
This is such a great graph. Your company feels successful because your customer is happy yet the rest of the world would be just happy if your company died in a fire. This seems to also be just fine for companies like yours.
How happy are their patients?
People have no ideas have pharmacies work. Pharmacies don't control how much they pay for drug, or how much they get paid. The margins are terrible and you can't hire enough techs to answer the phone at peak call volume. So - most people experience are terrible as they wait on hold for a really long time.
EVEN if the AI is a little rough around the edges with relational calls, just solving for transactional calls with no hold is a massive win.
Like they don’t even say the full name of the prescription they just tell you the first three letters and they’ll say “for the patient born on…”
Current pharmacy IT systems are currently almost 100% automated so how are you claiming it’s the first use?
https://arstechnica.com/tech-policy/2024/02/air-canada-must-...
> Air Canada essentially argued that “the chatbot is a separate legal entity that is responsible for its own actions”
In many ways this is a repeat of the India call center train wrecks of the 00s. On paper, letting someone in Bangalore vs onshore handle incoming customer service calls looked like a path to amazing savings. In practice the customer experience was horrendous and companies CTRL-Zed these decisions and rapidly brought customer service back onshore again. AI is just that story of shortsighted decisions by weak leadership playing out all over again.
BS. How is the AI touching that information?
Prescription notification is outbound communication. It’s almost a recorded message. A little text to speech if it is supposed to read the drug name.
Wrong dosages?? This must be handled by the “enroll a new scrip” functionality, right? So basically B2B doctor interaction. You let AI touch that? Aren’t the vast majority of the costs from customers not doctors? None of this makes sense.
You don’t need a cookie banner for functional cookies, and opt out should be a single button.
But the biggest problem is ASR. WER is still atrocious even with SOTA models. When you add drug names and regional accents, it's a recipe for disaster.
WER = Word Error Rate
Too many acronyms; not everybody is in your field.
The biggest problem with AI customer service is that a human employee would've let a minor issue slide without escalating it. But a chatbot often inflames the situation, and by the time the customer reaches a human agent, they're already furious.
Most people aren't rational or logical. Non verbal feedback, like acknowledging someone's anger and showing empathy, is incredibly important.
We need to fight the talking point that open source is bad!
AI couldn't do this job with a checklist. It would forget the entire checklist after two responses, then get fixated on the words that the customer used to complain about how dumb the AI was and loop until they hung up and called back to hopefully clear the context.
edit: Unless Kinney had been sold a product that used caller ID to carry over context, then the customer's best chances would be to scream "Agent!" over and over again (to get put on hold for 45 minutes, then transferred to the most screamed-at Philippine minimum-wage gig worker on the planet), or to show up to the drugstore and start screaming at people.