> I ran the agent 100 times. In 88% of the tests, it didn't open the external website. It either hallucinated a discount code or simply ignored the instructions.
Bad model? Don’t think I’ve ever seen a model ignore a link in instructions. It always wants to see what’s there
>The demo uses Claude Haiku rather than newer models like Opus or Fable. This is because I'm assuming that shopping agents of the future won't use newer, more expensive models,
Why are we assuming that shopping agents are going to be using the model that is easiest to fall for prompt injections? Only testing a year old, small model is going to lead to a misleading conclusion.
Also the author is deliberately chooses a model that is 1 year old and was supposed to be underpowered even then. If cost was deciding factor, they could’ve chosen Luna.
Model AA Index Cost/task
Claude 4.5 Haiku (thinking) 17 $0.21
GPT-6 Luna (low) 21 $0.0045
Luna (low) scores higher and costs about 47x less per task and performs better.
Question is: if cost were so crucial, why didn’t they choose a model that was 47x cheaper?
the sad thing is we need AI for shopping because it has been in the interests of online sellers to make a hostile experience on purpose - when I go to Amazon and search, the first organic result is almost scrolled off the screen past all the sponsored ones, or when want to buy from a random seller I'm almost definitely getting shunted through an account signup I didn't want or fooled into an affiliate purchase some other hostile experience beyond just "buying the thing".
So now we have AI to overcome the hostile sellers, but the fact the sellers introduced the friction in the first place strongly suggests it will just come back again in some other form. It wasn't there by accident, it was serving people's interests and once AI vendors have finished getting consumers hooked in, they will then turn around and enshittify by giving the sellers back some of the friction - for a cut. So you won't be able to just order what you want without the "would you like fries with that?" or "what about this other brand?" coming back.
Because people want to participate in activities that cost money. It's easy to see use cases like an AI agent notifying someone that a band they like is coming to town and asking if they want to secure tickets to the show.
The provided example is really bad and ignores why customer service is sometimes a good idea. I guess it's been so bad and corrupted by disingenuous marketing for so long that people have forgotten.
Suppose you need to buy some stuff for an upcoming project or vacation, but don't want to go down a research rabbit hole for the next hour(s) to cure your ignorance. Suppose you don't even want to have a conversation with the AI. You just want it to build a shopping cart. At most, hopefully, your effort is just pruning the cart before placing the order (if even that).
I'm not saying AI wouldn't be corrupted like human customer service. I'm saying that we're actually pretty close to this already on Amazon with their AI, but not all ecommerce can be or has to be Amazon.
> Picture this: it's 2027. A chilly spring is approaching, and you want a new coat before the Super El Niño. Instead of scrolling through endless online stores, comparing reviews, checking size charts, and hunting for discount codes, you tell your AI shopping agent what you're looking for — and it takes care of the rest.
Why would they AI slop this?
Is this supposed to be ironic and I’m missing the joke?
If it takes you less time to ‘write’ something than it does to read, I’m not reading it.
> The demo uses Claude Haiku rather than newer models like Opus or Fable. This is because I'm assuming that shopping agents of the future won't use newer, more expensive models, even though they're more reliable and secure. I suspect it'll come down to cutting costs, and using a cheaper model is one of the easiest ways to save money.
Stopped reading here. Author has little idea of the industry.
Bad model? Don’t think I’ve ever seen a model ignore a link in instructions. It always wants to see what’s there
"Cool! Will we get the money to do said 24/7 shopping as well?"
"No."
"Oh..."
"In fact, you'll have even less money to do the normal shopping you do now!"
"Oh..."
"There's more! The things you used to buy with the remaining money you have will cost even more!"
"Oh..."
"But you can do it 24/7 though."
"Sweet!"
Why are we assuming that shopping agents are going to be using the model that is easiest to fall for prompt injections? Only testing a year old, small model is going to lead to a misleading conclusion.
Author also needs to make a point, and can’t do so if they are using a powerful model.
next year, astra will be $2/1M or whatever (point is - cheaper) and whatever model is $10/1M will be insane like astra feels today.
also, in any agentic workflow, you use models of various levels together... not just one model of one level...
incapable of understanding speculation?
Question is: if cost were so crucial, why didn’t they choose a model that was 47x cheaper?
So now we have AI to overcome the hostile sellers, but the fact the sellers introduced the friction in the first place strongly suggests it will just come back again in some other form. It wasn't there by accident, it was serving people's interests and once AI vendors have finished getting consumers hooked in, they will then turn around and enshittify by giving the sellers back some of the friction - for a cut. So you won't be able to just order what you want without the "would you like fries with that?" or "what about this other brand?" coming back.
Suppose you need to buy some stuff for an upcoming project or vacation, but don't want to go down a research rabbit hole for the next hour(s) to cure your ignorance. Suppose you don't even want to have a conversation with the AI. You just want it to build a shopping cart. At most, hopefully, your effort is just pruning the cart before placing the order (if even that).
I'm not saying AI wouldn't be corrupted like human customer service. I'm saying that we're actually pretty close to this already on Amazon with their AI, but not all ecommerce can be or has to be Amazon.
Why would they AI slop this?
Is this supposed to be ironic and I’m missing the joke?
If it takes you less time to ‘write’ something than it does to read, I’m not reading it.
Stopped reading here. Author has little idea of the industry.