Agents of Bullshit

I didn't think this article in yesterday's Herald-Leader was worth referencing, but then Gary Marcus weighted in today on LLM-based Agents, which were supposed to be a Big Thing this year. Not going too well so far.

Here's the Herald-Leader article:

Will AI help or wreck your summer vacation?

The author, visiting Taiwan, asked ChatGPT to recommend a noodle shop nearby that took credit cards, as she was low on local currency. She had some delicious noodles, but was "oh shit" when the proprietor demanded payment in cash.

But, she points out, at least the place she got sent to existed. LLMs are sending people to places that don't exist. They call this "hallucinations", but ALL LLMs DO IS HALLUCINATE, ALL THE TIME! They do not have a model of the real world, all they have are words they have read on the internet.

Here's Gary Marcus's post:

AI Agents have, so far, mostly been a dud

Marcus quotes numerous bad reviews of LLM based "Agents". He summarizes it as:

to a paraphrase an old saying that goes back to 1969, “To err is human, to really screw up takes an AI agent.”

LLM "Agents" have "failure rates of 70% on some tasks".

1 thing I don't think is getting enough attention: as with almost every computer system going back pretty much forever, security sux.

As a consequence of their superficial understanding of what they’re being asked, current agents are grossly vulnerable to cyberattacks.

I like this name for the downward spiral that occurs when LLMs start eating their own dogfood, in a quote from Rohan Paul:

The team calls this road Degenerative AI, where models feed on their own flawed output until the answers collapse. [My bold]

So LLM-based Agents are, as ever, just hype, just bullshit, just techbros wanting to maximize their stock options.

LLMs are really good with words. As an assistant to creative types, in words, maybe science, they can be really useful.

Anyone thinking they are ready to really start doing stuff on their own, a la "Agents", is delusional. They are way, way too untrustworthy. And the limitations of the LLMs - no world model, no math engine, no physics engine, no chemistry engine, no biology engine, no social or moral engine - to me mean that they are a total dead end on the road to any kind of AGI.

I don't think agents can work without at least some rudimentary AGI. I think I discussed this in a little more detail in my post of 2025-02-17 "CommonSense™"

LLMs do have a great language engine, but that ain't enough.

---

Here's the home page/directory for my posts on Bullshit.