An interesting (and scary) week in the Bullshit Apocalypse. Perfect for Halloween!
October 20. Robert Reich just posted some Serious Thoughts on the future of "AI".
How to Protect Humanity from AI
5 essential guardrails
I commented:
You are drinking the koolaid. The effect of LLMs & Chatbots on the economy are going to be minimal.
The lowest error rate I have seen for LLMs & Chatbots is 9%. You can't build reliable systems with that high an error rate.
Agent error rates are up to 50%.
Already local governments are moving to ban the ever-so-annoying & defective customer service chatbots.
LLMs can be great tools for creatives who can check their outputs. Where is big $$$ in that?
All other uses, you wind up with slop, & a huge slop removal debt, wrong answers, systems that are more trouble than they are worth.
The economics are laughable. Spend $T to save maybe $B ?!?!?
It is a grift. Most of the lower tier hyperscalers are cryptocurrency grifters who moved to the newer & sexier grift.
Why is the Orange Turd all in? Because, just as with crypto, he smells a grift, & grifting is his speciality.
Plus, his sons, particularly Beavis Fredo & Butthead Fredo, have large investments in data center companies.
Stop drinking the koolaid.
Also October 20, from 404 Media.
‘Reverse-Engineer Anything’ Is the New Wild West of AI
Decompiled, cloned, and modified software is going viral and might slopify everything.
Whoa, this is indeed serious. Doctorow has mentioned how even in countries what have passed "right-to-repair" laws, users are still pretty screwed because the way the laws are written the user has to figure out how to do everything themself - it is still illegal to distribute jailbreaking tools, for example. Is this the answer to that?
The authors of this piece rightfully point out, it is still slop. Are the vibe coders capable of verifying that the generated code is handling all security issues correctly? I doubt it. Here's the 1st 3 paragraphs of the post. Definitely read the whole thing.
Over the weekend, it became fashionable to use AI to replace the operating systems on smart TVs with vibe coded or AI reverse-engineered ones to make them less bloated. People have taken proprietary internet-of-things devices that were bricked and abandoned by their creators and used AI to hack them and install new software on them, giving functionality the original developers took away.
In the last few days I have seen people hack smart picture frames to become AI-powered smart calendars, baby monitors to remove all ads, and bricked Lululemon mirrors to bring them back to life with new software. And dozens and dozens of TVs to install AI-written software that removes bloatware, ads, and simplifies the user interface.
What we’re seeing is vibe coding, but instead of making something from scratch, people are using AI to take an existing piece of software and replicate its features, its functionality, or cloning it wholesale. In some cases, people are using AI to take a proprietary piece of hardware and install some version of open source software on it. All of this is some mix of “decompiling,” “reverse-engineering,” “hacking,” vibe coding, AI slop, and stealing. It is the plagiarism machine being used for plagiarism, but also it is random people using AI to take control back over devices whose proprietary software was used to arbitrarily lock away basic functionality, to add advertisements, surveillance, and bloat to devices that probably shouldn’t have them, or whose functionality was bricked altogether.
In this post of October 9, in a section about arXiv being overwhelmed by slop, I concluded:
Ahh, the Bullshit Apocalypse! We've seen enough instances of this phenomenon - AI slop overwhelming a system that used to work - I should prolly think up a name for it & tag the posts.
Well, somebody beat me to it! From The Garden of Forking Paths website of Brian Klaas.
The Hidden Cost of AI “Effort Collapse”
Drowning in slop overloads everything from job applications to customer complaints. We need fresh mechanisms to sort people.
I: Effort collapse and effort barriers
II: Don’t offload anything that’s worth your effort
III: How do systems cope with a flood of slop?
IV: How to fix it
"Effort Collapse" is not particularly catchy IMO, but I think it does accurately describe the concept.
After reading the 1st 3 parts listed above, part IV was behind the "paid SubStack subscriber" paywall. So I went on & subscribed, $50/year. In the subscription upgrade process, there was a place to send a note to the author. I told him that we were on the same page. I mentioned that the concept of "having skin in the game" seemed applicable.
LOL, I read part IV, it has a section "Distributed Review: Skin in the Game" - nice!
I will tag appropriate posts with "Effort Collapse" as I run across them.
October 22, from Gary Marcus.
Breaking news: U.S. banks are taking hits from AI agents
Who knows what piece of infrastructure might be attacked next.
How do we get the genies back into their bottles?
Meanwhile, Marcus's conclusions are pessimisstic:
Here are three further predictions:
- No matter how much chaos generative AI causes, the AI industry will not take real responsibility
- The Trump administration will do little to hold them responsible.
- Things will get worse.
October 23, Yves Smith at naked capitalism also posts about the bank robber agents.
US Banks Scramble to Shut Down AI Access to Customer Accounts
Finally, this Doc Searls post discusses a few GenAI topics - people are insisting on taking Superhuman Bullshit Generators seriously, building new standards and coalitions. He includes the abstract to this article on arXiv, which I will do as well. My bold.
Agents of Chaos
We report an exploratory red-teaming study of autonomous language-model-powered agents deployed in a live laboratory environment with persistent memory, email accounts, Discord access, file systems, and shell execution. Over a two-week period, twenty AI researchers interacted with the agents under benign and adversarial conditions. Focusing on failures emerging from the integration of language models with autonomy, tool use, and multi-party communication, we document eleven representative case studies. Observed behaviors include unauthorized compliance with non-owners, disclosure of sensitive information, execution of destructive system-level actions, denial-of-service conditions, uncontrolled resource consumption, identity spoofing vulnerabilities, cross-agent propagation of unsafe practices, and partial system takeover. In several cases, agents reported task completion while the underlying system state contradicted those reports. We also report on some of the failed attempts. Our findings establish the existence of security-, privacy-, and governance-relevant vulnerabilities in realistic deployment settings. These behaviors raise unresolved questions regarding accountability, delegated authority, and responsibility for downstream harms, and warrant urgent attention from legal scholars, policymakers, and researchers across disciplines. This report serves as an initial empirical contribution to that broader conversation.
LLM Agents are clearly not even vaguely ready-for-prime-time. But it's hard to see how these
Nonpragmatic Genies go back in the bottle. Too much $$$ involved.
Here's the home page/directory for my posts on Bullshit. This is post #159.
