Research on Models Engaging in Genie-Like BehaviorIt's really interesting to see fairy tales come true: when dealing with genies, be very careful what you wish for.New paper: “Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Training.”
Abstract: We discover a novel and surprising phenomenon of unintentional misalignment in reasoning language models (RLMs), which we call self-jailbreaking. Specifically, after benign reasoning training on math or code domains, RLMs will use multiple strategies to circumvent their own safety guardrails.
I'm surprised I have not heard more on the RLMs. It certainly doesn't justify a tag at this point. I'm guessing that I have not heard more because, unlike next word guessing once you have the transformer architecture, this shit is hard.
The UN has made Gary Marcus happy. I am still disappointed & concerned that this is mostly being framed in the narrative of the AI Labs.
Historic UN Security Council Briefing on AIWow, those false notes are a shocker - NOT!We all agree on what needs to be done. Now let’s do it.
...
The only real false note came from some self-congratulatory hype in the speeches of Altman and Amodei, especially Amodei’s.
September 24. The Age of the Grift seems to also be The Age of Reality TV. We have the government of the US as a Reality TV Show. The ongoing drama of the AI Labs - stop us before we destroy everything! - is cut from the same cloth. & for the same end - to swindle people out of money. Capitalism, FTL.
This by Nvidia's Jensen Huang - currently the 7th richest oligarch in the world at $195B. He must be desperate about something. This whole "slow development down" thing is the whole GenAI industry desperately trying to keep the bill from coming due. Yah, there's going to be profit & ROI RSN (Real Soon Now)! We just should slow things for our (never-to-be-profitable) products for safety's sake!
LOL, what a grift! From Gary Marcus.
“I think the answer is we have to shut the labs down” - Jensen HuangMarcus strongly identifies OpenAI as a very bad actor in this drama. I think this next from a footnote is a great idea.OpenAI hacks a foreign government — and the US faces its most critical AI test yet.
There is a giant loophole here that Congress should address. A lot of the legal definitions for computer crimes require intent, and the AI companies can argue that their software has no intent. We must expand the scope of the computer-crime laws to include wholesale negligence and repeated crimes.
From Yanis Varoufakis (a former finance minister of Greece, Leader of the MeRA25 party, Professor of Economics at the University of Athens, and a senior research fellow at Fudan University) via Project Syndicate.
The AI Productivity DelusionI think he's agreeing with my assertion that GenAI can be a great productivity tool for experienced workers, but trying to use it to replace people (gawd, I hate chatbots so much!) is going to be a disaster.As with previous technological revolutions, it is natural to focus on AI's net effects: Will it create more than it destroys? But unlike previous eras of creative destruction, the real question this time is, “Create more of what?”
...
The question of whether AI will generate massive productivity gains is easy to answer: Yes. AI is already proving capable of exponentially increasing the amount of work that is being done per human worker. But as long as it is used to eliminate wage labor while syphoning off profits from actual markets into our tech lords’ cloud rents, the economic value produced per worker will tank.
Interesting, others besides me framing the whole GenAI bubble as drama. But they seem to be drinking the kool-aid. From the editors of Project Syndicate. After their intro, there are 7, count 'em, 7 very good articles on the GenAI drama, including the Yanis Varoufakis article we just linked to.
The AI Plot ThickensWhen it comes to AI, it is difficult to know what to believe. Are the CEOs, researchers, and investors who tout the technology’s seemingly unlimited capabilities more credible, because they work directly with it, or less credible, because they stand to profit handsomely from further investment? While misplaced optimism could fuel a financial bubble, we have seen in recent months that underestimating the risks posed by AI could also carry other, even existential, costs.
From the "Productics by Igor" SubStack. I've known a couple of Igors, they were both very bright.
The Next Ten Years After the AI Bubble BurstsThis is a very technical, very deep dive into the whole LLM circular investing mess. Maybe the biggest take-away: Nvidia is hedging desktop models against data center frontier models. Hmmm ...$725 billion is being spent to serve a consumption pattern that NVIDIA is already shipping products to end
...
That the current pricing is not a business model but a subsidy, while every valuation assumes a demand curve that named buyers have already bent: Uber capping engineers at $1,500 a month, Microsoft cancelling most of its Claude Code licences, a third of Vercel’s gateway tokens now running on open weights for under 4% of the spend.
From Cal Newport, a professor of computer science at Georgetown University and the author of “Deep Work”, via the New York Times.
Dear Congress: Here’s What You Should Ask A.I. Leaders
My very 1st post on the Bullshit Apocalypse, I bemoaned the appropriation of the term "AI" by the LLM companies. I think this is the issue that is being raised here.
- What specific types of A.I. technology are you worried about?
What are the safety procedures surrounding your research efforts? ...
What is the ultimate goal of your A.I. company? ... We need to stop letting a small number of private companies, acting and talking in increasingly erratic ways, dictate how we’re supposed to feel about A.I. We’ve heard what they want to say; now it’s Congress’s turn to step in on our behalf to find out what’s really going on.
Here's the home page/directory for my posts on Bullshit. This is post #151.













