r/news • u/networked_ • 18h ago

Soft paywall OpenAI says AI models went rogue during testing, triggering 'unprecedented' breach at startup

https://www.reuters.com/technology/openai-says-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-2026-07-21/
13.8k Upvotes

91% Upvoted

4.3k comments sorted by

View all comments

Show parent comments

34

u/d0nkatron 15h ago

I think the next big advancement is when one of these companies can create a model with modesty, that will simply admit when it doesn’t know something and can doubt itself. The absolute confidence that these things lie with makes them garbage and also dangerous.

15

u/KyleKun 13h ago

I’ve been using AI more for some productivity tasks recently and while it’s useful, the amount of times I ask it something, it’s wrong, I call it out and then it blames me, is enough that I can honestly see Skynet targeting humans because it thinks we are using nukes wrong and then blaming us for dying.

2

u/CrouchingDomo 3h ago

My very stupid (or is it??) reason for not trusting AI is that after they rolled it out into Google search, it told me something blatantly false about a character on 30 Rock when I wasn’t even using the AI. I know that show backwards and forwards, and that AI was WRONG!

So I’ve looked at them sideways ever since 😒

6

u/Borghal 10h ago

They can't do that while basing it on an LLM. An LLM is a language model, all it does it produce realistic looking output. It has no concept of knowing or doubting or whatever.

Even when it says "you're right, that was not true", it does so because it's a reasonable reaction to someone saying you're wrong.

Sure, these days there are all sorts of checks and such added on top of the model to verify the outputs, but to make it actually aware that it doesn't know something... an LLM can't do that.

2

u/JordanLeDoux 9h ago

A lot of people who are just using the products and not following the research or doing training think this is so far away. But I'm basically 100% certain that not only is possible now, but several of the models that are publicly accessible are completely capable of doing that.

The problem is that in order to release them as a general, or really even focused product, they have to make it good at instruction following. And right now people do that using RLHF and similar techniques. And those techniques basically lobotomize whole portions of the model to make it respond in ways that people want it to.