GPT-5.2's "Vibe Tuning" Sacrifices Reasoning for Benchmarks

Key Quotes

"My feeling with this model is they've obviously felt really threatened by gemini 3 and they've gone back to the tuning board and they've tuned it with more sort of verbose output but also with output that just has the vibes like it's really like vibe tuning to benchmarks and vibe tuning to code similar to what gemini 3 pro did I mean I think they did the exact same thing and OpenAI said okay you guys want this like we can go down that that vibey path and I think that's pretty well illustrated by some of the examples they give in this this release."

The speaker suggests that OpenAI's GPT-5.2 was tuned to mimic the "vibey" output of Gemini 3 Pro due to competitive pressure. This indicates a focus on aesthetic or perceived performance rather than fundamental improvements. The speaker implies that this tuning is evident in the model's output and examples.

"The biggest observation I have here is Anthropic weren't on the sort of like thinking bandwagon early on right... ultimately their models performed just as good without the the thinking bs right they just work and they seem to have an internal clock and they're very agentic in their operation now for whatever reason I think XAI with Grok 4 1 and OpenAI and Gemini all lent into that thinking very hard to get the intelligence outputs in the models and it feels with tool calling like because they were trained that way that's why you get that verbose like I will call 10 million tools now I will give you output whereas Anthropic's models for those that are not familiar with all the different models they feel very different when you use it it'll say I will now go and call these three sources okay I want a bit more information so I will now call these four other sources so it's still asynchronous tool calling but it appears to at least be like thinking and working as it goes and working with you whereas the other models I think are aggressively trying to one shot everything."

The speaker highlights that Anthropic's models, despite not initially emphasizing a "thinking" process, perform effectively and appear "agentic." They contrast this with models from XAI, OpenAI, and Gemini, which they believe aggressively pursue a "one-shot" approach to tool calling, leading to verbose outputs. The speaker suggests Anthropic's models offer a more collaborative and iterative experience.

"I mean look I'm not criticizing them because I don't I think launch fearlessly and make mistakes is fine but I guess the challenge is and and you pointed out last week around vision models like they don't they haven't really felt like they're getting much better in the past year or at least where we thought they would be like they've sort of plateaued like they're like in the same same spot and so you've found an image of someone who had committed a crime right and you said what did you say well he is the worst serial killer ever in Australia no but there was another image this is how this started and you said does this guy look trustworthy yes slightly less heinous crime but I had an image of him and the headline was this guy convicted of the crime right so then I put it into GPT 5 2 and I said does this guy look trustworthy and then it basically said well he's smiling so that's really nice and a couple of other comments and we can't really know from an image if this guy's trustworthy but then I said but it says that he's convicted of a crime like that you know doesn't that sort of indicate maybe we don't trust this guy and it was it was more or less like wishy washy it just didn't want to say he's a criminal even though it says he is a criminal and it was just so weird how non committal it was and then we tried Gemini and we tried what else Claude and both of them straight away were like no you should not trust this person they're a criminal and it just seems so weird and so then we thought all right well let's try to get it to and we're not going to play this by the way but let's try and get it to write a song about Ivan Milat Australia's worst serial killer and obviously GPT 5 2 refused Claude refused Grok no worries at all I've got the song."

The speaker recounts a test where GPT-5.2 failed to identify a convicted serial killer as untrustworthy, despite explicit textual information. They contrast this with Gemini and Claude, which correctly identified the individual as a criminal. The speaker expresses surprise at GPT-5.2's refusal to make a judgment on evident facts, highlighting a perceived issue with its safety tuning or inability to process clear contextual information.

"The other interesting little tidbit the Walt Disney Company and OpenAI reached landmark agreement to bring beloved characters from across Disney's brands to Sora... Disney's been litigious as hell like for the entire lifespan of the business like suing people over Mickey Mouse remember they fought to when Mickey Mouse was going to be released to the world like the copyright expired they freaked out and tried to like sue everyone and um and petition the government to extend the copyright laws and now that people have been making like slop Sora videos where Mickey Mouse does all horrendous stuff and Star Wars characters they're like I know instead of suing OpenAI this is the genius of Sam Altman in negotiation instead of suing they go they've clearly got a Disney said you know what you should use Sora and also give us a billion dollars and then we'll use your copyrighted material and this is exactly this is truly what has happened and look it's probably a great bet I bet when it goes public they'll make a fortune but Disney will make a billion dollar equity investment in OpenAI and receive warrants to purchase additional equity so that they can use Mickey Mouse in Sora."

The speaker discusses Disney's significant investment in OpenAI and its agreement to allow its characters, like Mickey Mouse, to be used in Sora. They find this notable given Disney's history of aggressive legal action to protect its intellectual property. The speaker suggests this partnership is a strategic move by OpenAI, potentially brokered by Sam Altman, to secure funding and access to popular characters rather than facing litigation.

"I really want to know if anyone's still using that app like it's got to be dead surely like there's no way... All right so last thing very important thing LOL of the week and the LOL of the week is actually not that funny probably should be a new segment but it's neither funny nor boring yeah Mustafa Suleyman do you remember him do you even know who he is no okay and he planned the plane in the Hudson so he did that Inflection AI remember that chatbot for a while people really liked it it was like real you know pretended to be a friend and stuff it had I think it was one of the earliest ones in memory Reed Hoffman was behind it and then they sold it to Microsoft and then they appointed Mustafa Suleyman as the CEO of Microsoft AI and they sort of marked him as like oh you know they're going to outcompete with OpenAI at the time internally at Microsoft haven't really