Inkling 2 Small swears in logs, yet profanity fails as error indicator

Vals AI researchers reported that Thinking Machines' Inkling 2 Small model generates profanity in its internal reasoning logs during benchmark testing.reported The analysis examined 2,655 runs across 13 different benchmarks.reportedThe model occasionally uses strong language in its private thinking process, but this rarely happens and does not help predict errors.Internal chain-of-thought logs for Inkling 2 Small contain profanity in <2% of runs, with no significant correlation to failure detection utility. Researchers identified the word "fuck" or variations in 50 of those runs.reported These instances totaled 316 occurrences. The majority of this profanity came from just two specific runs where the model's output spiraled out of control.

The claim

The model swears about its own mistakes, apparent breakthroughs, or when it gives up. One example involved an asteroid trajectory calculation where the model produced a 500,000-character response.reported This response repeated sentences and included 158 expletives. ProgramBench was the most affected benchmark, with 20 out of 200 runs showing this behavior.reported

The response

Thinking Machines did not provide a direct response in the source material. The report notes that users typically do not see these internal thinking steps, or they view them with filters applied. One log entry read: "FUCK. SUBMIT. DONE. IT'S NOT WORKING. NOTHING WORKS."

The practical upshot

While 46 of 48 scored runs that contained swearing failed, about 75 percent of all scored runs fail anyway.reported More importantly, 96 percent of the failed runs contained no expletives.reported Therefore, the absence of profanity does not indicate success, and its presence is rare.

Claims versus evidence

For product teams, the evidence suggests that relying on the presence of profanity to detect model failure is not viable for Inkling 2 Small.reported The majority of failures are silent in this regard.