A New Trick Reveals AI Models’ Inner Thoughts

Details

Computer scientists from University of Tübingen, Max Planck Institute, MATS Research, and security firm Snyk discovered a vulnerability affecting all major frontier AI models from Anthropic, OpenAI, and Google. The technique extracts hidden 'reasoning traces' — the step-by-step internal thinking models perform on complex problems — that providers consider proprietary. The vulnerability has since been patched by all affected providers. The most significant finding: Chinese open-weight model Kimi K3 from Moonshot AI produces reasoning traces strikingly similar to those of Claude Opus 4.8 and GPT 5.6 Sol for certain prompts. The researchers note this 'cannot causally establish distillation' but the similarity is substantial. By contrast, DeepSeek and Inkling from US firm Thinking Machines did not exhibit similar reasoning patterns. Moonshot AI did not respond to comment requests before publication. Beyond the IP implications, researchers demonstrated the technique could recover passwords and API keys from a model's inner reasoning — a critical security finding now fixed. The work adds empirical weight to congressional testimony: Anthropic told lawmakers in June that Alibaba had systematically distilled its models for Qwen; OpenAI made similar claims about DeepSeek's R1. Distillation — copying capabilities from existing models into new ones — is a standard, legitimate technique, but using hidden reasoning traces from closed models raises serious IP and competitive concerns.