Same Model, Different Harness: CUGA, Hermes, and Deep Agents — When to Pick CUGA?
| Source: Hugging Face Blog
Tags: IBM Research, CUGA, Hermes, Nous Research, Deep Agents, LangChain, AppWorld
IBM Research ran CUGA, Hermes and Deep Agents on the same open-weight model across 90 AppWorld tasks and found the harness alone moved completion from 30% to 81%.
Details
IBM Research compared three agent harnesses while keeping the model fixed: its open-source CUGA, Nous Research's Hermes, and LangChain's Deep Agents. On 90 AppWorld tasks, spanning 9 apps and 457 APIs, task completion ranged from 30% to 81% depending on the harness. The post cites a separate analysis where GLM-5.2 went from 23% to 52% on SWE-bench Pro when only the harness changed, arguing teams should pick a harness for their workload and model. IBM positions CUGA for enterprise use: rules enforced by the harness regardless of model output, and deployment inside your own environment, including sovereign clouds and air-gapped setups. Hermes targets always-on personal assistants and Deep Agents long multi-step research tasks.