Toward Personal Intelligence Through Cooperative Observation

Details

This paper argues that personal AI systems face a fundamental observation bottleneck: useful assistance requires a model of the user's goals and commitments, but broader observation alone is insufficient because bounded systems must compress information. The key insight is that observation access should be earned and maintained through demonstrated usefulness. The authors introduce 'cooperative observation' to name this feedback loop among usefulness, trust, and future observational access. Useful and inspectable behavior gives users reason to expand the observation channel; failures — or perceived overreach — can narrow or revoke it. This framing positions trust as an input variable in system design, not just a desirable property. The paper reports on Organizm, a prototype personal AI used by one subject over six months in a preliminary account. The single-subject nature limits generalizability, and the authors are explicit that this is a framework paper with evaluation directions, not a completed empirical study. For practitioners building personal productivity AI, this offers a principled alternative to maximizing data collection: design for inspectable, useful behavior that earns expanded access incrementally. Privacy and user control are architecturally central, not add-ons.