Perseus: A Full-Stack System for Homomorphic Model Inference

| Source: arXiv AI

Tags: Perseus, Fully Homomorphic Encryption, Llama-3-8B, GPT-2, private inference, Cachemir

Perseus is an open Python library for fully homomorphic encrypted LLM inference that reports 3.96x faster single-token decoding than other implementations on Llama-3-8B, within 3% of plaintext accuracy.

Details

Fully Homomorphic Encryption lets a provider run a model on encrypted user data, but it has been too slow and the tooling mostly closed-source. Perseus is a free, open Python library covering the whole stack, from cryptographic backend and data packing to encrypted nonlinearities and end-to-end execution. It adds in-kernel regeneration of key-switching keys, a complex variant of Cachemir packing, and a packing- and magnitude-aware bootstrap placer. The authors report state-of-the-art latency across models and modalities, accuracy and perplexity within 3% of plaintext, and 3.96x faster single-token decoding on Llama-3-8B than any other implementation. They also launch a public speedrun on encrypted GPT-2.