VaultGemma: The world's most capable differentially private LLM
| Source: Google Research Blog
Tags: VaultGemma, differential privacy, language models, Google Research, scaling laws, AI privacy
Google Research has introduced VaultGemma, a differentially private language model (LLM) with 1 billion parameters. This model aims to balance privacy and performance, addressing trade-offs in training stability and computational costs.
Details
VaultGemma is a new language model developed by Google Research, notable for being the largest model trained from scratch with differential privacy (DP). The research highlights the complexities of applying DP to LLMs, including trade-offs in training stability and increased computational demands. A key aspect of the research is the establishment of scaling laws that describe the compute-privacy-utility trade-offs in DP training. VaultGemma's development is guided by these findings, and the model's weights will be made available on platforms like Hugging Face and Kaggle to promote advancements in private AI.