Google's HEIR Compiler: Practical Private AI with Homomorphic Encryption

Google has released HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler that converts pre-trained AI models to operate on encrypted inputs. This enables cryptographically-secure private AI inference — the server processes ciphertexts without ever seeing the underlying data.

Why HEIR?

Homomorphic encryption has long been impractical due to massive computational overhead. HEIR aims to change that by providing a compiler toolchain that automates the conversion of existing models, eliminating the need for a team of cryptographers. The project is part of Google's Private Computing Toolkit, which already includes differential privacy, private set membership, and private information retrieval.

Key Features

• One-click goal: The vision is to make HEIR a one-click solution for non-experts to integrate encrypted inference into production.
• Hardware accelerators: Google has partnered with Belfort, Niobium, Cornami, and Optalysys to build dedicated hardware for homomorphic encryption. The latency benefits of these accelerators are planned for near-future demos.
• Research platform: HEIR is already used in collaborations with Georgia Tech, Carnegie Mellon, UC Santa Barbara, Purdue, and others, with four peer-reviewed publications built on it.

Demo Applications

Google shared four private inference applications compiled with HEIR, with latency numbers measured on a single-threaded CPU. Source code is available in the GitHub repository.

• Deep Learning Recommendation Model — private content recommendations (joint work with Belfort Labs, LG, NYU).
• Credit card fraud detection — compiled with Niobium and hardshell.ai.
• Threat intrusion detection — Kitsune system for anomaly detection on encrypted network traffic, without revealing packet contents.
• Hotword detector — enables audio-triggered AI agents to recognize hotwords privately (with Belfort Labs).

These demos show that homomorphic encryption is no longer theoretical — it's ready for real-world applications in sectors like healthcare and finance where data privacy is critical.

The code for all examples is available on GitHub. Check the full blog post for detailed latency numbers.