NVIDIA Releases TensorRT Model Connect in Public Preview: Hugging Face Checkpoint to Native C++ Inference in Two Commands

August 18, 2026 | Source: MarkTechPost

Tags: NVIDIA, TensorRT, TRTMC, Hugging-Face, C++, inference, edge-AI, embedded

NVIDIA released TensorRT Model Connect (TRTMC) in public preview — an Apache 2.0-licensed tool that converts Hugging Face checkpoints to native C++ TensorRT inference in two commands, skipping the ONNX export step and producing versioned .bundle artifacts for C++ services and embedded systems.

Details

NVIDIA's TensorRT Model Connect (TRTMC) addresses a persistent pain point in production AI deployment: getting from a Hugging Face model checkpoint to a performant C++ inference binary without navigating ONNX conversion. Two commands — trtmc build and trtmc run — produce a versioned .bundle artifact that loads natively in C++ via task-oriented APIs (generate(), transcribe(), embed(), solve()) with no PyTorch dependency at runtime. The project is Apache 2.0 licensed and fully open-source. Notably, NVIDIA reports the entire project — implementations, performance tuning, tests, and documentation — was built using OpenAI Codex agents under human direction and review. Practical constraints exist today. Public wheels currently target Linux aarch64 only (Python 3.10 or 3.12, glibc 2.39+, TensorRT 11.1.0.106). x86_64 users must take the Docker source-build path. This makes TRTMC best suited today for robotics, edge AI, automotive compute stacks, and medical device makers — anyone already running C++ inference infrastructure on ARM. Regulated enterprises should wait for tagged releases before standardizing; the project is currently public preview. Small teams shipping Python services gain little from it immediately.