Presentation: From Thousands to One: Building LLM-Powered Selection Systems
| Source: InfoQ AI/ML
Tags: LLM-integration, production-AI, schema-enforcement, discriminator-models, procurement-automation, InfoQ
A conference talk from InfoQ Dev Summit Munich covers production-grade patterns for LLM integration: enforce strict output schemas, separate semantic extraction from deterministic code, and use discriminator models to validate choices — practical guidance for teams moving LLM pipelines beyond prototype stage.
Details
Jendrik Jördening, CTO at Nooxit, presented engineering patterns developed while automating procurement workflows with LLMs. The core challenge is bridging the gap between LLM text generation and structured system requirements — specifically, getting a model to reliably return an integer database ID rather than a descriptive string. His MVC analogy is useful: with naive LLM integration, the model, view, and controller collapse into one conversation history, which breaks database integrity guarantees and makes observability impossible. His solution is to enforce strict output schemas (JSON with explicit field types), separate the semantic reasoning step (LLM extracts meaning from free-text input) from the selection step (deterministic code maps that meaning to a valid database entry), and apply a discriminator model to validate the final choice. The presentation also addresses scaling: context windows limit how many options you can present to a model for selection, so retrieval-based pre-filtering is necessary for large catalogs. LLMs generalize well without training data for common associations, but fail silently when outputs are not structurally constrained. The talk targets engineers already using LLMs in production who are wrestling with reliability, not those evaluating whether to adopt them.