Running SQL Concurrently Across Three Remote DuckDB Servers with Quack
| Source: Towards Data Science
Tags: DuckDB, Quack, SQL, data engineering, distributed systems
DuckDB's Quack protocol enables cross-server HTTP communication between DuckDB instances, and this hands-on experiment demonstrates concurrent SQL execution across three AWS-hosted databases—a practical test of distributed-style data access without full distributed query processing.
Details
DuckDB released the Quack protocol to allow DuckDB databases on separate servers to read and write each other's data over HTTP. Unlike attaching remote databases for federated queries, Quack creates a client/server channel enabling concurrent SQL execution across nodes. Author Thomas Reid tested this by standing up three AWS EC2 instances via CloudFormation, each running DuckDB 1.5.5 with the Quack extension, and built a GitHub project called cluster-duck to coordinate concurrent reads and writes. The setup uses a coordinator node pattern, Python 3.12, and a systemd service listening on port 9494. The experiment is notable because it probes the practical limits of Quack: it is not distributed query processing (DuckDB explicitly disclaimed that), but it does enable fan-out query patterns where results from multiple servers are aggregated centrally—useful for partitioned data architectures without full warehouse infrastructure. For data engineers evaluating DuckDB as a lightweight alternative to distributed systems, this is a useful proof-of-concept, though the experiment is exploratory and Quack is still new.