When One Process Becomes Too Much: Splitting a Pipeline into MCP Services

Details

The article walks through a common failure mode in Python pipelines: a single orchestrator class instantiating multiple engines (extraction, risk, fraud) that share one process and one virtual environment. The author shows that an unhandled exception in one engine silently kills the others, a version pin for one dependency can break another, and any bug fix requires a full application redeploy. The proposed solution is wrapping each engine in its own MCP server — not because MCP is special, but because a real process boundary around each service ensures crash isolation, independent dependency trees, and per-service release cadence. The author notes that plain REST endpoints achieve the same isolation; MCP adds standardized tool discovery and a cleaner client contract. The practical section covers what changed in the codebase: each service now has its own virtualenv, its own failure modes, and can be restarted independently. The article includes concrete code illustrating the before/after shape, though it notes the examples are illustrative rather than production-verbatim. For teams running multi-component AI pipelines — common in document processing, RAG pipelines, or multi-stage ML inference — this captures a real architectural inflection point that many hit six to twelve months after initial deployment.