One pipeline for dozens of enterprise apps

Problem

An enterprise with tens of services cannot afford a unique pipeline per repo. Copy-paste YAML drifts. Lint, image build, health checks, and deploy each get rewritten. Dev, staging, and production look similar enough to share a path — and different enough that a rigid pipeline breaks: Docker executor vs shell, Swarm vs Compose, local daemon vs a remote host over SSH.

Monorepos make it worse. A commit that only touches one app still rebuilds everything, unless the pipeline can read the message and skip the rest. Integration tests fail unless the temporary container can join the same overlay network as the database and resolve it by name.

The organization needed one catalog of jobs. Each project should declare names, ports, and branch rules — not invent a new CI design.

System Design

This project is a central template library. Apps include it and extend hidden jobs. Stages stay the same everywhere: lint, build, test, deploy.

Templates cover the variants that used to force a rewrite: executor checks, registry login, image build with optional build secrets for private dependencies, HTTP health tests, the same test attached to an existing network, Swarm stack deploy, Compose up/down, and deploy through SSH to a remote Docker host.

Branch rules and optional commit-message scopes decide which jobs run. Environment files come from CI variables or a path. Compose files are rendered at deploy time so the image tag and stack name stay consistent.

A consuming project is a short file: include the catalog, set stack name, image, compose file, ports, and which template to extend. Forty enterprise apps share the same path.

Architecture Decisions

• One shared catalog; apps extend jobs instead of copying pipelines
• Same stages for every service — only variables and tags change
• Executor-aware jobs so Docker and shell runners stay compatible
• Build, test, and deploy as separate templates so a service picks what it needs
• Health checks against a running container, including overlay-network attach for Swarm DNS
• Commit-message scopes so a monorepo rebuilds only the named app — or all
• Swarm, Compose, and SSH deploy as variants of the same contract, not three unrelated pipelines
• Image tag from the commit so environments promote the same artifact

Tradeoffs

• A catalog change can affect every consumer — version the include and roll out with care
• Hidden jobs are powerful; a project that overrides too much reintroduces drift
• Health checks assume an HTTP endpoint inside the image
• Overlay tests need an attachable network already on the runner
• Scope-by-commit-message is simple; it is not a full change-detection graph

Production Behavior

A team adds a new service by including the catalog and filling variables. Lint runs, the image is built and pushed, a throwaway container is health-checked (on the app network when needed), then the same tag is deployed to the named environment.

A new environment does not get a new CI design. It gets a branch rule, an env file, and a compose file — still extending the same templates. Built for an organization that has to keep tens of production apps on one delivery path.