Your Jenkins controller has a fixed pool of static build agents. When would switching to ephemeral, on-demand agents (e.g. via Kubernetes) actually be worth the added complexity?

It's worth it once your build load is genuinely variable and your static pool is either sitting mostly idle (wasting cost) or getting saturated during peak periods (queuing builds and slowing feedback) — ephemeral agents scale to match actual demand instead of a fixed capacity, at the cost of more complex setup and slightly slower individual build start times (provisioning a fresh agent takes longer than an already-running one picking up a job).

Static agents and ephemeral agents optimize for different things: static agents optimize for fast build start (a job can start immediately on an already-idle agent) and operational simplicity (nothing to provision, just a fixed known pool); ephemeral agents optimize for resource efficiency and elastic capacity, at the cost of both setup complexity and a per-build provisioning delay.

Static agents make sense when build load is predictable and roughly matches a fixed pool size: if your organization's build volume is fairly steady and a fixed number of agents keeps queue times low without excessive idle capacity, the operational simplicity of static agents (no orchestration layer to manage, no provisioning delay) is a real advantage not worth trading away for elasticity you don't actually need.

Ephemeral agents earn their complexity when load is genuinely variable or growing: a fixed pool sized for peak load sits mostly idle (and costs money) during quiet periods; a fixed pool sized for average load queues builds during peaks, slowing feedback exactly when developers are most active — ephemeral agents provisioned on demand (via the Kubernetes plugin or cloud-specific agent plugins) scale to match actual load in both directions, without either wasted idle cost or peak-time queuing.

Ephemeral agents also give you a cleaner build environment by default: each build gets a fresh agent/container rather than potentially inheriting state left behind by a previous build on a long-lived static agent (leftover files, cached dependencies with subtle version drift, environment pollution) — this can catch a class of "works on this agent but not that one" flakiness that static agents are more prone to over time.

The trade-offs are real, not just setup cost: provisioning a fresh agent per build adds startup latency to every build (pulling a container image, waiting for the pod to schedule, in a Kubernetes-backed setup) compared to an already-running static agent picking up a job immediately — for teams where fast build start matters more than elastic efficiency, this is a genuine downside, not just a one-time migration cost.

• Static agents optimize for fast build start and operational simplicity; ephemeral agents optimize for elastic capacity matching actual, variable load.
• Ephemeral agents are worth the added complexity once a fixed pool is either wasting idle cost or queuing builds during peaks — genuinely variable or growing load is the signal.
• Ephemeral agents also give a cleaner, more reproducible build environment per build, reducing a class of state-pollution flakiness static agents are prone to over time.
• The per-build provisioning delay is a real, ongoing trade-off, not just a migration cost — weigh it against how much fast build start actually matters for your team.

• Jenkins Docs: Kubernetes plugin
• Jenkins Docs: Distributed builds