Self-Hosted vs. API Breakeven

Find the request volume where renting or buying GPUs beats paying per token — including the MLOps overhead most comparisons leave out. From Appendix G of Production LLM Architecture.

Your Setup

API

Model tier

Input tokens/req

Output tokens/req

Cached input share

Defaults to 0% — Appendix G's worked example assumes no caching

Self-Hosted

GPU preset

GPU count

$/GPU-hour

Hours/month live

MLOps FTE

Loaded $/FTE/yr

Requests per month

Evaluate

Depends on output length, model size, and ops capacity

Cost Comparison

API / month

CHEAPER

Self-hosted / month

CHEAPER

Self-Hosted Breakdown

GPU infrastructure $0

MLOps overhead $0

Total self-hosted $0

—

Breakeven request volume/month at your current settings

Design Rule — Factor MLOps operational overhead into every self-hosted comparison. Running inference infrastructure requires model serving, monitoring, scaling, version management, and on-call coverage — typically 1–2 dedicated engineers. Many teams that "save money" on GPU inference spend more on the engineers operating it.

Cost by Volume

Self-hosted cost is fixed (GPU + ops); API cost scales linearly with volume. The crossover point is your breakeven.

API Self-hosted

This is Appendix G. The book has 61 more tables like this — plus the GPU capacity planning (Appendix H) and reference architecture (Appendix I) that turn this estimate into a deployable system.

Get notified when it's live →