CI/CD observability

CI/CD observability is the practice of collecting and analyzing data about pipeline execution — run outcomes, job durations, queue time, retries, failures and cost — so that questions about build health can be answered from evidence instead of from memory of last week's red builds.

Two ways to get it

Instrument the pipeline. You emit telemetry from the pipeline itself. OpenTelemetry has CI/CD semantic conventions that model a pipeline run as a SERVER span with task executions as child spans, plus matching metric and log conventions, and a dedicated CI/CD Observability SIG maintaining them. Pipelines are wired to export spans to a collector, which lands them in whatever backend already holds application traces.

Read the provider's API. You poll the CI provider's own REST API for the runs, jobs, steps and logs it already records, then compute metrics from that history. Nothing is added to the pipeline; the analysis reads what the provider stored.

What each can and cannot see

Instrumentation sees the most: custom spans inside a step, per-test results if the test runner is instrumented, spans that stitch pipeline traces to the application traces of the deploy they produced. The costs are structural. Every pipeline must be edited and kept edited — a workflow someone forgot to instrument is invisible, which is exactly the neglected pipeline worth looking at. Telemetry volume is real money in a usage-priced backend, and instrumentation typically runs inside the job, with access to whatever the job has.

The read-only API approach starts complete: every run the provider recorded is there, including history from before the tool existed, with no pipeline changes and nothing new running next to your build. What it gives up is depth and latency. It sees a step, not the function inside it; it has no per-test detail unless the provider exposes it; and with polling rather than webhooks, data arrives on a sync interval, not the second a run ends. It also inherits the provider's own retention and rate limits.

Neither is strictly better. Deep root-cause work on one service argues for instrumentation; cost, reliability and trend questions across an entire organization's pipelines argue for reading what the provider already has, because coverage matters more than resolution.

Why it matters

The permission model is the part teams underestimate. Pipeline telemetry is one of the easiest places for a tool to acquire broad repository access, and the access granted at install rarely gets revisited. Asking what a CI observability tool can read is a security question, not a procurement formality.

TrimCI takes the read-only API route deliberately: a GitHub App scoped to Actions and Metadata read-only — never Contents — and a GitLab client with an HTTP-layer endpoint allowlist, so source code, workflow YAML and diffs stay out of reach. The boundary is documented on security. The trade-offs come with the choice: TrimCI polls the provider on a sync interval rather than receiving webhooks, and its granularity is the run and the job, never the individual test. The metrics it produces are covered in CI pipeline failure analysis.