Calling an LLM API from Jenkins Without Leaking the Key

An LLM call in Jenkins is just an HTTP request, which is why people underestimate it. The request is easy. What breaks is everything around it: a key printed into a public build log, a stage that passes because curl exited zero on an HTTP 429, and a retry loop that turns one upstream blip into four hundred billable requests across a matrix build.

Jenkins is also the least forgiving place to debug this. There is no TTY, no interactive shell, the workspace is wiped between runs, and the person who wrote the Jenkinsfile is rarely the person watching it fail at 2am.

Store the key as a Secret Text credential

Do not put the key in the Jenkinsfile, in a job parameter with a default value, or in a global environment variable under Manage Jenkins. Job parameters are visible to anyone who can view the build, and global environment variables leak into every job on the controller including ones you did not write.

Add it under Manage Jenkins, Credentials as a Secret text entry with an ID you will recognise, then bind it in the stage that needs it. Scope it to a folder rather than globally if your Jenkins hosts more than one team.

The binding matters as much as the storage. Jenkins masks credential values in console output only when they arrive through the credentials binding, and only for exact string matches — a key that gets base64 encoded, JSON-escaped, or split across a line break in a log will appear in plain text. Never echo the variable, and never pass it as a command-line argument, because process arguments are visible to anything that can read the process table on the agent.

The stage, concretely

A declarative pipeline stage that calls an OpenAI-compatible chat completions endpoint looks like this. Note the single-quoted shell block — that is deliberate, so Groovy never interpolates the secret into the script text it logs.

pipeline {
  agent any
  stages {
    stage('LLM review') {
      steps {
        withCredentials([string(credentialsId: 'llm-api-key', variable: 'LLM_KEY')]) {
          sh '''
            set -euo pipefail
            git diff --unified=0 origin/main...HEAD > /tmp/diff.txt
            jq -n --rawfile d /tmp/diff.txt '{
              model: "kimi-k3",
              messages: [
                {role: "system", content: "You review diffs. Reply with findings only."},
                {role: "user", content: $d}
              ]
            }' > /tmp/req.json

            curl -sS --fail-with-body \
              --max-time 120 \
              -H "Authorization: Bearer $LLM_KEY" \
              -H "Content-Type: application/json" \
              -d @/tmp/req.json \
              https://api.example.com/v1/chat/completions > /tmp/resp.json

            jq -r '.choices[0].message.content' /tmp/resp.json
          '''
        }
      }
    }
  }
}

Building the request body with jq -n --rawfile rather than string concatenation is not fussiness. A diff contains quotes, backslashes and newlines, and hand-assembled JSON will break on the first file that contains a quotation mark. Let a tool do the escaping.

Make the step fail for the right reasons

Plain curl exits zero on an HTTP 500. Your stage then pipes an error envelope into jq, gets null, and reports success. Use --fail-with-body so a non-2xx status becomes a non-zero exit while still printing the response body you need to diagnose it.

Then add set -euo pipefail at the top of every shell block. Without pipefail a failing command in the middle of a pipe is invisible, because the exit status of a pipeline is the status of the last command. This single line catches more CI bugs than any amount of clever error handling.

Finally, decide explicitly whether an LLM failure should fail the build at all. For a code review comment it should not — wrap the stage in a catchError with buildResult: 'SUCCESS' so an upstream outage does not block merges. For a generated artefact the build depends on, it should. Pick one per stage rather than defaulting to whatever Jenkins does. The same reasoning applies to any LLM step you add to a CI pipeline.

Timeouts and retries need coordinating

Set --max-time on curl, a timeout block on the stage, and nothing else. The failure mode to avoid is three independent retry layers: curl retrying, a Jenkins retry block wrapping the stage, and the pipeline itself being retriggered by a webhook. Those multiply rather than add, and a brief 429 becomes a sustained load spike that keeps you rate limited.

If you do want retries, put them in exactly one place and make them respect Retry-After. A fixed one-second sleep in a loop is worse than no retry at all when the provider is asking for thirty. The general shape is covered in handling rate limits and retries, and it applies unchanged to a build agent.

Watch out for build matrices here. A stage that behaves fine on one branch runs concurrently across every axis of the matrix, so your effective request rate is the axis count multiplied by the number of open pull requests. Throttle with a Jenkins lock or a stage-level concurrency limit if that number is large.

Cost on a shared controller

CI is where token spend goes unnoticed, because nobody is watching a build log the way they watch a bill. Every push triggers the stage, and a busy repository pushes hundreds of times a week.

Two controls do most of the work. Gate the stage on something meaningful — only pull requests, only when the diff touches source files, only when the diff is under a size threshold — using a when block rather than running it on every commit to every branch. And truncate the input deliberately: a 40,000-line diff is not going to produce a better review than the first 2,000 lines, it is just going to cost twenty times more.

Log the token counts from the response usage block into the build description so the number is visible next to the build that caused it. Per-token billing across a CI fleet is genuinely hard to forecast, which is one reason flat-rate versus per-token pricing is a real decision rather than an accounting detail.

Debugging a stage that will not run

Reproduce outside Jenkins first. Copy the exact curl into a terminal with the key exported and confirm it works — that separates a broken request from a broken agent environment. The curl recipes for LLM APIs are useful as a known-good baseline.

If curl works locally but not on the agent, the usual causes are a missing binary (jq is not installed on a minimal agent image), an outbound proxy that needs HTTPS_PROXY, or TLS interception that curl rejects. A 401 on the agent with the same key that works locally almost always means the credential binding is not in scope for that block.

For anything that returns a status code you did not expect, the error code reference is faster than guessing. And keep the raw response file as a build artefact on failure; the error body is where the actual reason lives.

What not to do in a Jenkinsfile

Do not stream. Streaming buys perceived latency for a human watching output, and there is no human watching. Take the single JSON response and parse it once.

Do not put a multi-turn agent loop in a shell block. If the task needs iteration, tool calls and state, write it as a script in a real language, commit it to the repo, and have Jenkins invoke that script. Shell is a fine transport and a bad control flow language for anything with retries and branching.

And do not let the LLM decide whether the build passes. A model that returns a severity score is fine as information; a model whose output gates a deploy will eventually block one for the wrong reason and nobody will be able to explain why.