GitLab CI LLM Setup: Masked Variables and MR-Only Jobs

GitLab makes the mechanical parts of this easier than most CI systems and the security parts harder to get right by accident. Variables are a first-class concept with masking and protection flags, merge request pipelines are a supported trigger rather than a workaround, and the API for posting a note back is documented and stable. What is left to you is the discipline: not running the job on every push, not letting the key reach a shared runner, and not letting a slow provider hold a runner slot for an hour.

The policy question of what a model step should be allowed to decide is covered in running LLMs in CI. This is the GitLab-specific mechanics.

Masked, protected, and the difference that matters

The key goes in a CI/CD variable under Settings, CI/CD, Variables, with two flags set. Masked means GitLab redacts the value where it appears in job logs. Protected means the variable is only injected into pipelines running on protected branches and protected tags.

Masking has requirements the interface will tell you about only when it refuses: the value must be a single line, above a minimum length, and drawn from a restricted character set. A key containing a character GitLab cannot mask will be accepted as unmasked, and unmasked means it prints in full the first time somebody adds a debug echo. Check the flag actually stuck after saving.

Protected is the flag people skip, and it is the important one. Without it, anyone who can push a branch to the project can write a .gitlab-ci.yml that prints the variable, base64-encodes it past the masking, and reads it out of the job log. With it, the variable is simply absent on unprotected branches — which also means your merge request job will not have a key unless you plan for that, and the honest answer is usually to run the model step only on protected branches or to accept the exposure inside a private project with trusted developers.

Group-level variables are worth using if several projects share the credential, since rotation then happens once. Scope them to an environment if you want a different key for production pipelines.

The job, concretely

A merge request job that diffs against the target branch, calls the endpoint and keeps the answer as a file:

llm_review:
  stage: test
  image: alpine:3
  interruptible: true
  timeout: 6 minutes
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      changes:
        - "src/**/*.ts"
        - "src/**/*.py"
  before_script:
    - apk add --no-cache curl jq git
  script:
    - set -euo pipefail
    - git fetch origin "$CI_MERGE_REQUEST_TARGET_BRANCH_NAME" --depth=50
    - git diff --unified=0 "origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME"...HEAD > diff.txt
    - test -s diff.txt || exit 0
    - >
      jq -n --rawfile d diff.txt '{model:"kimi-k3", max_tokens:800,
      messages:[{role:"system",content:"Review this diff. Findings only."},
      {role:"user",content:$d}]}' > req.json
    - >
      curl -sS --fail-with-body --max-time 90
      -H "Authorization: Bearer $LLM_API_KEY"
      -H "Content-Type: application/json"
      -d @req.json "$LLM_BASE_URL/chat/completions" > resp.json
    - jq -r '.choices[0].message.content' resp.json > review.md
  artifacts:
    paths: [review.md]
    expire_in: 1 week
  allow_failure: true

Two details do a lot of work here. --fail-with-body makes curl exit non-zero on an HTTP error; without it the job passes on a 429 and you parse an error object into an empty review. allow_failure: true means a provider outage produces a yellow warning rather than a blocked merge request, which is the right severity for advisory output.

Rules, so it runs only where it should

The rules block is the whole cost story. The if clause restricts the job to merge request pipelines, so it never fires on a plain branch push or a scheduled pipeline. The changes clause restricts it further to pipelines where a file you care about actually moved.

Be aware that changes behaves differently outside merge request pipelines — on a branch pipeline it compares against the previous commit, which for a force push or a new branch can evaluate as true for everything. Keeping the if and the changes together in one rule avoids that class of surprise entirely.

Add interruptible: true and enable auto-cancel for redundant pipelines in the project settings. Someone iterating on a merge request will push five times in fifteen minutes, and without this you pay for five reviews of four commits that no longer exist. This is the GitLab equivalent of the concurrency group described in the GitHub Actions setup.

Artifacts versus a merge request note

An artifact is the low-friction option and the one to start with. The file is attached to the job, downloadable from the pipeline view, and expires on a schedule you set. Nothing needs write credentials, nothing spams the discussion, and if the output is bad for a week nobody has to clean up after it.

A note in the merge request discussion is what people actually want, because nobody clicks into a job to read an artifact. That needs a token with API write access, and the job token is not it — CI_JOB_TOKEN is deliberately limited and cannot post notes in most configurations. You need a project or group access token with the API scope, stored as another masked variable.

curl -sS --fail-with-body \
  -H "PRIVATE-TOKEN: $MR_NOTE_TOKEN" \
  --data-urlencode "[email protected]" \
  "$CI_API_V4_URL/projects/$CI_PROJECT_ID/merge_requests/$CI_MERGE_REQUEST_IID/notes"

Send the body with --data-urlencode reading from a file rather than interpolating the text into the command. Model output generated from a diff is untrusted text and will eventually contain a quote, a backslash or a newline that breaks a naive shell interpolation.

The good compromise is both: always write the artifact, post the note only when the review is non-empty, and update or delete the previous note instead of stacking a new one on every push. Nineteen stale review notes on one merge request is how a useful job gets disabled.

Runner considerations

Which runner executes the job changes your threat model. On GitLab.com shared runners each job gets a fresh ephemeral VM, so a leaked key does not persist past the job — but you are also sharing egress addresses with everyone else, which occasionally matters for provider allowlists.

Self-hosted runners are where care is needed. A shell executor runs jobs directly on the host with no isolation: a key written to a file, a shell profile or a git config on that machine is readable by every subsequent job, including jobs from projects you do not control. Prefer the Docker executor with a clean image per job, and tag the runner so only the projects that should reach your provider are scheduled onto it.

Network egress is the other runner question. A runner inside a private network usually needs an explicit proxy or allowlist entry to reach an external endpoint, and the failure looks like a connection timeout rather than a permission error. Test with a plain request from a job on that runner before debugging your YAML; the curl recipes cover the minimum shape.

Timeouts, retries and quota

Set the job timeout and a client timeout, because they protect different things. The job timeout stops a hung runner but tells you nothing about why; --max-time on the request produces a real error you can log. If you use an SDK rather than curl, check its default — several default to ten minutes with automatic retries built in, which means one slow call can eat an entire job budget silently. Tuning timeouts works through choosing the values.

GitLab has its own retry keyword, and it is the wrong tool for a rate limit. A job-level retry re-runs the whole job including the diff and the request, so a 429 becomes two or three full-cost calls spaced by however long the runner took to restart. Handle rate limits inside the script by reading Retry-After, as rate limits and retries describes, and reserve retry for genuine runner failures.

Use a resource_group if a monorepo fans out into many parallel jobs that would otherwise hit your provider quota simultaneously. It serialises jobs sharing the group name across pipelines, which is a blunt instrument but an effective one.

The takeaway

Masked and protected variable, and verify the mask actually applied. One rule combining a merge-request-only if with a changes filter. interruptible plus auto-cancel so iteration does not multiply the bill. --fail-with-body, both timeouts, allow_failure: true. Artifact always, note conditionally, with a project access token rather than the job token. Then track what it costs per merged request rather than per month, the way cost per code review lays out.