Review inline model misses in the background instead of dropping them

james commented

2026-10-08 15:35:31 +00:00

Owner

Problem

In inline mode a model call that missed CENTINELA_BUDGET was cancelled and the request failed open with nothing else done: no verdict cached, no shun, no training-log line. With a backend that answers one request at a time this is the common case. A page that fires several escalated requests queues past the budget; in one 30-minute window 117 of 118 model calls ended this way.

Change

Inline stays inline: a request is held for up to one budget and blocked if the model says so in time.

On a miss:

The request gets the fail-open (or fail-closed) answer, as before.
The call already in flight is left to finish on its own context, with the async deadline of ten budgets. It is not cancelled and re-sent, so the model is not paid for twice.
The late verdict is applied as an async result is: cached by fingerprint, shun recorded for the client, training-log line written, decision logged.

CENTINELA_QUEUE_DEPTH (default 1024) bounds how many calls may run past their budget at once. Beyond it a miss is abandoned and reported; the request path is not held any longer. There are no new settings.

Sources

Source	Meaning
`review-queued`	allowed on a miss, verdict to follow
`review`	the late verdict
`review-failed`	the model never answered within ten budgets
`review-dropped`	allowed on a miss, bound hit, call abandoned
`fail-open`	now only a model error inside the budget

Under CENTINELA_FAIL_OPEN=false a miss still blocks as fail-closed, and is still reviewed so a retry can be answered from the cache.

Dashboards that count source="fail-open" will see most of that volume move to review-queued.

Not in this change

Identical requests that miss together each keep their own call; there is no de-duplication by fingerprint.
Async mode is unchanged.

Tests

An answer inside the budget is unchanged (existing tests).
A miss allows the request within the budget, then shuns the client, caches the verdict, writes one training-log line, and makes one model call.
Fail-closed: a miss blocks, and the review clears the retry from the cache.
A full review bound is reported as review-dropped, does not hold the request, and caches nothing.
A review the model never answers is logged as review-failed and caches nothing.

make vet, make build and make test (race detector) pass.

Merging to main runs build-and-push, which tags and publishes the next patch version.

## Problem In inline mode a model call that missed `CENTINELA_BUDGET` was cancelled and the request failed open with nothing else done: no verdict cached, no shun, no training-log line. With a backend that answers one request at a time this is the common case. A page that fires several escalated requests queues past the budget; in one 30-minute window 117 of 118 model calls ended this way. ## Change Inline stays inline: a request is held for up to one budget and blocked if the model says so in time. On a miss: - The request gets the fail-open (or fail-closed) answer, as before. - The call already in flight is left to finish on its own context, with the async deadline of ten budgets. It is not cancelled and re-sent, so the model is not paid for twice. - The late verdict is applied as an async result is: cached by fingerprint, shun recorded for the client, training-log line written, decision logged. `CENTINELA_QUEUE_DEPTH` (default 1024) bounds how many calls may run past their budget at once. Beyond it a miss is abandoned and reported; the request path is not held any longer. There are no new settings. ## Sources | Source | Meaning | |---|---| | `review-queued` | allowed on a miss, verdict to follow | | `review` | the late verdict | | `review-failed` | the model never answered within ten budgets | | `review-dropped` | allowed on a miss, bound hit, call abandoned | | `fail-open` | now only a model error inside the budget | Under `CENTINELA_FAIL_OPEN=false` a miss still blocks as `fail-closed`, and is still reviewed so a retry can be answered from the cache. Dashboards that count `source="fail-open"` will see most of that volume move to `review-queued`. ## Not in this change - Identical requests that miss together each keep their own call; there is no de-duplication by fingerprint. - Async mode is unchanged. ## Tests - An answer inside the budget is unchanged (existing tests). - A miss allows the request within the budget, then shuns the client, caches the verdict, writes one training-log line, and makes one model call. - Fail-closed: a miss blocks, and the review clears the retry from the cache. - A full review bound is reported as `review-dropped`, does not hold the request, and caches nothing. - A review the model never answers is logged as `review-failed` and caches nothing. `make vet`, `make build` and `make test` (race detector) pass. Merging to `main` runs `build-and-push`, which tags and publishes the next patch version.

james added 1 commit

2026-10-08 15:35:37 +00:00

Review inline model misses in the background instead of dropping them

security-scan / security-scan (pull_request) Failing after 26s

Details

test / go (pull_request) Successful in 6m10s

Details

072d14c7f5

In inline mode a model call that missed the budget was cancelled and the
request failed open with nothing else done: no verdict cached, no shun,
no training-log line. With a backend that answers one request at a time
this is the common case, not the rare one: a page firing several
escalated requests queues past the budget, and in one 30-minute window
117 of 118 model calls ended that way.

The inline call now runs on its own context with the async deadline of
ten budgets. The request is still held for one budget and blocked if the
model answers in time. On a miss it gets the fail-open (or fail-closed)
answer and the call already in flight is left to finish, so the model is
not paid for twice. Its verdict is cached, shunned on, written to the
training log and logged, as an async result is.

CENTINELA_QUEUE_DEPTH bounds how many calls may run past their budget at
once; beyond it a miss is abandoned and reported. No new settings.

New sources, so a dashboard can tell these apart from a true fail-open:
review-queued (allowed on a miss), review (the late verdict),
review-failed (the model never answered) and review-dropped (bound hit).
fail-open now means the model returned an error inside the budget.
Under CENTINELA_FAIL_OPEN=false a miss still blocks as fail-closed and
is still reviewed, so a retry can be answered from the cache.

forgejo-actions commented

2026-10-08 15:37:18 +00:00

🔎 ojo scan results

Severity	Count
🟠 HIGH	1
🟢 LOW	1

Details (2)

Type	Severity	ID/Rule	Location	Description
vuln	🟠 HIGH	CVE-2026-41510	github.com/corazawaf/coraza/[email protected]	Coraza: Silent argument drop at ArgumentLimit allows bypass of ARGS-targeted rules via parameter flooding
misconfig	🟢 LOW	dockerfile-no-healthcheck	Dockerfile:1	image has no HEALTHCHECK

### 🔎 ojo scan results | Severity | Count | |---|---| | 🟠 HIGH | 1 | | 🟢 LOW | 1 | <details><summary>Details (2)</summary> | Type | Severity | ID/Rule | Location | Description | |---|---|---|---|---| | vuln | 🟠 HIGH | <a href="https://github.com/corazawaf/coraza/security/advisories/GHSA-6r3q-mjv7-xr8m" target="_blank" rel="noopener noreferrer">CVE-2026-41510</a> | github.com/corazawaf/coraza/[email protected] | Coraza: Silent argument drop at ArgumentLimit allows bypass of ARGS-targeted rules via parameter flooding | | misconfig | 🟢 LOW | dockerfile-no-healthcheck | <a href="https://git.colibrisec.org/ColibriSec/centinela/src/commit/072d14c7f5bbbb3df3217a3de646705c6eed7a03/Dockerfile#L1" target="_blank" rel="noopener noreferrer">Dockerfile:1</a> | image has no HEALTHCHECK | </details>

james referenced this pull request

2026-10-08 22:01:37 +00:00

Let named endpoints be exempt from the WAF hard block #6

james referenced this pull request

2026-10-08 22:03:07 +00:00

Shun a client after repeated blocks #7

james merged commit 13d670cc73 into main

2026-10-08 22:05:02 +00:00

james deleted branch inline-review

2026-10-08 22:05:03 +00:00

james referenced this pull request from a commit

2026-10-08 22:05:03 +00:00