Commit Graph
4 Commits
Author SHA1 Message Date
ZverGuy d3e6fd6aa8 fix(observability-proxy): keep the rate-limit buckets under their bound
When the bucket map was full and the sweep freed nothing, the new
user's bucket was added anyway, so the map grew past its bound and
every further newcomer paid a full scan under the limiter lock.

A newcomer is now refused with 429 until the first bucket refills,
and the Retry-After tells it when. Resetting the map instead would
hand every throttled user a fresh burst.

Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-11-10 00:45:06 +07:00
ZverGuy f19363ec62 fix(observability-proxy): keep Retry-After positive at tiny rates
The wait until the next token was converted from float to Duration
without a range check, so a rate limit below about 1e-10 requests per
second produced a negative Retry-After on amd64. The wait now saturates
at the largest Duration.

The earlier 429 change said range, step, bucket and limit violations
keep 422; on the logs root they are clamped or answer 400, and nothing
answers 422.

Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-11-10 00:45:06 +07:00
ZverGuy 163b079845 fix(observability-proxy): answer 429 to rate and concurrency limits
Clients such as Grafana and browser retry logic back off on 429 Too
Many Requests, not on 422, which they treat as a final rejection of
the query. The per-user rate limit, the per-user concurrency limit and
the open-tail caps are about the caller's pace, not the query, so they
now answer 429. The rate limiter knows when its bucket refills, so that
answer carries Retry-After; the concurrency caps cannot know when a
slot frees and omit it. Range, step, bucket and limit violations keep
422.

Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-11-09 15:48:57 +07:00
ZverGuy 61275df240 feat(observability-proxy): cap per-user concurrency, rate and open tails
The log stores are shared by every namespace routed to them, and the
root store also serves platform Grafana, so one tenant's dashboard must
not be able to monopolise vlselect. Live tails are counted apart from
request concurrency because they hold their slot for minutes and would
otherwise starve the same user's panels.

Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
2026-11-09 15:48:57 +07:00