An absent namespace resolves to the root store, and before the
informer's first list every namespace is absent, so a request served
in that window read a labeled namespace's logs from the wrong store.
Readiness keeps the Service from routing there, but a client that
addresses the pod directly is not stopped by it.
The resolver now fails while the cache is unsynced, which answers
503 like any other resolution error.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
vlselect widens the window to step boundaries and returns a
zero-filled bucket for each one, so a start off a boundary yields one
bucket more than the span divided by the step. Sizing the step from
the unaligned span let hits and stats_query_range return MaxBuckets+1
buckets.
The minimum step now spans MaxBuckets-1 steps, and the smallest
accepted --logs-max-buckets rises to 4 so that this step still fits
the third-of-range cap that keeps alignment inside the window.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
When the bucket map was full and the sweep freed nothing, the new
user's bucket was added anyway, so the map grew past its bound and
every further newcomer paid a full scan under the limiter lock.
A newcomer is now refused with 429 until the first bucket refills,
and the Retry-After tells it when. Resetting the map instead would
hand every throttled user a fresh burst.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
A TokenReview whose authenticator failed comes back unauthenticated
with status.error set, and a SubjectAccessReview whose authorizer
failed comes back with an evaluation error and no decision. Both were
cached as refusals, so a valid caller stayed locked out for the cache
TTL after the outage ended.
Such answers are now refused for the current request only. A review
that failed to decide still answers 401 or 403 rather than 503: RBAC
sets an evaluation error for any binding to a missing role, which a
retryable status would turn into a permanent 503 for that user.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Once the status line has gone out, a read error from the store left
the handler returning normally, so the chunked body ended cleanly and
the client took a partial answer for a complete one.
The handler now aborts the response in that case. The request is
counted before the abort, because the abort skips the rest of the
handler.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The tail idle timer started only once the store had sent its headers,
and a tail request carries no deadline, so a store that accepted the
request and stayed silent held the tail slot until the client left.
The timer now starts before the request, and the same timer bounds the
header wait and the gap between lines. A store that never answers
gets 504, like a non-tail request past its deadline.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
A client that stopped reading kept its slot for good: the first flush
ran before the tail idle timer started, and cancelling the upstream
request does not interrupt a write that is blocked on the client.
Every response now carries a write deadline. A tail renews it by the
idle timeout before each line, so a tail that keeps receiving stays
open while a stalled reader is cut and frees its slot; any other
response must be written by the request deadline.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The form was parsed before the concurrency slot was taken, and the
listener sets no body-read deadline, so a client that never finished a
POST body held a handler outside every limit for as long as it liked.
Tenant workloads reach the Service directly, so nginx buffering does
not cover this path.
The slot is now taken first and the body is read under the request
timeout; a body that does not arrive in time answers 408 and frees the
slot. The deadline is lifted once the body is read, because the
server's background read would take a later expiry for a client that
went away and cancel a live tail.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Only tails opt out of the transport's gzip negotiation; bounded
requests keep it on purpose. Assert that side too, so a change that
applies identity to every request fails a test instead of silently
dropping compression between the proxy and the store.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Go's transport asks for gzip on its own when a request sets no
Accept-Encoding. vlselect answers a gzip-negotiated tail through a gzip
writer that holds even the response headers until the first line, so the
proxy could not forward its 200 and the console kept showing a
connecting state for quiet applications; buffered lines were also
delayed.
The tail request now sends Accept-Encoding: identity, which also turns
off the transport's transparent gzip. Other endpoints keep it: they are
bounded responses read to the end, compression saves bandwidth between
the proxy and the store, and the transport decompresses before the body
is copied to the client.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The wait until the next token was converted from float to Duration
without a range check, so a rate limit below about 1e-10 requests per
second produced a negative Retry-After on amd64. The wait now saturates
at the largest Duration.
The earlier 429 change said range, step, bucket and limit violations
keep 422; on the logs root they are clamped or answer 400, and nothing
answers 422.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
parseDuration summed duration parts without an overflow check and
converted out-of-range floats to Duration, which Go leaves
implementation-defined. A start_offset such as
99999999999h99999999999h1s wrapped to 999ms and was accepted, and a
huge step saturated on some architectures and wrapped on others. Such
values now answer 400 like any other invalid duration.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Clients such as Grafana and browser retry logic back off on 429 Too
Many Requests, not on 422, which they treat as a final rejection of
the query. The per-user rate limit, the per-user concurrency limit and
the open-tail caps are about the caller's pace, not the query, so they
now answer 429. The rate limiter knows when its bucket refills, so that
answer carries Retry-After; the concurrency caps cannot know when a
slot frees and omit it. Range, step, bucket and limit violations keep
422.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Two of the three windows divided into whole milliseconds, so they
passed with the old truncating step and guarded nothing. Each window
now leaves a sub-millisecond step remainder.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The step is sent to vlselect in milliseconds and was truncated, so a
window that the bucket cap does not divide evenly got one bucket more
than --logs-max-buckets allows. Round it up before the window is
shrunk to fit the step, so both caps still hold.
Bare-seconds steps and start offsets are now tested on the accepting
side too: only the refused forms were, and disabling the digit branch
passed the suite.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Bare seconds were parsed with ParseFloat, which also takes signs,
exponents, Inf and NaN. Such a start fell through to the relative
duration form and silently moved the window, so 1.7937864e9 meant
1970 rather than an error, and NaN converted differently per CPU
architecture. Only digits with an optional fraction are taken now;
everything else is a 400.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
vlselect widens [start, end] outward to whole steps before scanning, so
a step of a hundred years turned the clamped week into the whole
retention. The step is now capped at the window and at a third of the
maximum range, and the window is shrunk from its start until the
widened span still fits the maximum range.
Numeric times are accepted only as unambiguous unix seconds, because
vlselect reads the unit from the digit count and a millisecond stamp or
a bare year would otherwise land in the wrong place. The standalone
ignore_global_time_filter substring check is dropped: the options block
check already covers it, and the substring only refused plain searches
for that word.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
A request that hits the proxy's own deadline was answered like an
unreachable store, so a heavy query looked like an outage; it now
answers 504, and a client that went away is no longer logged as a
store failure. A tail line is read through a fixed buffer, so a single
huge log line cannot make one stream hold unbounded memory.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The label value is spliced into the vlselect host name. A value with a
dot would address a host in some other namespace instead of a tenant's
store, so it is treated as a resolution error rather than followed.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The console nginx selects its location on the normalized path but
forwards the original request URI. A path such as
namespaces/tenant-b/../tenant-a/logs/ reaches the proxy as written, so
any interpretation the proxy makes of dot segments, empty segments or
percent-encoding could make the namespace in the SubjectAccessReview
differ from the one a normalizer downstream would see. Such paths are
answered 400 instead of being interpreted.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
A substring match on ignore_global_time_filter does not see the option
once it is written as a quoted name with an escape, or split by a
comment: the LogsQL lexer unquotes option names and joins words across
a comment, so the option still applies and lifts the time window the
proxy enforces. None of the query options is needed by a tenant, so
the options keyword followed by a parenthesis is refused outright,
skipping whitespace and comments exactly as the lexer does.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Tenants get a way to read their namespace's logs without Grafana and
without reaching vlselect directly, which would give them neither an
RBAC check nor a forced scope. The handler authenticates, authorizes
from the path, rebuilds the upstream request with fixed tenant headers
and no client headers, and returns VictoriaLogs' native response.
A live tail is flushed after every line, is not subject to the request
timeout, and is closed once idle, so it survives far past the apiserver
request timeout without leaving forgotten browser tabs holding streams
open. An unreachable store answers 502 so the dashboard can tell an
outage from an empty result.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The log stores are shared by every namespace routed to them, and the
root store also serves platform Grafana, so one tenant's dashboard must
not be able to monopolise vlselect. Live tails are counted apart from
request concurrency because they hold their slot for minutes and would
otherwise starve the same user's panels.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
vlselect has no notion of a tenant: whatever filter reaches it decides
what is returned. The upstream parameters are therefore rebuilt from a
per-endpoint allowlist with the namespace scope as the only
extra_stream_filters value, so a client cannot widen it through
extra_filters, tenant headers or an alternate parameter spelling.
VictoriaLogs ANDs extra_stream_filters into every subquery, so union
and in() queries pass through unparsed. What the query can still do is
cost: a missing start scans the whole retention, a tiny step returns a
bucket per millisecond, query has no default limit, and
ignore_global_time_filter lifts the window altogether. Each of those is
bounded or refused here, and a tail resume point becomes a capped
start_offset.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Fluent Bit writes a namespace's logs to the store its monitoring label
names, falling back to the root tenant, so exactly one store answers for
a namespace. Reading with any other rule would show an empty log view
or the wrong tenant's store, so the resolver repeats the collector's
rule and is tested against the same namespace set as its chart test.
It reads a namespace lister, keeping store lookups off the apiserver.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The proxy holds no credentials of its own: kube-apiserver answers who a
bearer token belongs to and whether that identity may read a scope, the
same way an aggregated API server would ask through delegated auth.
Both answers are cached briefly so a dashboard polling several panels
does not turn into a review per request, while transport errors are
left uncached so an apiserver blip does not lock users out for the TTL.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The proxy derives its single SubjectAccessReview from the request path,
so the path has to be split and validated before anything else runs.
Names are DNS-1123 checked because they end up quoted inside LogsQL and
PromQL scopes, and percent-encoding is refused so a decoded slash
cannot shift the segments authorization is built from.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
ZverGuy