With the lookback refusal, the 24h default turned common application
queries into 422: a day's increase, and month-long panels at the step
a dashboard picks for them. The lookback estimate is a loose upper
bound, made for the store choice, and counts every "(" as an implicit
subquery, so it far exceeds what such queries read.
The lookback is only the refusal threshold, since the lookup itself
spans the window the query reads, so its default now matches
--metrics-max-range. The chart passes it from values, so an operator
can tune it.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
vmselect answers 422 when a label lookup exceeds its own series
limits. The application root turned that into a 502, which reads as a
broken store, while the client could have fixed it by narrowing the
window. The 422 and vmselect's message now reach the client.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The pod and PVC lookup never started earlier than --metrics-app-lookback
before the window. A query that read further back, such as rate(x[7d])
with the 24h default, or whose lookback is unknown (an @ modifier, WITH,
a step-multiple window), answered 200 without the series of pods that
existed only before the lookup start.
Such queries now answer 422 on the application root, matching how a
query whose latest sample cannot be covered is handled; the namespace
root has no lookup and still serves them.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Only a deadline hit before the lookup's response headers was reported
as 504. When the store sent its headers and then stalled, the deadline
expired while the body was decoded, and the decode error came back as
a generic 502, so a slow store looked like a broken one.
The request deadline is now checked whichever step of the lookup it
cut, and both cases are covered by a test.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
vmselect rounds a range query's end up to a whole step before
evaluating it, so the last point can read samples up to one step past
the end the client sent. The pod and PVC lookup stopped at that end
and could miss a pod created within the last step.
The lookup end of a range query now adds one step.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
When the pod or PVC lookup failed, its branch was dropped and the
query still answered 200, without the cAdvisor and volume series of
the application, so a store error showed up as an empty graph. That
is the same silently incomplete answer that a lookup over its cap is
refused for.
A failed lookup now fails the request: 504 when the store missed the
deadline, 502 otherwise.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Reviews was inserted between ResourceAttributes and its doc comment,
so go doc showed the ResourceAttributes text on Reviews, and that text
still called it the single review a route needs, which application
roots no longer are.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The pod and PVC lookup of the application metrics root ended at the
evaluation time, but a negative offset reads samples after it. An
instant query evaluated two hours ago with "offset -1h" reads samples
from one hour ago, so a pod created in between was missing from the
scope and its series were dropped while the request succeeded.
The lookup now ends at the latest instant the query reads, its window
end plus the sum of its negative offsets. A query whose latest sample
cannot be bounded, an @ modifier other than start() or end() or an
offset that is not a plain duration, is refused with 422 on the
application root; the namespace root has no lookup and still serves
it.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The application roots asked only for the application subresource. Their
pod and PVC branches select by namespace and name across the whole
lookup window, so a caller granted only application "a" received the
older series of application "b" whenever "b" had used one of the same
pod or PVC names. The lineage labels the scope rests on can also be set
by whoever creates an unowned pod.
Until the scope carries application identity on every branch, the
application roots also require read access to the namespace root, the
same review the namespace root asks. Every tenant role that reads
applications already holds it, so no current grant loses access; a
grant narrowed by resourceNames alone is refused instead of trusted.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The store cuts a lookup at the cap plus one value. Dropping empty
values first could bring such a cut list back under the cap and serve
it as complete, hiding the application's other pods without the 422
that asks for a shorter range.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
An empty name in a lookup answer would turn the pod branch into a
pattern that matches every series without a pod label, widening the
application scope to the namespace; nothing pinned the guard against
it. Malformed answers must only drop their branch, and the lookup must
start at the query's own lookback, or at the configured limit when that
lookback is unknown.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Graphs of one application need its series without the caller knowing
which pods and volumes it ran on. Exporter series carry the identity
the scrape class stamps, but cAdvisor and kubelet volume series do not,
so the scope ORs three branches: the identity, and the application's
pods and PVCs as kube-state-metrics labelled them over the window. Each
branch also carries the namespace, so the scope is never wider than the
namespace root.
The lookup reaches back by the query's own lookback, bounded by a flag,
so pods replaced inside an expression's range still count. A failed
lookup only drops its branch. A list over its cap answers 422 instead
of being truncated, which would hide series without saying so.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
A viewer of one application gets that application's lines without
writing a namespace query and filtering by hand, and a grant narrowed to
one application by resourceNames is honoured on every logs endpoint,
tail and the pickers included. The namespace filter stays and the
application's identity is ANDed next to it as a second stream filter,
so the result can only be narrower than the namespace root.
The plural is mapped to its kind only after the caller is authorized,
so an anonymous client cannot probe which applications exist, and the
proxy is not ready until the ApplicationDefinitions cache is.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Paths and RBAC name an application by its plural, while the lineage
labels that scope its logs and series carry the kind. The mapping comes
from ApplicationDefinitions, kept in one place so both application roots
scope by the same kind. A plural declared for two kinds is refused
rather than resolved by cache order.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The metrics handler shares the bounded review calls with the logs
handler; a POST to it with an unread body must answer 503 as well
rather than wait on a stalled kube-apiserver.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
staticcheck flags the negated disjunctions; naming the duration
characters and spelling out both word boundaries reads the same way the
rule is stated.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Only the upper half of the refused 1970..2262 range had a test, so
the lower check could be dropped unnoticed. vmselect clamps a time
before 1970 to 0, which would answer for a window other than the one
asked for.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Times went to vmselect as "%d.%03d" unix seconds. vmselect reads a
number without "-" as a unix time whose unit it guesses from its size,
so a time in the first 24.8 days of 1970 arrived as a thousand times
as many seconds: a window the proxy checked as 8.8 days and 30 points
reached vmselect as 2001..2025 and 30000 points, and the label and
series APIs, which have no points check, scanned the whole retention.
Times past 2038 were read as microseconds.
Times now go out as RFC3339 with milliseconds, which vmselect parses
as a date and never as a guessed unit, and times outside 1970..2262,
which vmselect cannot hold as nanoseconds, are refused. A test repeats
vmselect's parsing of the exact strings the proxy sends.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The metrics slow-body and stalled-client tests probed the slot with
requests of their own, which competed for the one slot under test and
made them fail at random, under one shuffle seed reliably. They now
read the limiter's slot count, as the logs tests do; 300 runs of the
test and 20 shuffled runs of the package pass.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The metrics concurrency test only checked that a Retry-After was
present, while the rate limit test pins its value, so a change to the
promised hint passed. It now expects the literal value.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
vmselect reads step and timeout as float seconds multiplied by 1000
and truncated, so a step the proxy checked as 1001ms and sent as
"1.001" arrived as 1000ms. The points limit was then checked against
a coarser step than vmselect used: a limit of 11000 let 11010 points
through, and many steps between 1ms and 100000s were affected.
Durations are now written so that the truncation lands on the
milliseconds the limits were checked with: the shortest form when it
reads back exactly, otherwise with half a millisecond added. Step and
timeout share the formatter, and a sweep test mirrors vmselect's
parsing.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The metrics root has its own admission, so the hint added to the logs
root did not reach it, and a console graph polling into the limit had
no back-off hint.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
vmselect's sample limit counts raw samples read, not the work of a
rollup that revisits each sample across overlapping windows, so for
that work the forwarded timeout is the limit that applies.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The comment claimed the limits bound everything a request makes
vmselect do, but windows and offsets inside the expression pass
through unchecked: they are bounded by the store's retention and by
vmselect's own series and sample limits, not by the proxy. A reader
taking the comment at its word would rely on a cost cap that does not
exist.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The points limit was checked on the parsed bounds, but vmselect gets
both cut to the millisecond. A start cut back by more than the end
widened the forwarded range, so a 1ms step admitted at 11000 points
reached vmselect as 11001.
Times are now cut to the millisecond when they are parsed, so the
range and points checks and the forwarded request see one window.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The metrics root parsed its form before taking a concurrency slot,
under no read deadline, and wrote its answer under no write deadline,
so a client that sent its body slowly or stopped reading held the
handler outside every limit. A store answer that broke off after the
status line still ended as a clean 200.
The metrics handler now follows the logs root: the slot is taken
before the body is read under the request timeout, the answer must be
taken by the request deadline, and a failed copy cuts the connection.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The lookback scan assumed every window and offset applies once and
explicitly. A rollup function or offset over anything but a selector is
an implicit subquery in vmselect, so every "(" now counts as one level;
a WITH template can apply one window several times, so WITH makes the
bound unknown; and an offset written with digit separators was read as
its first digits only, so "_" now ends in an unknown bound instead.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
vmselect moves an instant query evaluated near the current time back by
-search.latencyOffset (30s). With a tiny explicit step the lookback
bound did not cover that shift, so a window ending at the retention
edge could still be sent to the short-term store.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The short-term store was picked from the request window alone, so an
instant increase(x[7d]) or x offset 10d went to the store that keeps
72h and silently returned a wrong number.
A lexical scan of the query now bounds how far back it reads: the sum
of every [...] window and subquery step, positive offsets, and the
step and silence interval vmselect reads before each rollup. An @
modifier, step-multiple units or any duration the scan cannot read
make the bound unknown and pick the long-term store, which receives
the same writes. The scan sees quoted label values too, so a false
hit only moves a query to the long-term store. The query is still
forwarded unchanged.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The logs root answers 429 when a user has too many requests in flight,
so clients back off instead of treating the refusal as a bad query.
The metrics root kept its own 422 for the same limit; it now matches.
The rate limit is shared and already answers 429 with Retry-After,
which the metrics test now expects too.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Rounding a duration up to the millisecond overflows for values within
a millisecond of the largest one, which turned such a step or timeout
negative and let it past the range, points and timeout limits. vmselect
refuses durations above 100 years itself, so the proxy now answers 400
for them before rounding.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The per-user concurrency cap on the metrics root had no test, so
dropping it from the handler went unnoticed. The new test holds one
request in the upstream, expects the same user's second one to be
refused and another user's to go through. Overrides of the metrics
flags were only checked for their defaults and refusals.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
vmselect reads step and timeout in whole milliseconds and drops the
remainder, so a step of 1.9ms passed the points check as 1.9ms and
then ran as 1ms, close to twice the points the limit allows. The
duration is now rounded up before the limits are checked, so they see
the value vmselect will use.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The namespace metrics root shares the logs root's authorization path:
the SubjectAccessReview comes from the path alone, so a POST query,
which Grafana sends by default, is authorized exactly like a GET. Both
roots also share the per-user rate and concurrency limits and the way
an unreachable store (502) is told apart from a slow one (504); the
upstream call moves into one helper so the two stay identical.
Every namespace resolves to the root tenant's vmselect for now,
because the series the graphs need are only scraped there.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
vmselect ORs repeated extra_filters, so a client-supplied value would
widen the tenant scope. The upstream request is therefore built from
scratch: only query, start, end, step, time, match[] and a clamped
timeout are read from the client, and the namespace selector is sent
as the single extra_filters[] value. The query itself is not parsed;
vmselect ANDs that filter into every selector.
The label endpoints always carry a match[] (the namespace selector
when the client sent none), because a vmselect started with
-search.ignoreExtraFiltersAtLabelsAPI applies extra filters there only
when a match[] is present.
Range, points per series and step are bounded with 422 per the cost
limits of the metrics proposal. Times are accepted only in the two
forms the Prometheus API defines, since vmselect guesses the unit of a
numeric timestamp from its digit count. The window start picks the
short-term store when it lies within that store's retention.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The flag now also bounds the body read and the token and access
reviews, and 0 lifts all of those at once; an operator raising it for
slow store queries needs to know that. The store comment named a
design document the repository does not have.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The package and serve comments described a delivery plan rather than
the code: the command named metrics it does not serve, and serve gave a
plan as the reason to refuse unknown roots before authenticating.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Both reviews ran on the request context, which net/http cancels for a
dropped client only once the body is read. A POST whose body was still
unread therefore waited on a stalled kube-apiserver past every request
limit. The reviews now share the request timeout and answer 503 when it
passes.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Polling for a slot that lives only until a 200ms deadline misses it when
the test goroutine is descheduled for longer, as under -race on a busy
runner. The tail and stalled-query tests now read the slot when the
request reaches the store, which the proxy does while holding it; the
slow-body test, which never reaches the store, gets a wider deadline.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Calling allowed twice in one condition reads as a duplicated expression
to staticcheck, which cannot see that each call takes a token.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
When both bounds were out of range the answer named one of them at
random, as the check walked a map, and it printed the instant without
its fraction, so a time one nanosecond past the last one vlselect can
hold was shown as a time inside the range. The end is now checked
first and the instant is printed in full; tests pin the inclusive
bound.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The 1970..2262 check refused times on parsing, including a start that
the window clamps to the maximum range before anything is sent and a
since that only becomes a capped start_offset, so an old start or a
tail resume from before 1970 answered 400 where the documented answer
is to clamp. It had also detached the parseTime doc comment.
The check now applies to the times actually sent, after clamping, and
uses the range vlselect can hold, int64 nanoseconds (1677..2262); 1970
is not a limit of vlselect.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The slow-body and stalled-client tests checked the slot by sending
requests of their own, which competed for the one slot under test: a
probe that held the slot when the request under test arrived got that
request refused, and the test failed. The tests now read the limiter's
slot count directly.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
An empty fields_limit answered 400, but an empty limit or facets cap
fell back to the default, so one malformed parameter got two answers
depending on its name. Every cap now refuses an empty value with 400:
a cap the client sent but left blank is a client bug, and reading it
as the default would hide it.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The proxy sends vlselect RFC3339 times, which it never reads as a
guessed-unit number, but it accepted any RFC3339 year and unix
seconds up to 2286. vlselect keeps times as int64 nanoseconds through
time.UnixNano, which is undefined outside 1678..2262, so such a
window landed anywhere. An absolute time outside 1970..2262 now
answers 400; nothing was logged before 1970.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
A request refused before its body was read (401, 403, 405, or a rate
or concurrency 429) had no read deadline yet, and net/http drains up
to 256 KiB of an unread body before it sends the headers. A client
that declared a body and never sent it got no answer, and held the
handler, for as long as it kept the connection; for a second form POST
arriving while the concurrency slot was taken, that was more than five
request timeouts.
The read deadline is now set when the request arrives and lifted once
the body is read, so tails are not cut. A refusal sent with the body
still unread closes the connection instead of draining it, so it is
answered at once.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The test only checked that a Retry-After was present, while the rate
limit test pins its value, so a change to the promised hint passed.
It now expects the literal value. The note on the empty fields_limit
case now gives the reason it is refused.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Parsing the scheme with a case-insensitive split changed which header
forms reach the token review; a bare scheme, an empty token and a
scheme run into the token must still answer 401.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
The fields_limit cap applied only when the client sent the parameter.
vlselect returns every group when it is absent, as it does for 0, so
leaving it out bypassed the cap. The capped value is now always sent.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>
Only the rate-limit 429 said when to retry. A concurrency or tail 429
came without it, so a client had no hint and could retry at once in a
loop. It now carries a one-second hint; when a slot frees depends on
the user's other requests, which the proxy cannot predict.
Assisted-by: LLM
Signed-off-by: ZverGuy <maximbel2003@gmail.com>