The apiserver defaults SendInitialEvents on for every watch whose
resourceVersion is empty or "0", and leaves AllowWatchBookmarks off. The
aggregated storages read SendInitialEvents alone as a WatchList request,
so a plain `kubectl get -w` on applications, tenant modules, namespaces,
secrets or security groups got a bookmark annotated as the end of
initial events.
The k8s.io/apiserver v0.35 watch handler lets that through. From v0.36
it treats the annotated bookmark as the end of a WatchList and calls the
WatchList completion hook, which it only sets up when SendInitialEvents
and AllowWatchBookmarks are both on, so the handler dereferences nil and
the watch dies. The handler is the one compiled into cozystack-api, so
bumping the library is what breaks it, whatever the cluster version.
The marker now requires both flags, the same condition the watch cache
uses, and the backing watches ask for bookmarks only when the client
did, since every backing bookmark is forwarded. Security groups still
forward SendInitialEvents to their backing watch so it replays existing
objects as ADDED.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <f@lex.la>
The aggregated apiserver resources in group core.cozystack.io
(TenantSecret, TenantModule, TenantNamespace) ignored the
SendInitialEvents field of ListOptions. Clients using the WatchList /
streaming-list protocol (client-go's WatchListClient feature, on by
default since v1.35) never received the required
k8s.io/initial-events-end bookmark, so their reflectors stayed stuck in
the initial-stream phase and logged "hasn't received required bookmark
event marking the end of initial events stream" every ~10s, never
reaching HasSynced.
apps.cozystack.io/Application already implemented this contract inline.
Extract that logic into a shared registry.InitialEventsBookmarker helper
(pkg/registry/utils.go) and wire it into all four translating Watch
implementations: after the initial ADDED events, the first backing
bookmark is annotated as the initial-events-end marker, with fallbacks
that emit it before the first live event and on watcher close. Application
is refactored onto the shared helper with its WorkloadMonitor
pending-event draining preserved.
Adds unit tests for the bookmarker and watch tests for each of the three
core resources asserting ADDED -> annotated Bookmark -> live event.
```release-note
[apiserver] core.cozystack.io resources (TenantSecret, TenantModule,
TenantNamespace) now emit the k8s.io/initial-events-end bookmark for
SendInitialEvents=true (WatchList) watches, so client-go informers reach
HasSynced instead of looping on the missing-bookmark warning.
```
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Simon Koudijs <simonkoudijs@gmail.com>
- Add resourceVersion handling for Watch requests by filtering ADDED events
based on the resourceVersion provided by the client
- Forward bookmark events from underlying HelmRelease watchers to clients
for proper resourceVersion synchronization
- Extract MaxResourceVersion helper using meta.EachListItem for cleaner code
- This ensures clients don't receive duplicate objects they already have
from List+Watch patterns
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Andrei Kvapil <kvapss@gmail.com>
Aleksei Sviridkin