The relay server force-sets its UDP socket buffer sizes. The kernel
requires the NET_ADMIN capability for this. Without it, tailscaled logs
a warning and uses the kernel defaults. Dogfooding showed lower
throughput and more dropped packets in that state.
This commit adds NET_ADMIN to the tailscaled container in peer relay
StatefulSets. A ProxyClass that sets a security context on the
Tailscale container replaces the default, as for other proxy types.
Fixes#21745
Change-Id: Ic84e243d7e6184afa4d0bb0c38c4be8eb899261b
Signed-off-by: David Bond <davidsbond93@gmail.com>
The reconciler forced the AWS load balancer address type to ipv4, so on
an IPv6 EKS cluster, where pod targets are IPv6, no Network Load
Balancer was ever provisioned.
Make the address type a default that spec.service.annotations can
override, defaulting IPv6 source NAT on when dualstack is requested.
Resolve load balancer hostnames for both address families and requeue
ready PeerRelays every five minutes, since DNS records for a new load
balancer arrive over several minutes. Add spec.service.ipFamilyPolicy
and ipFamilies so dual-stack GKE and AKS clusters can request a
dual-stack load balancer too.
Shorten the PeerRelay field descriptions and link to the PeerRelay docs
for the detail.
Fixes#21399
Change-Id: Idf22eb02f8ed9a46ff4abea8d4faef4b8c12c111
Signed-off-by: David Bond <davidsbond93@gmail.com>
* k8s-operator,sessionrecording: fix SPDY protocol DoS risks
There was a risk that a malicious kubectl client could cause a DoS issue for the Tailscale proxy performing a k8s recording by defining large quantities of headers, or using a decompression bomb.
This change adds limits of 1000 header pairs, and a maximum of 1MB for the decompressed header block size. In addition fuzz testing is added and integrated with oss-fuzz to review for other protocol issues.
Credit to Eresus Security for reporting the header quantity DoS risk.
Fixestailscale/corp#45112
Updates tailscale/corp#46608
Change-Id: Ie27b241b9f1cac58198560a88a44ffaa0927b96e
Signed-off-by: Mike Jensen <mikej@tailscale.com>
* k8s-operator/sessionrecording/spdy: test consecutive header blocks share one zlib context
kubectl compresses every stream's header block against one zlib stream for the
life of the connection, so the proxy decodes them all through a single
zlibReader (conn.zlibReqReader). This test compresses two blocks onto one writer
and decodes both from one reader.
It fails on the second block: parseHeaders' ended() reads one byte past the
first block, which on the shared context is the start of the next block, leaving
the reader unable to decode it ("unexpected EOF"). In a real session that means
every stream after the first fails to parse and its output is never recorded.
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* k8s-operator,sessionrecording: parse full 31-bit SPDY stream IDs
dataFrameStreamID masked stream IDs to 7 bits while storeStreamID kept all 32, so output on any stream with ID >= 128 was never matched against a stored stdout/stderr stream and went unrecorded. Keep the 31 spec bits in both places. controlFrameVersion gets the matching 15-bit mask for its version field.
Mask storeStreamID to 31 bits as well: the reserved flag bit in a SYN_STREAM ID field can never appear in a data frame, so an ID stored with the bit set matches nothing.
Change-Id: Icf05a29a2838f2f2492b8d2da9e821f4e2459877
Signed-off-by: Mike Jensen <mikej@tailscale.com>
---------
Signed-off-by: Mike Jensen <mikej@tailscale.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
Previously ProxyClass spec.staticEndpoints was only implemented for
ProxyGroups and silently ignored when the ProxyClass was referenced by
a Connector. Extract the NodePort Service provisioning, port
allocation, and node ExternalIP discovery logic shared with ProxyGroup
into a new k8s-operator/reconciler/staticendpoints package, and wire
it into the StatefulSet reconciler so that each Connector replica gets
a per-pod NodePort Service, the discovered endpoints are written to
its tailscaled config as static endpoints, and tailscaled listens on
the Services' target port via the PORT env var. Also reconcile
Connectors on Node changes, clean up NodePort Services on scale down,
Connector deletion, and when static endpoints are removed, and
surface the endpoints in the Connector's device status.
Add e2e tests for static endpoints on both Connectors and ProxyGroups.
They verify the per-replica NodePort Services, the PORT env var, the
endpoints reported in status and advertised to control, and the
cleanup on scale down and on removal of the static endpoints
configuration. The tests skip on clusters whose Nodes have no
ExternalIP addresses, such as kind.
Fixes#18819
Change-Id: Ia134916062471e6c0088692834257aafabd0e713
Signed-off-by: David Bond <davidsbond93@gmail.com>
An Ingress annotated with
`tailscale.com/experimental-forward-cluster-traffic-via-ingress` forwards
cluster traffic to the proxy's Pod IP, which is DNATed to the node's Tailscale IP
where serve answers it. On Linux the serve listener is bound to the tunnel
interface and drops that traffic, so set TS_SERVE_ALLOW_ALL_INTERFACES on the
proxy when this annotation is used, which makes serve answer it again.
Document on the annotation how the traffic reaches serve and that it bypasses
tailnet ACLs, and regenerate the CRD and operator manifests.
Updates tailscale/corp#48248
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This is the output of the new misc/bumpdeps tool (#21325) run with
--exclude-newer-than-days=7, which asks proxy.golang.org for the newest
version of every direct dependency, ignoring releases younger than a
week in favor of the newest older one, and runs a single go get.
gvisor tracks its "go" branch, wireguard-go its "tailscale" branch,
and golang-x-crypto its "main" branch (the proxy's @latest for it is
a stray v0.91.0 tag from 2024 that predates our acme fork changes).
Indirect deps only moved as far as MVS pulled them.
The week-long cooldown held back gvisor, the gokrazy modules,
chromedp/cdproto, and hashicorp/raft-boltdb/v2, whose only newer
versions are days old; they'll come along next time.
Several upstream changes needed small fixes: nfpm's PrepareForPackager
takes a modification time now (a zero time keeps the old behavior of
using the source file's mtime), esbuild's ServeOptions.Port became an
int while ServeResult.Host became a Hosts slice, client-go's
EventRecorder.Eventf is now recognized by vet as a printf wrapper (so
the k8s-operator calls that passed a preformatted message switch to
Event), google/nftables v0.3.0 reads back the kernel's
NF_NAT_RANGE_PROTO_SPECIFIED flag into a new expr.NAT.Specified field
(so the port map DNAT rule now sets it too or findRule never matches
the rule it just added), and staticcheck v0.8.1 knows encoding/json/v2's
embed tag option, so the two SA5008 suppressions for it are gone.
Two tests assumed old library behavior. client-go's fake clientset now
replays existing objects when a watch starts, as a real apiserver does,
so the k8s-proxy config test must tolerate the loader ignoring that
no-op event before the real reload arrives. fyne.io/systray moved its
dbusmenu object path and answers the first GetLayout with depth 1, so
the systray test now finds the menu via the item's Menu property and
polls until the submenu entries appear.
Then make tidy, make updatedeps, and make kube-generate-all (the
controller-gen bump to v0.22.0 changes doc strings, stops listing
top-level metadata as required, and crd-ref-docs now marks optional
fields).
Updates #8043
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I3f9a2c6e8b1d4705a9e2c7b8d1f4e6a0c2b5d8e3
The PeerRelay resource hardcoded UDP port 41641 in the generated
Service, the tailscaled config, status.endpoints and the advertised
static endpoints. That breaks deployments where the load balancer
address or port is not what peers can actually reach, e.g. behind NAT.
Add spec.service.port to choose the UDP port, and spec.staticEndpoints
to advertise extra address:port pairs alongside the discovered load
balancer endpoints. A replica whose only endpoint is a static one
counts as addressed, and a static entry's port wins when it names an
address the load balancer already provides.
The e2e tests now supply static endpoints, which lets them assert the
PeerRelayReady condition goes True in kind, where no cloud controller
ever gives the LoadBalancer Services an address.
Fixes#20821
Change-Id: I463012cd447c81c2849fa653fa18eb82662aaf4f
Signed-off-by: David Bond <davidsbond93@gmail.com>
Guard optional annotations maps in generated CEL expressions before
looking up expose or proxy-group annotations. This lets ordinary
annotationless Services and Ingresses pass admission while preserving
deny-all and allowlist behavior for present proxy-group annotations.
RELNOTE: Kubernetes ProxyGroupPolicy now permits annotationless resources.
Fixes#20906
Change-Id: I8b9475d34c003ca6d233246c021ac656e0530fb1
Signed-off-by: joshrzemien <joshua.k.rzemien@gmail.com>
This commit changes how PeerRelay services are exposed on AWS. A Network
Load Balancer only forwards to targets in an availability zone enabled
on it, and spec.aws.elasticIPs pins each service to a single subnet,
which enables just one zone. A replica scheduled anywhere else silently
receives nothing while still reporting PeerRelayReady with an address in
status.endpoints.
Without spec.aws we now leave the subnet unpinned, so the AWS Load
Balancer Controller spreads the load balancer over every zone it finds,
and cross-zone load balancing is on by default so any of its addresses
reach the pod. Hostname resolution is no longer gated on the
eip-allocations annotation, which had left these unpinned services in
EndpointsPending forever, and a failure to resolve now logs at debug
since it is expected while a load balancer provisions.
Such a load balancer has an address per zone, and AWS bills for each, so
every one of them is now advertised rather than only the lowest sorted.
That also lets a peer reach the relay when one zone is unreachable.
status.endpoints gains address as a second list map key so a replica can
hold an entry per address; no field changes, so existing readers of
endpoints[].address keep working. Readiness counts the replicas that
have an endpoint rather than the entries, so a replica with several
addresses cannot mask one that has none.
The pods now serve containerboot's health check endpoint and the load
balancer is pointed at it over HTTP. A peer relay listens only on UDP,
so the default TCP check against the port the load balancer forwards
could never succeed and every target sat unhealthy while relaying
perfectly well. /healthz reports 200 once the device has tailnet
addresses, which is the condition that actually matters.
The CRD docs now describe spec.aws as the exception, note that it also
needs a ProxyClass pinning pods to the zone of the subnets it names, and
drop the claim that an Elastic IP has an availability zone of its own.
Fixes: https://github.com/tailscale/tailscale/issues/20833
Signed-off-by: David Bond <davidsbond93@gmail.com>
authkey.go references tsclient.Client in getAuthKey but never imported
the package, breaking main.
Updates #20544
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
ProxyGroup, Recorder, and PeerRelay each carried a near-identical copy of
the auth key re-issuance state machine (in-flight tracking, per-parent rate
limiting, stale-device cleanup). A bug fix had to land in three places and
could silently drift.
Extract it into a shared tailscaled.Reissuer, alongside the other tailscaled
workload helpers (NewAuthKey, AuthKeyFromConfigSecret, DeviceIDFromStateSecret)
that the callers already use. It owns its own mutex, tracks in-flight reissues
per replica keyed by parent, and rate-limits re-issuance per parent; the three
reconcilers drive it via EnsureState/RemoveState/ShouldReissue. The device
deletion helper is shared too, so the reissue state machine and its tests now
live in one place.
Updates #20544
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Fix the small number of existing violations of this check, and enable it for
future runs. The fixes needed were:
- Clean up a few misspelled package names (probably renames).
- Clean up a few lexical nits ("Package x" instead of "The x package").
- Add lint directives to some files affected by build tag variance.
- Add a missing package comment and re-generate the k8s docs.
The lint overrides are a little ugly, but there are only a few places where we
need them, and it's probably worthwhile to enable the check on the rest of the
repo. Rather than replicate the docs around the build tag, I made the lint
diagnotics reference the "correct" file.
Updates #cleanup
Change-Id: I0d97f2f468542af456a0396cf9a023f04f23e436
Signed-off-by: M. J. Fromberger <fromberger@tailscale.com>
Package tailcfg defines the types and constants used by the Tailscale
protocol, but since everything is all in one package, it’s difficult
to sift through the docs: https://pkg.go.dev/tailscale.com/tailcfg
We define and enumerate capabilities as string constants for
tailcfg.NodeCapability and tailcfg.PeerCapability. This PR extracts
them into their own packages:
- tailcfg.CapabilityFileSharing becomes nodecap.FileSharing
- tailcfg.NodeAttrOnlyTCP443 becomes nodecap.OnlyTCP443
- tailcfg.PeerCapabilityTaildrive becomes peercap.Taildrive
We originally intended for CapabilityFoo to grant an entitlement or
permission for Foo, and for NodeAttrBar to configure Bar in the
nodeAttrs section of the policy file. However, there was no technical
enforcement of this convention, so new capabilities have used the
NodeAttr prefix regardless of meaning. Therefore, this PR unifies
tailcfg.CapabilityFoo and tailcfg.NodeAttrBar into a single package as
nodecap.Foo and nodecap.Bar.
Ran `go fix -inline ./...` and committed the changes that replaced
uses of the tailcfg aliases with the authoritative ones.
Updates #20259
Change-Id: Ieb7e7e6c8247c39faf42fdf15c68cdc7c621c730
Signed-off-by: Simon Law <sfllaw@tailscale.com>
The nameserver Deployment created for a DNSConfig was the only
operator-managed workload with no way to configure imagePullSecrets, so its
pods could not pull the nameserver image from a private registry.
Adds imagePullSecrets to NameserverPod and thread it through to the nameserver
Deployment pod spec.
Updates #16772
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit adds a new end-to-end test for the `PeerRelay` custom
resource.
This test is currently quite limited due to the fact that we are running
our tests within a kind cluster within github actions. This means it's
not really possible to give the peer relays a proper public IP address
via the `LoadBalancer` type services that we spin up.
That being said, we intend to expand our e2e test suite with actual
real clusters in future so this can be expanded upon at a later date.
For now, this test spins up a single and multi-replica deployment of
a peer relay and confirms that it has been registered with control
and is configured to act as a peer relay.
This test also caught a small bug where the server URL was not being
passed into the peer relay's configuration, which has been fixed here.
Closes: https://github.com/tailscale/corp/issues/45731
Closes: https://github.com/tailscale/corp/issues/45737
Signed-off-by: David Bond <davidsbond93@gmail.com>
PeerRelay replicas got a single-use auth key minted on config Secret
creation and never again, so a replica that lost its login (device
deleted, state Secret lost, key expiry) could never re-auth despite
containerboot writing the reissue_authkey signal.
Port the shouldReissueAuthKey pattern already used by ProxyGroup and
Recorder: track in-flight reissues per replica, rate-limit re-issuance
per PeerRelay, and delete the stale device before minting a new key.
Updates #20544
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit moves the proxyclass reconciliation logic from its original
home in the rather large main package of the k8s-operator to a dedicated
package within `k8s-operator/reconciler/proxyclass`.
Closes: https://github.com/tailscale/corp/issues/37085
Signed-off-by: David Bond <davidsbond93@gmail.com>
The session recording WebSocket parser rejected frames whose extended
payload length field had not fully arrived yet, tearing down 'kubectl
exec' sessions whenever a read from the network delivered only the
first two or three bytes of a frame header. Every data frame carrying
126 or more bytes of payload uses an extended length field, so bulk
stdin (such as piping a file into a pod through the API server proxy)
failed with near-certainty on large transfers, while small interactive
sessions were rarely affected.
Teach fragmentDimensions to report an incomplete frame header
distinctly, so that both the data and control frame paths buffer and
wait for more bytes instead of erroring out and killing the session.
Add tests covering both extended length forms and control frames being
split at every boundary within the frame header, into two and three
segments, on both the read (client to server) and write (server to
client) paths, plus a bulk stdin passthrough test approximating how
'kubectl exec' frames and chunks a large stdin stream.
Updates tailscale/corp#39583
Change-Id: I9c28a17bc6bee898abfa2af2519c99fff9144217
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
This commit contains the Kubernetes implementation of peer relays via the new `PeerRelay` CRD. It's a mega branch consisting of the commits of other PRs gone into this work:
1. https://github.com/tailscale/tailscale/pull/20211
2. https://github.com/tailscale/tailscale/pull/20329
3. https://github.com/tailscale/tailscale/pull/20423
4. https://github.com/tailscale/tailscale/pull/20503
An instance of the `PeerRelay` CRD deploys a `StatefulSet` of containerboot instances configured to advertise themselves as peer relays using the IP addresses configured via `LoadBalancer` services on each cloud provider (with some AWS specifics as it's less automatic than its competing cloud providers).
Per replica, a `LoadBalancer` type `Service` resource is provisioned and its IP address is used to configure the respective relay.
This has been tested with success in AWS, GCP & Azure and provides additional modification to `Service` resources via the CRD for any other kinds of deployment environments. It also contains some work that may appear to be duplication of what already exists within `cmd/k8s-operator` so we can start building an appropriate migration path for `Connector`, `ProxyGroup` etc into respective `k8s-operator/reconciler/*` packages.
Closes https://github.com/tailscale/corp/issues/34524
Add support for configuring egress to destinations reachable via 4via6
subnet routes, using either the synthesized 4via6 address or the MagicDNS
name (in the form <IPv4-with-hyphens>-via-<siteID>[.*]).
Also update the Connector to validate and advertise 4via6 subnet routes.
Export net/netutil.ValidateViaPrefix so it can be reused by the Connector
validation logic.
This change only affects standalone egress proxies — ProxyGroup egress
requires IPv6 support before it can use 4via6.
Updates #19334
Change-Id: I6faecd6eb61ab55fc0cd97fe417af6b6a12fe7fc
Signed-off-by: Becky Pauley <becky@tailscale.com>
This commit modifies the reconciler for the `Tailnet` custom resource
to allow referenced secrets to specify an `audience` field. If a
referenced secret contains both an `audience` and `client_id` we assume
the user's intention is to use workload identity.
In that case, we configure the tailscale API client to authenticate
using the Kubernetes token request API against the operator's service
account. This requires the operator to be aware of its own service
account name.
A small change has also been made to the messages added to the `Tailnet`
CRD's status field in the even that it is missing scopes to make it
clearer that certain scopes may not be applied.
Closes: #19090
Updates: #19471
Signed-off-by: David Bond <davidsbond93@gmail.com>
Several packages built their HTTP transports with
http.DefaultTransport.(*http.Transport).Clone()
The standard library only documents http.DefaultTransport as an
http.RoundTripper, so an application is free to replace it with a
RoundTripper that is not a *http.Transport (e.g. an instrumented or
tracing wrapper). When such an application embeds tsnet.Server, the
unchecked type assertion panics as soon as tsnet brings up its control
connection, DNS bootstrap, or log uploader.
Add netutil.NewDefaultTransport, which returns a clone of the global
when it is still the standard *http.Transport (preserving existing
behavior) and otherwise returns a fresh transport mirroring the stdlib
defaults. Route every clone site through it.
Updates #19937
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Achille Roussel <achille.roussel@gmail.com>
* cmd/k8s-operator: truncate long label values in metrics resources
Kubernetes label values have a 63-character limit, but resource names
can be up to 253 characters. When a Service or Ingress with a long
name is exposed via Tailscale, the operator fails to reconcile because
it uses the parent resource name directly as label values on metrics
Services.
Truncate label values that may exceed the limit by keeping the first
54 characters and appending a SHA256-based hash suffix to preserve
uniqueness.
Fixes#18894
Signed-off-by: Daniel Pañeda <daniel.paneda@clickhouse.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* cmd/k8s-operator: move TruncateLabelValue to shared k8s-operator package
Move the label truncation helper to k8s-operator/utils.go so it can be
reused by other components that need to produce valid Kubernetes labels.
Signed-off-by: Daniel Pañeda <daniel.paneda@clickhouse.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* cmd/k8s-operator: truncate long domain label values in cert resources
Applies TruncateLabelValue to certResourceLabels in order to prevent API
server validation failures. This covers both the HA Ingress and kube-apiserver
proxy reconcilers, as both flow through certResourceLabels.
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* cmd/k8s-operator: remove empty metrics_resources_test.go, use hyphens in test names to satisfy go vet
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
---------
Signed-off-by: Daniel Pañeda <daniel.paneda@clickhouse.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit modifies the kubernetes operator to use the `tailscale-client-go-v2`
package instead of the internal tailscale client it was previously using. This
now gives us the ability to expand out custom resources and features as they
become available via the API module.
The tailnet reconciler has also been modified to manage clients as tailnets
are created and removed, providing each subsequent reconciler with a single
`ClientProvider` that obtains a tailscale client for the respective tailnet
by name, or the operator's default when presented with a blank string.
Fixes: https://github.com/tailscale/corp/issues/38418
Signed-off-by: David Bond <davidsbond93@gmail.com>
Consolidate the duplicated WebSocket frame-parsing logic from Read
and Write into a shared processFrames loop, fixing several bugs in
the process:
- Mixed control and data frames in a single Read/Write call buffer
were not handled: a control frame would cause merged data frames
to be skipped.
- Multiple data frames into one Write call weren't being correctly
parsed: only the first frame was processed, ignoring the rest in
the buffer.
- msg.isFinalized was being set before confirming the fragment was
complete, so an incomplete msg fragment, could've been sometimes
marked as finalized.
- Continuation frames without any payload were being treated as if
they didn't have stream ID, even thought the id is already known
from the initial fragment.
Fixestailscale/corp#39583
Signed-off-by: Fernando Serboncini <fserb@tailscale.com>
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
Add a new vet analyzer that checks t.Run subtest names don't contain
characters requiring quoting when re-running via "go test -run". This
enforces the style guide rule: don't use spaces or punctuation in
subtest names.
The analyzer flags:
- Direct t.Run calls with string literal names containing spaces,
regex metacharacters, quotes, or other problematic characters
- Table-driven t.Run(tt.name, ...) calls where tt ranges over a
slice/map literal with bad name field values
Also fix all 978 existing violations across 81 test files, replacing
spaces with hyphens and shortening long sentence-like names to concise
hyphenated forms.
Updates #19242
Change-Id: Ib0ad96a111bd8e764582d1d4902fe2599454ab65
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
I omitted a lot of the min/max modernizers because they didn't
result in more clear code.
Some of it's older "for x := range 123".
Also: errors.AsType, any, fmt.Appendf, etc.
Updates #18682
Change-Id: I83a451577f33877f962766a5b65ce86f7696471c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
considerable latency was seen when using k8s-proxy with ProxyGroup
in the kubernetes operator. Switching to L4 TCPForward solves this.
Fixes tailscale#18171
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Co-authored-by: chaosinthecrd <tom@tmlabs.co.uk>
Remove the TS_EXPERIMENTAL_KUBE_API_EVENTS env var from the operator and its
helm chart. This has already been marked as deprecated, and has been
scheduled to be removed in release 1.96.
Add a check in helm chart to fail if the removed variable is set to true,
prompting users to move to ACLs instead.
Fixes: #18875
Signed-off-by: Becky Pauley <becky@tailscale.com>
This commit implements a reconciler for the new `ProxyGroupPolicy`
custom resource. When created, all `ProxyGroupPolicy` resources
within the same namespace are merged into two `ValidatingAdmissionPolicy`
resources, one for egress and one for ingress.
These policies use CEL expressions to limit the usage of the
"tailscale.com/proxy-group" annotation on `Service` and `Ingress`
resources on create & update.
Included here is also a new e2e test that ensures that resources that
violate the policy return an error on creation, and that once the
policy is changed to allow them they can be created.
Closes: https://github.com/tailscale/corp/issues/36830
Signed-off-by: David Bond <davidsbond93@gmail.com>
This commit adds a new custom resource definition to the kubernetes
operator named `ProxyGroupPolicy`. This resource is namespace scoped
and is used as an allow list for which `ProxyGroup` resources can be
used within its namespace.
The `spec` contains two fields, `ingress` and `egress`. These should
contain the names of `ProxyGroup` resources to denote which can be
used as values in the `tailscale.com/proxy-group` annotation within
`Service` and `Ingress` resources.
The intention is for these policies to be merged within a namespace and
produce a `ValidatingAdmissionPolicy` and `ValidatingAdmissionPolicyBinding`
for both ingress and egress that prevents users from using names of
`ProxyGroup` resources in those annotations.
Closes: https://github.com/tailscale/corp/issues/36829
Signed-off-by: David Bond <davidsbond93@gmail.com>
This file was never truly necessary and has never actually been used in
the history of Tailscale's open source releases.
A Brief History of AUTHORS files
---
The AUTHORS file was a pattern developed at Google, originally for
Chromium, then adopted by Go and a bunch of other projects. The problem
was that Chromium originally had a copyright line only recognizing
Google as the copyright holder. Because Google (and most open source
projects) do not require copyright assignemnt for contributions, each
contributor maintains their copyright. Some large corporate contributors
then tried to add their own name to the copyright line in the LICENSE
file or in file headers. This quickly becomes unwieldy, and puts a
tremendous burden on anyone building on top of Chromium, since the
license requires that they keep all copyright lines intact.
The compromise was to create an AUTHORS file that would list all of the
copyright holders. The LICENSE file and source file headers would then
include that list by reference, listing the copyright holder as "The
Chromium Authors".
This also become cumbersome to simply keep the file up to date with a
high rate of new contributors. Plus it's not always obvious who the
copyright holder is. Sometimes it is the individual making the
contribution, but many times it may be their employer. There is no way
for the proejct maintainer to know.
Eventually, Google changed their policy to no longer recommend trying to
keep the AUTHORS file up to date proactively, and instead to only add to
it when requested: https://opensource.google/docs/releasing/authors.
They are also clear that:
> Adding contributors to the AUTHORS file is entirely within the
> project's discretion and has no implications for copyright ownership.
It was primarily added to appease a small number of large contributors
that insisted that they be recognized as copyright holders (which was
entirely their right to do). But it's not truly necessary, and not even
the most accurate way of identifying contributors and/or copyright
holders.
In practice, we've never added anyone to our AUTHORS file. It only lists
Tailscale, so it's not really serving any purpose. It also causes
confusion because Tailscalars put the "Tailscale Inc & AUTHORS" header
in other open source repos which don't actually have an AUTHORS file, so
it's ambiguous what that means.
Instead, we just acknowledge that the contributors to Tailscale (whoever
they are) are copyright holders for their individual contributions. We
also have the benefit of using the DCO (developercertificate.org) which
provides some additional certification of their right to make the
contribution.
The source file changes were purely mechanical with:
git ls-files | xargs sed -i -e 's/\(Tailscale Inc &\) AUTHORS/\1 contributors/g'
Updates #cleanup
Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d
Signed-off-by: Will Norris <will@tailscale.com>
This commit contains the implementation of multi-tailnet support within the Kubernetes Operator
Each of our custom resources now expose the `spec.tailnet` field. This field is a string that must match the name of an existing `Tailnet` resource. A `Tailnet` resource looks like this:
```yaml
apiVersion: tailscale.com/v1alpha1
kind: Tailnet
metadata:
name: example # This is the name that must be referenced by other resources
spec:
credentials:
secretName: example-oauth
```
Each `Tailnet` references a `Secret` resource that contains a set of oauth credentials. This secret must be created in the same namespace as the operator:
```yaml
apiVersion: v1
kind: Secret
metadata:
name: example-oauth # This is the name that's referenced by the Tailnet resource.
namespace: tailscale
stringData:
client_id: "client-id"
client_secret: "client-secret"
```
When created, the operator performs a basic check that the oauth client has access to all required scopes. This is done using read actions on devices, keys & services. While this doesn't capture a missing "write" permission, it catches completely missing permissions. Once this check passes, the `Tailnet` moves into a ready state and can be referenced. Attempting to use a `Tailnet` in a non-ready state will stall the deployment of `Connector`s, `ProxyGroup`s and `Recorder`s until the `Tailnet` becomes ready.
The `spec.tailnet` field informs the operator that a `Connector`, `ProxyGroup`, or `Recorder` must be given an auth key generated using the specified oauth client. For backwards compatibility, the set of credentials the operator is configured with are considered the default. That is, where `spec.tailnet` is not set, the resource will be deployed in the same tailnet as the operator.
Updates https://github.com/tailscale/corp/issues/34561
* k8s-operator,kube: removing enableSessionRecordings option. It seems
like it is going to create a confusing user experience and it's going to
be a very niche use case, so we have decided to defer this for now.
Updates tailscale/corp#35796
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
* k8s-operator: adding metric for env var deprecation
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
---------
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This commit adds the `spec.replicas` field to the `Recorder` custom
resource that allows for a highly available deployment of `tsrecorder`
within a kubernetes cluster.
Many changes were required here as the code hard-coded the assumption
of a single replica. This has required a few loops, similar to what we
do for the `Connector` resource to create auth and state secrets. It
was also required to add a check to remove dangling state and auth
secrets should the recorder be scaled down.
Updates: https://github.com/tailscale/tailscale/issues/17965
Signed-off-by: David Bond <davidsbond93@gmail.com>
This commit modifies the k8s-operator's api proxy implementation to only
enable forwarding of api requests to tsrecorder when an environment
variable is set.
This new environment variable is named `TS_EXPERIMENTAL_KUBE_API_EVENTS`.
Updates https://github.com/tailscale/corp/issues/32448
Signed-off-by: David Bond <davidsbond93@gmail.com>
The hijacker on k8s-proxy's reverse proxy is used to stream recordings
to tsrecorder as they pass through the proxy to the kubernetes api
server. The connection to the recorder was using the client's
(e.g., kubectl) context, rather than a dedicated one. This was causing
the recording stream to get cut off in scenarios where the client
cancelled the context before streaming could be completed.
By using a dedicated context, we can continue streaming even if the
client cancels the context (for example if the client request
completes).
Fixes#17404
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
DNS configuration support to ProxyClass, allowing users to customize DNS resolution for Tailscale proxy pods.
Fixes#16886
Signed-off-by: Raj Singh <raj@tailscale.com>
Simon Law