Add Docker integration tests for TCP and Unix forwarding, pty size,
resize and modes, session env, envknobs, x11 rejection, session
teardown, SessionDuration, policy changes and RuleExpires on active
sessions, Shutdown, host keys and utmp, and run every integrationtest
test in the Dockerfile. Extend the vmtest SSH tests to use real
OpenSSH clients across the tailnet for pty, sftp/scp, agent and port
forwarding, AcceptEnv, ControlMaster, large transfers, banners, user
mapping, revocation, check mode (via a new testcontrol handler),
recording, and the tailscale ssh CLI. Fix make sshintegrationtest,
which only ever tested one image and always exited 0, and trigger the
workflow on go.mod changes and a schedule.
Updates #13038
Updates tailscale/corp#50380
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: Ie4fffad7e9298b6b77c3ec3a6ae81fd784d182ce
A Reader is shared by all effective policies that include its source.
A forced read through one of them updated the Reader's cached snapshot
and marked it up to date, but didn't notify the other sessions, so
their policies kept the old settings.
Since a89b32904, "tailscale syspolicy reload" reloads the caller's user
policy rather than the device policy used by LocalBackend. On Windows,
a registry change made without a Group Policy refresh was therefore
reported as configured by the CLI but never applied by the backend.
Updates tailscale/corp#51820
Signed-off-by: Nick Khyl <nickk@tailscale.com>
When an update is requested via LocalAPI, updateWindows runs inside
tailscaled, and its os.Exit(0) after launching the tailscale.exe copy
made the service restart the tailscaled child process immediately.
Exit only when running as tailscale.exe. In tailscaled.exe,
wait for the installer instead, which stops the service as part
of the upgrade. As a side effect, an install that fails before
stopping the service is now reported as a failed update.
Updates tailscale/corp#40430
Change-Id: I8af60fa12740dafceb7c9ea4f50d87c1cad27fa4
Signed-off-by: Nick Khyl <nickk@tailscale.com>
Implement the new peerapi hook and add the tokens authenticating that
the destination IP addresses were seen by this connector when forwarding
a DNS query for the given app.
Updates tailscale/corp#46169
Signed-off-by: Fran Bull <fran@tailscale.com>
To allow logic to be shared between client and connector, to allow the
connector to parse a DNS response to add its tokens that authenticate it
returned a given address for a given app.
Updates tailscale/corp#46169
Signed-off-by: Fran Bull <fran@tailscale.com>
Setting a deadline on a Pipe starts a goroutine that wakes blocked
readers or writers when it passes, and only a later SetDeadline or Close
cancels it. But Close only cancels the timers it finds, so setting a
deadline after Close started a goroutine nothing would ever cancel,
which lived until the deadline passed.
An http.Server does exactly that if it's closed between accepting a
conn and serving it: conn.serve sets the TLS handshake deadlines on the
already-closed conn, then fails the handshake. Under synctest, the
leftover goroutines make the test fail with "main bubble goroutine has
exited but blocked goroutines remain", as seen in a lansport test in
tailscale/tb on Windows CI.
A closed Pipe has no blocked readers or writers left to wake, so don't
arm a timer for it. The deadline itself is still recorded.
Updates tailscale/corp#51705
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I772ca698fc81f7389542e87d108eec0c04b98df1
The write replica of an HA ProxyGroup (cert share "rw") doesn't keep TLS
certs in memory, so that it notices a recreated, empty cert Secret and
issues a new cert. As a result, it read the cert Secret from the API
server on every TLS handshake, with a 30s timeout. When the API server
was slow or unavailable, handshakes on that replica waited for the read,
and feature/acme repeated the read under the per-domain lock. Clients
with a 10s handshake timeout, such as kubectl, failed.
This commit adds a small per-domain cache of the last-known-good cert
and key for write replicas. The replica serves a cached cert for 30s
before it reads the Secret again. When it has a cached cert, the read
uses a 3s timeout, and on an error other than not found or forbidden
the replica serves the cached cert and waits another 30s before the
next read. A read that finds the Secret missing, forbidden or empty
removes the cached cert and returns ipn.ErrStateNotExist as before, so
a recreated Ingress still gets a new cert, at most 30s later.
Concurrent reads of the same Secret share one GET, and a read that is
in flight when a cert is written does not replace the written cert.
A Secret watch remains the long-term fix.
Fixes#22009
Change-Id: Id107bf0a7628a1a026f94e621173d7c5d02fb451
Signed-off-by: David Bond <davidsbond93@gmail.com>
* k8s-operator: stop PortRanges.All() at the last port in the range
PortRanges.All() walks a range with a uint16 counter and a port <= end
condition. When end is 65535 the increment wraps back to zero instead
of ending the loop, so the iterator restarts at 0 and yields the whole
port space again, and again.
ClashesWith is the caller that drains All(), so it is where this
shows. With one ProxyClass holding 40000-65535, validating a second
one holding 30000-30100 walks up to 65535, wraps, and reaches 30000 on
the next pass, where Contains returns true. The two ranges are
disjoint, but the second ProxyClass is rejected as clashing and never
goes valid. allocatePorts stops at the first free port so it returns,
but once past 65535 the port it picks is outside the range that was
configured.
endPort is only bounded by the cluster's --service-node-port-range, so
65535 is reachable on a cluster configured up to it.
Yield end and then stop. Also skip a range whose Port is above its
EndPort, which the old condition did for free and the new one would
not.
Fixes#21304
Change-Id: I7d60b133a4a83678bc6aba76c5f9b6b6e9e4e984
Signed-off-by: leoca <leo.camus23@gmail.com>
* k8s-operator: fix copyright header on types_proxyclass_test.go
TestLicenseHeaders wants "Tailscale Inc & contributors"; the new test
file had the older "Tailscale Inc & AUTHORS" wording and failed the
licenses check.
Change-Id: I3217a0795d1c23aba5eb01b3b62ee825efa0f137
Signed-off-by: leoca <leo.camus23@gmail.com>
* k8s-operator: count PortRanges.All() with an int
An int counter cannot wrap at 65535 and an inverted range still
yields nothing, so the explicit guard and the break on end are not
needed.
Change-Id: Ia3de3c0e1336f821a6c99af4e23f221024582e53
Signed-off-by: leoca <leo.camus23@gmail.com>
---------
Signed-off-by: leoca <leo.camus23@gmail.com>
ClientPrepareNewFlowForMagicIP folded resolving a Magic IP and counting the
flow into one call, and the datapath dropped its own IsValid check on the
resolved Transit IP in favor of trusting the new ok return. That left the
validity requirement stated only in the Conn25Datapath doc comment, and an
implementation that violates it panics the packet path two different ways.
A zero Transit IP is neither Is4 nor Is6, so it slips past both of
checksum.UpdateDstAddr's family-mismatch panics and reaches As4, which
panics in turn; a valid Transit IP in the other family hits one of those
family-mismatch panics outright. Check for both, releasing the counted
reference since no flow is installed, so the packet passes through
unmodified as before.
Updates https://github.com/tailscale/corp/issues/44273
Change-Id: Ib0c0eaa92bc4f0470299da08d48c865e1d101849
Signed-off-by: Mazdak Nasab <mazdak.nasab@gmail.com>
newIPTablesRunner returned the error from the IPv6 go-iptables constructor,
which made tailscaled exit at startup. The nft flavor of ip6tables exits 1
from "--version" on kernels without nf_tables even though the binary is
present. Treat that like a missing ip6tables and disable tunneled IPv6.
Fixes#20348
Signed-off-by: Brendan Creane <bcreane@gmail.com>
And associated tests to cover WireGuard handshake messages with the
retry bit set.
Updates tailscale/corp#50068
Signed-off-by: Jordan Whited <jordan@tailscale.com>
In controlknobs.Knobs, each atomic field represents a knob controlled
by the existence of a capability in the node’s CapMap. This patch adds
a doc comment for each field referencing which nodecap.Cap sets it.
This patch also cleans up UpdateFromNodeAttributes to unify the
CapMap lookup with the setting of the appropriate Knobs field.
Previously, it was easier to accidentally mismatch them because they
were split in two places.
Updates #cleanup
Change-Id: I329b0c5732f1c13f2e0d8609c35f42ad645b6bd7
Signed-off-by: Simon Law <sfllaw@tailscale.com>
Covers a new ProxyGroup getting a Parallel StatefulSet, and an existing
OrderedReady one being deleted and then recreated as Parallel.
Updates tailscale/corp#38175
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
With a readiness probe and the default OrderedReady policy, one unready
ProxyGroup Pod stops the rest from starting, so ProxyGroup StatefulSets now
use Parallel. The policy can't change in place, so an existing StatefulSet is
deleted with its Pods orphaned and recreated, adopting them.
Updates tailscale/corp#38175
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Covers the ProxyClass probe defaults and overrides, the env cases that skip or
move the probes, probes on ProxyGroup proxies, and rejecting probes on the
init container or with an env that leaves the endpoints unreachable.
Updates tailscale/corp#38175
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Proxies now serve /livez and /readyz, but nothing points Kubernetes probes at
them. This adds livenessProbe and readinessProbe fields to the ProxyClass
Tailscale container, each off unless set, with tunable timing (older proxy
images don't serve the endpoints, so a probe would fail against them), and
wires them into ProxyGroup proxies. A ProxyGroup is rejected if its ProxyClass
sets probes but its env would leave the endpoints unreachable, such as
TS_ENABLE_HEALTH_CHECK=false, TS_LOCAL_ADDR_PORT on loopback, or the
deprecated TS_HEALTHCHECK_ADDR_PORT. Probes follow a custom TS_LOCAL_ADDR_PORT
port, and liveness gets a 5s timeout by default, as a restart drops traffic
that a slow tailscaled still carries.
Updates tailscale/corp#38175
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Covers the Probe handlers and conditions, AllCertsPresent with present,
missing, expired and unreadable certs, and the containerboot checks for
tailscaled failing or timing out, certs for advertised HTTPS-only Services and
a serve config file that doesn't exist.
Updates tailscale/corp#38175
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
Proxies only serve /healthz, which reports whether the node has a tailnet IP,
so it can't back a liveness or readiness probe. When TS_ENABLE_HEALTH_CHECK is
set, containerboot now also serves /livez, which fails only when tailscaled
stops answering LocalAPI, and /readyz, which passes once the node has tailnet
IPs and, with TS_SERVE_CONFIG set, has applied the serve config and has a cert
for each HTTPS-only Tailscale Service it advertises. Certs are checked through
the read-only cert-status endpoint so the probe never triggers issuance. An
expired or unreadable cert counts as present, as failing readiness on it would
stop a StatefulSet rollout before it reaches the replica that renews certs.
Updates tailscale/corp#38175
Signed-off-by: chaosinthecrd <tom@tmlabs.co.uk>
This device policy hides entry points to the new windowed UI and uses the legacy tray based menu instead
Updates tailscale/corp#50029
Signed-off-by: kari-ts <kari@tailscale.com>
Add an optional auth-exchange-attempts flag to up that allows for
controlling the number of attempts made on transient errors (e.g, 5XX or
client-side deadline exceeded errors) during OAuth or workload identity
federation auth exchange operations.
This defaults to 1 (e.g., no retries). Specifying a larger number of
attempts enables retrying the auth exchange with exponential backoff
instead of the default of failing after a single try.
Updates https://github.com/tailscale/tailscale/issues/20339
Signed-off-by: Mario Minardi <mario@tailscale.com>
The current command fails with:
invalid value "15" for flag -timeout: parse error
because the timeout flag wants a time.Duration value.
Updates #cleanup
Change-Id: I645974078ce60c42b9d5d7bd227226de5fff26ad
Signed-off-by: Francois Marier <francois@tailscale.com>
In 2c7aeadb1d we cleaned up some extra
logs but did not remove all logs of disco keys moving from zero to non
zero keys. Remove more log lines that we do not need to see.
Updates #cleanup
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
The hook was given (node ID, node key) pairs, but its result is keyed
by node ID, and its only implementation, conn25, now tracks connector
transit IPs by node ID too. Drop the node key from the sequence.
Updates tailscale/corp#51237
Signed-off-by: Michael Ben-Ami <mzb@tailscale.com>
The client tracked which transit IPs belong to which connector in a
map keyed by the connector's node key. The ExtraWireGuardAllowedIPs
hook's results and the route manager are keyed by node ID, though, and
a node key can rotate while the node ID stays the same, which would
orphan a connector's transit IPs under the stale key.
Key the map by node ID instead, and rename byConnKey and
lookupTransitIPsByConnKey to match. addTransitIPForConnector now
rejects connectors with a zero node ID rather than a zero node key,
since the ID is what the mapping is keyed on.
Updates tailscale/corp#51237
Signed-off-by: Michael Ben-Ami <mzb@tailscale.com>
The resolv.conf watcher needs one watch on one directory and a
blocking read. gonotify v3.0.2 has a bug where it loops forever
without advancing when a record's name length runs past the end of a
read. It also never closes its fd on cancel and allocates 32 KB per
Read. Use x/sys/unix directly, return an error on a malformed record,
and stop when the kernel removes the watch.
Fixes#16576
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: If2bc5e94f8aab2922ee9e07b3c2b75a76f4f9236
The web client and tsconnect now use tool/pnpm, so nothing uses yarn.
Remove tool/yarn, tool/yarn.rev, the deprecated dist.Build.Yarn field
and yarn from the flake dev shell.
Updates tailscale/corp#51439
Change-Id: Ic27506cc77c1ada96c40a2978903204418d82d3d
Signed-off-by: Jim Wright <jimw@tailscale.com>
When prefs gets logged, we weren't recording whether the node had enabled
posture checking. Not enabling this is a common cause of device posture
failing, so include it here.
Updates #cleanup
Change-Id: I9e16c9b56ac70f51598172aa717e7e2a33c4fa7c
Signed-off-by: Alex Chan <alexc@tailscale.com>
Log which data keys the operator Secret contains when the expected state is
missing.
Discard logs for tsnet.Servers created as part of test setup and during
tests to reduce log noise.
Updates tailscale/corp#34562
Change-Id: I37d730a46304004fb1579350a755bc92cdf46e42
Signed-off-by: Becky Pauley <becky@tailscale.com>
An egress Service without the tailscale.com/hostname annotation gets the
default proxy hostname <namespace>-<name>. When that hostname exceeds 63
bytes, the operator rejects the Service. This includes ProxyGroup egress
Services, which do not use the hostname.
For egress Services, shorten a default that is too long to a 54-char
prefix and an 8-char hash. Defaults that fit do not change, so no
existing proxy is renamed. Explicit annotations and ingress hostnames
keep the current validation.
Rename TruncateLabelValue to TruncateWithHash, as it now also truncates
hostnames.
Fixes#22080
Change-Id: Ib1589a5e9c750cd37d0641829a84fabd0c4352a7
Signed-off-by: David Bond <davidsbond93@gmail.com>
This commit widens the egress match port range from [10000, 11000) to
[10000, 30000). The old limit of 1000 ports was arbitrary, and users
with many egress Services reach it. The new range ends below the Linux
ephemeral port range, so match ports do not collide with ports that the
proxy itself uses.
The port count also no longer includes the health check port, which
lowered the real limit by one. A new test covers allocation up to the
limit.
Fixes#22195
Change-Id: Ie20feb2744b40dce052d3732f9f486b1bff31b32
Signed-off-by: David Bond <davidsbond93@gmail.com>
gonotify.NewDirWatcher registers IN_ALL_EVENTS on the directory and
every subdirectory below it, and applies the caller's mask only in
userspace. Watching /etc that way wakes tailscaled for every open or
read of any file under /etc, including /etc/ld.so.cache on each exec.
Under a busy exec or open loop tailscaled used 150-230% CPU.
Use a plain inotify watch on the directory itself with the events we
care about, so the kernel drops everything else. Call cb once at
startup, which NewDirWatcher's synthetic events for existing files
used to do.
Rework TestWatchFile to wait for the startup callback first, so it
still checks that a write triggers one, and add a test that reads and
writes to other files don't.
Fixes#21743
Updates #16576
Change-Id: Idf1b13a6dffde4191ce8744cbb0e24a74b7a5b82
Signed-off-by: Yerassyl Urazalin <erasil.urazalin@gmail.com>
The os case failed before the previous commit: a link with no os
parameter and an HMAC over an empty OS name validated successfully.
Change-Id: I087c66749475ac4c60130d700f4dc74dde60987f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
A Magic IP gets its address assignment when the DNS response is rewritten,
but the Transit IP it maps to only becomes usable later: handleAddressAssignment
registers it with the connector over PeerAPI, and only then does
addTransitIPForConnector put it in byConnKey, where extraWireGuardAllowedIPs
picks it up and hands it to WireGuard as an allowed IP for that peer.
Until then, transitIPForMagicIP would happily return the Transit IP, the
datapath would DNAT to it, and WireGuard would drop the packet because no
peer claims the address. The client application saw a black hole and hung.
That window stays open indefinitely when the registration never lands: a
failed PeerAPI request, a full assignment queue, or a reset that clears
byConnKey.
transitIPForMagicIP now checks that the Transit IP is registered under its
connector's node key, and returns the new ErrTransitIPNotRoutable if it
isn't. The datapath treats that like an unknown Magic IP and injects an
ICMP host unreachable, so the application fails fast instead of hanging.
Fixes https://github.com/tailscale/corp/issues/50405
Change-Id: Id232e27063f27278e65279d9b1adad5d67d3b6ef
Signed-off-by: Mazdak Nasab <mazdak.nasab@gmail.com>
Import yarn.lock into pnpm with the same supply-chain settings as
cmd/tsconnect, plus overrides replacing yarn resolutions, two narrow
trustPolicyExclude entries for packages published without provenance,
and denied esbuild/@swc/core install scripts.
Switch assets.go, cmd/build-webclient, release/dist and CI to
tool/pnpm. release/dist now looks up pnpm only when building web
client assets; Build.Yarn is deprecated.
Updates tailscale/corp#51439
Change-Id: Ib472fc8439e916bfe41158e3952f9fa8c02f522d
Signed-off-by: Jim Wright <jimw@tailscale.com>
inc_counter looked up the per-CPU counter, incremented it in place,
and then called bpf_map_update_elem to store the same value back. For
a BPF_MAP_TYPE_PERCPU_HASH the pointer returned by the lookup already
is this CPU's slot, so the write-back was redundant, and it is the one
step that takes the hash bucket spinlock. The program runs for every
received packet, not just STUN, and updates two counters each time, so
a busy DERP paid two locked map updates per incoming TCP segment.
Keep the update only for the miss path, which runs once per key.
While here, make inc_counter and handle_counters return void. Neither
caller looked at the result.
TestXDP cleared the counters map before every case, so it only ever
exercised the key insertion path. Add a subtest that runs the same
packet repeatedly without clearing, to cover the in-place increment
of an existing key.
Regenerate the bpf2go outputs with clang 18.1.8 and the bpf2go from
the cilium/ebpf v0.22.0 already in go.mod, which is also why the
generated Go files pick up structs.HostLayout and the object name
constants.
Fixes#22463
Signed-off-by: Mike O'Driscoll <mikeo@tailscale.com>
Co-authored-by: Brad Fitzpatrick <bradfitz@tailscale.com>