This is a follow-up to #21029 (aa2681ac5f) to make it a bit stricter
and not cache DNS results until they've passed TLS cert validation,
to weed out DNS servers that are lying (like captive portals).
Because this is done via dnscache.TLSDialer we only catch the control
connection, but that's fine. That's all we need to come back alive
if DNS was down because real system DNS is itself over Tailscale.
The DERP connections should come via IPv4/IPv6 fields in the DERPMap.
And the logging connection isn't important; it'll buffer and catch up
later as needed when DNS is back up.
Updates #21028
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I75f176a0222f04ed52c9de1247deeed9b911f92c
* net/dns: export OSConfigurationReadWarnable
A natlab vmtest checks that a node is not reporting this warning. Exporting
the Warnable lets the test take the warning's text from it, instead of
keeping a copy of the wording that could stop matching without failing.
Updates #20825
Updates tailscale/corp#44793
Signed-off-by: Brendan Creane <bcreane@gmail.com>
* tstest/natlab/vmtest: add VM coverage for the openresolv DNS backend
dnsMode() picks one of five Linux DNS backends, and natlab could provision
only systemd-resolved and direct. Add a DNSOpenresolv mode and a VM test for
it, so the backend behind #20825 is covered.
No cloud image ships openresolv and a guest cannot download it, so its two
source files are vendored under testdata and installed with cloud-init's
write_files. openresolv's build is a set of sed substitutions with nothing to
compile, so resolvconf.go does the substitutions in process.
Updates #20825
Updates tailscale/corp#44793
Signed-off-by: Brendan Creane <bcreane@gmail.com>
---------
Signed-off-by: Brendan Creane <bcreane@gmail.com>
isAddressValid rejected all non-masquerade destination addresses
whenever any masquerade address was set for the peer. With a v6-only
masquerade pair, the peer's client still dials the v4 (native, not
masqueraded) peerapi URL, so every fresh peerapi connection got a 403.
The bug was masked by HTTP keep-alive: peerNode is snapshotted per
connection, so connections established before the masquerade was
configured kept validating against the old node view. It surfaced when
a newer tailscale/go toolchain started closing idle netstack
connections, forcing fresh peerapi connections and failing the
TestNATPing v6=true NAT subtests with 403s.
A masquerade address for one family says nothing about the other
family, so require a masquerade address match only for the address
family it applies to, and fall through to the self-addresses check for
the other family.
Fixes#21194
Change-Id: Idd2ed82b81e805b47b6aa6af4b5938984c6fa208
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Route packets to dedicated WireGuard, host, and loopback queues when
gVisor writes them. Drain each queue in a separate goroutine, preparing
the outbound path for batching and multiqueue WireGuard delivery.
Updates tailscale/corp#37878
Signed-off-by: Jordan Whited <jordan@tailscale.com>
TestUDPConcurrent closed its UDP connection as soon as the send loop
finished, and then required that at least one reply had arrived. With
GOMAXPROCS=1 the send loop runs to completion before the proxy's relay
goroutines are scheduled at all, so the close discarded every response.
The test failed on all 20 runs of "go test -race -count=20 -cpu=1".
Wait for the first reply, with a 10 second timeout, before closing the
connection.
This also makes the test catch the race it was written for more often.
With the fix in 0301c7493 reverted, "-cpu=2" reports the race where it
previously reported none.
Fixes#21182
Signed-off-by: Brendan Creane <bcreane@gmail.com>
Serve copied the raw Server.Logf field into each Conn, so the UDP error
paths called a nil func instead of falling back to log.Printf. Conn
already holds the Server, so drop the duplicated field and log through
the server's method.
Fixes#21047
Signed-off-by: Brendan Creane <bcreane@gmail.com>
handleUDPRequest recorded the client's UDP source address in
Conn.udpClientAddr on the goroutine that reads from the client.
handleUDPResponse read the field to address the responses, and it runs
on a separate goroutine per target. Nothing synchronized the two.
Guard the field with syncs.MutexValue. The most recently written
address still wins, which is what the code did before.
TestUDPConcurrent keeps datagrams in flight to four targets at once.
Without the fix it reports the race on every run.
Fixes#21048
Signed-off-by: Brendan Creane <bcreane@gmail.com>
Android has no /var, so DefaultTailscaledSocket fell through to the
cwd-relative "tailscaled.sock": the daemon created its socket wherever
it was started and the CLI looked for one wherever it was started, and
the two only agreed by coincidence (reproduced on Termux in #21161).
Return os.TempDir()/tailscaled.sock on Android - the shared per-app
temp directory, following the same pattern as the existing QNAP /tmp
default - so daemon and CLI agree on an absolute path.
Fixes#21161
Signed-off-by: breken <hi@breken.ai>
As of Linux 6.2 (torvalds/linux@f1f1f25699),
the stat size of the /proc/self/fd directory is the number of open file
descriptors, so CurrentFDs no longer needs to walk the directory on
modern kernels. The walk is O(n) in the number of open file descriptors,
which adds up on servers like derper that hold many connections and
report this metric on every scrape: at 10k open fds, the walk takes
about 2.2ms while an fstat takes a constant 300ns.
We fstat a directory file descriptor held open for the process lifetime
rather than calling stat with the path, because syscall.Stat copies the
path string to a NUL-terminated buffer on every call and the metric
must remain allocation-free. Kernels older than 6.2 report a size of
zero, in which case we fall back to the directory walk.
Updates #2784
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I1742db2ae1938c77f9b7d137b167ab4a2b6b6167
Adds an opt-in, in-memory aggregator of recent connection-rejection
events (TSMP rejects received from peers, outbound TSMP rejects we emit
on ACL-blocked inbound flows, and pendopen timeouts) keyed by
(direction, proto, peer-address, reason). The aggregated data is exposed
over a new debug-rejects LocalAPI endpoint and a GET /debug/rejects c2n
endpoint, intended for future GUI/CLI consumption when diagnosing why a
connection failed.
Architecture:
- net/connreject holds the data types and a per-LocalBackend
Aggregator (LRU-bounded, default 256 entries on desktop / 32 on
mobile, per direction).
- feature/connreject is a self-registering ipnext.Extension that owns
one Aggregator per LocalBackend, installs note callbacks on the
tundev and engine, subscribes to OnSelfChange to flip the runtime
gate, and serves the LocalAPI/c2n endpoints.
- wgengine.Engine and *tstun.Wrapper each gain a SetConnRejectNote
setter; data-plane sites use a single atomic.Pointer load + nil
check, so the cost when no consumer is installed is one MOV.
Gating:
- Compile-time: ts_omit_connreject build tag (standard
feature/buildfeatures + condregister plumbing). Trims ~41 KB.
- Runtime: nodecap.ConnReject node attribute, off by default
at the control plane. May be removed once the feature is enabled
by default.
Updates CapabilityVersion to 146 (clients understand nodecap.ConnReject
and can serve GET /debug/rejects).
Adds Proto/Src/Dst accessors on flowtrack.Tuple (used by pendopen to
construct events without exposing the tuple's internals to the
aggregator).
Updates #1094
Updates #14802
Change-Id: I83e8f24a7e66fa2d158d128bd25fbe851134941b
Signed-off-by: James Tucker <james@tailscale.com>
Add a new modular dnsresolvecache feature that records every successful
DNS resolution from net/dnscache as a JSON file per hostname in
$statedir/dns-cache/, so a later boot with misconfigured DNS can still
find last-known-good IPs for critical hostnames like the control plane.
Files are rewritten only when their contents change, so a file's
modification time records when the answer last changed.
When regular DNS resolution fails, the disk cache is now consulted
before the DERP-based bootstrap DNS in net/dnsfallback. This is the
first step toward removing the DERP-based mechanism: new clientmetrics
(dnscache_disk_fallback_hit, dnscache_disk_fallback_miss,
dnscache_derp_fallback_ok, dnscache_derp_fallback_dial_ok) will tell us
when the DERP path no longer fires in the fleet and can be deleted.
The feature is linked into tailscaled by default (omittable with
ts_omit_dnsresolvecache) and is not included in tsnet.
Updates #21028
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I59228cbf68e3b48dfb1215cdd12bd8166ab14034
Instead of opening all DNS queries with conn25, only permit clients to
access domains of the apps that they have permission to use. Previous
changes ensure that clients report which app they are making a DNS query
for, simplifying the check.
Updates tailscale/corp#40076
Updates tailscale/corp#47585
Change-Id: I40fee220b3f190b2a0db9b3e6cc79f323ef16b73
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
Android denies app UIDs both NETLINK_ROUTE (golang/go#40569, #2293)
and /proc/net, so net.Interfaces always fails and netmon.New errors
out before a standalone binary can do anything. The Android app solves
this from Java via netmon.RegisterInterfaceGetter, but raw binaries
run under Termux or a rooted shell have no Java to lean on. This is
the second half of #21129, following the androiddns feature.
I added a netmon fallback hook, consulted only when no interface
getter was registered and net.Interfaces failed, and a new androidbin
feature (ts_omit_androidbin) that implements it: report a single
synthetic interface whose v4 and v6 addresses come from asking the
kernel to route an outbound UDP socket, which sends no packets and is
permitted in the app sandbox. That's enough for magicsock to discover
local endpoints. The fallback also requires runtime evidence of
Android (GOOS=android, or /dev/__properties__ existing for GOOS=linux
binaries running under an Android kernel), so it's inert on regular
Linux.
The androidbin feature also blank imports androiddns, and fixes a
third gap I found while testing: GOOS=linux binaries on Android have
an empty CA root pool, because Go's unix root loader doesn't know
Android's /system/etc/security/cacerts (the GOOS=android loader
does), so all TLS verification fails. On Android it points
SSL_CERT_DIR there unless the user already set it, as Termux's
ca-certificates package does.
The feature is on by default in tailscaled builds on Linux and
Android via condregister, and deliberately not linked into tsnet by
default; tsnet apps and other programs opt in with a blank import of
tailscale.com/feature/androidbin.
I verified on an Android 13 emulator with SELinux enforcing, running
GOOS=linux static binaries under the app UID (run-as), where
net.Interfaces fails with the exact netlinkrib permission denial from
the issue: netmon.New succeeds with the synthetic interface, and a
tailcat binary importing this feature does DNS via dnsproxyd, fetches
its DERP map over TLS using the Android cert store, completes STUN,
selects a DERP region, and prints its address.
Updates #21129
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: If7dbcbb825ecd24bcf6d9d64b1e334dd00700d51
Previously the DERP handler served each connection for its lifetime
on its hijacked connection's net/http handler goroutine. That
goroutine's conn.serve stack frame kept the dead HTTP/1 server state
reachable for the whole DERP connection: the http.conn and its 4KB
bufio.Writer (hijack hands over c.bufw but conn.serve still references
it, so derpserver returning it to its flush pool never made it
collectable), the 4KB bufio.Reader, the upgrade *http.Request with its
parsed headers, and the request context chain. The goroutine also kept
the stack growth from the TLS handshake and HTTP request parsing.
Instead, hand the connection off to a new goroutine and return from
the handler (ala tailscale/corp@dc09e27aef), letting all the HTTP
upgrade state be collected. Give Accept a smaller 1KB frame reader,
draining and releasing the hijacked reader if it contains buffered
bytes from a fast-start client, and a nil bufio.Writer so writes go
through pooled buffers held only for the duration of a write instead
of a per-connection buffer.
Because the handler now returns at handoff time, cmd/derper's
gauge_derper_tls_active_version decrement can no longer be deferred
to handler return: intercept Hijack in the TLS metrics wrapper and,
for hijacked connections (DERP, its WebSocket flavor, and CONNECT),
decrement the gauge once when the hijacked connection closes,
restoring the gauge's connection-lifetime semantics. Teach
derpserver's TCP RTT stats to unwrap the close-hook conn so they
still find the underlying *net.TCPConn.
Also soften the UntypedHexString deprecation notices in types/key to
warnings: the untyped hex string format is the DERP wire protocol's
key encoding, so these call sites are legitimate and permanent, and
a Deprecated marker just makes them light up in editors and linters.
The cautionary text about the format's risks remains.
Measured with 100,000 idle TLS DERP connections on linux/amd64:
standing memory drops from 55.6KB to 32.6KB per connection (-41%).
Updates #21064
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: If05a0c6ea79134807e9e8872861db216
The PeerRelay resource hardcoded UDP port 41641 in the generated
Service, the tailscaled config, status.endpoints and the advertised
static endpoints. That breaks deployments where the load balancer
address or port is not what peers can actually reach, e.g. behind NAT.
Add spec.service.port to choose the UDP port, and spec.staticEndpoints
to advertise extra address:port pairs alongside the discovered load
balancer endpoints. A replica whose only endpoint is a static one
counts as addressed, and a static entry's port wins when it names an
address the load balancer already provides.
The e2e tests now supply static endpoints, which lets them assert the
PeerRelayReady condition goes True in kind, where no cloud controller
ever gives the LoadBalancer Services an address.
Fixes#20821
Change-Id: I463012cd447c81c2849fa653fa18eb82662aaf4f
Signed-off-by: David Bond <davidsbond93@gmail.com>
Move the container image build logic out of the e2e test setup into
cmd/k8s-operator/e2e/internal/build, with a thin CLI wrapper at
cmd/k8s-operator/e2e/build, so CI can publish images once per commit and
fan out into multiple test jobs that consume them. The command skips
images that already exist in the registry, so a retried run doesn't
fail if the registry is immutable.
Also adds support for the test harness leveraging workload identity
federation credentials so we can use GitHub's token issuer for the test
code itself, and AWS' OIDC provider for the operator, and avoid using
any secrets in CI.
Updates tailscale/corp#46577
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
Android doesn't have /etc/resolv.conf. This causes problems for people
running GOOS=linux binaries (or GOOS=android binaries without cgo, so
they don't use Android's bionic libc) in Termux, adb shell, etc.
(Go binaries built with cgo use bionic on Android: golang/go#10714)
16 years ago when I was on the Android team I added a system-wide DNS
cache (dnsproxyd) and made bionic query that, so each Android app
wasn't doing its own DNS resolution. That interface was never meant to
be stable, and I thought that code would be surely dead by now 16
years later, but apparently it lives on, and is more stable now: both
empirically (time, ossification?), and because of how Android's split system
updates work nowadays, the dnsproxyd lives on the other side of bionic,
so they seem to keep it pretty stable. The old bionic<->dnsproxyd APIs
I added 16 years ago are still there, but 8 years ago it got some additional
APIs to query by a DNS packet instead.
So use it! If we find ourselves on Android and without libc access
(and because we don't want to pull in ebitengine/purego with all its
side effects), just query the DNS server like bionic does.
This can be disabled in Linux binaries with ts_omit_androiddns.
Old links:
LineageOS/android_system_netd@007e987feehttps://android.googlesource.com/platform/system/netd/+/007e987fee7e815e0c4bc820f434a632b7a69a9d
("DNS proxy thread in netd.")
aosp-mirror/platform_bionic@a1dbf0b453https://android.googlesource.com/platform/bionic/+/a1dbf0b453801620565e5911f354f82706b0200d
("DNS proxy: the start. proxies getaddrinfo calls.")
Back then I found it cleaner to proxy at the getaddrinfo level rather
than speak in terms of DNS packets. The raw-packet resnsend command I
use here came eight years later, added in November 2018 for Android
10's android_res_nsend NDK API:
LineageOS/android_system_netd@c0c818f448https://android.googlesource.com/platform/system/netd/+/c0c818f448efa90ab1f9b1733fb86c5e22fb894c
("Add resNetworkSend cmd in DnsProxyListener")
Android 10 (codename Q, API level 29, released September 2019) is
therefore the minimum OS version for this to work.
I verified this against the DnsResolver module on an Android 13
emulator with SELinux enforcing, from the shell UID, with both a pure
Go GOOS=android binary and a static GOOS=linux binary: raw queries,
NXDOMAIN handling, the runtime Android detection, and a tailcat binary
reaching DERP with lookups visible in the daemon's logcat output, some
served from my 2010 DNS cache.
Updates #21129
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I0d63763e255a077e4e5745b3e64ba0d78dab6d69
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
A full netmap carries only the profiles of users with a currently
visible peer (netmapForResponse), and nodeBackend replaces its live
profile set wholesale on every full netmap install. A full netmap that
arrives while a user has no visible peer therefore drops that user's
profile downstream.
When a peer of that user later returns as an incremental upsert,
control does not resend the profile, because MapResponse.UserProfiles
has carried only new or updated profiles since mapver 5. The upsert
indexes the node by address and key, so WireGuard admits its traffic,
but WhoIs then fails one step later at the user profile lookup,
surfacing as "peerapi: unknown peer" until the next full netmap. It is
a second, independent cause of the symptom fixed by the recent index
eviction change.
mapSession.lastUserProfile holds the profile the whole time, so when
handling a response incrementally, also deliver the profiles of
upserted peers' users (and sharers) from that store, before the
mutations that reference them.
Updates tailscale/corp#47435
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I9c4b2f6a8e0d47d3b1a5c2e7f4098d61b23a7c50
Refresh the expiry timer netmap from the live peer state before
reinstalling it, preventing delta updates from being rolled back.
Updates tailscale/corp#47686
Change-Id: Idc738acea82bab5a8ba772084a41e55b38a06bcc
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
When applying netmap deltas, nodeBackend evicted its index entries
(nodeByAddr, nodeByKey, nodeByWGString, nodeByStableID, nodeByName)
derived from a node's last-known value without checking that the entry
still pointed at that node. Control can reassign a churning ephemeral
peer's Tailscale IP (or MagicDNS name) to a newer peer and deliver the
new peer's upsert before the old peer's removal, either in an earlier
MapResponse or reordered within one batch by the NodeID sort in
netmap.MutationsFromMapResponse. The removal then wiped the new
owner's entry.
The peers map itself stayed correct in every ordering, so WireGuard
kept the peer and handshakes succeeded, but WhoIs lookups by IP failed
until the next full netmap rebuilt the indexes. On App Connectors that
surfaced as "peerapi: unknown peer" and refused DNS connections from
affected clients, with a toggle of Tailscale (forcing a full netmap)
as the only recovery.
Make every index eviction conditional on the entry still mapping to
the node being removed or replaced, and add a regression test covering
the cross-batch, intra-batch, and upsert-eviction orderings.
Also add an end-to-end test in tstest/integration showing that a
MapResponse reusing an address is handled incrementally rather than as
a full netmap, and that LocalBackend.WhoIs still resolves the reused
address afterwards, which is the lookup PeerAPI makes before it
accepts a connection.
Updates tailscale/corp#47435
Co-authored-by: Brendan Creane <bcreane@gmail.com>
Signed-off-by: Brendan Creane <bcreane@gmail.com>
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I3f8c2a9d41e07b6a5cd2e94f78b013c6ad2f5e91
This commit bumps the wireguard-go dependency to incorporate changes to
the packet memory model and the tun.Device.Read and conn.ReceiveFunc I/O
interfaces. It updates their implementations accordingly.
These changes improve throughput in all measured benchmarks and reduce
peak RSS in six of eight cases. The two regressions will be addressed in
a follow-up commit that reduces peak RSS below the baseline measured at
1e69418. That work is kept separate to simplify review.
The following throughput and peak RSS benchmarks were performed with
iperf3 between two Intel i5-12400 nodes running Ubuntu 24.04 (Linux 6.8).
The UDP benchmarks did not use UDP GSO on the sender, so they were
roughly equivalent to single packet I/O through wireguard-go.
TCP/1 signifies one TCP stream; TCP/128 signifies 128 parallel TCP
streams.
Throughput (Mb/s)
Test 1e69418 After Change
TCP/1 10,371 11,354 +9.5%
TCP/128 7,886 8,404 +6.6%
UDP/1 2,111 2,853 +35.1%
UDP/128 1,747 2,235 +28.0%
Peak memory (VmHWM, kB)
Test Side 1e69418 After Change
TCP/1 TX 98,240 52,596 -46.5%
RX 287,748 73,384 -74.5%
TCP/128 TX 101,196 52,812 -47.8%
RX 290,420 63,620 -78.1%
UDP/1 TX 58,864 160,840 +173.2%
RX 137,516 49,900 -63.7%
UDP/128 TX 66,148 116,096 +75.5%
RX 154,384 56,556 -63.4%
Updates tailscale/corp#46716
Updates tailscale/corp#22467
Updates tailscale/corp#36989
Updates tailscale/corp#37878
Signed-off-by: Jordan Whited <jordan@tailscale.com>
Rename the tailcat job added in 6c06e00ad to
heads-up-hypothetical-tailcat-build-using-this-commit and print an
explanation on failure: the job builds tailcat against tailscale.com at
this commit rather than the version tailcat pins, so a failure doesn't
break tailcat until it bumps its dependency. It only matters if the API
or behavior breakage is unexpected. People were misreading failures as
something more urgent.
Updates tailscale/corp#24454
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I3f8a1c92d47e60b5a2e9c14f7d08b361
Upates to controlClient learned keys was logged as coming from TSMP.
Also, nil changes on nil keys were not filtered.
Updates #20494
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
Once the pkg-types script has generated pkg.d.ts and checked them
against the tsconfig.json and node_modules in cmd/tsconnect, copy
them into the pkgDir so they get bundled into the package.
Updates #19707
Signed-off-by: Gesa Stupperich <gesa@tailscale.com>
Replace the allowed-IP-only peer callback result with wgcfg.PeerConfig.
It carries allowed IPs and an optional pre-shared key through lazy peer
creation and active peer synchronization.
Update wireguard-go for the new peer PSK APIs.
Updates tailscale/tailcat#84
Change-Id: Iacd9d2c74b0b64d690f3cbdf93918686ac6076d7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
freebsdRouter.Close ran "ifconfig tailscale0 destroy" while this process
still holds the tun open: the engine closes the router before the tun
device, and FreeBSD's tun(4) makes SIOCIFDESTROY sleep until the last
descriptor closes. tailscaled therefore deadlocks against its own child
ifconfig on every shutdown, and "service tailscaled restart" hangs
forever:
51709 - I /usr/local/bin/tailscaled -port 41641 -tun tailscale0 ...
52455 - I ifconfig tailscale0 destroy
52403 1 I+ /bin/sh /usr/local/etc/rc.d/tailscaled restart
Do only the PF cleanup at Close time. The interface left behind at
process exit is destroyed by the next startup's cleanUp hook
(router.HookCleanUp), which exists for exactly that and runs when
nothing holds the device.
Updates #5573
Change-Id: I7b739d948718bfb460240d2c0dd74658fc40ac82
Signed-off-by: Martin Minkus <martin.minkus@sonic.com>
The natlab workflows cached only the VM images, so every job re-downloaded
the tsgo toolchain and built Go from scratch. Cache the toolchain, module
cache, and build cache in both workflows. natlab-test's prepare job is the
only writer of the build cache; everything else, including its 54 matrix
jobs and natlab-basic, restores it.
The warm step compiles with both toolchains on purpose: ./tool/go builds the
test binaries, while the bare `go` that vmtest.go and gokrazy/build shell out
to is the runner's stock Go. GOCACHE keys embed the compiler's build ID, so
those entries don't interchange, but both land in ~/.cache/go-build.
Updates #13038
Signed-off-by: Brendan Creane <bcreane@gmail.com>
runProbe records IPv4CanSend or IPv6CanSend after SendPacket returns
successfully. A sufficiently fast STUN response can be received and
processed before that return, however, and the report can be cloned in
the intervening window. That produces a contradictory report with UDP
and a valid mapping, but CanSend false. Magicsock treats that as a
send failure and unnecessarily rebinds, perturbing tailcat tests.
A received STUN response itself proves that its address family sent
successfully, so also mark the family sendable while recording the
response.
Before this, the tailcat test flaked after ~3700 runs under
flakestress. Now it can run 30 minutes (12,203 successful runs).
Updates tailscale/tailcat#73
Change-Id: I54fe2e81fc703fac36f693e9dbf0c3fa27d119b1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Previously, the new disco keys entering the client via TSMP was routed
into controlClient to allow for deduplication and filtering of disco
keys and avoid control overwriting an active TSMP learned key with a
stale key.
A system supporting multiple disco keys was introduced to allow these
keys living side by side, along side a system for selecting an active
egress key with a bias towards keys learned via TSMP. On ingress any
known key is accepted.
This PR makes the switch to key routing, by sending new TSMP learned
disco keys directly into the userspace engine and in turn magicsock,
removing the need for a full layer of deduplication in the
controlClient.
Additionally, the system that previously fully reconfigured clients on
disco key updates coming from control is now using an optimistic
handshake througha newly introduced wireguard-go method,
ScheduleHandshakeOnUserSend implemented in:
https://github.com/tailscale/wireguard-go/pull/81
What this PR does not do is revert the mapSession back to being single
writer. Bringing the mapSession back to this state is desired, however
to ensure the plumbing is done right and easier to reason about, that
will be done in a separate PR.
Updates #20590
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
Add a workflow that checks out tailscale/tailcat alongside this repo and
runs its tests in a go.work workspace where tailscale.com resolves to the
tailscale.com repo at head instead of tailcat's pinned version. This gives
us an early warning when a change here breaks tailcat's API or behavior
expectations. Such a failure might be fine if the change is intentional
and coordinated with a tailcat fix, but the goal is to catch accidents.
Updates tailscale/corp#24454
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I0566437ee86bd0403c708b2f44e7ef169abf2570
go-118-fuzz-build (fc5dc53b) overlays every sibling _test.go in the fuzzer's directory onto a non-test path (<base>_libFuzzer.go). Sibling test files declaring an external test package (package foo_test) then collide with the fuzzer's package, failing the build with "found packages foo and foo_test".
One possible fix would be to stop using external _test package for our tests, but this change attempts to address this issue without changing our test packaging structure.
This change wraps each compile_native_go_fuzzer_v2 call in build_fuzzer, which hides the files in `go list .XTestGoFiles` for the build. Internal test files stay visible since the fuzz target may use their helpers.
Updates https://github.com/tailscale/corp/issues/46608
Change-Id: Ic6805854f76c597a5edff60674b5ea05b2d18ea9
Signed-off-by: Mike Jensen <mikej@tailscale.com>
MikroTik routers only support permanent UPnP leases. Previously, a
mapping attempt could proceed as follows:
1. Add a temporary mapping.
2. Retry with a permanent mapping after error 725.
3. Successfully create the permanent mapping.
4. Fail to query the external IP.
5. Forget the mapping and retry with another random port.
Each retry left another permanent NAT rule behind.
To prevent this, query and validate the external IP before creating
the mapping. When service selection has already queried the address,
reuse that result. This leaves no fallible network request after
AddPortMapping succeeds.
I tested this on my own MikroTik router running RouterOS 7.23.2. A
successful mapping queried the external IP before creating the
permanent rule, and a forced external-IP failure created no rule.
Updates #10602
RELNOTE: Prevent UPnP rule accumulation on MikroTik routers.
Change-Id: Id925ed9cc3a3da6ecb06fc892f2a934addfac37b
Signed-off-by: Jake Bailey <jacob.b.bailey@gmail.com>
The golang:N-alpine Docker image lags behind Go minor releases by a
day or two. When we bump go.mod to a new minor version before the
image catches up, the required "Build Docker image" CI check fails
with "go.mod requires go >= 1.27.1 (running go 1.27.0;
GOTOOLCHAIN=local)" and blocks the toolchain bump from merging.
Instead, base the build stage on plain alpine and download the
Tailscale Go toolchain release for the revision in go.toolchain.rev,
matching how everything else in this repo is built.
Fixes#21072
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I1ccb77fa3c7a49532ea87dbdbf9e3340880ec94e
Prior to this change there were two problems with our fuzzing for oss-fuzz:
1. There was an issue if the fuzzing spanned two files (mingled with the testing).
2. The fuzzing needs to be part of the implementation package (no _test packages).
This change fixes that by moving all package fuzzing into a common `fuzz_test.go` within the package.
Updates https://github.com/tailscale/corp/issues/46608
Change-Id: I0b95edcd0df946f723eea32f575c679214c0b202
Signed-off-by: Mike Jensen <mikej@tailscale.com>
When the wasmbuild is run from external workflows, it can't derive
the version stamps from its build context. Allow setting the
ProdLDFlags version.longStamp and version.shortStamp from the
VERSION_LONG and VERSION_SHORT environment variables when present.
This way, if the wasmbuild caller already knows them (e.g. from
mkversion), it can pass them in.
This avoids "x.y.z-ERR-BuildInfo" showing up in the admin console's
machine version column.
Updates #19707
Signed-off-by: Gesa Stupperich <gesa@tailscale.com>
When receiving disco traffic from a node, mark that node as having been
seen. Should that disco key not be the active use disco key, switch to
that one as being the active key. Additionally, clear states on the
magicsock connection and prepare for sending a new WG handshake whenever
user data is transmitted.
Sets up for:
- Routing TSMP keys directly into magicsock
- Switching the active connection reset mechanism to the optimistic
handshake
- Cleaning up paths into controlClient
Updates #20494
Signed-off-by: Claus Lensbøl <claus@tailscale.com>
Use whatever traffic steering's notion of the best connector for the
client is, rather than picking arbitrarily.
Fixestailscale/corp#46766
Signed-off-by: Fran Bull <fran@tailscale.com>
From Go 1.27.0 to Go 1.27.0 + latest upstream release branches,
plus a cherry-pick of an x/net http2 hpack memory optimization
that didn't make Go 1.27 (https://go.dev/cl/807260)
Updates #29053
Change-Id: I5c694d8c4aebf4854773ea2856177aaebb2c39a4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
The existing TestSubnetRouterFreeBSD makes a single HTTP request, which
cannot catch a NAT rule whose translation address is wrong on average
but right occasionally: with a PF "-> (self)" rule, pf round-robins new
states across every address on the machine, so one request has decent
odds of drawing the working address while most flows hang at SYN. That
made the single-flow test an unreproducible coin flip rather than a
regression gate.
Open several fresh flows and require that all of them complete and that
the backend sees exactly one source address. PF rule counters and the
state table are dumped along the way, so a failure shows which address
each flow was translated to.
Fails 4/8 (alternating) against a "-> (self)" NAT rule, with the state
table showing the dead flows translated to the QEMU debug NIC's address
and to tailscale0's own address. Passes 8/8 with per-egress-interface
rules, and under netstack subnet routing.
Updates #5573
Change-Id: I1739ceefbab7764f1db8d07424832947b1d962a3
Signed-off-by: Martin Minkus <martin.minkus@sonic.com>
Moving FreeBSD subnet routing from netstack to the kernel changes the
behavior of every existing FreeBSD subnet router, and the kernel path is
not yet ready to be the default:
- The pf NAT rule we install never matches. On a production FreeBSD
15.0 subnet router, "pfctl -vsn -a tailscale" reports 97071
evaluations with 0 packets and 0 translations, and "pfctl -s info"
reports translate: 0. So --snat-subnet-routes=true, the default,
silently performs no source NAT at all.
- Inserting the pf anchor at runtime requires reloading the main
ruleset, which cannot preserve the contents of any table that
ruleset references. See the comment in osrouter.loadPFMainRuleset.
Netstack does its own SNAT in userspace, touches no system state, and is
what FreeBSD has always used, so keep it as the default. The kernel path
can be opted into with TS_DEBUG_NETSTACK_SUBNETS=false; only that path
can serve --snat-subnet-routes=false, which netstack cannot do because
it must rewrite the source address.
With this, TestSubnetRouterFreeBSD passes.
Updates #5573
Signed-off-by: Martin Minkus <martin.minkus@sonic.com>
removePFAnchorRef ran at shutdown (and from the startup cleanup hook) and
unconditionally rewrote the main PF ruleset to strip the "tailscale"
anchor references. That is wrong twice over for an operator who
configured those references statically in /etc/pf.conf -- the durable
setup the ensurePFAnchorRef error message recommends:
- it drifts the running ruleset away from /etc/pf.conf on every
tailscaled shutdown, and
- the rewrite reconstructs the ruleset from "pfctl -s" output, which
names PF tables but never their contents, so any table the operator
populates out-of-band is silently emptied (same failure mode
ensurePFAnchorRef now refuses).
Track whether this process inserted the references and only remove what
we added, and even then leave them alone if the ruleset now references
tables. Skipped removal is harmless: callers flush the anchor's contents
first, and a reference to an empty anchor has no effect on traffic. The
flag is process-local by design, so the cross-process startup cleanup
hook never removes references it cannot prove are tailscaled's.
The only-remove-what-we-added semantics were first identified and
implemented by Ross Williams (@overhacked) on a fork of this branch;
this is an independent implementation of the same idea alongside the
table guard.
Updates #5573
Change-Id: Idf579a8b4190c37211782feac4e5bfd057c3ace0
Signed-off-by: Martin Minkus <martin.minkus@sonic.com>