mirror of
https://github.com/Cacti/spine.git
synced 2026-11-03 10:37:56 +09:00
develop
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Thomas Vincent
|
9be5f8ad2e |
fix(results): [develop] normalize output and unify SNMP batch persistence (#641)
Classify numeric, undefined, multipart and Cacti octet results through one parser, normalize undefined agent values before persistence and flush every SNMP batch consistently. Validate whole non-negative uptimes, retain unknown reindex baselines and free script results through their owning pointer. The mocked poll_host suite checks session-profile boundaries, spike handling, output_regex input and completion writes. Originally step 4/14 of the C17/native-correctness series; its preceding steps have merged. The current base is develop. Earlier validation, before this review follow-up, in isolated copied snapshots on Linux arm64: - Clean GCC build and make check: 15 passed, 1 existing prerequisite skips, 0 failures. - ASan/UBSan: 14 passed, 2 existing privilege/allocator-interposition prerequisite skips, 0 failures; no sanitizer findings. - Production source/profile integrity verification against disposable MariaDB and loopback SNMP fixtures passed. - Release archive extracted into a separate source tree and built/tested out of tree; fuzz checks passed. Earlier local profiles retain source hashes for the copied validation tree; GitHub checks validate the published commit. No production database/device, host installation or live privilege change is part of validation. Existing prerequisite skips were retained; no test or finding was suppressed to obtain a pass. Review follow-up (2026-10-09): Restacked on the corrected preceding branches. The SNMP status fix is introduced in #638, the reindex/completion fixes in #640, and focused regression coverage in #650. Restacked tip validation: isolated Linux arm64 build and make check passed (18 passed, 1 setuid prerequisite skip, no failures). GitHub checks validate the updated published commits; earlier profile/archive/sanitizer evidence above belongs to the previous revisions. ## Current merge dependencies Current base: `develop`. Immediate dependency: `develop` (earlier steps already merged). Next open PR: [#644](https://github.com/Cacti/spine/pull/644). Merge order: #641 → #644 → #654 → #646 → #648 → #649 → #650 → #651 → #655 → #658 → #659. This repository uses squash-only merges and deletes merged branches. Record the old parent SHA, squash one PR at a time, then restack and retarget dependents before merging the next PR. Re-run merge validation and checks on the resulting heads; an ordinary clean merge does not replace squash/restack validation. --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> |
||
|
Thomas Vincent
|
3065c6fc23 |
refactor(core): add bounded native interfaces and polling contracts (#638)
Introduce bounded native interfaces and grouped runtime state, then link regression and fault contracts against the real poller implementations. The contracts capture CLI/configuration input, availability combinations, SNMP request/response ownership, script framing, reindexing, output persistence, worker completion and shutdown. This foundational change is larger because the shared interfaces and their existing production contracts must move together; later changes have separate review boundaries. Step 1/14 of the complete C17/native-correctness series. Base: develop. Earlier validation, before this review follow-up, in isolated copied snapshots on Linux arm64: - Clean GCC build and make check: 13 passed, 0 existing prerequisite skips, 0 failures. - ASan/UBSan: 12 passed, 1 existing privilege/allocator-interposition prerequisite skips, 0 failures; no sanitizer findings. - Production source/profile integrity verification against disposable MariaDB and loopback SNMP fixtures passed. - Release archive extracted into a separate source tree and built/tested out of tree; fuzz checks passed. Earlier local profiles retain source hashes for the copied validation tree; GitHub checks validate the published commit. No production database/device, host installation or live privilege change is part of validation. Existing prerequisite skips were retained; no test or finding was suppressed to obtain a pass. The worker contracts publish owned heap state and release it only after joining workers and restoring the runtime table. Coverage counters use atomic updates; cppcheck receives an explicit model of the fatal exit contract. Fixture images use public mirrors to avoid shared Docker Hub pull limits. Series navigation: review and merge in order. Every PR targets the preceding branch so its diff contains one step. 1. [refactor(core): add bounded native interfaces and polling contracts](https://github.com/Cacti/spine/pull/638) 2. [fix(privileges): drop root before input and retain owned ICMP sockets](https://github.com/Cacti/spine/pull/639) 3. [fix(poller): isolate worker failures and bound database reconnects](https://github.com/Cacti/spine/pull/640) 4. [fix(results): normalize output and unify SNMP batch persistence](https://github.com/Cacti/spine/pull/641) 5. [fix(runtime): bound probe retries and validate CLI and configuration](https://github.com/Cacti/spine/pull/642) 6. [build: select GNU C17 explicitly and probe hardening support](https://github.com/Cacti/spine/pull/643) 7. [fix(threads): size worker stacks and guard size arithmetic](https://github.com/Cacti/spine/pull/644) 8. [refactor(c17): simplify ownership guards and share test support](https://github.com/Cacti/spine/pull/645) 9. [refactor(core): extract configuration polling and runtime modules](https://github.com/Cacti/spine/pull/646) 10. [fix(boundaries): validate script slots and make cleanup portable](https://github.com/Cacti/spine/pull/647) 11. [refactor(layout): organize responsibilities and separate main from runtime](https://github.com/Cacti/spine/pull/648) 12. [docs(license): adopt SPDX notices and retain contributor credits](https://github.com/Cacti/spine/pull/649) 13. [test(runtime): measure all suites and cover protocol boundaries](https://github.com/Cacti/spine/pull/650) 14. [build(ci): validate connectors time width and native C17 modules](https://github.com/Cacti/spine/pull/651) Review follow-up (2026-10-09): Publish the decoded GET/GETNEXT status back to the host so availability checks observe exception and missing-response failures, while preserving the legacy transport-success behavior for agent PDU errors. Updated step 1 snapshot: isolated Linux arm64 build and make check passed (13 passed, no skips or failures). The restacked tip passes all five clang-tidy gate regression tests and real clang-tidy analysis (26 reviewed baseline findings, no new findings). Actionlint and matching DCO trailers pass. The setuid test also passes when run in the separate disposable root container. GitHub checks validate the updated published commits; earlier profile/archive/sanitizer evidence above belongs to the previous revisions. --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> |
||
|
TheWitness
|
ca662c1be7 |
fix: cap Solaris reap budget, bound EINTR retries, kill before parking on waitpid error (#635)
## What Follow-up to #633 (already merged), from the code review findings on the equivalent 1.2.x change (#634): - **SOLAR_THREAD sleep budget**: `spine_reap_child_bounded()` has no `usleep()`-paced spin phase on `SOLAR_THREAD` builds - it sleeps a flat 1s per attempt. #633 raised the shared attempt budget (`NFT_PCLOSE_SPIN_ATTEMPTS`) to 100 for `nft_pclose()`'s brief allowance, which made it sleep up to **~100 seconds** on that platform instead of the intended ~20ms, reintroducing the exact poller stall #633 was fixing. `NFT_PCLOSE_SPIN_ATTEMPTS` is now platform-specific (100 elsewhere, 2 under `SOLAR_THREAD`). - **Unbounded EINTR retries**: bounded the `EINTR` retry loop in `spine_reap_child_bounded()` and in `nft_sweep_abandoned()` (which runs with `ListMutex` held), so a stream of caught signals can no longer spin either indefinitely. - **Kill before park on waitpid() error**: `nft_pclose()` now SIGKILLs the child before parking it when `waitpid()` itself errors (previously only done when the child was confirmed still running), so a still-running child isn't left outside the abandoned-pid sweep's reach. Preserves `waitpid()`'s errno across the call. - **Docs**: `nft_popen.h`'s `nft_pclose()` comment still described the old blocking-wait behavior and didn't mention `ETIMEDOUT` - #633 updated `nft_popen.c`'s comment but missed the header. Both now document the bounded-check/parked-child contract. ## Why These were flagged by review of #634 (the same fix ported to 1.2.x, where the affected code showed up as new rather than a diff against pre-existing lines). Fixing here too since develop has the identical code. ## What was tested - `./bootstrap && ./configure --enable-warnings && make` - clean build, no new warnings. - `make check` - all 11 suites pass. - `cppcheck --enable=all --std=c11` on `nft_popen.c` - no new errors. Closes #632. --------- Signed-off-by: TheWitness <thewitness@cacti.net> |
||
|
TheWitness
|
08e2ae0b42 |
fix: reap script/PHP children without blocking the poller, drop USING_TPOPEN (#633)
## What - `nft_pclose()` (`nft_popen.c`) and `php_terminate_and_reap()` (`php.c`) no longer escalate through a spin/sleep/SIGTERM/SIGKILL sequence. Each now does one bounded, non-blocking check (~20ms allowance for normal fork/exec/exit scheduling latency), kills anything still running, and hands it to the existing abandoned-pid sweep instead of waiting on it in the calling (polling) thread. - Removed the `USING_TPOPEN`/`--enable-popen` native-`popen()` fallback in `exec_poll()` (`poller.c`, `configure.ac`). It could not track or kill a timed-out child (no pid available from a real `popen()`), so a timeout there could only ever abandon the descriptor without terminating the process. - Extended `tests/regression/test_child_process_safety.sh` with structural guards against reintroducing the blocking spin/sleep escalation in either function. - Updated `tests/unit/test_linked.c`'s `nft_pclose()` coverage for the new, faster contract (the old test asserted a graceful-SIGTERM-then-wait exit code that the new design intentionally no longer provides). ## Why Investigating a reported poller slowdown surfaced that `nft_pclose()` (added by #614) could block a poller thread for up to ~5 seconds per script whose process hadn't exited by the time its pipe was closed, and `php_terminate_and_reap()` had the same shape (up to ~2s), reachable mid-poll-cycle via `php_fail_read()`, not just at shutdown. Separately, the `USING_TPOPEN` fallback's inability to kill a timed-out child was flagged as out of scope in #574. Tracked in #632; design principles written up in #630. ## What was tested - `./bootstrap && ./configure --enable-warnings && make -j` — clean build, no new warnings. - `make check` — all 11 suites pass (unit, regression, integration), including the extended `test_child_process_safety.sh` guard and the updated `test_linked` case. - `cppcheck --enable=all --std=c11` on the three changed `.c` files — no new errors (pre-existing style-only notes unrelated to this change). Signed-off-by: TheWitness <thewitness@cacti.net> |
||
|
|
86a062dd09 |
ci: enforce cross-platform quality gates and connector coverage (#618)
## Summary Consolidates the connector/build work from #613 and the broader test-infrastructure work extracted from #597 into one review: - discover MariaDB Connector/C first while retaining MySQL and Percona-compatible support - test MariaDB and MySQL Connector/C with strict GCC and Clang builds on Ubuntu 24.04 - add a real macOS 15 build and unit-test lane with portable Autotools platform detection - run unit tests under ASan/UBSan and ThreadSanitizer - enforce the measured 7% global coverage baseline and 70% changed-line coverage - add libFuzzer coverage for output parsing alongside the existing hostname parser - exercise the SNMPv3 integration suite against MariaDB and MySQL 8.4 - add cppcheck, shellcheck, DCO verification, CODEOWNERS, distcheck, and Linux-capability checks while retaining the repository-managed default CodeQL setup - expose a stable `CI / required` aggregate check for branch protection - pin runners and third-party actions and keep workflow permissions read-only by default ## Correctness fixes found by the new gates - reject truncated capability strings and poller-output SQL tuples instead of publishing partial data - make `bootstrap` fail closed and select `libtoolize`/`glibtoolize` portably - use Autoconf's canonical host for Darwin detection instead of an empty native-build alias - remove stale initialization/name-shadow findings and make read-only interfaces const-correct - make the ICMP parser test portable across Linux and BSD header names ## Verification - strict GCC and Clang builds: 4/4 unit targets passed - ASan/UBSan: 4/4 passed - ThreadSanitizer: 4/4 passed - macOS Clang/Homebrew build: 4/4 passed - MariaDB integration: passed - MySQL 8.4 integration: passed - both libFuzzer targets: passed sustained local runs - coverage: 7.2% global and 84% on changed lines - cppcheck, shellcheck, actionlint, DCO verification, and `git diff --check`: passed locally ## Post-merge repository settings After these workflows exist on `develop`, an administrator should require `CI / required`, Fuzz, Integration, Rocky Linux, and the repository-managed CodeQL/code-scanning result in the active ruleset; enable code-owner approval; and require the branch to be current (or use a merge queue). Enabling those settings before this PR merges would create required checks that the base branch cannot yet emit. Consolidates and supersedes #613. Closes #612. --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Signed-off-by: TheWitness <thewitness@cacti.net> Signed-off-by: TheWitness <copilot@users.noreply.github.com> Co-authored-by: TheWitness <thewitness@cacti.net> Co-authored-by: TheWitness <copilot@users.noreply.github.com> |
||
|
|
dbbccc5263 |
fix: extract core safety fixes from #597 (#621)
Extracted from #597 into a focused core correctness and memory-safety change. This checks startup allocations, bounds full log lines and query construction, frees settings result sets on NULL-row paths, caches the configured timestamp format, replaces advancing snprintf cursors with a bounded append contract, fixes remote batch boundary loss, and converts delimited hexadecimal OctetStrings without overflow. poller_output and remote status pushes use the cross-vendor VALUES() form until #590 supplies an independently detected main-server capability. Behavior worth calling out: - colon/dash-delimited hexadecimal values that were previously stored as zero now convert correctly - values wider than 64 bits become U and count as collection errors instead of producing undefined numeric conversion - remote batches flush before the boundary-triggering row and refuse truncated statements Verification on macOS arm64: - warning-enabled C99 configure/build - forced make -B check on the final tree: 6 pass, 1 expected skip - the skipped allocation-failure integration uses Linux LD_PRELOAD and runs in Linux CI - full VPATH make distcheck passed earlier in this branch - mandatory independent pre-push review: PASS Closes #564 Closes #565 Closes #566 Closes #567 Closes #588 Refs #590 Refs #597 --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Signed-off-by: TheWitness <copilot@users.noreply.github.com> Co-authored-by: TheWitness <copilot@users.noreply.github.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
|
478d330985 |
fix(ping): make IPv4 and IPv6 ICMP polling safe (#619)
## Summary Consolidates the overlapping ICMP work from #555 and #619 into one review of the complete socket and resolver state machine. - add ICMPv6 echo polling and dispatch IPv6 devices to it - prefer unprivileged datagram ICMP sockets when the kernel permits them - retain bounded raw-socket fallback for IPv4 and IPv6 - give packet and socket resources one cleanup owner - leave caller-owned resolver state untouched on lookup failure - keep every retry inside one cached capability and privilege-transition decision - prevent LOCK_SETEUID retry deadlocks - validate raw and datagram reply layouts, source, sequence, identifiers, and payload - retain fuzz and unprivileged-socket integration coverage The consolidation also fixed an integration defect found locally: the controlled successful-reply test constructed only a raw IPv4 packet even when the production path selected SOCK_DGRAM. It now models the selected socket format, and IPv6 raw retries use the same privilege-safety invariant as IPv4. ## Verification - warning-enabled local build completed with MariaDB Connector/C, Net-SNMP, OpenSSL 3, and cmocka - make check: 5/5 test binaries passed - focused ping_icmp suite: 9/9 cases passed - actionlint and git diff --check passed - all commits are signed - GitHub CI is authoritative for Linux capability, sanitizer, fuzz, and unprivileged-network coverage Closes #127. Closes #250. Closes #593. Closes #611. Supersedes #555. Refs #597 and #618. --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Signed-off-by: TheWitness <copilot@users.noreply.github.com> Co-authored-by: TheWitness <thewitness@cacti.net> Co-authored-by: TheWitness <copilot@users.noreply.github.com> |
||
|
Thomas VincentandTheWitness
|
90da94b94f |
fix(process): unify subprocess ownership and child lifecycle safety (#614)
## Summary - make subprocess-registry detach atomic so exactly one closer owns each entry - make cancellation cleanup release only the detached node it owns - replace `vfork`/`execve` child setup with `posix_spawn` and explicit file actions - mark every pipe and stdio-collision duplicate close-on-exec - restore the child's default SIGPIPE disposition while preserving the parent's handling - bound TERM/KILL reaping and park unreaped children in a fixed-capacity sweep list - harden PHP script-server slot ownership, startup, restart, signal, descriptor, and cleanup paths - retire and restart failed or oversized reads so stale response tails cannot poison later data sources - revalidate every command retry and cap recovery handshakes at two seconds This folds the child-process lifecycle portion of #597 into the narrower ownership PR. It does not absorb unrelated poll-host, SQL, SNMP, ICMP, or build-infrastructure work. ## Review-driven corrections - close-on-exec is explicitly restored after `dup()` in both read and write stdio-collision branches - the source invariant check now uses portable POSIX character classes - direct tests cover duplicate flags, both collision branches, failure after pipe creation, abandoned-child sweeping, and capacity overflow - collision cleanup preserves current-child redirects even when an older registry entry occupies fd 0 or 1 - script children receive SIGTERM before SIGKILL and unreaped PHP children transfer to the bounded sweep registry - the Docker integration suite now starts a real PHP script server, verifies its startup protocol and database result, and checks that shutdown leaves no zombie PHP process ## Validation - Ubuntu 24.04 container with GCC and GNU ld: `make check`, 6/6 targets passed with no skips, including `test_php_runtime` - real PHP script-server Docker integration: 6/6 checks passed; exact value `42` reached `poller_output` and no zombie remained - macOS Homebrew dependency build passed - macOS `make check`: 5/5 available binaries passed; GNU `--wrap` runtime suite is Linux-only - PHP fixture syntax checked through mise PHP 8.3 - actionlint, shellcheck, and `git diff --check` passed Closes #610. --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas Vincent
|
8208fcb593 |
test: link the shipped objects instead of copies of the logic (#577)
The three unit binaries in `tests/unit/` do not test the code that ships. `test_safety_fixes.c` says so in its own header: > the functions under test are inlined (or their exact predicate replicated) so this translation unit compiles standalone without linking util.o/ping.o/php.o and dragging in MySQL/SNMP That keeps them cheap, but a fix can land in `util.c` while the test carries on passing against its private copy. The 39 assertions currently guarantee less than they appear to. This adds a fourth binary that links the real translation units. `tests/fuzz/stubs.c` already solves the hard part for the fuzz targets by supplying the globals `spine.c` would define, so the same file is reused here. Everything except `spine.c` is linked as built. **Demonstration.** Reverting `strncopy()` to the issue#447 off-by-one and running the suite: ``` PASS: tests/unit/test_util_strings PASS: tests/unit/test_build_fixes PASS: tests/unit/test_safety_fixes FAIL: tests/unit/test_linked # TOTAL: 4 # PASS: 3 # FAIL: 1 ``` The three existing suites do not notice. Only the linked one does. Seven cases to start, covering `strncopy()` bounds and termination and `regex_replace()` match/no-match/bad-pattern. The point of the PR is the harness rather than the case count: with this in place, future tests can assert against real behaviour, and the existing binaries can migrate over one at a time. Verified in a clean `ubuntu:24.04` container: builds at 4 warnings matching develop, `make check` passes 4/4, `make distcheck` passes. --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> |