mirror of
https://github.com/Cacti/cacti.git
synced 2026-10-30 08:37:01 +09:00
fix/graph-wrapper-date-range-develop
47
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 TheWitness
|
d57431371b |
perf(poller/boost): cached per-data-source field-name lookups, remove batched prefetch (#8078)
* perf(poller/boost): replace batched field-name prefetch with cached per-data-source lookups poller_prefetch_rrd_field_names() (issue#7534) batched the unused/nt field-name lookups into large IN() lists and rebuilt them on every process_poller_output()/boost_process_local_data_ids() call, so the completion-wait loop re-ran them each pass (DB thrash) and the IN() lists risk crashing some MariaDB/MySQL releases and do not scale to multi-million-data-source installs. Option B (full): remove poller_prefetch_rrd_field_names() and replace it with two small per-data-source helpers - poller_get_unused_data_source_names() and poller_get_nt_rrd_field_names() - each a single indexed WHERE local_data_id = ? query cached by local_data_id in a caller-held static, so every data source is queried at most once per poller process. Templated field names resolve from the static poller_data_template_field_mappings cache (the Data Template dictates); only non-templated (data_template_id == 0) manual sources hit the per-instance query - a rare exception. Both process_poller_output() (lib/poller.php) and boost_process_local_data_ids() (poller_boost.php) now use the helpers. Added a WARNING documentation block to both bulk hot-path functions spelling out the query-discipline rules for future maintainers/AI (minimal queries, no query in the per-row loop, avoid large IN() lists and full-table scans, single pass, static template-dictated caching). Updated BoostProcessBugsTest assertions to the new helpers and removed the now-obsolete BoostPrefetchRrdFieldNamesTest integration file. * fix(poller/boost): split GROUP_CONCAT'd field mappings; guard orphan-name query to templated sources; add coverage * boost: normalize orphan-name guard to ternary at third call site The reset_template branch in boost_process_local_data_ids() guarded the poller_get_unused_data_source_names() call with an if/else block rather than the ternary used at the other two call sites. Normalize it so all three sites read identically and the template-id guard is uniform. |
||
|
Thomas VincentandTheWitness
|
d3d99bb08a |
test(cdef): enforce resolver coverage (#7974)
* test(cdef): enforce resolver coverage Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci(cdef): pin cache action correctly Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: isolate CDEF coverage bootstrap * fix(cdef): fail closed on invalid recursive definitions Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(cdef): log corrupt resolver records Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(cdef): bound and isolate recursive resolution Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(cdef): preserve resolver failure state Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(cdef): isolate invalid graph expressions Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(cdef): abort unsafe consumer operations Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(cdef): close consumer failure paths Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(cdef): make aggregate updates atomic Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(cdef): keep repair queries auditable Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(cdef): bind bulk deletes and escape item types Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(cdef): guard corrupt types and repair reporting Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(aggregate): propagate push-out failures Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(aggregate): document mutable message checks Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
cb49f31ff9 |
Replace csrf-magic with maintained CSRF protection (#7744)
* build: add symfony/security-csrf Backs the CSRF token core that replaces the vendored csrf-magic fork. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: fix security-csrf package path assertion symfony/security-csrf ships CsrfTokenManager.php at the package root, not under src/. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * refactor: register the composer autoloader before the CSRF include CSRF validation ran before the autoloader existed, which blocks any class-based guard. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * feat(csrf): add CactiCsrfGuard over symfony/security-csrf Issues and validates session-scoped tokens; not yet wired into the request path. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * feat(csrf): accept pre-upgrade csrf-magic tokens during the grace window Legacy tokens carry their own expiry, so the fallback closes itself without a config flag. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * feat(csrf): port the output buffer rewriter to CactiCsrfGuard Keeps the hidden field name and JS globals that third-party plugins rely on. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(csrf): use json_encode for the JS script context in rewriteBuffer htmlspecialchars() protects the HTML attribute context but not the <script> string literal, which relied on Symfony's token generator never emitting a quote rather than being safe by construction. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * refactor(csrf): move the XHR shim to include/js as Cacti-owned code It has been maintained by the Cacti Group since 2020 and is not an upstream package. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * refactor(csrf): retire the vendored csrf-magic fork The procedural csrf_* surface stays as a deprecated shim so plugins keep working. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * feat(csrf): validate requests through CactiCsrfGuard csrf-magic self-invoked from its own file; the guard is called explicitly. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(install): stop gating the permission step on the csrf secret Retiring csrf-magic removed the directory its default path lives in, and setStep() clamps to the first errored step, so no fresh install could advance. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(csrf): key the legacy fallback on the raw secret bytes refresh_csrf.php writes PHP source with a trailing newline, and trimming that yields a different HMAC key than csrf-magic used. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(csrf): log when the guard degrades to disabled in a web context The installer is the only web path that can reach it, and failing open in silence is not something csrf-magic could do. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: changelog for the CSRF library replacement Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(csrf): mint the token before the session write callback closes it csrf_startup() minted lazily from the output buffer callback, which runs after session_write_close()'s shutdown-function registration, so a token minted there on a legacy-fallback POST was silently dropped. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(csrf): make refresh_csrf.php a no-op Its default secret path no longer ships and live tokens are session-bound, not HMACs over the secret, so rotation neither worked nor did anything meaningful; running it during the upgrade grace window used to delete the legacy secret and kill validation for pre-upgrade tokens outright. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(csrf): fall back to the old default secret path when reading it csrf-magic tried its own directory as a second candidate regardless of $path_csrf_secret; the new reader only tried the configured path, so a site pointing $path_csrf_secret at a file that was never created lost the legacy-token grace window entirely. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs(csrf): point the JS header at its BSD license sidecar The header reduces the original BSD terms to prose and the actual text sits in an unreferenced csrf.js.LICENSE, so add a pointer to it. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(csrf): restore csrf_check()'s bool return and $fatal contract The shim ignored $fatal and always returned void, so audit/audit.php's `!csrf_check(false)` evaluated `!null` (always true) and denied every request instead of getting a real pass/fail result back. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: reference the pull request number in the changelog entry Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: resolve includes through Cacti path constants Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(csrf): enforce full coverage Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Harden CSRF fallback handling Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(csrf): assert the component stays out of the index Committing it would leave a second copy to drift against the lock. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: align CSRF checks with PHP 8.2 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: normalize the CSRF changelog entry * Fix CHANGELOG entry ordering Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(csrf): support constrained test environments * fix(csrf): narrow nullable guard state * test: place CSRF coverage in security domain Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * security(test): match disabled exec exactly Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: run CSRF coverage on PHP 8.3 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
157a4458ae |
fix(auth): reject malformed remember cookies (#7976)
* fix(auth): reject malformed remember cookies Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: load cookie contracts lazily * test(cookie): fail clearly when contract markers move Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(auth): align persistent credential lifecycle Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(auth): find cookie function boundaries robustly Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
055c5ef5fa |
security: reject LDAP logins that skip a domain bind (#8009)
* hardening: validate the login realm and escape the LDAP username per context domains_login_process() now rejects a realm get_auth_realms() does not list and fails closed on one that is listed but has no domain behind it, and the Ldap class escapes the username for the DN and the search filter separately rather than relying on a filter-only blocklist. - an unlisted realm sent with a non-empty password used to match no branch at all and returned an empty user with no error, which auth_login.php reads as an authenticated user and provisions from the template account Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * security: reject LDAP logins that skip a domain bind Posted realms outside user_domains no longer fall through to template or guest login. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(ldap): align escaping contract with username lookup Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: cover LDAP domain login through OpenLDAP The e2e stack now binds a real directory user and checks that a forged realm cannot skip that bind. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(ldap): address Copilot review feedback Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: substitute LDAP filter placeholders in one pass LDAP_ESCAPE_FILTER leaves angle brackets intact, so a looped str_replace can rescan one value as another key. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Change realm retrieval function from gnrv to gfrv * tests: add gfrv() stub to harnesses that exercise gfrv()-calling code domains_login_process() and api_aggregate_convert_template() call gfrv() for realm/aggregate_template_id, but their test harnesses only stubbed gnrv(), so Pest failed with 'Call to undefined function gfrv()'. --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
182b21dc53 |
fix(csv): align export and graph sample windows (#7987)
* fix(csv): align export and graph sample windows Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci(csv): satisfy release policy checks * test(csv): align legacy capacity contract * test(csv): add standard license headers * fix(rrd): infer step from trailing fetch row Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci(csv): trigger on resolution query coverage Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(rrd): mirror fetch helper arguments Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(rrd): isolate percentile globals Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(rrd): annotate intentional eval Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(rrd): restore integration config Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(rrd): enable strict types Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(rrd): restore font config Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: align CSV coverage with supported PHP versions Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
3c20f4ce7e |
Harden Boost data handoffs and graph cache writes (#7729)
* fix(boost): harden data handoffs and cache writes Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Define $updates_ok when there is nothing to flush PHPStan level 6 also flags the poller ownership check, which is the POLLER_ID constant case the file already annotates above. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Use a line comment to match the fixer Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Record the boost process columns in the audit schema Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: resolve includes through Cacti path constants Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Fix Boost handoff review concerns Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: seed settings in Boost database fixture Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: refresh SQL interpolation baseline Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: record Boost handoff hardening Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: stop boost discarding a cycle and wedging a shard Three defects in the handoff hardening. A rejected ownership handoff returned false, which the poller reads as boost having consumed the data, so it skipped the RRD write. The poller_output rows were already deleted by then, so a stale poller_item copy on a remote poller silently lost the whole cycle. Return true instead and let the poller write directly, matching the database failure path. Any non-OK RRDtool response retained the shard, and the retry then replayed rows already written, which RRDtool rejects as an illegal update time. That failed again on every pass, so the archive tables accumulated and boost never drained. Treat an at-or-behind timestamp as applied, since the sample is in the RRD; genuine errors still retain the shard. The two new poller_output_boost_processes columns passed 'null' where db_install_update_table reads 'NULL', so NOT NULL was never emitted and upgraded installs diverged from cacti.sql. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: spell the NULL column key so upgrades emit NOT NULL db_add_column() reads $column['NULL'] and PHP array keys are case sensitive, so the thirteen definitions in the 1.3.0 upgrade that passed a lowercase 'null' key had the flag discarded and created nullable columns. Fresh installs built from cacti.sql got NOT NULL, upgraded installs did not, and Cacti's own schema audit reports the difference. The same file already spells the key in upper case 144 times, and the wrapper's own docblock example uses the upper case form. Guard the class with a test over every upgrade file rather than only the one that was wrong. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: assert the NULL key plainly toContain() takes needles rather than a failure message. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): close remaining handoff review gaps Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): guard nullable graph output Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): keep in-flight completion rows through the 1.3.0 upgrade The truncate only existed so the run_child UNIQUE key could build over rows that predate run_id; deleting just those rows leaves a running boost its completion records. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * refactor(boost): drop the poller ownership gate Its boost_poller_on_demand() call sits behind the boost_redirect early return that poller.php forces on for every multi-collector install, and its recovery call turns a device deletion on main into a recovery that never brings the collector back online. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): bound atomic cache write allocations Guard upgrade source reads so failures report clearly, write graph cache data in fixed-size chunks, and exercise a multi-chunk payload. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): support Windows cache replacement Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(boost): expand handoff failure coverage Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(boost): support ANSI_QUOTES upgrade checks Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): stop staging after chunk failure Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(recovery): use numeric poller status Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(sql): support ANSI_QUOTES in recovery paths Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): preserve cache on replacement failure Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): retain readable cache permissions Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): repair completion schema at runtime Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(recovery): bound timestamp batch fetches Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): satisfy runtime schema analysis Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(recovery): restore status after transfer failure Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(boost): rely on unit bootstrap helpers Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): refresh generated cache secret Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): normalize cache path separators Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(boost): fail clearly on unreadable fixtures * fix(boost): name ignored duplicate sample keys The staging warning only said rows were dropped, so a retry could not be told from a colliding writer. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Lack of pre-commit hook execution allowed lint failure --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas Vincent
|
95060c49ba | Close test-suite discovery and coverage gaps (#7970) | ||
|
Thomas Vincent
|
a155419ff3 |
Serialize process registry admission with portable locks (#7771)
* Serialize process registry admission with Symfony Lock Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Fix PDO process lock mode handling Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: run process lock coverage on PHP 8.2 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: confirm a pid is still ours before signalling it #7574 added the still-running guard to dsstats but left six byte-identical siblings signalling a pid straight out of the processes table. After a crash or a power loss the row survives, the pid is recycled by an unrelated daemon, and the next start SIGTERMs a stranger. That is issue #7425, still live in rrdcheck, boost, poller_commands, float_rrdfiles, poller_reindex_hosts and rebuild_poller_cache. Wrap each kill in cacti_process_still_running(), matching the dsstats shape, and leave unregister_process() outside the guard so a stale row is still cleaned up. Cover all seven with a test that walks every kill site, so a new variant cannot reintroduce it. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: assert the guard plainly toContain() takes needles rather than a failure message, so the file name was being asserted as a second required substring and the test always failed. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs(poller): describe required lock parameters --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> |
||
|
Thomas Vincent
|
ca2e4f07c2 |
Build releases from managed dependencies (#7888)
* chore: manage front-end JavaScript libraries with npm and Dependabot Move all third-party front-end JavaScript to npm so Dependabot tracks it, and drop the committed copies in favour of a build. package.json pins jquery, jquery-ui, jstree (3.3.12 -> 3.3.17), billboard.js, pace-js, tablesorter and jquery-validation; build/sync-js.mjs copies their dist files into include/js/ and runs on npm install via postinstall. pace-js carries a one-line local fix (a stray regex alternation that made it match the empty string) applied through patch-package (patches/pace-js+1.2.4.patch); the fix should be sent upstream so the patch can be dropped. Only Cacti's own themed jquery-ui/jstree CSS stays committed. An npm ecosystem is added to dependabot.yml for develop. The release pipeline and any workflow that serves Cacti pages must run 'npm ci' (which builds via postinstall) before use, since the generated files are no longer in the tree. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * hardening: remove vendored development artifacts * feat(release): attach the built artifact to the release The workflow already builds an attested, checksummed archive carrying the Composer dependencies, then discards it after thirty days. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: fail the publish step when the bundle is empty An unmatched glob would otherwise reach gh as a literal path. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: treat the maintenance lock notice as expected log output The line records the lock working, so a second maintenance run overlapping in CI failed unrelated pull requests. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: consolidate release dependency changelog * ci: build managed JavaScript in release artifacts * test: cover managed JavaScript build end to end * deps: complete phpseclib 4 migration * deps: update patch-package to audited release * ci: satisfy workflow shell policy * chore: source DOMPurify from npm and update it to 3.4.14 Add dompurify to the npm-managed set (was committed at 3.4.7) and bump to 3.4.14. DOMPurify is the client-side XSS sanitizer used on tree labels, device tooltips and plugin messages; keeping it current with upstream bypass fixes matters more than for the UI libraries. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: preserve release changelog ordering * test: preserve DOMPurify source map in npm build * chore: source htmx, d3, big.js, lzjs and screenfull from npm Add the remaining unmodified third-party libraries to the npm-managed build: htmx.org 2.0.6, d3 7.8.2, big.js 6.2.2, lzjs 1.3.1 and screenfull 3.3.3. Each was verified byte-identical (lzjs differs only in a JSDoc comment) to the copies being removed. htmx in particular moves from a hand-tracked SHA pin to npm with lockfile integrity. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: cover phpseclib 4 integration * ci: build managed assets before htmx tests --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> |
||
|
Thomas Vincent
|
a4010e64a6 |
security: confine rrd data source paths to the RRA directory (#7665)
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> |
||
|
Thomas Vincent
|
964c1fb1f4 |
Harden RRDtool execution and data handoff (#7615)
* fix(rrdtool): quote RRDfile paths passed to the command line __rrd_execute() hands the assembled line to shell_exec(), and its docblock already required callers to quote each variable argument. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(tests): widen bare RRD path sweep Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(rrdtool): quote binary path as one argument Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(rrd): preserve caller logging context Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Make RRD XML mutations transactional Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Harden RRDtool proxy transport handling Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(rrdproxy): reject invalid command argument lists Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(rrdproxy): preserve binary graph payloads Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Keep RRDtool proxy sessions encrypted Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Harden local RRDtool command and output handoff Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Eliminate graph rendering metadata lookups Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Batch DSStats metadata and last-value reads Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Batch interface speed metadata reads Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Update architectural hotspot baseline for RRD execution Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Batch RRD resolution profile lookups Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Batch DSStats summary aggregates Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Narrow phpseclib key types for the proxy encrypt and decrypt calls Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(rrd): bound the forked proxy peers so a stall cannot hang the suite The children accepted and read with no deadline while the parent used a 2s socket timeout, so any mismatch left the child blocked in read and the parent blocked in a bare pcntl_waitpid(). Note that accept() ignores SO_RCVTIMEO, so readiness has to be waited on with socket_select() rather than a socket option. Also corrects the array-command expectation, which still assumed develop's escape-everything behaviour rather than rrdtool_build_command() leaving the operation unquoted. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(rrd): seed proxy settings where the real read_config_option looks UnitStubs only declares its stub when lib/functions.php has not been loaded yet, so during a full run the real database-backed function is the one in effect and $unit_config_options was being ignored. The parent then dialled an address the child was not listening on, which is what stalled the handshake. Seeding the option cache keeps the settings visible to either implementation. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(rrd): assert the shell-free RRDtool launch instead of the old quoting __rrd_init now hands proc_open an argv array with bypass_shell, so escaping the binary path would put the quotes inside the filename. The assertion still expected the pre-argv line and had been failing unnoticed behind the suite hang. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(dsstats): derive the expected stamps instead of assuming UTC dsstats_poller_output() formats with date(), so the hardcoded UTC strings only matched on a UTC runner. The failure was masked by the suite hang. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(rrd): keep RRDtool control codes intact in stdin commands Escaping the backslash when converting a shell-quoted argument to an RRDtool token left legends such as " IO Wait\:" ending in a dangling control code, so RRDtool refused every graph carrying an escaped colon. Verified against RRDtool 1.7.2 in pipe mode; values with no unambiguous quoted form are now refused rather than mangled. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: use Cacti test bootstrap and config Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: resolve includes through Cacti path constants Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: match the escaping helpers by token, not by text lib/rrd.php documents its escaping behaviour in a comment, and the regex over raw source counted that as a call site. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh sink inventory after develop rebase Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(rrd): prove argv isolation and concurrent pipe draining Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(security): refresh sink inventory baseline after rebase onto develop Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> |
||
|
Thomas Vincent
|
5c6e3d21cb | Add DI-friendly CactiFilesystem and adopt atomic CLI writes (#7189) | ||
|
Thomas Vincent
|
7159ce5115 | Add a Cacti-native form API and migrate the Site editor (#7368) | ||
|
Thomas Vincent
|
ff475bef6d | feat(installer): add installation domain foundation (#7291) | ||
|
Thomas Vincent
|
49f5212181 |
fix(tests): restore paths and isolate test doubles (#7764)
* fix(tests): correct path depths after the domain reorganisation The category move left nine files resolving paths relative to their old depth, so the bootstrap could not read tests/Helpers/UnitStubs.php and no test ran at all. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix tests without replacing production functions * remove unrelated failing integration tests * remove unrelated broken test workflows --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> |
||
|
Sean Mancini
|
96033267cb |
refactor(tests): reorganize unit tests into domain category folders (#7733)
* refactor(tests): reorganize unit tests into domain category folders - Consolidate per-advisory GHSA regression tests into Security/ subfolders by vuln class - Group all 186 test files into domain folders: DataCollection/, Ui/, Core/, Database/, Installer/, Scripts/, Security/, Plugin/ - Strip GitHub issue/GHSA numbers from filenames; keep IDs in test descriptions - Fix all __DIR__ relative paths to match new folder nesting depth - Update CI workflow path triggers and phpunit XML configs for new file locations - Update .github/copilot-instructions.md with unit test organization guidelines - Add tests/vendor/ to .gitignore * add gaurds * fix up guard for unit tests * fix redeclaration of cacti_log in tests * fix updates to unit tests * fix test patjhs * fix: update test __DIR__ paths for reorganized test structure * fix: conditionally define read_user_setting to prevent redeclaration in tests * fix: conditionally define functions to prevent redeclaration in tests - Wrap clean_up_lines, raise_message, cacti_debug_backtrace, and get_debug_prefix with function_exists() checks to prevent conflicts when test stubs are loaded before lib/functions.php in the same PHP process * fix: add unit test bootstrap to prevent function redeclaration errors - Create tests/bootstrap-unit.php to load core libraries before tests - Ensures lib/functions.php loads before test stubs define stub functions - Test stubs use function_exists() checks, so they detect pre-loaded functions - Update phpunit.xml to use bootstrap-unit.php for Unit testsuite - Fixes 'Cannot redeclare read_user_setting()' errors when tests run in same process * fix: load test stubs before global.php in unit test bootstrap - Reorder bootstrap to load CactiStubs.php and UnitStubs.php first - Ensures stub functions like __() are available when global.php loads - global.php calls get_cacti_version_text() which needs __() for translations - Fixes 'Call to undefined function __()' error during bootstrap * fix: remove function conflicts from test stubs - Remove stub definitions for functions already in lib/functions.php: cacti_sizeof, cacti_log, cacti_strtolower, read_config_option, read_user_setting, cacti_count - Keep translation function stubs (__(), __esc()) needed by global.php - Update bootstrap to load lib/functions.php after stubs - This prevents function redeclaration errors while maintaining test isolation * fix: prevent constant and variable redeclaration in bootstrap - Remove explicit PHP_TESTING and CACTI_TEST_BOOTSTRAP defines from bootstrap (CactiStubs.php now handles this with guards) - Add guards to CactiStubs.php to prevent redefinition of PHP_TESTING - Only set CACTI_TEST_BOOTSTRAP env vars if not already set - Fixes PHP warnings about 'Constant X already defined' * fix: remove redundant lib/functions.php load from bootstrap include/global.php already loads lib/functions.php at lines 287-288. The bootstrap was trying to load it again after global.php, but global.php needs format_cacti_version() to be available on lines 293-294 to define CACTI_VERSION and CACTI_VERSION_FULL constants. Solution: Let global.php handle loading lib/functions.php itself, and don't load it separately in the bootstrap. * fix: remove more conflicting function stubs from test helpers - Remove stubs for functions also defined in lib/functions.php: raise_message, clean_up_lines, cacti_debug_backtrace, get_debug_prefix - Remove POLLER_ID constant stub (defined unconditionally by global.php) - Keep only translation functions and test-specific helpers - This prevents function/constant redeclaration errors * fix: minimize test stubs to only translation functions Remove ALL helper functions and constants from UnitStubs.php that are: - Defined in lib/functions.php (raise_message, clean_up_lines, etc.) - Defined in lib/html_utility.php (srv, set_request_var) - Defined in global_constants.php (POLLER_VERBOSITY_*, MESSAGE_LEVEL_*) - Defined in global.php (POLLER_ID) Keep ONLY translation functions (__(), __esc()) that are: - Called by global.php during initialization - Not defined elsewhere - Required for proper bootstrap sequence This eliminates all constant and function redeclaration errors. * fix: remove __esc stub - already defined in global_languages.php Keep only __() translation function stub, which is needed by global.php and not defined elsewhere in Cacti core. * fix: handle no-arg case in __() translation stub The __() function may be called with just a string and no format arguments. Previously, vsprintf() would fail with 'arguments array must contain 2 items'. Now, if no args are provided, return the text as-is. |
||
|
Thomas Vincent
|
d4e255b5d9 |
security: stage nonce CSP migration and gated enforcement (#7662)
* security: add nonce CSP mode for inline JavaScript protection Reworks the content_security_policy_script setting into a three-mode Inline JavaScript Protection dropdown (HTMLX default, Nonce, None) and ports the per-request nonce, strict-dynamic policy, and CSP violation report endpoint from 1.2.x. The stored option key and its default are unchanged, so existing installs keep their current policy byte-for-byte; nonce mode is opt-in. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * security: stage nonce CSP enforcement safely Add report-only migration, gated enforcement, nonce-aware shared emitters, Linux Docker validation, and operator documentation.\n\nRefs #7703 * security: validate CSP header sources Reject malformed alternate-source configurations at the CSP boundary, validate fallback report URIs, and remove the obsolete Psalm header-taint baseline entry.\n\nRefs #7705 * chore: drop the composer.json change from this PR Dependency and manifest changes are a maintainer call and do not belong in this one. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> |
||
|
Thomas VincentandTheWitness
|
f2898730e1 |
fix: guard process registration and cleanup against PID reuse (#7425) (#7574)
* fix: guard process registration and cleanup against PID reuse A bare posix_kill($pid, 0) only proves some process owns the pid, so a recycled one blocked new tasks or drew a SIGTERM meant for something else. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: spawn probe children without a wrapping shell proc_open with a string command runs it via sh, so the child's /proc comm was sh and the identity check correctly refused to signal it on Linux. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: append the changelog entry at the end of the issue block Matches .github/copilot-instructions.md, which appends rather than inserting in numeric order. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: limit the process identity check to CLI callers The registry holds CLI tasks, so a web request comparing command names would never match and would report a running task dead. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: use the full licence header on the new test files Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: treat CACTI_PATH_LOG as a directory and pass ctor args to the statement class Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: make process identity checks self-contained * fix(batchgapfix): guard forced process termination * fix(processes): preserve live pids on permission errors * Refresh security baselines after develop rebase Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Wait for the spawned child to exec before the identity check Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Serve fetch() from the buffered rows in the process registry fake db_table_exists() probes with SHOW TABLES, and db_fetch_row_prepared() reads a single row with fetch(); the fake drained the cursor on execute() and answered neither, so every registry query came back empty and the tests exercised the no-processes-table path instead of the guard. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(changelog): drop entries duplicated by an earlier rebase Only the issue#7425 line belongs to this branch. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(poller): identify a registered process by its executable, not its name Children are spawned from path_php_binary, so a versioned child and a parent started as plain php reported different command names for one interpreter. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(poller): skip the alias check where both paths share a command name Whether a second path to the interpreter yields a second name is the runner's business; asserting it failed where php is already the plain name. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
d0fb966247 |
test: make the poller cache batching test independent of load order (#7582)
* test: make the poller cache batching test independent of load order Its set_config_option() stub only applied when nothing had already loaded lib/functions.php, which the unit suite always has by the time it runs. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: note why the constants are guarded defines rather than a require Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(batching): fail on conflicting constant fixtures * Make batching test log path order independent --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
a75bf823bb |
feat: consolidate htmx loader and renderer abstraction global.php includes (#7517)
* feat(rendering): add view renderer abstraction Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * isolate renderer traversal test file * Standardize renderer headers and endpoint guard * style(renderer): use standard unit-test header in CactiRendererTest Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(renderer): reject null-byte paths before realpath Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh sink and hotspot baselines for line shifts Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(renderer): guard root base path and template-dir redirect Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(renderer): canonicalize both paths in index.php direct-access guard Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * hardening: normalize renderer base path to one separator Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(renderer): reject filesystem root as template path A /-rooted renderer collapsed the containment prefix to /, which str_starts_with() matches for every absolute path, so any file was includable. Fail closed by refusing a root template path. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * hardening: reject windows drive-root template path Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * refactor: adopt PHP 8.4 idioms in view renderer Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: cover renderer abstraction to full coverage Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * hardening(renderer): reject UNC template names and fix sink baselines Refuse a leading-backslash (UNC) template name the same way absolute POSIX paths are refused, skip the direct-request redirect when SCRIPT_FILENAME carries a null byte, and correct the renderer include and views redirect line numbers in the security baselines. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * style: apply php-cs-fixer Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(renderer): use PHP 8.1-compatible readonly property in CactiRenderer Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * hardening(rendering): drop template context array before include Unset the internal context array after extract so an included template only sees the extracted variables, not the full context. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines for renderer line shift Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(renderer): simplify containment check and drain nested output buffers The redundant real_path !== template_path comparison in validateTemplateFile() can never be false once is_file()/is_dir() have run, so drop it and keep the str_starts_with() prefix check as the actual containment guard. renderFile() also only unwound a single output buffer, so a template that opens (and leaves) its own buffer could leak an ob level or drop its content; unwind every buffer opened during rendering and reassemble them in the order they were written. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines for renderer buffer/containment fix The dynamic_include line for lib/renderer.php's include shifted again after adding drainOutputBuffers(). Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * feat(htmx): vendor htmx 2.0.6 and add opt-in filter loader Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: cover htmx filter loader Add missing-vendored-file case for htmx_script_tag and mark the pre-bootstrap read_config_option guard as uncoverable. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: harden htmx loader harness and tighten enable check Skip anything but an exact 'on' for htmx_enabled, reject 4xx/5xx fixture responses, fail fast when the fixture base path is unresolved, pin setup-php to a commit SHA, and reindent the e2e specs to tabs. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * style: apply php-cs-fixer to htmx test files Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(htmx): drop duplicate script-tag src assertion The subdir-install test repeated the preceding assertion; CACTI_PATH_URL is fixed at bootstrap so it could not exercise a distinct scenario. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(e2e): drop sqlite3 from docker-php-ext-install php:8.2-apache already ships sqlite3 built in, and its ext/sqlite3 source only carries config0.m4 (not config.m4), so building it standalone always fails with "Cannot find config.m4", breaking the e2e image build before Playwright ever runs. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(htmx): clear stat cache around version-file rename Earlier tests already stat the version file via file_exists()/ file_get_contents(), so PHP's stat cache can report it present right after these tests rename it away, making the assertions flaky. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(htmx): recompute script-tag cache-buster per call The md5 cache-buster was cached in a static, so FPM/script_server processes kept serving a stale ?v=... hash after an htmx.min.js upgrade. Recompute it per call, matching get_md5_include_js() and get_theme_paths() in lib/functions.php, and drop the 'v=' parameter name to match their path + '?' + hash shape. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(e2e): drop obsolete MariaDB options * ci(htmx): run the htmx integration test by path instead of a testsuite filter Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * build: vendor htmx unminified Debian rejects minified sources. The file is the htmx 2.0.6 dist source; its minified sibling hashes identically to what was committed, and the pinned SRI is recomputed over the unminified file. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: address htmx review feedback * test(e2e): resolve Cacti version through PHP * test(htmx): harden portable server cleanup * ci(actions): update checkout action pins --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
e8ec3ca0ec |
ci: consolidate pest coverage gate re-enable and validator constraint ratchet (#7516)
* ci: consolidate coverage and validator quality gates * fix(audit): handle opening and multiline comments * style(tests): align shared Orb assertions * test(orb): require the configured runtime machine * revert(tests): leave macOS Orb integration out of scope * fix: Fix ci update * fix: Remove imagedestroy calls from merge conflict --------- Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas Vincent
|
cde946d9dd |
fix: repair regressions found in recent develop audit (#7614)
* Fix regressions found in recent commit audit * fix(database): fetch only the requested first row * fix(tests): use native PDO fetch behavior * test(orb): clarify unavailable runtime skips * test(database): assert malformed metadata query path * revert(tests): leave macOS Orb integration out of scope |
||
|
Thomas Vincent
|
906d19ca72 | fix(reports): consolidate image conversion and notification payload hardening (#7515) | ||
|
Thomas VincentandTheWitness
|
fd111a1900 |
fix: reset loop-scoped $oid and $script_path when building the poller cache (#7135) (#7570)
* fix: reset loop-scoped $oid and $script_path when building the poller cache Both output loops assigned inside a guard but read after it, so an output with no mapping was cached against the previous output's OID or script. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: append the changelog entry at the end of the issue block Matches .github/copilot-instructions.md, which appends rather than inserting in numeric order. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: use the full licence header on the new test files Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: treat CACTI_PATH_LOG as a directory and pass ctor args to the statement class Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: reset data-source output loop state * test(poller): isolate SQL dialect in cache regression --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
4d211093c0 |
fix: remove plugin table entries left behind by uninstalled plugins (#7202) (#7579)
* fix: remove plugin table entries left behind by uninstalled plugins A plugin directory can disappear without its uninstall running, and the realm rows it leaves carry user and group grants a later plugin would inherit. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: append the changelog entry at the end of the issue block Matches .github/copilot-instructions.md, which appends rather than inserting in numeric order. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: use the full licence header on the new test files Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: treat CACTI_PATH_LOG as a directory and pass ctor args to the statement class Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: delete orphan realm rows set-based rather than three statements each The cleanup runs on every render of the plugin page, so the per-orphan loop was an N+1 in the page path. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: bind the orphan realm ids instead of interpolating them db_in_clause() builds the list into the statement, which the SQL interpolation ratchet counts; a sized placeholder list keeps them bound. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
96f03780f9 |
fix: roll monthly schedules over to the next year instead of resolving to false (#7429) (#7568)
* fix: roll monthly schedules over to the next year instead of resolving to false A schedule whose selected dates had all passed returned false, which the caller stored as a 1970 next_start and then read as overdue every cycle. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: add phpdoc to the issue 7429 test helpers Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: append the changelog entry at the end of the issue block Matches .github/copilot-instructions.md, which appends rather than inserting in numeric order. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: check the calculated start strictly against false empty() would also treat a timestamp of 0 as unschedulable, which is a different state from having nothing to schedule. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: use the full licence header on the new test files Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: treat CACTI_PATH_LOG as a directory and pass ctor args to the statement class Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
453e747d45 |
fix: update device status by host id instead of hostname (#7427) (#7567)
* fix: update device status by host id instead of hostname Hostnames are not unique in Cacti, so keying the status write on hostname let sibling devices overwrite each other's availability every poll. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: add phpdoc to the issue 7427 test helpers Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: append the changelog entry at the end of the issue block Matches .github/copilot-instructions.md, which appends rather than inserting in numeric order. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: use the full licence header on the new test files Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: treat CACTI_PATH_LOG as a directory and pass ctor args to the statement class Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: restore host-status config state exactly --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
TheWitness
|
096b002da5 | ci: Disable two tests until fixed (#7587) | ||
|
Thomas VincentandTheWitness
|
db39178735 |
fix: harden the poller_item cache and Boost RRD staging subsystems (#7538)
* fix(boost): forward still-open-round archive rows instead of dropping them boost_process_poller_output() correctly excluded rows from an in-progress poll round when building the RRD write, but the archive-table cleanup right after deleted every row for the local_data_id anyway, with no time filter. Those rows now get INSERT IGNORE'd into the live poller_output_boost table before the archive table is cleared, so a still-open round's data survives. Also fixes the duplicate-record guard in the same function: $last_item was initialized once before the results loop and never reassigned, so the comparison against it could never be true. It is now updated after each processed row. Fixes #7519 Fixes #7523 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): gate archive-table drop on shard completeness poller_boost.php decided whether to drop a run's archive tables using SELECT SUM(status) FROM poller_output_boost_processes, an aggregate over whatever completion rows landed. If one child crashed before writing its completion row, the sum from the surviving shards was still positive and every archive table for the run got dropped, including the crashed shard's unprocessed data. The drain loop already computes boost_completed_children() vs $expected_children to detect a crashed child; the drop now uses that same completeness check instead, and leaves the archive tables in place with a warning when it is not met. Fixes #7520 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(dsstats,rrdcheck): apply Boost's startup barrier to launch loops dsstats_launch_children()/rrdcheck_launch_children() fork children via exec_background() (non-blocking) and the callers then wait with a bare while ($running = ..._processes_running($type)) { sleep(2); }. If the parent evaluates that before any child has registered in the processes table, $running reads 0 and the loop body never runs, treating "not started yet" as "already done" -- the same race Boost's boost_all_children_registered() barrier was added to close. dsstats_launch_children()/rrdcheck_launch_children() now return the launched count, and dsstats_wait_for_children()/ rrdcheck_wait_for_children() wait for the running count to reach it (bounded by a deadline) before trusting the drain loop's running-count poll. Fixes #7526 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * refactor(poller): consolidate poller_item deletes behind a shared helper Extract poller_item_delete_for_data_source() and poller_item_delete_for_host() in lib/poller.php and migrate the existing local+remote delete pairs in lib/api_device.php and lib/api_data_source.php onto them, so the local-delete-plus-remote-mirror pattern lives in one place instead of eight. Refs #7535 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(poller): mirror poller_item deletes to the remote poller on handoff api_data_source_change_host(), data_query_remove_disabled_items(), and data_source_to_data_template() each moved or removed a data source's poller_item row locally but never on the remote Data Collector that still owned it, so that Collector kept polling an item it no longer should. All three now resolve the owning remote poller before the local change and purge the row there too via the shared helper. Fixes #7521 Fixes #7524 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(poller): correct poller_item join semantics and cron reset in cmd.php poller_item and host share ~13 identically-named snmp_*/hostname columns; SELECT * across the join let PDO FETCH_ASSOC silently collapse them to host's live values instead of poller_item's cached copies. Select pi.* plus explicitly aliased host/site disabled columns instead. The now already-fetched $poller_items also makes the separate script-server-call COUNT(*) re-join redundant, replaced with an in-PHP count. Also apply the same cron_interval-aware rrd_next_step reset used by the ranged branch to the --allhost branch, which previously ignored it. Fixes #7522, #7525, #7528. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(poller): thread known poller_id through update_poller_cache() callers update_poller_cache() re-derived poller_id via a host/data_local JOIN on every call, even when the caller already had it: repopulate_poller_cache() and push_out_data_input_method() both already select h.poller_id and discarded it, push_out_host() has the host row cached per data source, and api_device_enable_devices()/update_poller_cache_from_query() compute it once outside their loops. Add an optional $poller_id argument (and honor an already-present 'poller_id' array key) so these callers skip the redundant query. Fixes #7527. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(poller): scope global aggregate scans and make process-leveling sargable Three poller.php queries (total_snmp_ports and the mutually-exclusive active_profiles pair) had no poller_id filter, unlike the rest of the file's central-only blocks, so every remote poller redundantly recomputed and overwrote the same global config options; the snmp_port scan is also an unindexed full table scan. Gate them behind poller_id == 1, with remote pollers reading back the value the central poller already stored. Also rewrite the process-leveling $sql_where's rrd_next_step predicate from the non-sargable `rrd_next_step - $poller_interval <= 0` to the algebraically equivalent `rrd_next_step <= $poller_interval`, restoring use of the poller_id_rrd_next_step index. Fixes #7529, #7530. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(poller): batch update_data_query_cache()'s poller cache writes Each changed data source called update_poller_cache($data_source, true), which flushes immediately via poller_update_poller_cache_from_buffer() -- its own present=0 UPDATE, INSERT ON DUPLICATE KEY UPDATE, and stale-row DELETE -- on every iteration. Re-indexing a host where every port changed issued up to 3x as many statements as ports instead of one batched flush. Accumulate changed rows across the loop and call update_poller_cache() with commit=false, then flush once via poller_update_poller_cache_from_buffer() after the loop, matching the batched pattern already used by push_out_data_input_method(). Fixes #7531. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): add USING BTREE to poller_output_boost_local_data_ids indexes MEMORY-engine indexes default to HASH unless USING BTREE is specified, unlike the sibling poller_output/poller_output_boost tables in the same schema file. boost_output_rrd_data()'s ORDER BY/LIMIT query and boost_process_local_data_ids()'s ranged DELETE both need to scan local_data_id in order, which a HASH index can't serve, forcing a filesort on every boost worker pass. Verified against a real MariaDB 11 instance: EXPLAIN on the ORDER BY query shows "Using filesort" with the old HASH schema and no filesort with USING BTREE, using the primary key for ordering instead. Also aligns boost_prepare_process_table()'s dead CREATE TABLE IF NOT EXISTS fallback with the corrected schema, and adds an upgrade step that rebuilds both indexes as BTREE on existing installs (checked via information_schema.STATISTICS, since db_install_add_key() only compares index columns, not algorithm, and would otherwise skip an existing HASH index with the right column list). Closes #7532 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): skip the UNION ALL wrapper for the single-archive-table case boost_process_local_data_ids() always wrapped its flush query in "SELECT * FROM (... UNION ALL ...) t ORDER BY ...", even when there was only one archive table -- the steady-state case, since old archive tables are dropped at the end of each cycle. UNION ALL results wrapped in a derived table are one of the documented cases MySQL/MariaDB's derived-table merging doesn't apply to, so every pass paid for a filesort over an indexless derived table regardless of how well-ordered the underlying archive table's rows already were. Extracts the per-table SELECT into boost_archive_select_sql() and uses it directly, without the wrapper, when there's exactly one archive table. The UNION ALL/derived-table shape is kept as the fallback for the genuinely rare case where a prior crashed run left more than one archive table behind. Verified against a real MariaDB 11 instance and via a SQLite-backed integration test that both shapes return correctly filtered and sorted rows. Closes #7533 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): batch-prefetch RRD field name lookups instead of per-row queries Two N+1 patterns in the Boost hot paths: boost_process_local_data_ids() (poller_boost.php) re-ran the unused_data_source_names/nt_rrd_field_names lookups on every local_data_id boundary in the sorted flush result set. process_poller_output() (lib/poller.php) had the equivalent lookup, but without even a boundary check -- it re-ran per matching row whenever the output contained a ':', including consecutive rows for the same data source. Adds poller_prefetch_rrd_field_names() (lib/poller.php, shared by both callers since poller_boost.php already requires lib/poller.php) to batch-fetch these mappings once per invocation via IN()-batched queries keyed by the full set of local_data_ids in the batch, chunked at 1000 ids to match the array_chunk(..., 1000) convention used elsewhere (lib/api_data_source.php, lib/api_graph.php). Both call sites now look up the prefetched maps in-memory inside their loops. While consolidating the lookups, fixes a latent bug the old process_poller_output() code had in its non-templated fallback branch: a query referencing gti.task_item_id without joining graph_templates_item, which throws an unknown-column error whenever that path is hit. The batched query always joins correctly. Integration test proves both correctness (against a real SQLite-backed schema) and the query-count reduction: 5 local_data_ids that would previously cost up to 10 queries for these two lookups now cost exactly 3, regardless of how many ids are in the batch. Closes #7534 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * refactor(boost): consolidate poller_output_boost inserts into one helper The INSERT INTO poller_output_boost (...) VALUES ... statement was hand-written four times: three in cmd.php (behind the same boost_redirect == 'on' && boost_rrd_update_enable == 'on' guard) using INSERT IGNORE, and once in boost_poller_on_demand() (lib/boost.php) using ON DUPLICATE KEY UPDATE. Beyond the duplication, that was a real data-integrity inconsistency between two paths writing the same shape of data: silently-drop vs last-write-wins on the same key. Since boost_redirect gates cmd.php's writes and boost_poller_on_demand() returns early when boost_redirect is on, the two paths never run concurrently for the same install; they're alternate write paths selected by that one setting, not competing writers. A genuine collision on (local_data_id, rrd_name, time) in either path represents a retry of an already-collected sample -- a batch resent after a partial failure, or two writers racing on the same rounded second -- not two independently meaningful readings, since RRD only stores one value per timestamp regardless. Standardizes on INSERT IGNORE (first write wins), matching the sibling poller_output insert this table shadows, which has always used INSERT IGNORE, and 3 of the 4 existing call sites. Extracts boost_flush_output_batch() (lib/boost.php) as the one shared helper, handling both the batched INSERT and the max_allowed_packet-safe chunking cmd.php's three sites previously didn't do at all (they only chunked by item count). Integration test exercises a genuine duplicate-key collision against a real (SQLite-backed) table and asserts the chosen first-write-wins behavior. Closes #7536 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(tests): resolve cross-branch test interactions found by full-suite verification Two test-only bugs surfaced only when all four fix branches for the poller_item/Boost hardening pass ran together, not when each ran in isolation: - BoostArchiveHandoffIntegrationTest extracted the archive-forward INSERT IGNORE by searching lib/boost.php for the first occurrence of "INSERT IGNORE INTO poller_output_boost". The Boost SQL-optimization branch adds a second, earlier occurrence (the new boost_flush_output_batch() helper's single-line prefix), so the naive search now grabbed the wrong snippet. Anchor on the multi-line "...poller_output_boost\n\tSELECT *" shape that only the archive-forward statement has. - PollerItemRemotePurgeHandOffTest calls push_out_host() (via api_data_source_change_host()), which needs CACTI_PATH_LIBRARY. The test never requires include/global.php (which normally defines it), and was relying on some other file in a full multi-thousand-file suite run having defined it first as a process-wide side effect. Define it directly instead of depending on unguaranteed load order. No production code changed. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * style(poller): satisfy php-cs-fixer strict_param in the script-server count Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): order the single-archive-table flush by the table's PK columns Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): key the max_allowed_packet cache per connection Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(tests): restore the default db_* session after swapping it for a fake handle Left swapped, the fake handle answered every later read_config_option() in the run; on PHP 8.4 the extra deprecation traffic routed through CactiErrorHandler turned that into an uncaught PDOException and killed the pest process. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
7c5e4c102a |
feat(support): consolidate diagnostics, lockout CSRF, process-table hook, and safe-triage (#7514)
* feat(support): add redaction toggle and recent-log panel Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: cover support safe triage Unit-test the redaction helper, severity log tail and environment table, and add a Docker probe that the redacted report masks the RSA fingerprint and hides the database host and password. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): escape rsa fingerprint and default unknown poller type Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): redact compressed IPv6 addresses Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: skip tests tree in hardening pattern guard Test fixtures assign request superglobals to exercise product code; the guard only protects product code. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): drop redundant isset flagged by phpstan Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * feat(support): guard snmpget path before executing Reject a null byte and require a regular executable file for path_snmpget so a misconfigured path cannot throw a ValueError or match a directory. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines for snmpget path guard Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): redact FQDNs before the node name and stop truncating tool output Masking the node name first could leave a domain suffix unmasked when the name was a prefix of an FQDN, and strip_tags() on installed SNMP output dropped legitimate angle-bracket content instead of just masking it. Also tighten the redact toggle to a strict '1' match, carry it through page-refresh URLs, and correct the diagnostics report comment to describe what actually gets masked. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): guard poller type lookup, log stats, and binary path checks Fall back to an "Unknown" label when poller_type doesn't match a known poller instead of an undefined-index warning, suppress warnings from filesize()/fopen() if the log file disappears between the caller's check and the open, and require is_file() plus a null-byte guard on the required-binaries version check (matches the snmpget check next to it). Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs(tests): align triage probe comments with actual assertions The header comments claimed the unredacted report exposes the DB host and password, but the probe only ever asserts the RSA fingerprint; DB credentials are checked for absence when present, not required to appear unmasked. Correct the wording in both files. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): require POST + CSRF token for maintenance lockout, compare-and-set Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): require a valid expected state for lockout compare-and-set Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(security): refresh sink/hotspot baselines for upstream line shifts Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): clamp page offset, quote SQL literals, suppress lockout URL state Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: cover support lockout csrf guard Exercise the POST-only guard and compare-and-set branches, and add a Docker probe that a GET or tokenless POST cannot change the lockout state. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(support): guard lockout probe extraction and correct probe scope note Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: skip tests tree in hardening pattern guard Test fixtures assign request superglobals to exercise product code; the guard only protects product code. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): keep lockout e2e health poll alive under pipefail Tolerate grep returning non-zero before the Health field appears so the poll loop waits instead of exiting the script. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(support): add integration coverage for the lockout compare-and-set The Unit test cannot reach the "currently locked" branch (forced reads return the PHP_TESTING default) and the e2e probe deliberately never locks the system, so the real read/write round trip against cacti_lockout_status has no coverage. Add a fast integration test that drives the lock, a stale-expected rejection, and the unlock against a real database, skipping gracefully when none is reachable. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(support): match the page-clamp assertion whitespace-tolerantly Whitespace-exact substr_count() breaks on reformatting; the regex enforces the same guard without depending on exact spacing. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): harden the lockout e2e probe's auth and cookie-jar handling Assert a CSRF token was extracted and the session actually authenticated before running the lockout checks, otherwise a failed login still passes. Also handle tempnam() failure and clean up the cookie jar on every exit path via a shutdown handler. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * feat(support): add diagnostics export and environment checks Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: cover support diagnostics export Unit-test the environment table and report redaction, and add a Docker probe that checks the report never leaks the database host, password or full RSA fingerprint. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): guard null snmp version and copy-status styling Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): keep diagnostics textarea visible on copy failure and report real extension load state Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): verify PHP extensions in both cli and web contexts show_tech_environment() only checked utility_php_verify_extensions()'s web flag, which seeds CLI-only extensions like pcntl as satisfied without checking whether they're actually loaded. Shell out to cli_check.php for the real CLI state and combine it with the web state the same way lib/installer.php does, so status reflects reality in both directions. Also guard the source-extraction helpers in SupportDiagnosticsTest and the e2e probe against a moved/unreadable support.php instead of letting strpos(false, ...)/eval() fail confusingly. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): require POST + CSRF token for maintenance lockout, compare-and-set Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * refactor(support): register process tables via support_process_tables hook Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): guard non-array hook result and empty process-table set Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): cast poller_interval to int in SQL and validate hook entry types Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(security): refresh sink/hotspot baselines for upstream line shifts Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): reuse forced admin lookup and coerce expected state Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: cover support process-table hook Verify the hook drops malformed plugin definitions and keeps valid ones, and add a Docker probe that the hook returns well-formed tables against a real database. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(support): guard hook extraction and tolerate empty health probe Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): escape plugin-contributed process-table cells Plugins can now supply the process-table SELECT fragments via the support_process_tables hook, so escape the rendered cell values, and install the e2e cleanup trap before backing up config.php. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines for support.php line shift Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): use form_selectable_ecell for process rows and cast pid in row id Plugin-contributed process-table values were only escaped via manual html_escape() calls, and the row id was built from an unsanitized pid; switch to the file's existing form_selectable_ecell() helper and cast pid to int before it goes into the id attribute. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(support): harden hook-call rewrite and narrow validation comment Replace the exact-string str_replace() with a regex plus a match-count assertion so the probe fails loudly instead of silently no-op'ing if support_process_tables() formatting drifts, and correct the header comment's claim that malformed definitions can't corrupt the UNION -- the validation only checks types, not SQL validity. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(support): make e2e hook extraction regex whitespace-tolerant The extraction pattern required exact spacing around the function signature; match flexible whitespace instead and fail loudly if preg_match doesn't return exactly one match. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(support): add integration coverage for the process tables SQL path Unit tests eval() support_process_tables() with the plugin hook stubbed and never touch a database; the e2e Docker probe calls the same function directly but never builds or executes the UNION show_cacti_processes() runs. Add a fast, DB-only integration test that executes the real query against a live schema, skipping gracefully when no database is reachable. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(support): harden process table hook contracts * docs: add changelog entry * style: satisfy support hook fixer rules * fix(support): reconcile escaping contract and diagnostics test after merge tech_env_status() now escapes its $text argument (safe-triage's design, tested by SupportSafeTriageTest); strip the now-redundant html_escape() calls at its call sites in show_tech_environment() to avoid double encoding. Update SupportDiagnosticsTest's source-extraction marker and eval() inputs to match the merged, redact-toggle-aware report block from safe-triage. Refresh the sink inventory baseline for the resulting line shift. * fix(support): escape the PHP uname value on both redact paths Hostnames and kernel strings can carry markup, and support_redact() itself substitutes <host>, so redact first and escape the result either way. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(support): locate the diagnostics report block by regex, not literal The $snmp_line assignment became a conditional and the literal start marker stopped matching, aborting the probe before it built a report. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: append the process tables entry to the feature block Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
66c00774c6 |
feat: batch the graph-create task_item_id wiring (#7380) (#7383)
* feat(template): batch the graph-create task_item_id wiring (#7380) Pre-fetch the graph-item and data-source id maps in bulk and extract the wiring into graph_template_connect_task_items(), replacing the per-item SELECT pair plus UPDATE with one UPDATE per item. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * perf(template): build task_item_id IN() via db_in_clause Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(7380): assert bulk rrd fetch via db_in_clause marker Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: order bulk graph-item prefetch by id Add ORDER BY id to the new-item prefetch queries so the first-match per key is deterministic and matches the previous db_fetch_cell() behavior. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(7380): add integration coverage for real ORDER BY dedupe behavior Exercises graph_template_connect_task_items() from lib/template.php directly (not an eval'd extract) against a real SQLite-backed PDO connection, proving the ORDER BY id clauses -- not insertion order -- decide which row wins the first-match dedupe. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: add changelog entry * fix(template): revert local_data_id IN() to implode idiom TheWitness asked to revert the bulk rrd prefetch to the previous implode/intval style used elsewhere in this file instead of db_in_clause(); update the matching test assertion accordingly. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * style(template): hoist bulk graph-create fetches out of the foreach headers Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
aa93dc247d |
fix(aggregate): restore item-override saves and bulk-action input names (#7338)
* fix(aggregate): restore item-override saves and bulk-action input names The form_save() gate dropped save_component_item during the develop refactor, so item-override saves redirected away unsaved, and the migrate/combine confirmation dialogs posted their input as title_format while the handlers read aggregate_template_id and aggregate_name. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: relax aggregate form-wiring regex assertions Match the gate and extra-array checks without pinning term order or sub-array key position. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(aggregate): surface error when aggregate template is missing Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(aggregate): catch title_format extra input in any key position Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: add space before return-type colon Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(aggregate): validate request input on convert and combine paths Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh security baselines Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(aggregate): verify convert-template guard writes nothing on bad template tests/Unit/AggregateFormFieldNamesTest.php only regex-matches the guard clause in source; this runs api_aggregate_convert_template() against a real sqlite-backed connection and confirms a missing/unmatched aggregate_template_id leaves aggregate_graphs and aggregate_graphs_items untouched. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci(hardening): allow test-fixture superglobal writes in aggregate guard test tests/integration/AggregateConvertTemplateGuardIntegrationTest.php sets $_REQUEST directly to simulate request state before calling the real api_aggregate_convert_template(), which reads aggregate_template_id via gnrv(). Same pattern already used in tests/HandOff/CactiDispatchHandOffTest.php and tests/HandOff/CactiSettingsHandOffTest.php. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(aggregate): restore db globals after convert-template guard test issue7338_wire_default_connection() overwrote $database_hostname/ $database_port/$database_default and added a fake PDO to $database_sessions without ever resetting them, so any later test in the same Pest process that hit a db_* helper resolved against the stale sqlite handle instead of its own fixture -- which crashed the whole suite before the HandOff tests ran. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Change function to retrieve aggregate template ID --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Abdul MohammadandTheWitness
|
7cb994d36c |
fix(installer): recover from expired CSRF tokens (#7392)
* fix(installer): recover from expired CSRF tokens Return a replacement token for expired installer AJAX requests, retry the failed action once, and roll tokens forward after successful responses. Fixes #7343 Signed-off-by: Abdul Mohammad <abdul.mohammad9087@gmail.com> * test(installer): exercise CSRF recovery round trip Signed-off-by: Abdul Mohammad <abdul.mohammad9087@gmail.com> --------- Signed-off-by: Abdul Mohammad <abdul.mohammad9087@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas Vincent
|
2d623c66f8 |
Parameterise and harden aggregate_graphs.php (#7261)
* security(aggregate): parameterise form_actions and purge_old_graphs IN() lists
The template-migration queries in form_actions() and the purge_old_graphs()
DELETE queries built IN() lists with implode(','); they now use db_*_prepared()
with placeholder arrays. aggregate_items() rfilter and aggregate_graph_id
parameterisation already landed on develop via the #7254/#7260 filter refactor,
so this keeps only the remaining unique fixes plus the gfrv()/rawurlencode() URL
hardening.
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
* test: use whitespace-tolerant regex for grv/gfrv assertion
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
* test(security): verify purge_old_graphs' parameterized deletes against real SQL
tests/Unit/InClausePlaceholderTest.php only checks the placeholder-count
arithmetic; it never runs a prepared statement, so it can't catch a
mismatched id list feeding the wrong query. This extracts the real
purge_old_graphs() from aggregate_graphs.php and runs it against a
sqlite-backed connection across five joined tables.
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
* ci: refresh SQL interpolation ratchet baseline for aggregate_graphs parameterization
Rebasing onto develop pulled in prior parameterization work (#7262,
with; aggregate_graphs.php now drops out entirely after this branch's
own db_* parameterization.
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
* test(aggregate): restore db globals after purge_old_graphs integration test
issue7261_wire_default_connection() overwrote $database_hostname/
$database_port/$database_default and added a fake PDO to
$database_sessions without ever resetting them, so any later test in the
same Pest process that hit a db_* helper resolved against the stale
sqlite handle instead of its own fixture -- which crashed the whole
suite before the HandOff tests ran.
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
* docs: add changelog entry
* docs: fix changelog entry format
---------
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
|
||
|
Thomas Vincent
|
cbd711869c | test: run the Integration and HandOff suites in isolated processes (#7550) | ||
|
Thomas VincentandTheWitness
|
e96d791e2b |
fix(auth): validate measure/order before ORDER BY concatenation in get_allowed_graphs() (#7438)
* fix(auth): validate measure/order before ORDER BY concatenation in get_allowed_graphs()
get_allowed_graphs() built its dsstats ORDER BY clause by concatenating
$sql_order['measure'] and $sql_order['order'] directly into SQL. Both
values trace back to unfiltered request variables (grv('measure'),
grv('graph_order')) at several call sites in html_graph.php and
html_tree.php, none of which register the vars in a filter array.
Reachable pre-auth via graph_view.php when dsstats_enable and the guest
account are on.
Add get_dsstats_order_string() in lib/html_utility.php, next to the
existing sort_column/sort_direction validator get_order_string(), and
have get_allowed_graphs() call it instead of trusting the raw array.
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
* refactor(auth): validate dsstats measure/order via CactiValidator + backed enum
get_dsstats_order_string() used a hand-rolled in_array() whitelist instead
of this codebase's established CactiValidator/Symfony Assert pattern
(~334 existing call sites). Adds a DsstatsMeasure backed enum as the
single source of truth for legal measure values, feeding an
Assert\Choice constraint checked via CactiValidator::isValid() (not
validateInput(), which writes to $_SESSION and isn't appropriate for
this pure helper).
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
* fix: guard dsstats order/measure against array input and missing require
Addresses Copilot review: require html_utility.php where
get_dsstats_order_string() is called from lib/auth.php, and reject
non-scalar measure/order values before validating/concatenating them.
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
* test(auth): add integration coverage for dsstats ORDER BY fragment build
Exercises get_dsstats_order_string() together with dsstats_get_best_partition()
and db_qstr(), the other real functions get_allowed_graphs() calls on the
same path, reproducing the JOIN + ORDER BY fragment it concatenates.
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
* style(auth): order use import before require_once in html_utility.php
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
---------
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Co-authored-by: TheWitness <thewitness@cacti.net>
|
||
|
Thomas VincentandTheWitness
|
bc12e4492e |
Fix TypeError in boost_graph_set_file when sending reports (#7268)
* Fix TypeError in boost_graph_set_file when sending reports reports.php passes '' for rra_id; normalize to int|null in rrdtool_function_graph before it reaches boost. Ref #7214 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Restrict rra_id normalization to whole numbers Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: bound page crawler network retries * test: match rra_id source anchors to ctype_digit or filter_var Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: accept ?int spelling of the rra_id contract Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: refresh sink and hotspot baselines for line shifts Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
1a599a3700 |
Fix aggregate totalling legend showing untotalled values (develop) (#7269)
* Fix aggregate totalling legend showing untotalled values Same fix as the 1.2.x change for #7216; the exclusion came from #6842. Ref #7216 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: place CHANGELOG entry in release order --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
53d7962b88 |
Allow installer to repair missing Composer vendor tree (#7325)
* Allow installer to repair missing vendor dependencies * Apply project formatting to guarded autoload --------- Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
438ae8cdb8 |
Warn when data input commands contain unresolved substitution variables (#7270)
* Warn when data input commands contain unresolved substitution variables Unresolved <field> tokens were silently nulled out, shifting the script's positional arguments with no trace in the log. Ref #3744 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Add device context to the unresolved substitution warning Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: place CHANGELOG entry in release order --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
1565f495a7 |
Correct the Nth percentile divisor on base-1024 graphs (#7273)
* Correct the percentile divisor on base-1024 graphs 10.24^power is not 1024^(power/3); printed values ran about 9% low at power 6. Ref #7211 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: place CHANGELOG entry in release order * docs: keep issue entries before feature entries --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas Vincent
|
10fba168a0 | Guard startup on missing vendor and repair incomplete trees at install (#7272) | ||
|
Thomas VincentandTheWitness
|
a610887bc9 |
Add Composer binary path setting with installer detection (#7271)
* Add Composer binary path setting with installer detection First half of #6456: the path setting, auto-detection, and a runnability probe on the Binary Locations step. Ref #6456 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Normalize Windows path in composer probe Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Use line comment per cs-fixer Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Re-ordering issue per spec ChangeLog is order as follows: 1. Issue first, in numeric order 2. Features next in numeric order 3. For issues and features that do not have a number, they should be at the bottom of each section. --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
Thomas VincentandTheWitness
|
37fe6eab9d |
Gate new unparameterised SQL value interpolation (#7260)
* test(security): add SQL interpolation classifier and ratchet baseline Tokenizes each db_fetch/db_execute call and classifies its SQL argument as prepared, static, qstr-escaped, opaque, identifier or value. The value bucket is the migration target: a raw variable interpolated into value position. The baseline records the current per-file value count so CI can block new ones. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci(security): gate new SQL value interpolation in quality workflow Runs the interpolation audit in --check mode so a pull request that adds an unparameterised value interpolation fails until it uses db_*_prepared(). Pins setup-php to a commit SHA. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * refactor(security): split prebuilt SQL fragments from scalar interpolation Cacti names query fragments consistently (sql_where, sql_join, ...). A variable with such a name is a SQL snippet, not a bindable scalar, so it now lands in a separate fragment category instead of inflating the value count. Extracts scan_source() and a library-only include guard so the classifier can be unit tested. Baseline value count drops from 373 to 300. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(security): cover the SQL interpolation classifier Unit tests exercise each category (prepared, static, qstr, value, identifier, fragment, opaque) against snippets, including IF EXISTS identifiers and mixed scalar/fragment statements. An integration test runs the CLI against the tree and asserts the committed baseline stays in sync. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(tooling): gate chdir to CLI runs and align checkout version The audit is require'd by the unit tests in library mode; running chdir() at load time leaked a working-directory change into other tests. Gate it behind the library sentinel. Also aligns the ratchet job's checkout to the version the rest of the workflow uses. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * style(tooling): use PHP 8 string helpers and short array syntax in audit Switches the classifier to str_starts_with/str_ends_with/str_contains and short array and list syntax, matching the idiom the rest of the codebase uses. Output is unchanged. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix installer failure handling * style(lib): align installer assignment per cs-fixer Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |
||
|
|
61a542a670 |
fix(auth): correct auth system bugs (#7187)
* fix(auth): correct auth system bugs Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Address auth PR review findings Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Guard remember cookie cache deletion Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(auth): prevent remote agent lockout on FCrDNS failure and add regression tests Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: update CHANGELOG for auth system correctness fixes Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(auth): stop returning 2fa secret from verification Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(auth): reject remote agent requests that fail forward-confirmed reverse DNS Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci: ignore phpstan function.alreadyNarrowedType identifier Clears the level 6 alreadyNarrowedType error on lib/data_query.php:2591 that develop currently emits, matching the sibling narrowing identifiers already ignored. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: reword FCRDNS change as a refactor, not a security fix Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
Thomas VincentandTheWitness
|
ae4c147f04 |
chore(test): unified Pest test infrastructure with HandOff support (#7124)
* test: shared hand-off + mutation infrastructure for feature PRs Consolidates infection/infection dev dep, allow-plugins entry, baseline infection.json5, tests/HandOff/HandOffHelpers.php with reusable stubs (cacti_log capture, temp-file fixtures, minimal-zip builder), HandOff testsuite registration, and the documented pattern. Feature PRs that add hand-off coverage rebase onto this and contribute only ONE HandOffTest file plus an optional infection.json5 filter override. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: consolidate per-feature hand-off suites into shared infra Each test guards on its feature file and skips when not present on develop. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(test): replace eval stub, unify log buffer ref, guard ZipArchive add Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(ci): correct infection vendor path in infection.json5 Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * ci(phpstan): baseline upstream undefined-variable and isset errors Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: add HandOff and Integration testsuites; add regression guards - phpunit.xml: add Integration testsuite alongside existing Unit/HandOff - tests/Integration/: DbDumpIntegrationTest, PingIntegrationTest (from develop), CactiProcessIntegrationTest, SqlScriptsIntegrationTest - tests/HandOff/RegressionGuardTest: guards against backsliding on shell_exec, password-in-argv, RLIKE concat, and open-redirect patterns Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * chore(test): pest test infrastructure and PSR-4 autoload Add Pest 2 dev dependency, regenerate composer.lock under PHP 8.1, declare PSR-4 autoload for the Cacti namespace, and seed the test bootstrap with three unit tests plus an orb integration check. Pest stays on the v2 line because v3 transitively requires PHP 8.2 and breaks the 8.1 matrix entry. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * chore(phpstan): catch up baseline with 11 upstream-detected entries Cacti Commit Audit fails on develop with 11 PHPStan errors that are not yet in phpstan-baseline.neon. Append the matching entries so the PR-A branch passes Run PHPStan at Level 6. Same set of entries already exists on PR #7077; this is a transient catch-up that upstream will absorb when those entries land on develop. Files: aggregate_graphs.php, color_templates.php, graph_templates.php, graphs.php, lib/html.php (3 entries). Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * chore(test): drop composer.lock; let CI resolve per PHP version The lock file pinned brianium/paratest v7.3.1 which only supports PHP 8.1-8.3, so the 8.4 matrix entry rejected the lock with "Your lock file does not contain a compatible set of packages". A single lock cannot satisfy 8.1 (paratest 7.3.x) and 8.4 (paratest 7.4.x) at the same time because pestphp/pest 2.36.0 is the last 2.x release that supports 8.1 and the 8.1-compatible paratest tag predates 8.4 support. Match upstream develop's behaviour: no committed lock; let `composer install` in CI resolve per matrix entry. Pest stays on the v2 line in composer.json so 8.1 still gets a compatible Pest. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(test): apply Copilot review feedback for test infra Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * chore(test): drop psr-4 autoload and align deps with canonical PRs Match the flat un-namespaced lib/CactiX.php convention used by #7088, #7073, #7077, #7075. Drop symfony/process (added by #7073) and symfony/validator (added by #7077) to avoid composer.json conflicts at merge time. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: skip integration suites and regression guards pending feature PRs Guards integration tests and regression guards from #7083 against absence of CactiProcess and other feature-PR hardening, so the consolidated foundation PR is green on develop. Suites activate automatically once their feature PRs merge. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(ci): drop infection/infection to unbreak PHP 8.4 matrix infection/infection ^0.27 transitively pins thecodingmachine/safe ^2.1.2, resolving to v2.5.0. Its generated stubs use implicit-nullable parameter declarations, which PHP 8.4 emits as deprecations into cacti.log and trips the log-quiet assertion. infection has no config or test wiring in this repo, so removing the dev dep clears the transitive pin without losing functionality. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net> |