mirror of
https://github.com/Cacti/cacti.git
synced 2026-10-29 08:06:58 +09:00
* fix(boost): harden data handoffs and cache writes Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Define $updates_ok when there is nothing to flush PHPStan level 6 also flags the poller ownership check, which is the POLLER_ID constant case the file already annotates above. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Use a line comment to match the fixer Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Record the boost process columns in the audit schema Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: resolve includes through Cacti path constants Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Fix Boost handoff review concerns Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: seed settings in Boost database fixture Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: refresh SQL interpolation baseline Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * docs: record Boost handoff hardening Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: stop boost discarding a cycle and wedging a shard Three defects in the handoff hardening. A rejected ownership handoff returned false, which the poller reads as boost having consumed the data, so it skipped the RRD write. The poller_output rows were already deleted by then, so a stale poller_item copy on a remote poller silently lost the whole cycle. Return true instead and let the poller write directly, matching the database failure path. Any non-OK RRDtool response retained the shard, and the retry then replayed rows already written, which RRDtool rejects as an illegal update time. That failed again on every pass, so the archive tables accumulated and boost never drained. Treat an at-or-behind timestamp as applied, since the sample is in the RRD; genuine errors still retain the shard. The two new poller_output_boost_processes columns passed 'null' where db_install_update_table reads 'NULL', so NOT NULL was never emitted and upgraded installs diverged from cacti.sql. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix: spell the NULL column key so upgrades emit NOT NULL db_add_column() reads $column['NULL'] and PHP array keys are case sensitive, so the thirteen definitions in the 1.3.0 upgrade that passed a lowercase 'null' key had the flag discarded and created nullable columns. Fresh installs built from cacti.sql got NOT NULL, upgraded installs did not, and Cacti's own schema audit reports the difference. The same file already spells the key in upper case 144 times, and the wrapper's own docblock example uses the upper case form. Guard the class with a test over every upgrade file rather than only the one that was wrong. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test: assert the NULL key plainly toContain() takes needles rather than a failure message. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): close remaining handoff review gaps Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): guard nullable graph output Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): keep in-flight completion rows through the 1.3.0 upgrade The truncate only existed so the run_child UNIQUE key could build over rows that predate run_id; deleting just those rows leaves a running boost its completion records. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * refactor(boost): drop the poller ownership gate Its boost_poller_on_demand() call sits behind the boost_redirect early return that poller.php forces on for every multi-collector install, and its recovery call turns a device deletion on main into a recovery that never brings the collector back online. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): bound atomic cache write allocations Guard upgrade source reads so failures report clearly, write graph cache data in fixed-size chunks, and exercise a multi-chunk payload. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): support Windows cache replacement Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(boost): expand handoff failure coverage Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(boost): support ANSI_QUOTES upgrade checks Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): stop staging after chunk failure Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(recovery): use numeric poller status Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(sql): support ANSI_QUOTES in recovery paths Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): preserve cache on replacement failure Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): retain readable cache permissions Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): repair completion schema at runtime Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(recovery): bound timestamp batch fetches Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): satisfy runtime schema analysis Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(recovery): restore status after transfer failure Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(boost): rely on unit bootstrap helpers Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): refresh generated cache secret Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * fix(boost): normalize cache path separators Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * test(boost): fail clearly on unreadable fixtures * fix(boost): name ignored duplicate sample keys The staging warning only said rows were dropped, so a retry could not be told from a colliding writer. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> * Lack of pre-commit hook execution allowed lint failure --------- Signed-off-by: Thomas Vincent <thomasvincent@gmail.com> Co-authored-by: TheWitness <thewitness@cacti.net>
253 lines
9.9 KiB
PHP
253 lines
9.9 KiB
PHP
<?php
|
|
/*
|
|
+-------------------------------------------------------------------------+
|
|
| Copyright (C) 2004-2026 The Cacti Group |
|
|
| |
|
|
| This program is free software; you can redistribute it and/or |
|
|
| modify it under the terms of the GNU General Public License |
|
|
| as published by the Free Software Foundation; either version 2 |
|
|
| of the License, or (at your option) any later version. |
|
|
+-------------------------------------------------------------------------+
|
|
| Cacti: The Complete RRDtool-based Graphing Solution |
|
|
+-------------------------------------------------------------------------+
|
|
*/
|
|
|
|
/*
|
|
* upgrade_boost_process_table() has to add the run_child UNIQUE key to
|
|
* poller_output_boost_processes, and nothing stops cron from starting
|
|
* poller_boost.php while the installer is working. Rows written before this
|
|
* upgrade carry no run identifier, so the ALTERs default every one of them to
|
|
* ('', 0) and the key cannot build over them; rows written by a child of an
|
|
* in-flight run carry a real run_id and are what the parent counts when it
|
|
* decides whether every child finished.
|
|
*
|
|
* These cases run against a real server because the discriminating behaviour
|
|
* is the engine's: whether the UNIQUE key builds, and which rows survive.
|
|
*
|
|
* Requires a MySQL/MariaDB instance; see BoostArchiveHandoffIntegrationTest.php
|
|
* for the same env vars. Skips (does not fail) if unreachable.
|
|
*/
|
|
|
|
// Production upgrade entry points define this before loading upgrade files.
|
|
// It also makes schema probes bypass their runtime cache while this test
|
|
// deliberately drops and recreates the same table between cases.
|
|
if (!defined('IN_CACTI_INSTALL')) {
|
|
define('IN_CACTI_INSTALL', 1);
|
|
}
|
|
|
|
require_once __DIR__ . '/../Helpers/UnitStubs.php';
|
|
require_once dirname(__DIR__, 2) . '/lib/database.php';
|
|
require_once dirname(__DIR__, 2) . '/lib/boost.php';
|
|
require_once dirname(__DIR__, 2) . '/install/functions.php';
|
|
require_once dirname(__DIR__, 2) . '/install/upgrades/1_3_0.php';
|
|
|
|
beforeEach(function () {
|
|
global $database_sessions, $database_hostname, $database_port, $database_default;
|
|
|
|
$this->db_globals = [$database_sessions, $database_hostname, $database_port, $database_default];
|
|
|
|
$dsn = getenv('CACTI_TEST_MYSQL_DSN') ?: 'mysql:host=127.0.0.1;port=33061;dbname=cacti_test';
|
|
$user = getenv('CACTI_TEST_MYSQL_USER') ?: 'root';
|
|
$pass = getenv('CACTI_TEST_MYSQL_PASS') ?: 'root';
|
|
|
|
try {
|
|
$this->pdo = new PDO($dsn, $user, $pass, [PDO::ATTR_ERRMODE => PDO::ERRMODE_SILENT, PDO::ATTR_TIMEOUT => 3]);
|
|
} catch (\PDOException $e) {
|
|
$this->pdo = null;
|
|
test()->markTestSkipped('No MySQL/MariaDB test instance reachable at ' . $dsn . ': ' . $e->getMessage());
|
|
|
|
return;
|
|
}
|
|
|
|
$database_hostname = 'unit-test-host';
|
|
$database_port = 3306;
|
|
$database_default = 'cacti_boost_upgrade_test';
|
|
|
|
$database_sessions["$database_hostname:$database_port:$database_default"] = $this->pdo;
|
|
$this->pdo->exec("SET SESSION sql_mode = CONCAT_WS(',', @@sql_mode, 'ANSI_QUOTES')");
|
|
|
|
// db_install_add_cache() stamps install_updated on every statement.
|
|
$this->pdo->exec('CREATE TABLE IF NOT EXISTS settings (
|
|
name varchar(50) NOT NULL default \'\',
|
|
value varchar(2048) NOT NULL default \'\',
|
|
PRIMARY KEY (name))');
|
|
|
|
$this->pdo->exec('DROP TABLE IF EXISTS poller_output_boost_processes');
|
|
});
|
|
|
|
afterEach(function () {
|
|
global $database_sessions, $database_hostname, $database_port, $database_default;
|
|
|
|
if (isset($this->pdo) && $this->pdo instanceof PDO) {
|
|
$this->pdo->exec('DROP TABLE IF EXISTS poller_output_boost_processes');
|
|
}
|
|
|
|
// Put the default db_* connection back so the test handle does not answer
|
|
// every later read_config_option() in the run.
|
|
[$database_sessions, $database_hostname, $database_port, $database_default] = $this->db_globals;
|
|
});
|
|
|
|
function boostProcessIndexColumns(PDO $pdo) : array {
|
|
$rows = $pdo->query("SHOW INDEX FROM poller_output_boost_processes WHERE Key_name = 'run_child'");
|
|
|
|
if ($rows === false) {
|
|
return [];
|
|
}
|
|
|
|
$columns = [];
|
|
|
|
foreach ($rows->fetchAll(PDO::FETCH_ASSOC) as $row) {
|
|
$columns[(int) $row['Seq_in_index']] = $row['Column_name'] . ':' . $row['Non_unique'];
|
|
}
|
|
|
|
ksort($columns);
|
|
|
|
return array_values($columns);
|
|
}
|
|
|
|
test('the 1.2.x table gains the run_child key even though its rows cannot satisfy it', function () {
|
|
if (!isset($this->pdo) || !$this->pdo instanceof PDO) {
|
|
return;
|
|
}
|
|
|
|
$this->pdo->exec('CREATE TABLE poller_output_boost_processes (
|
|
sock_int_value bigint(20) unsigned NOT NULL auto_increment,
|
|
status varchar(255) default NULL,
|
|
PRIMARY KEY (sock_int_value)) ENGINE=MEMORY');
|
|
|
|
$this->pdo->exec("INSERT INTO poller_output_boost_processes (status) VALUES ('118'), ('92'), ('0')");
|
|
|
|
upgrade_boost_process_table();
|
|
|
|
expect(boostProcessIndexColumns($this->pdo))->toBe(['run_id:0', 'child_id:0'])
|
|
->and((int) $this->pdo->query('SELECT COUNT(*) FROM poller_output_boost_processes')->fetchColumn())->toBe(0);
|
|
});
|
|
|
|
test('an in-flight run keeps its completion rows while the legacy rows are dropped', function () {
|
|
if (!isset($this->pdo) || !$this->pdo instanceof PDO) {
|
|
return;
|
|
}
|
|
|
|
// A collector already carrying the 1.3.0 columns but not yet the key, which
|
|
// is what a develop-to-develop upgrade finds.
|
|
$this->pdo->exec("CREATE TABLE poller_output_boost_processes (
|
|
sock_int_value bigint(20) unsigned NOT NULL auto_increment,
|
|
run_id char(32) NOT NULL default '',
|
|
child_id int(10) unsigned NOT NULL default 0,
|
|
status varchar(255) default NULL,
|
|
PRIMARY KEY (sock_int_value)) ENGINE=MEMORY");
|
|
|
|
$run_id = '3f2a91c4d80b47e6a15c9f0e7b32d5a8';
|
|
|
|
$this->pdo->exec("INSERT INTO poller_output_boost_processes (run_id, child_id, status) VALUES
|
|
('$run_id', 1, '4210'),
|
|
('$run_id', 2, '3987'),
|
|
('', 0, '77')");
|
|
|
|
upgrade_boost_process_table();
|
|
|
|
$survivors = $this->pdo->query('SELECT run_id, child_id, status
|
|
FROM poller_output_boost_processes
|
|
ORDER BY child_id')->fetchAll(PDO::FETCH_ASSOC);
|
|
|
|
expect($survivors)->toBe([
|
|
['run_id' => $run_id, 'child_id' => 1, 'status' => '4210'],
|
|
['run_id' => $run_id, 'child_id' => 2, 'status' => '3987'],
|
|
]);
|
|
|
|
// The parent drains on COUNT(*) for its own run_id, so those two rows are
|
|
// the difference between finishing the run and warning about a crashed
|
|
// child while holding the archive tables.
|
|
$completed = $this->pdo->prepare('SELECT COUNT(*) FROM poller_output_boost_processes WHERE run_id = ?');
|
|
$completed->execute([$run_id]);
|
|
|
|
expect((int) $completed->fetchColumn())->toBe(2)
|
|
->and(boostProcessIndexColumns($this->pdo))->toBe(['run_id:0', 'child_id:0']);
|
|
|
|
// What the upgrade used to do, on the same rows.
|
|
$this->pdo->exec('TRUNCATE TABLE poller_output_boost_processes');
|
|
$completed->execute([$run_id]);
|
|
|
|
expect((int) $completed->fetchColumn())->toBe(0);
|
|
});
|
|
|
|
test('a missing process table is recreated with the complete current schema', function () {
|
|
if (!isset($this->pdo) || !$this->pdo instanceof PDO) {
|
|
return;
|
|
}
|
|
|
|
upgrade_boost_process_table();
|
|
|
|
$columns = $this->pdo->query('SHOW COLUMNS FROM poller_output_boost_processes')->fetchAll(PDO::FETCH_COLUMN);
|
|
$engine = $this->pdo->query("SELECT ENGINE FROM information_schema.TABLES
|
|
WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'poller_output_boost_processes'")->fetchColumn();
|
|
|
|
expect($columns)->toBe(['sock_int_value', 'run_id', 'child_id', 'status'])
|
|
->and(boostProcessIndexColumns($this->pdo))->toBe(['run_id:0', 'child_id:0'])
|
|
->and($engine)->toBe('MEMORY');
|
|
});
|
|
|
|
test('an already upgraded process table and its completion row are unchanged', function () {
|
|
if (!isset($this->pdo) || !$this->pdo instanceof PDO) {
|
|
return;
|
|
}
|
|
|
|
$this->pdo->exec("CREATE TABLE poller_output_boost_processes (
|
|
sock_int_value bigint(20) unsigned NOT NULL auto_increment,
|
|
run_id char(32) NOT NULL default '',
|
|
child_id int(10) unsigned NOT NULL default 0,
|
|
status varchar(255) default NULL,
|
|
PRIMARY KEY (sock_int_value),
|
|
UNIQUE KEY run_child (run_id, child_id)) ENGINE=MEMORY");
|
|
$this->pdo->exec("INSERT INTO poller_output_boost_processes (run_id, child_id, status)
|
|
VALUES ('3f2a91c4d80b47e6a15c9f0e7b32d5a8', 4, '123')");
|
|
|
|
upgrade_boost_process_table();
|
|
|
|
$row = $this->pdo->query('SELECT run_id, child_id, status FROM poller_output_boost_processes')->fetch(PDO::FETCH_ASSOC);
|
|
|
|
expect($row)->toBe([
|
|
'run_id' => '3f2a91c4d80b47e6a15c9f0e7b32d5a8',
|
|
'child_id' => 4,
|
|
'status' => '123',
|
|
]);
|
|
});
|
|
|
|
test('the run_child index rejects duplicate completion reports for one child', function () {
|
|
if (!isset($this->pdo) || !$this->pdo instanceof PDO) {
|
|
return;
|
|
}
|
|
|
|
upgrade_boost_process_table();
|
|
|
|
$insert = $this->pdo->prepare('INSERT INTO poller_output_boost_processes (run_id, child_id, status) VALUES (?, ?, ?)');
|
|
expect($insert->execute(['3f2a91c4d80b47e6a15c9f0e7b32d5a8', 1, '10']))->toBeTrue();
|
|
|
|
$duplicate = $insert->execute(['3f2a91c4d80b47e6a15c9f0e7b32d5a8', 1, '20']);
|
|
|
|
expect($duplicate)->toBeFalse()
|
|
->and((int) $this->pdo->query('SELECT COUNT(*) FROM poller_output_boost_processes')->fetchColumn())->toBe(1);
|
|
});
|
|
|
|
test('runtime repair makes a legacy process table safe before a child records completion', function () {
|
|
if (!isset($this->pdo) || !$this->pdo instanceof PDO) {
|
|
return;
|
|
}
|
|
|
|
$this->pdo->exec('CREATE TABLE poller_output_boost_processes (
|
|
sock_int_value bigint(20) unsigned NOT NULL auto_increment,
|
|
status varchar(255) default NULL,
|
|
PRIMARY KEY (sock_int_value)) ENGINE=MEMORY');
|
|
$this->pdo->exec("INSERT INTO poller_output_boost_processes (status) VALUES ('legacy')");
|
|
|
|
expect(boost_ensure_process_table(true))->toBeTrue()
|
|
->and(boostProcessIndexColumns($this->pdo))->toBe(['run_id:0', 'child_id:0'])
|
|
->and((int) $this->pdo->query('SELECT COUNT(*) FROM poller_output_boost_processes')->fetchColumn())->toBe(0);
|
|
|
|
$insert = $this->pdo->prepare('INSERT INTO poller_output_boost_processes
|
|
(run_id, child_id, status) VALUES (?, ?, ?)
|
|
ON DUPLICATE KEY UPDATE status = VALUES(status)');
|
|
|
|
expect($insert->execute(['3f2a91c4d80b47e6a15c9f0e7b32d5a8', 1, '10']))->toBeTrue();
|
|
});
|