From 3065c6fc23b1fab39d252ddbe3cb5c9d0517fe81 Mon Sep 17 00:00:00 2001 From: Thomas Vincent Date: Sat, 10 Oct 2026 17:03:21 -0700 Subject: [PATCH] refactor(core): add bounded native interfaces and polling contracts (#638) Introduce bounded native interfaces and grouped runtime state, then link regression and fault contracts against the real poller implementations. The contracts capture CLI/configuration input, availability combinations, SNMP request/response ownership, script framing, reindexing, output persistence, worker completion and shutdown. This foundational change is larger because the shared interfaces and their existing production contracts must move together; later changes have separate review boundaries. Step 1/14 of the complete C17/native-correctness series. Base: develop. Earlier validation, before this review follow-up, in isolated copied snapshots on Linux arm64: - Clean GCC build and make check: 13 passed, 0 existing prerequisite skips, 0 failures. - ASan/UBSan: 12 passed, 1 existing privilege/allocator-interposition prerequisite skips, 0 failures; no sanitizer findings. - Production source/profile integrity verification against disposable MariaDB and loopback SNMP fixtures passed. - Release archive extracted into a separate source tree and built/tested out of tree; fuzz checks passed. Earlier local profiles retain source hashes for the copied validation tree; GitHub checks validate the published commit. No production database/device, host installation or live privilege change is part of validation. Existing prerequisite skips were retained; no test or finding was suppressed to obtain a pass. The worker contracts publish owned heap state and release it only after joining workers and restoring the runtime table. Coverage counters use atomic updates; cppcheck receives an explicit model of the fatal exit contract. Fixture images use public mirrors to avoid shared Docker Hub pull limits. Series navigation: review and merge in order. Every PR targets the preceding branch so its diff contains one step. 1. [refactor(core): add bounded native interfaces and polling contracts](https://github.com/Cacti/spine/pull/638) 2. [fix(privileges): drop root before input and retain owned ICMP sockets](https://github.com/Cacti/spine/pull/639) 3. [fix(poller): isolate worker failures and bound database reconnects](https://github.com/Cacti/spine/pull/640) 4. [fix(results): normalize output and unify SNMP batch persistence](https://github.com/Cacti/spine/pull/641) 5. [fix(runtime): bound probe retries and validate CLI and configuration](https://github.com/Cacti/spine/pull/642) 6. [build: select GNU C17 explicitly and probe hardening support](https://github.com/Cacti/spine/pull/643) 7. [fix(threads): size worker stacks and guard size arithmetic](https://github.com/Cacti/spine/pull/644) 8. [refactor(c17): simplify ownership guards and share test support](https://github.com/Cacti/spine/pull/645) 9. [refactor(core): extract configuration polling and runtime modules](https://github.com/Cacti/spine/pull/646) 10. [fix(boundaries): validate script slots and make cleanup portable](https://github.com/Cacti/spine/pull/647) 11. [refactor(layout): organize responsibilities and separate main from runtime](https://github.com/Cacti/spine/pull/648) 12. [docs(license): adopt SPDX notices and retain contributor credits](https://github.com/Cacti/spine/pull/649) 13. [test(runtime): measure all suites and cover protocol boundaries](https://github.com/Cacti/spine/pull/650) 14. [build(ci): validate connectors time width and native C17 modules](https://github.com/Cacti/spine/pull/651) Review follow-up (2026-10-09): Publish the decoded GET/GETNEXT status back to the host so availability checks observe exception and missing-response failures, while preserving the legacy transport-success behavior for agent PDU errors. Updated step 1 snapshot: isolated Linux arm64 build and make check passed (13 passed, no skips or failures). The restacked tip passes all five clang-tidy gate regression tests and real clang-tidy analysis (26 reviewed baseline findings, no new findings). Actionlint and matching DCO trailers pass. The setuid test also passes when run in the separate disposable root container. GitHub checks validate the updated published commits; earlier profile/archive/sanitizer evidence above belongs to the previous revisions. --------- Signed-off-by: Thomas Vincent --- .github/workflows/ci.yml | 46 +- .github/workflows/integration.yml | 2 +- .github/workflows/regressions.yml | 91 + .github/workflows/rocky-linux.yml | 4 +- Dockerfile | 5 +- Makefile.am | 50 +- Makefile.in | 1056 +++++++- common.h | 6 - configure.ac | 33 + copyright_year.sh | 32 +- error.c | 2 +- nft_popen.c | 34 +- php.c | 79 +- ping.c | 1220 +++------ ping.h | 12 +- poller.c | 3573 ++++++++++--------------- poller.h | 25 +- scripts/cppcheck/spine.cfg | 31 + scripts/verify.sh | 1 + snmp.c | 1771 +++++------- snmp.h | 37 +- spine.c | 1779 ++++++------ spine.h | 584 +--- spine_types.h | 668 +++++ sql.c | 400 ++- sql.h | 18 +- src/core/locks.c | 53 +- src/core/locks.h | 1 + tests/fixtures/configuration.sql | 111 + tests/fixtures/poller.sql | 153 ++ tests/fixtures/snmpd.conf | 7 + tests/fixtures/snmpv3-agent.conf | 9 + tests/fixtures/snmpv3-users.conf | 7 + tests/fuzz/stubs.c | 6 +- tests/regression/test_remote_push.sh | 23 +- tests/snmpv3/Dockerfile.coverage | 2 +- tests/snmpv3/docker-compose.mysql.yml | 4 +- tests/snmpv3/docker-compose.yml | 4 +- tests/snmpv3/snmpd/Dockerfile | 2 +- tests/test_spine_cli_contracts.c | 109 + tests/test_spine_contracts.c | 305 +++ tests/test_spine_faults.c | 712 +++++ tests/test_spine_output_contracts.c | 609 +++++ tests/test_spine_regressions.c | 2523 +++++++++++++++++ tests/test_spine_reindex_contracts.c | 189 ++ tests/test_spine_settings_contracts.c | 60 + tests/test_spine_snmpv3_contracts.c | 166 ++ tests/test_spine_stream_contracts.c | 82 + tests/tools/coverage_process_exit.h | 15 + tests/tools/run_coverage.sh | 182 ++ tests/tools/run_snmpv3.sh | 27 + tests/tools/test_coverage_verifier.sh | 49 + tests/tools/verify_coverage.sh | 63 + tests/unit/test_linked.c | 30 +- tests/unit/test_php_runtime.c | 66 +- tests/unit/test_ping_icmp.c | 22 +- tests/unit/test_snmpv3_session.c | 12 +- tests/unit/test_util_paths.c | 109 +- tests/unit/test_util_strings.c | 2 +- util.c | 2146 +++++++-------- util.h | 25 +- 61 files changed, 12267 insertions(+), 7177 deletions(-) create mode 100644 .github/workflows/regressions.yml create mode 100644 scripts/cppcheck/spine.cfg create mode 100644 spine_types.h create mode 100644 tests/fixtures/configuration.sql create mode 100644 tests/fixtures/poller.sql create mode 100644 tests/fixtures/snmpd.conf create mode 100644 tests/fixtures/snmpv3-agent.conf create mode 100644 tests/fixtures/snmpv3-users.conf create mode 100644 tests/test_spine_cli_contracts.c create mode 100644 tests/test_spine_contracts.c create mode 100644 tests/test_spine_faults.c create mode 100644 tests/test_spine_output_contracts.c create mode 100644 tests/test_spine_regressions.c create mode 100644 tests/test_spine_reindex_contracts.c create mode 100644 tests/test_spine_settings_contracts.c create mode 100644 tests/test_spine_snmpv3_contracts.c create mode 100644 tests/test_spine_stream_contracts.c create mode 100644 tests/tools/coverage_process_exit.h create mode 100755 tests/tools/run_coverage.sh create mode 100755 tests/tools/run_snmpv3.sh create mode 100755 tests/tools/test_coverage_verifier.sh create mode 100755 tests/tools/verify_coverage.sh diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 7418019..ff6151d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -4,7 +4,7 @@ on: push: branches: [develop, '1.2.x'] pull_request: - branches: [develop, '1.2.x'] + branches: [develop, '1.2.x', 'refactor/c17-series-*'] permissions: contents: read @@ -166,6 +166,7 @@ jobs: --suppress=constVariablePointer \ --suppress=redundantAssignment \ --suppress=toomanyconfigs \ + --library=scripts/cppcheck/spine.cfg \ ./*.c ./*.h shellcheck: @@ -187,6 +188,19 @@ jobs: coverage: runs-on: ubuntu-24.04 timeout-minutes: 20 + services: + database: + image: mirror.gcr.io/library/mariadb:10.11 + env: + MARIADB_ALLOW_EMPTY_ROOT_PASSWORD: '1' + MARIADB_DATABASE: spine_regressions + ports: + - 3306:3306 + options: >- + --health-cmd="mariadb-admin ping --silent" + --health-interval=5s + --health-timeout=5s + --health-retries=20 steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: @@ -194,8 +208,8 @@ jobs: - uses: ./.github/actions/build-spine with: - extra-packages: gcovr python3-pip - cflags: -g -O0 --coverage + extra-packages: gcovr python3-pip mariadb-client snmpd libcap2-bin + cflags: -g -O0 --coverage -fprofile-update=atomic ldflags: --coverage - name: Build instrumented @@ -208,6 +222,32 @@ jobs: set -euo pipefail make check + - name: Cover production database and protocol scenarios + env: + SPINE_TEST_DB_HOST: 127.0.0.1 + SPINE_TEST_SNMP_HOST: 127.0.0.1 + SPINE_SNMPV3_EVIDENCE_DIR: snmpv3-evidence + run: | + set -euo pipefail + mariadb --protocol=TCP --host=127.0.0.1 --user=root spine_regressions < tests/fixtures/configuration.sql + mariadb --protocol=TCP --host=127.0.0.1 --user=root spine_regressions < tests/fixtures/poller.sql + snmpd -f -Lo -C -c tests/fixtures/snmpd.conf > coverage-snmp-agent.log 2>&1 & + agent_pid=$! + cleanup() { + sudo setcap -r ./test_spine_regressions + kill "$agent_pid" + wait "$agent_pid" || true + } + trap cleanup EXIT + sleep 1 + ./test_spine_regressions --database + ./test_spine_regressions --snmp-agent + ./test_spine_faults --database + tests/tools/run_snmpv3.sh + ./test_spine_regressions --icmp-no-capability + sudo setcap cap_net_raw+ep ./test_spine_regressions + ./test_spine_regressions --raw-icmp + - name: Report coverage run: | set -euo pipefail diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml index be649f8..6b75c6d 100644 --- a/.github/workflows/integration.yml +++ b/.github/workflows/integration.yml @@ -2,7 +2,7 @@ name: Integration on: pull_request: - branches: [develop] + branches: [develop, 'refactor/c17-series-*'] paths: ['**.c', '**.h', 'Dockerfile', '.dockerignore', 'tests/integration/**', 'tests/test-harness.sh', 'tests/snmpv3/**', '.github/workflows/integration.yml'] workflow_dispatch: diff --git a/.github/workflows/regressions.yml b/.github/workflows/regressions.yml new file mode 100644 index 0000000..2172a8a --- /dev/null +++ b/.github/workflows/regressions.yml @@ -0,0 +1,91 @@ +name: Production regressions + +on: + push: + branches: [develop] + pull_request: + workflow_dispatch: + +permissions: + contents: read + +jobs: + regressions: + runs-on: ubuntu-24.04 + timeout-minutes: 15 + services: + database: + image: mirror.gcr.io/library/mariadb:10.11 + env: + MARIADB_ALLOW_EMPTY_ROOT_PASSWORD: '1' + MARIADB_DATABASE: spine_regressions + ports: + - 3306:3306 + options: >- + --health-cmd="mariadb-admin ping --silent" + --health-interval=5s + --health-timeout=5s + --health-retries=20 + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 + - name: Install build dependencies + run: | + set -eu + sudo apt-get update + sudo apt-get install -y autoconf automake libtool help2man libmariadb-dev mariadb-client libsnmp-dev snmpd libssl-dev libcap2-bin lcov + - name: Build and test production code + run: | + set -eu + mkdir -p config m4 + autoreconf -fi + ./configure + make -j2 check + - name: Verify database configuration loading + env: + SPINE_TEST_DB_HOST: 127.0.0.1 + run: | + set -eu + mariadb --protocol=TCP --host=127.0.0.1 --user=root spine_regressions < tests/fixtures/configuration.sql + mariadb --protocol=TCP --host=127.0.0.1 --user=root spine_regressions < tests/fixtures/poller.sql + ./test_spine_regressions --database + - name: Verify SNMP availability against local agent + env: + SPINE_TEST_SNMP_HOST: 127.0.0.1 + SPINE_TEST_DB_HOST: 127.0.0.1 + run: | + set -eu + snmpd -f -Lo -C -c tests/fixtures/snmpd.conf > snmp-agent.log 2>&1 & + agent_pid=$! + trap 'kill "$agent_pid"; wait "$agent_pid" || true' EXIT + sleep 1 + ./test_spine_regressions --snmp-agent + ./test_spine_regressions --database + ./test_spine_faults --database + - name: Verify authenticated SNMPv3 against owned agent + env: + SPINE_SNMPV3_EVIDENCE_DIR: snmpv3-evidence + run: tests/tools/run_snmpv3.sh + - name: Verify ICMP against local loopback + run: | + set -eu + ./test_spine_regressions --icmp-no-capability + sudo setcap cap_net_raw+ep ./test_spine_regressions + ./test_spine_regressions --raw-icmp + - name: Measure every production translation unit + env: + SPINE_TEST_SNMP_HOST: 127.0.0.1 + SPINE_TEST_DB_HOST: 127.0.0.1 + run: | + set -eu + snmpd -f -Lo -C -c tests/fixtures/snmpd.conf > coverage-snmp-agent.log 2>&1 & + agent_pid=$! + trap 'kill "$agent_pid"; wait "$agent_pid" || true' EXIT + sleep 1 + tests/tools/run_coverage.sh + - name: Preserve coverage and source provenance + if: always() + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 + with: + name: production-coverage-${{ github.sha }} + path: coverage/ + if-no-files-found: error diff --git a/.github/workflows/rocky-linux.yml b/.github/workflows/rocky-linux.yml index de8be03..ff27ddf 100644 --- a/.github/workflows/rocky-linux.yml +++ b/.github/workflows/rocky-linux.yml @@ -5,7 +5,7 @@ on: schedule: - cron: "37 3 * * 6" pull_request: - branches: [develop] + branches: [develop, 'refactor/c17-series-*'] paths: - "configure.ac" - "Makefile.am" @@ -35,7 +35,7 @@ jobs: docker run --rm \ --volume "$GITHUB_WORKSPACE:/src:ro" \ --workdir /workspace \ - rockylinux:9@sha256:d7be1c094cc5845ee815d4632fe377514ee6ebcf8efaed6892889657e5ddaaa6 \ + mirror.gcr.io/library/rockylinux:9@sha256:d7be1c094cc5845ee815d4632fe377514ee6ebcf8efaed6892889657e5ddaaa6 \ bash -euo pipefail -c ' cp -a /src/. /workspace/ dnf -y group install "Development Tools" diff --git a/Dockerfile b/Dockerfile index afb12ef..e84156a 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,5 +1,4 @@ -# syntax=docker/dockerfile:1 -FROM debian:bookworm-slim AS builder +FROM mirror.gcr.io/library/debian:bookworm-slim AS builder RUN apt-get update && apt-get install -y --no-install-recommends \ gcc \ @@ -20,7 +19,7 @@ RUN autoreconf -fi \ && ./configure --prefix=/usr/local \ && make -j"$(nproc)" spine -FROM debian:bookworm-slim +FROM mirror.gcr.io/library/debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ libmariadb3 \ diff --git a/Makefile.am b/Makefile.am index 27e5773..416f2d8 100644 --- a/Makefile.am +++ b/Makefile.am @@ -22,11 +22,12 @@ AUTOMAKE_OPTIONS = foreign ACLOCAL_AMFLAGS = -I m4 -spine_SOURCES = sql.c spine.c util.c snmp.c src/core/locks.c poller.c \ +spine_shared_sources = sql.c util.c snmp.c src/core/locks.c poller.c \ nft_popen.c php.c ping.c src/core/keywords.c error.c +spine_SOURCES = $(spine_shared_sources) spine.c configdir = $(sysconfdir) -config_DATA = spine.conf.dist +dist_config_DATA = spine.conf.dist # Seed a live config on first install without ever overwriting an existing one. install-data-hook: @@ -46,12 +47,35 @@ uninstall-hook: bin_PROGRAMS = spine man_MANS = spine.1 +DISTCLEANFILES = spine.1 + +check_PROGRAMS = test_spine_regressions +dist_noinst_DATA = tests/fixtures/configuration.sql tests/fixtures/poller.sql tests/fixtures/snmpd.conf tests/fixtures/snmpv3-agent.conf tests/fixtures/snmpv3-users.conf +TESTS = $(check_PROGRAMS) +test_spine_regressions_SOURCES = $(spine_SOURCES) tests/test_spine_regressions.c tests/test_spine_contracts.c tests/test_spine_reindex_contracts.c tests/test_spine_snmpv3_contracts.c tests/test_spine_settings_contracts.c tests/test_spine_cli_contracts.c tests/test_spine_stream_contracts.c tests/test_spine_output_contracts.c +# Link the real production translation units and globals, with a separate test +# entry point. Only spine.c maps its own entry point in the test build. +test_spine_regressions_CPPFLAGS = $(AM_CPPFLAGS) -I$(srcdir) -DSPINE_TEST_PROGRAM_ENTRY + +if HAVE_LINKER_WRAP +check_PROGRAMS += test_spine_faults +# Reuse the real renamed spine object for its production globals, avoiding +# another production main-renaming flag on the fault-test translation units. +test_spine_faults_SOURCES = $(spine_shared_sources) tests/test_spine_faults.c +test_spine_faults_CPPFLAGS = $(AM_CPPFLAGS) -I$(srcdir) +test_spine_faults_LDADD = test_spine_regressions-spine.$(OBJEXT) +test_spine_faults_LDFLAGS = $(AM_LDFLAGS) -Wl,--wrap=strftime -Wl,--wrap=mysql_query -Wl,--wrap=mysql_ping -Wl,--wrap=mysql_errno \ + -Wl,--wrap=pipe -Wl,--wrap=socketpair -Wl,--wrap=posix_spawn \ + -Wl,--wrap=malloc -Wl,--wrap=strdup \ + -Wl,--wrap=snmp_sess_open -Wl,--wrap=snmp_sess_close -Wl,--wrap=pthread_create +endif + # Headers are not referenced by any _SOURCES, so automake will not distribute # them unless they are listed. Without this the release tarball builds nothing. noinst_HEADERS = common.h error.h src/core/keywords.h src/core/locks.h \ - nft_popen.h php.h ping.h poller.h snmp.h spine.h spine_sem.h sql.h \ - uthash.h util.h + nft_popen.h php.h ping.h poller.h snmp.h spine.h spine_sem.h spine_types.h \ + sql.h uthash.h util.h EXTRA_DIST = spine.1 spine.conf.dist \ tests/README.md \ @@ -59,6 +83,18 @@ EXTRA_DIST = spine.1 spine.conf.dist \ tests/integration/test_concurrency.sh \ tests/integration/test_network_faults.sh +EXTRA_DIST += tests/tools/run_snmpv3.sh tests/tools/run_coverage.sh tests/tools/coverage_process_exit.h \ + tests/tools/verify_coverage.sh tests/tools/test_coverage_verifier.sh +.PHONY: coverage-source-list coverage-test-source-list +coverage-source-list: + @printf '%s\n' $(spine_SOURCES) + +coverage-test-source-list: + @printf '%s\n' $(test_spine_regressions_SOURCES) + +.PHONY: coverage-fault-source-list +coverage-fault-source-list: + @printf '%s\n' $(test_spine_faults_SOURCES) # Docker targets — require Dockerfile and Dockerfile.dev (from PR #401) .PHONY: docker docker-dev verify cppcheck @@ -76,8 +112,6 @@ cppcheck: docker-dev AM_CPPFLAGS = -I$(top_srcdir) -I$(top_srcdir)/src/core -I$(top_builddir) -check_PROGRAMS = - if HAVE_CMOCKA check_PROGRAMS += \ tests/unit/test_util_strings \ @@ -162,6 +196,8 @@ tests_unit_test_php_runtime_DEPENDENCIES = tests/unit/php_test_server$(EXEEXT) TEST_EXTENSIONS = .sh SH_LOG_COMPILER = $(SHELL) -TESTS = $(check_PROGRAMS) tests/regression/test_remote_push.sh tests/integration/test_alloc_failure.sh tests/regression/test_child_process_safety.sh +TESTS += tests/regression/test_remote_push.sh tests/integration/test_alloc_failure.sh tests/regression/test_child_process_safety.sh TESTS_ENVIRONMENT = top_builddir='$(abs_top_builddir)' EXTRA_DIST += tests/regression/test_remote_push.sh tests/integration/test_alloc_failure.sh tests/regression/test_child_process_safety.sh + +EXTRA_DIST += scripts/cppcheck/spine.cfg diff --git a/Makefile.in b/Makefile.in index 65d1762..c53c9fd 100644 --- a/Makefile.in +++ b/Makefile.in @@ -1,7 +1,7 @@ -# Makefile.in generated by automake 1.16.2 from Makefile.am. +# Makefile.in generated by automake 1.16.5 from Makefile.am. # @configure_input@ -# Copyright (C) 1994-2020 Free Software Foundation, Inc. +# Copyright (C) 1994-2021 Free Software Foundation, Inc. # This Makefile.in is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -112,8 +112,10 @@ POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ bin_PROGRAMS = spine$(EXEEXT) -check_PROGRAMS = $(am__EXEEXT_1) $(am__EXEEXT_2) -@HAVE_CMOCKA_TRUE@am__append_1 = \ +check_PROGRAMS = test_spine_regressions$(EXEEXT) $(am__EXEEXT_1) \ + $(am__EXEEXT_2) $(am__EXEEXT_3) +@HAVE_LINKER_WRAP_TRUE@am__append_1 = test_spine_faults +@HAVE_CMOCKA_TRUE@am__append_2 = \ @HAVE_CMOCKA_TRUE@ tests/unit/test_util_strings \ @HAVE_CMOCKA_TRUE@ tests/unit/test_build_fixes \ @HAVE_CMOCKA_TRUE@ tests/unit/test_safety_fixes \ @@ -124,7 +126,8 @@ check_PROGRAMS = $(am__EXEEXT_1) $(am__EXEEXT_2) @HAVE_CMOCKA_TRUE@noinst_PROGRAMS = \ @HAVE_CMOCKA_TRUE@ tests/unit/php_test_server$(EXEEXT) -@HAVE_CMOCKA_TRUE@@HAVE_LD_WRAP_TRUE@am__append_2 = tests/unit/test_php_runtime +@HAVE_CMOCKA_TRUE@@HAVE_LD_WRAP_TRUE@am__append_3 = tests/unit/test_php_runtime +@HAVE_LD_WRAP_TRUE@am__append_4 = -DSPINE_TEST_WRAP_WAITPID subdir = . ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \ @@ -134,7 +137,8 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/libtool.m4 \ am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ $(ACLOCAL_M4) DIST_COMMON = $(srcdir)/Makefile.am $(top_srcdir)/configure \ - $(am__configure_deps) $(noinst_HEADERS) $(am__DIST_COMMON) + $(am__configure_deps) $(dist_config_DATA) $(dist_noinst_DATA) \ + $(noinst_HEADERS) $(am__DIST_COMMON) am__CONFIG_DISTCLEAN_FILES = config.status config.cache config.log \ configure.lineno config.status.lineno mkinstalldirs = $(install_sh) -d @@ -143,7 +147,8 @@ CONFIG_CLEAN_FILES = CONFIG_CLEAN_VPATH_FILES = am__installdirs = "$(DESTDIR)$(bindir)" "$(DESTDIR)$(man1dir)" \ "$(DESTDIR)$(configdir)" -@HAVE_CMOCKA_TRUE@am__EXEEXT_1 = \ +@HAVE_LINKER_WRAP_TRUE@am__EXEEXT_1 = test_spine_faults$(EXEEXT) +@HAVE_CMOCKA_TRUE@am__EXEEXT_2 = \ @HAVE_CMOCKA_TRUE@ tests/unit/test_util_strings$(EXEEXT) \ @HAVE_CMOCKA_TRUE@ tests/unit/test_build_fixes$(EXEEXT) \ @HAVE_CMOCKA_TRUE@ tests/unit/test_safety_fixes$(EXEEXT) \ @@ -151,19 +156,66 @@ am__installdirs = "$(DESTDIR)$(bindir)" "$(DESTDIR)$(man1dir)" \ @HAVE_CMOCKA_TRUE@ tests/unit/test_util_paths$(EXEEXT) \ @HAVE_CMOCKA_TRUE@ tests/unit/test_snmpv3_session$(EXEEXT) \ @HAVE_CMOCKA_TRUE@ tests/unit/test_ping_icmp$(EXEEXT) -@HAVE_CMOCKA_TRUE@@HAVE_LD_WRAP_TRUE@am__EXEEXT_2 = tests/unit/test_php_runtime$(EXEEXT) +@HAVE_CMOCKA_TRUE@@HAVE_LD_WRAP_TRUE@am__EXEEXT_3 = tests/unit/test_php_runtime$(EXEEXT) PROGRAMS = $(bin_PROGRAMS) $(noinst_PROGRAMS) am__dirstamp = $(am__leading_dot)dirstamp -am_spine_OBJECTS = sql.$(OBJEXT) spine.$(OBJEXT) util.$(OBJEXT) \ - snmp.$(OBJEXT) src/core/locks.$(OBJEXT) poller.$(OBJEXT) \ - nft_popen.$(OBJEXT) php.$(OBJEXT) ping.$(OBJEXT) \ - src/core/keywords.$(OBJEXT) error.$(OBJEXT) +am__objects_1 = sql.$(OBJEXT) util.$(OBJEXT) snmp.$(OBJEXT) \ + src/core/locks.$(OBJEXT) poller.$(OBJEXT) nft_popen.$(OBJEXT) \ + php.$(OBJEXT) ping.$(OBJEXT) src/core/keywords.$(OBJEXT) \ + error.$(OBJEXT) +am_spine_OBJECTS = $(am__objects_1) spine.$(OBJEXT) spine_OBJECTS = $(am_spine_OBJECTS) spine_LDADD = $(LDADD) AM_V_lt = $(am__v_lt_@AM_V@) am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) am__v_lt_0 = --silent -am__v_lt_1 = +am__v_lt_1 = +am__test_spine_faults_SOURCES_DIST = sql.c util.c snmp.c \ + src/core/locks.c poller.c nft_popen.c php.c ping.c \ + src/core/keywords.c error.c tests/test_spine_faults.c +am__objects_2 = test_spine_faults-sql.$(OBJEXT) \ + test_spine_faults-util.$(OBJEXT) \ + test_spine_faults-snmp.$(OBJEXT) \ + src/core/test_spine_faults-locks.$(OBJEXT) \ + test_spine_faults-poller.$(OBJEXT) \ + test_spine_faults-nft_popen.$(OBJEXT) \ + test_spine_faults-php.$(OBJEXT) \ + test_spine_faults-ping.$(OBJEXT) \ + src/core/test_spine_faults-keywords.$(OBJEXT) \ + test_spine_faults-error.$(OBJEXT) +@HAVE_LINKER_WRAP_TRUE@am_test_spine_faults_OBJECTS = \ +@HAVE_LINKER_WRAP_TRUE@ $(am__objects_2) \ +@HAVE_LINKER_WRAP_TRUE@ tests/test_spine_faults-test_spine_faults.$(OBJEXT) +test_spine_faults_OBJECTS = $(am_test_spine_faults_OBJECTS) +@HAVE_LINKER_WRAP_TRUE@test_spine_faults_DEPENDENCIES = \ +@HAVE_LINKER_WRAP_TRUE@ test_spine_regressions-spine.$(OBJEXT) +test_spine_faults_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC \ + $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CCLD) \ + $(AM_CFLAGS) $(CFLAGS) $(test_spine_faults_LDFLAGS) $(LDFLAGS) \ + -o $@ +am__objects_3 = test_spine_regressions-sql.$(OBJEXT) \ + test_spine_regressions-util.$(OBJEXT) \ + test_spine_regressions-snmp.$(OBJEXT) \ + src/core/test_spine_regressions-locks.$(OBJEXT) \ + test_spine_regressions-poller.$(OBJEXT) \ + test_spine_regressions-nft_popen.$(OBJEXT) \ + test_spine_regressions-php.$(OBJEXT) \ + test_spine_regressions-ping.$(OBJEXT) \ + src/core/test_spine_regressions-keywords.$(OBJEXT) \ + test_spine_regressions-error.$(OBJEXT) +am__objects_4 = $(am__objects_3) \ + test_spine_regressions-spine.$(OBJEXT) +am_test_spine_regressions_OBJECTS = $(am__objects_4) \ + tests/test_spine_regressions-test_spine_regressions.$(OBJEXT) \ + tests/test_spine_regressions-test_spine_contracts.$(OBJEXT) \ + tests/test_spine_regressions-test_spine_reindex_contracts.$(OBJEXT) \ + tests/test_spine_regressions-test_spine_snmpv3_contracts.$(OBJEXT) \ + tests/test_spine_regressions-test_spine_settings_contracts.$(OBJEXT) \ + tests/test_spine_regressions-test_spine_cli_contracts.$(OBJEXT) \ + tests/test_spine_regressions-test_spine_stream_contracts.$(OBJEXT) \ + tests/test_spine_regressions-test_spine_output_contracts.$(OBJEXT) +test_spine_regressions_OBJECTS = $(am_test_spine_regressions_OBJECTS) +test_spine_regressions_LDADD = $(LDADD) am__tests_unit_php_test_server_SOURCES_DIST = \ tests/unit/php_test_server.c @HAVE_CMOCKA_TRUE@am_tests_unit_php_test_server_OBJECTS = \ @@ -177,14 +229,26 @@ tests_unit_test_build_fixes_OBJECTS = \ $(am_tests_unit_test_build_fixes_OBJECTS) am__DEPENDENCIES_1 = tests_unit_test_build_fixes_DEPENDENCIES = $(am__DEPENDENCIES_1) -am_tests_unit_test_linked_OBJECTS = tests/unit/test_linked.$(OBJEXT) \ - tests/fuzz/stubs.$(OBJEXT) sql.$(OBJEXT) util.$(OBJEXT) \ - snmp.$(OBJEXT) src/core/locks.$(OBJEXT) poller.$(OBJEXT) \ - nft_popen.$(OBJEXT) php.$(OBJEXT) ping.$(OBJEXT) \ - src/core/keywords.$(OBJEXT) error.$(OBJEXT) +am_tests_unit_test_linked_OBJECTS = \ + tests/unit/test_linked-test_linked.$(OBJEXT) \ + tests/fuzz/unit_test_linked-stubs.$(OBJEXT) \ + tests_unit_test_linked-sql.$(OBJEXT) \ + tests_unit_test_linked-util.$(OBJEXT) \ + tests_unit_test_linked-snmp.$(OBJEXT) \ + src/core/tests_unit_test_linked-locks.$(OBJEXT) \ + tests_unit_test_linked-poller.$(OBJEXT) \ + tests_unit_test_linked-nft_popen.$(OBJEXT) \ + tests_unit_test_linked-php.$(OBJEXT) \ + tests_unit_test_linked-ping.$(OBJEXT) \ + src/core/tests_unit_test_linked-keywords.$(OBJEXT) \ + tests_unit_test_linked-error.$(OBJEXT) tests_unit_test_linked_OBJECTS = $(am_tests_unit_test_linked_OBJECTS) tests_unit_test_linked_DEPENDENCIES = $(am__DEPENDENCIES_1) \ $(am__DEPENDENCIES_1) +tests_unit_test_linked_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC \ + $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CCLD) \ + $(AM_CFLAGS) $(CFLAGS) $(tests_unit_test_linked_LDFLAGS) \ + $(LDFLAGS) -o $@ am_tests_unit_test_php_runtime_OBJECTS = \ tests/unit/test_php_runtime-test_php_runtime.$(OBJEXT) \ tests/fuzz/unit_test_php_runtime-stubs.$(OBJEXT) \ @@ -247,17 +311,42 @@ am__v_P_1 = : AM_V_GEN = $(am__v_GEN_@AM_V@) am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) am__v_GEN_0 = @echo " GEN " $@; -am__v_GEN_1 = +am__v_GEN_1 = AM_V_at = $(am__v_at_@AM_V@) am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) am__v_at_0 = @ -am__v_at_1 = +am__v_at_1 = DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir)/config depcomp = $(SHELL) $(top_srcdir)/config/depcomp am__maybe_remake_depfiles = depfiles am__depfiles_remade = ./$(DEPDIR)/error.Po ./$(DEPDIR)/nft_popen.Po \ ./$(DEPDIR)/php.Po ./$(DEPDIR)/ping.Po ./$(DEPDIR)/poller.Po \ ./$(DEPDIR)/snmp.Po ./$(DEPDIR)/spine.Po ./$(DEPDIR)/sql.Po \ + ./$(DEPDIR)/test_spine_faults-error.Po \ + ./$(DEPDIR)/test_spine_faults-nft_popen.Po \ + ./$(DEPDIR)/test_spine_faults-php.Po \ + ./$(DEPDIR)/test_spine_faults-ping.Po \ + ./$(DEPDIR)/test_spine_faults-poller.Po \ + ./$(DEPDIR)/test_spine_faults-snmp.Po \ + ./$(DEPDIR)/test_spine_faults-sql.Po \ + ./$(DEPDIR)/test_spine_faults-util.Po \ + ./$(DEPDIR)/test_spine_regressions-error.Po \ + ./$(DEPDIR)/test_spine_regressions-nft_popen.Po \ + ./$(DEPDIR)/test_spine_regressions-php.Po \ + ./$(DEPDIR)/test_spine_regressions-ping.Po \ + ./$(DEPDIR)/test_spine_regressions-poller.Po \ + ./$(DEPDIR)/test_spine_regressions-snmp.Po \ + ./$(DEPDIR)/test_spine_regressions-spine.Po \ + ./$(DEPDIR)/test_spine_regressions-sql.Po \ + ./$(DEPDIR)/test_spine_regressions-util.Po \ + ./$(DEPDIR)/tests_unit_test_linked-error.Po \ + ./$(DEPDIR)/tests_unit_test_linked-nft_popen.Po \ + ./$(DEPDIR)/tests_unit_test_linked-php.Po \ + ./$(DEPDIR)/tests_unit_test_linked-ping.Po \ + ./$(DEPDIR)/tests_unit_test_linked-poller.Po \ + ./$(DEPDIR)/tests_unit_test_linked-snmp.Po \ + ./$(DEPDIR)/tests_unit_test_linked-sql.Po \ + ./$(DEPDIR)/tests_unit_test_linked-util.Po \ ./$(DEPDIR)/tests_unit_test_php_runtime-error.Po \ ./$(DEPDIR)/tests_unit_test_php_runtime-nft_popen.Po \ ./$(DEPDIR)/tests_unit_test_php_runtime-php.Po \ @@ -268,13 +357,29 @@ am__depfiles_remade = ./$(DEPDIR)/error.Po ./$(DEPDIR)/nft_popen.Po \ ./$(DEPDIR)/tests_unit_test_php_runtime-util.Po \ ./$(DEPDIR)/util.Po src/core/$(DEPDIR)/keywords.Po \ src/core/$(DEPDIR)/locks.Po \ + src/core/$(DEPDIR)/test_spine_faults-keywords.Po \ + src/core/$(DEPDIR)/test_spine_faults-locks.Po \ + src/core/$(DEPDIR)/test_spine_regressions-keywords.Po \ + src/core/$(DEPDIR)/test_spine_regressions-locks.Po \ + src/core/$(DEPDIR)/tests_unit_test_linked-keywords.Po \ + src/core/$(DEPDIR)/tests_unit_test_linked-locks.Po \ src/core/$(DEPDIR)/tests_unit_test_php_runtime-keywords.Po \ src/core/$(DEPDIR)/tests_unit_test_php_runtime-locks.Po \ + tests/$(DEPDIR)/test_spine_faults-test_spine_faults.Po \ + tests/$(DEPDIR)/test_spine_regressions-test_spine_cli_contracts.Po \ + tests/$(DEPDIR)/test_spine_regressions-test_spine_contracts.Po \ + tests/$(DEPDIR)/test_spine_regressions-test_spine_output_contracts.Po \ + tests/$(DEPDIR)/test_spine_regressions-test_spine_regressions.Po \ + tests/$(DEPDIR)/test_spine_regressions-test_spine_reindex_contracts.Po \ + tests/$(DEPDIR)/test_spine_regressions-test_spine_settings_contracts.Po \ + tests/$(DEPDIR)/test_spine_regressions-test_spine_snmpv3_contracts.Po \ + tests/$(DEPDIR)/test_spine_regressions-test_spine_stream_contracts.Po \ tests/fuzz/$(DEPDIR)/stubs.Po \ + tests/fuzz/$(DEPDIR)/unit_test_linked-stubs.Po \ tests/fuzz/$(DEPDIR)/unit_test_php_runtime-stubs.Po \ tests/unit/$(DEPDIR)/php_test_server.Po \ tests/unit/$(DEPDIR)/test_build_fixes.Po \ - tests/unit/$(DEPDIR)/test_linked.Po \ + tests/unit/$(DEPDIR)/test_linked-test_linked.Po \ tests/unit/$(DEPDIR)/test_php_runtime-test_php_runtime.Po \ tests/unit/$(DEPDIR)/test_ping_icmp.Po \ tests/unit/$(DEPDIR)/test_safety_fixes.Po \ @@ -291,7 +396,7 @@ LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ AM_V_CC = $(am__v_CC_@AM_V@) am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@) am__v_CC_0 = @echo " CC " $@; -am__v_CC_1 = +am__v_CC_1 = CCLD = $(CC) LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ @@ -299,8 +404,10 @@ LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ AM_V_CCLD = $(am__v_CCLD_@AM_V@) am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) am__v_CCLD_0 = @echo " CCLD " $@; -am__v_CCLD_1 = -SOURCES = $(spine_SOURCES) $(tests_unit_php_test_server_SOURCES) \ +am__v_CCLD_1 = +SOURCES = $(spine_SOURCES) $(test_spine_faults_SOURCES) \ + $(test_spine_regressions_SOURCES) \ + $(tests_unit_php_test_server_SOURCES) \ $(tests_unit_test_build_fixes_SOURCES) \ $(tests_unit_test_linked_SOURCES) \ $(tests_unit_test_php_runtime_SOURCES) \ @@ -309,7 +416,8 @@ SOURCES = $(spine_SOURCES) $(tests_unit_php_test_server_SOURCES) \ $(tests_unit_test_snmpv3_session_SOURCES) \ $(tests_unit_test_util_paths_SOURCES) \ $(tests_unit_test_util_strings_SOURCES) -DIST_SOURCES = $(spine_SOURCES) \ +DIST_SOURCES = $(spine_SOURCES) $(am__test_spine_faults_SOURCES_DIST) \ + $(test_spine_regressions_SOURCES) \ $(am__tests_unit_php_test_server_SOURCES_DIST) \ $(tests_unit_test_build_fixes_SOURCES) \ $(tests_unit_test_linked_SOURCES) \ @@ -354,7 +462,7 @@ am__uninstall_files_from_dir = { \ man1dir = $(mandir)/man1 NROFF = nroff MANS = $(man_MANS) -DATA = $(config_DATA) +DATA = $(dist_config_DATA) $(dist_noinst_DATA) HEADERS = $(noinst_HEADERS) am__tagged_files = $(HEADERS) $(SOURCES) $(TAGS_FILES) $(LISP) # Read a list of newline-separated strings from the standard input, @@ -373,9 +481,6 @@ am__define_uniq_tagged_files = \ unique=`for i in $$list; do \ if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ done | $(am__uniquify_input)` -ETAGS = etags -CTAGS = ctags -CSCOPE = cscope AM_RECURSIVE_TARGETS = cscope check recheck am__tty_colors_dummy = \ mgn= red= grn= lgn= blu= brg= std=; \ @@ -532,6 +637,7 @@ am__set_TESTS_bases = \ bases='$(TEST_LOGS)'; \ bases=`for i in $$bases; do echo $$i; done | sed 's/\.log$$//'`; \ bases=`echo $$bases` +AM_TESTSUITE_SUMMARY_HEADER = ' for $(PACKAGE_STRING)' RECHECK_LOGS = $(TEST_LOGS) TEST_SUITE_LOG = test-suite.log LOG_DRIVER = $(SHELL) $(top_srcdir)/config/test-driver @@ -557,9 +663,9 @@ am__DIST_COMMON = $(srcdir)/Makefile.in $(top_srcdir)/config/compile \ $(top_srcdir)/config/config.sub $(top_srcdir)/config/depcomp \ $(top_srcdir)/config/install-sh $(top_srcdir)/config/ltmain.sh \ $(top_srcdir)/config/missing $(top_srcdir)/config/test-driver \ - INSTALL config/compile config/config.guess config/config.sub \ - config/depcomp config/install-sh config/ltmain.sh \ - config/missing + INSTALL README.md config/compile config/config.guess \ + config/config.sub config/depcomp config/install-sh \ + config/ltmain.sh config/missing DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) distdir = $(PACKAGE)-$(VERSION) top_distdir = $(distdir) @@ -573,6 +679,8 @@ am__post_remove_distdir = $(am__remove_distdir) DIST_ARCHIVES = $(distdir).tar.gz GZIP_ENV = --best DIST_TARGETS = dist-gzip +# Exists only to be overridden by the user if desired. +AM_DISTCHECK_DVI_TARGET = dvi distuninstallcheck_listfiles = find . -type f -print am__distuninstallcheck_listfiles = $(distuninstallcheck_listfiles) \ | sed 's|^\./|$(prefix)/|' | grep -v '$(infodir)/dir$$' @@ -591,6 +699,8 @@ CFLAGS = @CFLAGS@ CMOCKA_LIBS = @CMOCKA_LIBS@ CPP = @CPP@ CPPFLAGS = @CPPFLAGS@ +CSCOPE = @CSCOPE@ +CTAGS = @CTAGS@ CYGPATH_W = @CYGPATH_W@ DEFS = @DEFS@ DEPDIR = @DEPDIR@ @@ -601,8 +711,10 @@ ECHO_C = @ECHO_C@ ECHO_N = @ECHO_N@ ECHO_T = @ECHO_T@ EGREP = @EGREP@ +ETAGS = @ETAGS@ EXEEXT = @EXEEXT@ FGREP = @FGREP@ +FILECMD = @FILECMD@ GREP = @GREP@ HELP2MAN = @HELP2MAN@ INSTALL = @INSTALL@ @@ -699,25 +811,50 @@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ AUTOMAKE_OPTIONS = foreign ACLOCAL_AMFLAGS = -I m4 -spine_SOURCES = sql.c spine.c util.c snmp.c src/core/locks.c poller.c \ +spine_shared_sources = sql.c util.c snmp.c src/core/locks.c poller.c \ nft_popen.c php.c ping.c src/core/keywords.c error.c +spine_SOURCES = $(spine_shared_sources) spine.c configdir = $(sysconfdir) -config_DATA = spine.conf.dist +dist_config_DATA = spine.conf.dist man_MANS = spine.1 +DISTCLEANFILES = spine.1 +dist_noinst_DATA = tests/fixtures/configuration.sql tests/fixtures/poller.sql tests/fixtures/snmpd.conf tests/fixtures/snmpv3-agent.conf tests/fixtures/snmpv3-users.conf +TESTS = $(check_PROGRAMS) tests/regression/test_remote_push.sh \ + tests/integration/test_alloc_failure.sh \ + tests/regression/test_child_process_safety.sh +test_spine_regressions_SOURCES = $(spine_SOURCES) tests/test_spine_regressions.c tests/test_spine_contracts.c tests/test_spine_reindex_contracts.c tests/test_spine_snmpv3_contracts.c tests/test_spine_settings_contracts.c tests/test_spine_cli_contracts.c tests/test_spine_stream_contracts.c tests/test_spine_output_contracts.c +# Link the real production translation units and globals, with a separate test +# entry point. Only spine.c maps its own entry point in the test build. +test_spine_regressions_CPPFLAGS = $(AM_CPPFLAGS) -I$(srcdir) -DSPINE_TEST_PROGRAM_ENTRY +# Reuse the real renamed spine object for its production globals, avoiding +# another production main-renaming flag on the fault-test translation units. +@HAVE_LINKER_WRAP_TRUE@test_spine_faults_SOURCES = $(spine_shared_sources) tests/test_spine_faults.c +@HAVE_LINKER_WRAP_TRUE@test_spine_faults_CPPFLAGS = $(AM_CPPFLAGS) -I$(srcdir) +@HAVE_LINKER_WRAP_TRUE@test_spine_faults_LDADD = test_spine_regressions-spine.$(OBJEXT) +@HAVE_LINKER_WRAP_TRUE@test_spine_faults_LDFLAGS = $(AM_LDFLAGS) -Wl,--wrap=strftime -Wl,--wrap=mysql_query -Wl,--wrap=mysql_ping -Wl,--wrap=mysql_errno \ +@HAVE_LINKER_WRAP_TRUE@ -Wl,--wrap=pipe -Wl,--wrap=socketpair -Wl,--wrap=posix_spawn \ +@HAVE_LINKER_WRAP_TRUE@ -Wl,--wrap=malloc -Wl,--wrap=strdup \ +@HAVE_LINKER_WRAP_TRUE@ -Wl,--wrap=snmp_sess_open -Wl,--wrap=snmp_sess_close -Wl,--wrap=pthread_create + # Headers are not referenced by any _SOURCES, so automake will not distribute # them unless they are listed. Without this the release tarball builds nothing. noinst_HEADERS = common.h error.h src/core/keywords.h src/core/locks.h \ - nft_popen.h php.h ping.h poller.h snmp.h spine.h spine_sem.h sql.h \ - uthash.h util.h + nft_popen.h php.h ping.h poller.h snmp.h spine.h spine_sem.h spine_types.h \ + sql.h uthash.h util.h EXTRA_DIST = spine.1 spine.conf.dist tests/README.md \ tests/test-harness.sh tests/integration/test_concurrency.sh \ tests/integration/test_network_faults.sh \ + tests/tools/run_snmpv3.sh tests/tools/run_coverage.sh \ + tests/tools/coverage_process_exit.h \ + tests/tools/verify_coverage.sh \ + tests/tools/test_coverage_verifier.sh \ tests/regression/test_remote_push.sh \ tests/integration/test_alloc_failure.sh \ - tests/regression/test_child_process_safety.sh + tests/regression/test_child_process_safety.sh \ + scripts/cppcheck/spine.cfg AM_CPPFLAGS = -I$(top_srcdir) -I$(top_srcdir)/src/core -I$(top_builddir) @HAVE_CMOCKA_TRUE@tests_unit_php_test_server_SOURCES = tests/unit/php_test_server.c @@ -739,6 +876,18 @@ tests_unit_test_linked_SOURCES = tests/unit/test_linked.c tests/fuzz/stubs.c \ tests_unit_test_linked_LDADD = $(CMOCKA_LIBS) $(LIBS) +# waitpid() failing with anything other than ECHILD/EINTR has no reachable +# real-syscall trigger (WNOHANG is always a valid options value), so covering +# nft_pclose()'s defensive "waitpid itself failed" path needs a wrapped call. +# +# The base assignment must stay unconditional: once a target defines its own +# _CPPFLAGS anywhere, automake stops falling back to AM_CPPFLAGS for it on +# every configuration, including ones where HAVE_LD_WRAP is false (e.g. +# macOS's linker doesn't support --wrap) - that dropped the -I flags spine.h +# needs to find locks.h there. +tests_unit_test_linked_CPPFLAGS = $(AM_CPPFLAGS) $(am__append_4) +@HAVE_LD_WRAP_TRUE@tests_unit_test_linked_LDFLAGS = -Wl,--wrap=waitpid + # includes util.c, so it needs the same libraries spine links against tests_unit_test_util_paths_SOURCES = tests/unit/test_util_paths.c tests_unit_test_util_paths_LDADD = $(CMOCKA_LIBS) $(LIBS) @@ -769,7 +918,6 @@ tests_unit_test_php_runtime_CPPFLAGS = $(AM_CPPFLAGS) \ tests_unit_test_php_runtime_DEPENDENCIES = tests/unit/php_test_server$(EXEEXT) TEST_EXTENSIONS = .sh SH_LOG_COMPILER = $(SHELL) -TESTS = $(check_PROGRAMS) tests/regression/test_remote_push.sh tests/integration/test_alloc_failure.sh tests/regression/test_child_process_safety.sh TESTS_ENVIRONMENT = top_builddir='$(abs_top_builddir)' all: all-am @@ -816,7 +964,7 @@ config/config.h: config/stamp-h1 config/stamp-h1: $(top_srcdir)/config/config.h.in $(top_builddir)/config.status @rm -f config/stamp-h1 cd $(top_builddir) && $(SHELL) ./config.status config/config.h -$(top_srcdir)/config/config.h.in: $(am__configure_deps) +$(top_srcdir)/config/config.h.in: $(am__configure_deps) ($(am__cd) $(top_srcdir) && $(AUTOHEADER)) rm -f config/stamp-h1 touch $@ @@ -901,9 +1049,49 @@ src/core/locks.$(OBJEXT): src/core/$(am__dirstamp) \ src/core/keywords.$(OBJEXT): src/core/$(am__dirstamp) \ src/core/$(DEPDIR)/$(am__dirstamp) -spine$(EXEEXT): $(spine_OBJECTS) $(spine_DEPENDENCIES) $(EXTRA_spine_DEPENDENCIES) +spine$(EXEEXT): $(spine_OBJECTS) $(spine_DEPENDENCIES) $(EXTRA_spine_DEPENDENCIES) @rm -f spine$(EXEEXT) $(AM_V_CCLD)$(LINK) $(spine_OBJECTS) $(spine_LDADD) $(LIBS) +src/core/test_spine_faults-locks.$(OBJEXT): src/core/$(am__dirstamp) \ + src/core/$(DEPDIR)/$(am__dirstamp) +src/core/test_spine_faults-keywords.$(OBJEXT): \ + src/core/$(am__dirstamp) src/core/$(DEPDIR)/$(am__dirstamp) +tests/$(am__dirstamp): + @$(MKDIR_P) tests + @: > tests/$(am__dirstamp) +tests/$(DEPDIR)/$(am__dirstamp): + @$(MKDIR_P) tests/$(DEPDIR) + @: > tests/$(DEPDIR)/$(am__dirstamp) +tests/test_spine_faults-test_spine_faults.$(OBJEXT): \ + tests/$(am__dirstamp) tests/$(DEPDIR)/$(am__dirstamp) + +test_spine_faults$(EXEEXT): $(test_spine_faults_OBJECTS) $(test_spine_faults_DEPENDENCIES) $(EXTRA_test_spine_faults_DEPENDENCIES) + @rm -f test_spine_faults$(EXEEXT) + $(AM_V_CCLD)$(test_spine_faults_LINK) $(test_spine_faults_OBJECTS) $(test_spine_faults_LDADD) $(LIBS) +src/core/test_spine_regressions-locks.$(OBJEXT): \ + src/core/$(am__dirstamp) src/core/$(DEPDIR)/$(am__dirstamp) +src/core/test_spine_regressions-keywords.$(OBJEXT): \ + src/core/$(am__dirstamp) src/core/$(DEPDIR)/$(am__dirstamp) +tests/test_spine_regressions-test_spine_regressions.$(OBJEXT): \ + tests/$(am__dirstamp) tests/$(DEPDIR)/$(am__dirstamp) +tests/test_spine_regressions-test_spine_contracts.$(OBJEXT): \ + tests/$(am__dirstamp) tests/$(DEPDIR)/$(am__dirstamp) +tests/test_spine_regressions-test_spine_reindex_contracts.$(OBJEXT): \ + tests/$(am__dirstamp) tests/$(DEPDIR)/$(am__dirstamp) +tests/test_spine_regressions-test_spine_snmpv3_contracts.$(OBJEXT): \ + tests/$(am__dirstamp) tests/$(DEPDIR)/$(am__dirstamp) +tests/test_spine_regressions-test_spine_settings_contracts.$(OBJEXT): \ + tests/$(am__dirstamp) tests/$(DEPDIR)/$(am__dirstamp) +tests/test_spine_regressions-test_spine_cli_contracts.$(OBJEXT): \ + tests/$(am__dirstamp) tests/$(DEPDIR)/$(am__dirstamp) +tests/test_spine_regressions-test_spine_stream_contracts.$(OBJEXT): \ + tests/$(am__dirstamp) tests/$(DEPDIR)/$(am__dirstamp) +tests/test_spine_regressions-test_spine_output_contracts.$(OBJEXT): \ + tests/$(am__dirstamp) tests/$(DEPDIR)/$(am__dirstamp) + +test_spine_regressions$(EXEEXT): $(test_spine_regressions_OBJECTS) $(test_spine_regressions_DEPENDENCIES) $(EXTRA_test_spine_regressions_DEPENDENCIES) + @rm -f test_spine_regressions$(EXEEXT) + $(AM_V_CCLD)$(LINK) $(test_spine_regressions_OBJECTS) $(test_spine_regressions_LDADD) $(LIBS) tests/unit/$(am__dirstamp): @$(MKDIR_P) tests/unit @: > tests/unit/$(am__dirstamp) @@ -922,7 +1110,8 @@ tests/unit/test_build_fixes.$(OBJEXT): tests/unit/$(am__dirstamp) \ tests/unit/test_build_fixes$(EXEEXT): $(tests_unit_test_build_fixes_OBJECTS) $(tests_unit_test_build_fixes_DEPENDENCIES) $(EXTRA_tests_unit_test_build_fixes_DEPENDENCIES) tests/unit/$(am__dirstamp) @rm -f tests/unit/test_build_fixes$(EXEEXT) $(AM_V_CCLD)$(LINK) $(tests_unit_test_build_fixes_OBJECTS) $(tests_unit_test_build_fixes_LDADD) $(LIBS) -tests/unit/test_linked.$(OBJEXT): tests/unit/$(am__dirstamp) \ +tests/unit/test_linked-test_linked.$(OBJEXT): \ + tests/unit/$(am__dirstamp) \ tests/unit/$(DEPDIR)/$(am__dirstamp) tests/fuzz/$(am__dirstamp): @$(MKDIR_P) tests/fuzz @@ -930,12 +1119,17 @@ tests/fuzz/$(am__dirstamp): tests/fuzz/$(DEPDIR)/$(am__dirstamp): @$(MKDIR_P) tests/fuzz/$(DEPDIR) @: > tests/fuzz/$(DEPDIR)/$(am__dirstamp) -tests/fuzz/stubs.$(OBJEXT): tests/fuzz/$(am__dirstamp) \ +tests/fuzz/unit_test_linked-stubs.$(OBJEXT): \ + tests/fuzz/$(am__dirstamp) \ tests/fuzz/$(DEPDIR)/$(am__dirstamp) +src/core/tests_unit_test_linked-locks.$(OBJEXT): \ + src/core/$(am__dirstamp) src/core/$(DEPDIR)/$(am__dirstamp) +src/core/tests_unit_test_linked-keywords.$(OBJEXT): \ + src/core/$(am__dirstamp) src/core/$(DEPDIR)/$(am__dirstamp) tests/unit/test_linked$(EXEEXT): $(tests_unit_test_linked_OBJECTS) $(tests_unit_test_linked_DEPENDENCIES) $(EXTRA_tests_unit_test_linked_DEPENDENCIES) tests/unit/$(am__dirstamp) @rm -f tests/unit/test_linked$(EXEEXT) - $(AM_V_CCLD)$(LINK) $(tests_unit_test_linked_OBJECTS) $(tests_unit_test_linked_LDADD) $(LIBS) + $(AM_V_CCLD)$(tests_unit_test_linked_LINK) $(tests_unit_test_linked_OBJECTS) $(tests_unit_test_linked_LDADD) $(LIBS) tests/unit/test_php_runtime-test_php_runtime.$(OBJEXT): \ tests/unit/$(am__dirstamp) \ tests/unit/$(DEPDIR)/$(am__dirstamp) @@ -952,6 +1146,8 @@ tests/unit/test_php_runtime$(EXEEXT): $(tests_unit_test_php_runtime_OBJECTS) $(t $(AM_V_CCLD)$(tests_unit_test_php_runtime_LINK) $(tests_unit_test_php_runtime_OBJECTS) $(tests_unit_test_php_runtime_LDADD) $(LIBS) tests/unit/test_ping_icmp.$(OBJEXT): tests/unit/$(am__dirstamp) \ tests/unit/$(DEPDIR)/$(am__dirstamp) +tests/fuzz/stubs.$(OBJEXT): tests/fuzz/$(am__dirstamp) \ + tests/fuzz/$(DEPDIR)/$(am__dirstamp) tests/unit/test_ping_icmp$(EXEEXT): $(tests_unit_test_ping_icmp_OBJECTS) $(tests_unit_test_ping_icmp_DEPENDENCIES) $(EXTRA_tests_unit_test_ping_icmp_DEPENDENCIES) tests/unit/$(am__dirstamp) @rm -f tests/unit/test_ping_icmp$(EXEEXT) @@ -984,6 +1180,7 @@ tests/unit/test_util_strings$(EXEEXT): $(tests_unit_test_util_strings_OBJECTS) $ mostlyclean-compile: -rm -f *.$(OBJEXT) -rm -f src/core/*.$(OBJEXT) + -rm -f tests/*.$(OBJEXT) -rm -f tests/fuzz/*.$(OBJEXT) -rm -f tests/unit/*.$(OBJEXT) @@ -998,6 +1195,31 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/snmp.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/spine.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/sql.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_faults-error.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_faults-nft_popen.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_faults-php.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_faults-ping.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_faults-poller.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_faults-snmp.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_faults-sql.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_faults-util.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_regressions-error.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_regressions-nft_popen.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_regressions-php.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_regressions-ping.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_regressions-poller.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_regressions-snmp.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_regressions-spine.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_regressions-sql.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/test_spine_regressions-util.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_linked-error.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_linked-nft_popen.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_linked-php.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_linked-ping.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_linked-poller.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_linked-snmp.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_linked-sql.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_linked-util.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_php_runtime-error.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_php_runtime-nft_popen.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/tests_unit_test_php_runtime-php.Po@am__quote@ # am--include-marker @@ -1009,13 +1231,29 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/util.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@src/core/$(DEPDIR)/keywords.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@src/core/$(DEPDIR)/locks.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@src/core/$(DEPDIR)/test_spine_faults-keywords.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@src/core/$(DEPDIR)/test_spine_faults-locks.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@src/core/$(DEPDIR)/test_spine_regressions-keywords.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@src/core/$(DEPDIR)/test_spine_regressions-locks.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@src/core/$(DEPDIR)/tests_unit_test_linked-keywords.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@src/core/$(DEPDIR)/tests_unit_test_linked-locks.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@src/core/$(DEPDIR)/tests_unit_test_php_runtime-keywords.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@src/core/$(DEPDIR)/tests_unit_test_php_runtime-locks.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/$(DEPDIR)/test_spine_faults-test_spine_faults.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/$(DEPDIR)/test_spine_regressions-test_spine_cli_contracts.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/$(DEPDIR)/test_spine_regressions-test_spine_contracts.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/$(DEPDIR)/test_spine_regressions-test_spine_output_contracts.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/$(DEPDIR)/test_spine_regressions-test_spine_regressions.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/$(DEPDIR)/test_spine_regressions-test_spine_reindex_contracts.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/$(DEPDIR)/test_spine_regressions-test_spine_settings_contracts.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/$(DEPDIR)/test_spine_regressions-test_spine_snmpv3_contracts.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/$(DEPDIR)/test_spine_regressions-test_spine_stream_contracts.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@tests/fuzz/$(DEPDIR)/stubs.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/fuzz/$(DEPDIR)/unit_test_linked-stubs.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@tests/fuzz/$(DEPDIR)/unit_test_php_runtime-stubs.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@tests/unit/$(DEPDIR)/php_test_server.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@tests/unit/$(DEPDIR)/test_build_fixes.Po@am__quote@ # am--include-marker -@AMDEP_TRUE@@am__include@ @am__quote@tests/unit/$(DEPDIR)/test_linked.Po@am__quote@ # am--include-marker +@AMDEP_TRUE@@am__include@ @am__quote@tests/unit/$(DEPDIR)/test_linked-test_linked.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@tests/unit/$(DEPDIR)/test_php_runtime-test_php_runtime.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@tests/unit/$(DEPDIR)/test_ping_icmp.Po@am__quote@ # am--include-marker @AMDEP_TRUE@@am__include@ @am__quote@tests/unit/$(DEPDIR)/test_safety_fixes.Po@am__quote@ # am--include-marker @@ -1053,6 +1291,594 @@ am--depfiles: $(am__depfiles_remade) @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< +test_spine_faults-sql.o: sql.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-sql.o -MD -MP -MF $(DEPDIR)/test_spine_faults-sql.Tpo -c -o test_spine_faults-sql.o `test -f 'sql.c' || echo '$(srcdir)/'`sql.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-sql.Tpo $(DEPDIR)/test_spine_faults-sql.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='sql.c' object='test_spine_faults-sql.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-sql.o `test -f 'sql.c' || echo '$(srcdir)/'`sql.c + +test_spine_faults-sql.obj: sql.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-sql.obj -MD -MP -MF $(DEPDIR)/test_spine_faults-sql.Tpo -c -o test_spine_faults-sql.obj `if test -f 'sql.c'; then $(CYGPATH_W) 'sql.c'; else $(CYGPATH_W) '$(srcdir)/sql.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-sql.Tpo $(DEPDIR)/test_spine_faults-sql.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='sql.c' object='test_spine_faults-sql.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-sql.obj `if test -f 'sql.c'; then $(CYGPATH_W) 'sql.c'; else $(CYGPATH_W) '$(srcdir)/sql.c'; fi` + +test_spine_faults-util.o: util.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-util.o -MD -MP -MF $(DEPDIR)/test_spine_faults-util.Tpo -c -o test_spine_faults-util.o `test -f 'util.c' || echo '$(srcdir)/'`util.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-util.Tpo $(DEPDIR)/test_spine_faults-util.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='util.c' object='test_spine_faults-util.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-util.o `test -f 'util.c' || echo '$(srcdir)/'`util.c + +test_spine_faults-util.obj: util.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-util.obj -MD -MP -MF $(DEPDIR)/test_spine_faults-util.Tpo -c -o test_spine_faults-util.obj `if test -f 'util.c'; then $(CYGPATH_W) 'util.c'; else $(CYGPATH_W) '$(srcdir)/util.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-util.Tpo $(DEPDIR)/test_spine_faults-util.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='util.c' object='test_spine_faults-util.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-util.obj `if test -f 'util.c'; then $(CYGPATH_W) 'util.c'; else $(CYGPATH_W) '$(srcdir)/util.c'; fi` + +test_spine_faults-snmp.o: snmp.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-snmp.o -MD -MP -MF $(DEPDIR)/test_spine_faults-snmp.Tpo -c -o test_spine_faults-snmp.o `test -f 'snmp.c' || echo '$(srcdir)/'`snmp.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-snmp.Tpo $(DEPDIR)/test_spine_faults-snmp.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='snmp.c' object='test_spine_faults-snmp.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-snmp.o `test -f 'snmp.c' || echo '$(srcdir)/'`snmp.c + +test_spine_faults-snmp.obj: snmp.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-snmp.obj -MD -MP -MF $(DEPDIR)/test_spine_faults-snmp.Tpo -c -o test_spine_faults-snmp.obj `if test -f 'snmp.c'; then $(CYGPATH_W) 'snmp.c'; else $(CYGPATH_W) '$(srcdir)/snmp.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-snmp.Tpo $(DEPDIR)/test_spine_faults-snmp.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='snmp.c' object='test_spine_faults-snmp.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-snmp.obj `if test -f 'snmp.c'; then $(CYGPATH_W) 'snmp.c'; else $(CYGPATH_W) '$(srcdir)/snmp.c'; fi` + +src/core/test_spine_faults-locks.o: src/core/locks.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/test_spine_faults-locks.o -MD -MP -MF src/core/$(DEPDIR)/test_spine_faults-locks.Tpo -c -o src/core/test_spine_faults-locks.o `test -f 'src/core/locks.c' || echo '$(srcdir)/'`src/core/locks.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/test_spine_faults-locks.Tpo src/core/$(DEPDIR)/test_spine_faults-locks.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/locks.c' object='src/core/test_spine_faults-locks.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/test_spine_faults-locks.o `test -f 'src/core/locks.c' || echo '$(srcdir)/'`src/core/locks.c + +src/core/test_spine_faults-locks.obj: src/core/locks.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/test_spine_faults-locks.obj -MD -MP -MF src/core/$(DEPDIR)/test_spine_faults-locks.Tpo -c -o src/core/test_spine_faults-locks.obj `if test -f 'src/core/locks.c'; then $(CYGPATH_W) 'src/core/locks.c'; else $(CYGPATH_W) '$(srcdir)/src/core/locks.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/test_spine_faults-locks.Tpo src/core/$(DEPDIR)/test_spine_faults-locks.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/locks.c' object='src/core/test_spine_faults-locks.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/test_spine_faults-locks.obj `if test -f 'src/core/locks.c'; then $(CYGPATH_W) 'src/core/locks.c'; else $(CYGPATH_W) '$(srcdir)/src/core/locks.c'; fi` + +test_spine_faults-poller.o: poller.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-poller.o -MD -MP -MF $(DEPDIR)/test_spine_faults-poller.Tpo -c -o test_spine_faults-poller.o `test -f 'poller.c' || echo '$(srcdir)/'`poller.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-poller.Tpo $(DEPDIR)/test_spine_faults-poller.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='poller.c' object='test_spine_faults-poller.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-poller.o `test -f 'poller.c' || echo '$(srcdir)/'`poller.c + +test_spine_faults-poller.obj: poller.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-poller.obj -MD -MP -MF $(DEPDIR)/test_spine_faults-poller.Tpo -c -o test_spine_faults-poller.obj `if test -f 'poller.c'; then $(CYGPATH_W) 'poller.c'; else $(CYGPATH_W) '$(srcdir)/poller.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-poller.Tpo $(DEPDIR)/test_spine_faults-poller.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='poller.c' object='test_spine_faults-poller.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-poller.obj `if test -f 'poller.c'; then $(CYGPATH_W) 'poller.c'; else $(CYGPATH_W) '$(srcdir)/poller.c'; fi` + +test_spine_faults-nft_popen.o: nft_popen.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-nft_popen.o -MD -MP -MF $(DEPDIR)/test_spine_faults-nft_popen.Tpo -c -o test_spine_faults-nft_popen.o `test -f 'nft_popen.c' || echo '$(srcdir)/'`nft_popen.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-nft_popen.Tpo $(DEPDIR)/test_spine_faults-nft_popen.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='nft_popen.c' object='test_spine_faults-nft_popen.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-nft_popen.o `test -f 'nft_popen.c' || echo '$(srcdir)/'`nft_popen.c + +test_spine_faults-nft_popen.obj: nft_popen.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-nft_popen.obj -MD -MP -MF $(DEPDIR)/test_spine_faults-nft_popen.Tpo -c -o test_spine_faults-nft_popen.obj `if test -f 'nft_popen.c'; then $(CYGPATH_W) 'nft_popen.c'; else $(CYGPATH_W) '$(srcdir)/nft_popen.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-nft_popen.Tpo $(DEPDIR)/test_spine_faults-nft_popen.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='nft_popen.c' object='test_spine_faults-nft_popen.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-nft_popen.obj `if test -f 'nft_popen.c'; then $(CYGPATH_W) 'nft_popen.c'; else $(CYGPATH_W) '$(srcdir)/nft_popen.c'; fi` + +test_spine_faults-php.o: php.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-php.o -MD -MP -MF $(DEPDIR)/test_spine_faults-php.Tpo -c -o test_spine_faults-php.o `test -f 'php.c' || echo '$(srcdir)/'`php.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-php.Tpo $(DEPDIR)/test_spine_faults-php.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='php.c' object='test_spine_faults-php.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-php.o `test -f 'php.c' || echo '$(srcdir)/'`php.c + +test_spine_faults-php.obj: php.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-php.obj -MD -MP -MF $(DEPDIR)/test_spine_faults-php.Tpo -c -o test_spine_faults-php.obj `if test -f 'php.c'; then $(CYGPATH_W) 'php.c'; else $(CYGPATH_W) '$(srcdir)/php.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-php.Tpo $(DEPDIR)/test_spine_faults-php.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='php.c' object='test_spine_faults-php.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-php.obj `if test -f 'php.c'; then $(CYGPATH_W) 'php.c'; else $(CYGPATH_W) '$(srcdir)/php.c'; fi` + +test_spine_faults-ping.o: ping.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-ping.o -MD -MP -MF $(DEPDIR)/test_spine_faults-ping.Tpo -c -o test_spine_faults-ping.o `test -f 'ping.c' || echo '$(srcdir)/'`ping.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-ping.Tpo $(DEPDIR)/test_spine_faults-ping.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='ping.c' object='test_spine_faults-ping.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-ping.o `test -f 'ping.c' || echo '$(srcdir)/'`ping.c + +test_spine_faults-ping.obj: ping.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-ping.obj -MD -MP -MF $(DEPDIR)/test_spine_faults-ping.Tpo -c -o test_spine_faults-ping.obj `if test -f 'ping.c'; then $(CYGPATH_W) 'ping.c'; else $(CYGPATH_W) '$(srcdir)/ping.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-ping.Tpo $(DEPDIR)/test_spine_faults-ping.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='ping.c' object='test_spine_faults-ping.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-ping.obj `if test -f 'ping.c'; then $(CYGPATH_W) 'ping.c'; else $(CYGPATH_W) '$(srcdir)/ping.c'; fi` + +src/core/test_spine_faults-keywords.o: src/core/keywords.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/test_spine_faults-keywords.o -MD -MP -MF src/core/$(DEPDIR)/test_spine_faults-keywords.Tpo -c -o src/core/test_spine_faults-keywords.o `test -f 'src/core/keywords.c' || echo '$(srcdir)/'`src/core/keywords.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/test_spine_faults-keywords.Tpo src/core/$(DEPDIR)/test_spine_faults-keywords.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/keywords.c' object='src/core/test_spine_faults-keywords.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/test_spine_faults-keywords.o `test -f 'src/core/keywords.c' || echo '$(srcdir)/'`src/core/keywords.c + +src/core/test_spine_faults-keywords.obj: src/core/keywords.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/test_spine_faults-keywords.obj -MD -MP -MF src/core/$(DEPDIR)/test_spine_faults-keywords.Tpo -c -o src/core/test_spine_faults-keywords.obj `if test -f 'src/core/keywords.c'; then $(CYGPATH_W) 'src/core/keywords.c'; else $(CYGPATH_W) '$(srcdir)/src/core/keywords.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/test_spine_faults-keywords.Tpo src/core/$(DEPDIR)/test_spine_faults-keywords.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/keywords.c' object='src/core/test_spine_faults-keywords.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/test_spine_faults-keywords.obj `if test -f 'src/core/keywords.c'; then $(CYGPATH_W) 'src/core/keywords.c'; else $(CYGPATH_W) '$(srcdir)/src/core/keywords.c'; fi` + +test_spine_faults-error.o: error.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-error.o -MD -MP -MF $(DEPDIR)/test_spine_faults-error.Tpo -c -o test_spine_faults-error.o `test -f 'error.c' || echo '$(srcdir)/'`error.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-error.Tpo $(DEPDIR)/test_spine_faults-error.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='error.c' object='test_spine_faults-error.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-error.o `test -f 'error.c' || echo '$(srcdir)/'`error.c + +test_spine_faults-error.obj: error.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_faults-error.obj -MD -MP -MF $(DEPDIR)/test_spine_faults-error.Tpo -c -o test_spine_faults-error.obj `if test -f 'error.c'; then $(CYGPATH_W) 'error.c'; else $(CYGPATH_W) '$(srcdir)/error.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_faults-error.Tpo $(DEPDIR)/test_spine_faults-error.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='error.c' object='test_spine_faults-error.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_faults-error.obj `if test -f 'error.c'; then $(CYGPATH_W) 'error.c'; else $(CYGPATH_W) '$(srcdir)/error.c'; fi` + +tests/test_spine_faults-test_spine_faults.o: tests/test_spine_faults.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_faults-test_spine_faults.o -MD -MP -MF tests/$(DEPDIR)/test_spine_faults-test_spine_faults.Tpo -c -o tests/test_spine_faults-test_spine_faults.o `test -f 'tests/test_spine_faults.c' || echo '$(srcdir)/'`tests/test_spine_faults.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_faults-test_spine_faults.Tpo tests/$(DEPDIR)/test_spine_faults-test_spine_faults.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_faults.c' object='tests/test_spine_faults-test_spine_faults.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_faults-test_spine_faults.o `test -f 'tests/test_spine_faults.c' || echo '$(srcdir)/'`tests/test_spine_faults.c + +tests/test_spine_faults-test_spine_faults.obj: tests/test_spine_faults.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_faults-test_spine_faults.obj -MD -MP -MF tests/$(DEPDIR)/test_spine_faults-test_spine_faults.Tpo -c -o tests/test_spine_faults-test_spine_faults.obj `if test -f 'tests/test_spine_faults.c'; then $(CYGPATH_W) 'tests/test_spine_faults.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_faults.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_faults-test_spine_faults.Tpo tests/$(DEPDIR)/test_spine_faults-test_spine_faults.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_faults.c' object='tests/test_spine_faults-test_spine_faults.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_faults_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_faults-test_spine_faults.obj `if test -f 'tests/test_spine_faults.c'; then $(CYGPATH_W) 'tests/test_spine_faults.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_faults.c'; fi` + +test_spine_regressions-sql.o: sql.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-sql.o -MD -MP -MF $(DEPDIR)/test_spine_regressions-sql.Tpo -c -o test_spine_regressions-sql.o `test -f 'sql.c' || echo '$(srcdir)/'`sql.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-sql.Tpo $(DEPDIR)/test_spine_regressions-sql.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='sql.c' object='test_spine_regressions-sql.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-sql.o `test -f 'sql.c' || echo '$(srcdir)/'`sql.c + +test_spine_regressions-sql.obj: sql.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-sql.obj -MD -MP -MF $(DEPDIR)/test_spine_regressions-sql.Tpo -c -o test_spine_regressions-sql.obj `if test -f 'sql.c'; then $(CYGPATH_W) 'sql.c'; else $(CYGPATH_W) '$(srcdir)/sql.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-sql.Tpo $(DEPDIR)/test_spine_regressions-sql.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='sql.c' object='test_spine_regressions-sql.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-sql.obj `if test -f 'sql.c'; then $(CYGPATH_W) 'sql.c'; else $(CYGPATH_W) '$(srcdir)/sql.c'; fi` + +test_spine_regressions-util.o: util.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-util.o -MD -MP -MF $(DEPDIR)/test_spine_regressions-util.Tpo -c -o test_spine_regressions-util.o `test -f 'util.c' || echo '$(srcdir)/'`util.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-util.Tpo $(DEPDIR)/test_spine_regressions-util.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='util.c' object='test_spine_regressions-util.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-util.o `test -f 'util.c' || echo '$(srcdir)/'`util.c + +test_spine_regressions-util.obj: util.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-util.obj -MD -MP -MF $(DEPDIR)/test_spine_regressions-util.Tpo -c -o test_spine_regressions-util.obj `if test -f 'util.c'; then $(CYGPATH_W) 'util.c'; else $(CYGPATH_W) '$(srcdir)/util.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-util.Tpo $(DEPDIR)/test_spine_regressions-util.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='util.c' object='test_spine_regressions-util.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-util.obj `if test -f 'util.c'; then $(CYGPATH_W) 'util.c'; else $(CYGPATH_W) '$(srcdir)/util.c'; fi` + +test_spine_regressions-snmp.o: snmp.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-snmp.o -MD -MP -MF $(DEPDIR)/test_spine_regressions-snmp.Tpo -c -o test_spine_regressions-snmp.o `test -f 'snmp.c' || echo '$(srcdir)/'`snmp.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-snmp.Tpo $(DEPDIR)/test_spine_regressions-snmp.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='snmp.c' object='test_spine_regressions-snmp.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-snmp.o `test -f 'snmp.c' || echo '$(srcdir)/'`snmp.c + +test_spine_regressions-snmp.obj: snmp.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-snmp.obj -MD -MP -MF $(DEPDIR)/test_spine_regressions-snmp.Tpo -c -o test_spine_regressions-snmp.obj `if test -f 'snmp.c'; then $(CYGPATH_W) 'snmp.c'; else $(CYGPATH_W) '$(srcdir)/snmp.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-snmp.Tpo $(DEPDIR)/test_spine_regressions-snmp.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='snmp.c' object='test_spine_regressions-snmp.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-snmp.obj `if test -f 'snmp.c'; then $(CYGPATH_W) 'snmp.c'; else $(CYGPATH_W) '$(srcdir)/snmp.c'; fi` + +src/core/test_spine_regressions-locks.o: src/core/locks.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/test_spine_regressions-locks.o -MD -MP -MF src/core/$(DEPDIR)/test_spine_regressions-locks.Tpo -c -o src/core/test_spine_regressions-locks.o `test -f 'src/core/locks.c' || echo '$(srcdir)/'`src/core/locks.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/test_spine_regressions-locks.Tpo src/core/$(DEPDIR)/test_spine_regressions-locks.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/locks.c' object='src/core/test_spine_regressions-locks.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/test_spine_regressions-locks.o `test -f 'src/core/locks.c' || echo '$(srcdir)/'`src/core/locks.c + +src/core/test_spine_regressions-locks.obj: src/core/locks.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/test_spine_regressions-locks.obj -MD -MP -MF src/core/$(DEPDIR)/test_spine_regressions-locks.Tpo -c -o src/core/test_spine_regressions-locks.obj `if test -f 'src/core/locks.c'; then $(CYGPATH_W) 'src/core/locks.c'; else $(CYGPATH_W) '$(srcdir)/src/core/locks.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/test_spine_regressions-locks.Tpo src/core/$(DEPDIR)/test_spine_regressions-locks.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/locks.c' object='src/core/test_spine_regressions-locks.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/test_spine_regressions-locks.obj `if test -f 'src/core/locks.c'; then $(CYGPATH_W) 'src/core/locks.c'; else $(CYGPATH_W) '$(srcdir)/src/core/locks.c'; fi` + +test_spine_regressions-poller.o: poller.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-poller.o -MD -MP -MF $(DEPDIR)/test_spine_regressions-poller.Tpo -c -o test_spine_regressions-poller.o `test -f 'poller.c' || echo '$(srcdir)/'`poller.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-poller.Tpo $(DEPDIR)/test_spine_regressions-poller.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='poller.c' object='test_spine_regressions-poller.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-poller.o `test -f 'poller.c' || echo '$(srcdir)/'`poller.c + +test_spine_regressions-poller.obj: poller.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-poller.obj -MD -MP -MF $(DEPDIR)/test_spine_regressions-poller.Tpo -c -o test_spine_regressions-poller.obj `if test -f 'poller.c'; then $(CYGPATH_W) 'poller.c'; else $(CYGPATH_W) '$(srcdir)/poller.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-poller.Tpo $(DEPDIR)/test_spine_regressions-poller.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='poller.c' object='test_spine_regressions-poller.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-poller.obj `if test -f 'poller.c'; then $(CYGPATH_W) 'poller.c'; else $(CYGPATH_W) '$(srcdir)/poller.c'; fi` + +test_spine_regressions-nft_popen.o: nft_popen.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-nft_popen.o -MD -MP -MF $(DEPDIR)/test_spine_regressions-nft_popen.Tpo -c -o test_spine_regressions-nft_popen.o `test -f 'nft_popen.c' || echo '$(srcdir)/'`nft_popen.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-nft_popen.Tpo $(DEPDIR)/test_spine_regressions-nft_popen.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='nft_popen.c' object='test_spine_regressions-nft_popen.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-nft_popen.o `test -f 'nft_popen.c' || echo '$(srcdir)/'`nft_popen.c + +test_spine_regressions-nft_popen.obj: nft_popen.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-nft_popen.obj -MD -MP -MF $(DEPDIR)/test_spine_regressions-nft_popen.Tpo -c -o test_spine_regressions-nft_popen.obj `if test -f 'nft_popen.c'; then $(CYGPATH_W) 'nft_popen.c'; else $(CYGPATH_W) '$(srcdir)/nft_popen.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-nft_popen.Tpo $(DEPDIR)/test_spine_regressions-nft_popen.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='nft_popen.c' object='test_spine_regressions-nft_popen.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-nft_popen.obj `if test -f 'nft_popen.c'; then $(CYGPATH_W) 'nft_popen.c'; else $(CYGPATH_W) '$(srcdir)/nft_popen.c'; fi` + +test_spine_regressions-php.o: php.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-php.o -MD -MP -MF $(DEPDIR)/test_spine_regressions-php.Tpo -c -o test_spine_regressions-php.o `test -f 'php.c' || echo '$(srcdir)/'`php.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-php.Tpo $(DEPDIR)/test_spine_regressions-php.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='php.c' object='test_spine_regressions-php.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-php.o `test -f 'php.c' || echo '$(srcdir)/'`php.c + +test_spine_regressions-php.obj: php.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-php.obj -MD -MP -MF $(DEPDIR)/test_spine_regressions-php.Tpo -c -o test_spine_regressions-php.obj `if test -f 'php.c'; then $(CYGPATH_W) 'php.c'; else $(CYGPATH_W) '$(srcdir)/php.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-php.Tpo $(DEPDIR)/test_spine_regressions-php.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='php.c' object='test_spine_regressions-php.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-php.obj `if test -f 'php.c'; then $(CYGPATH_W) 'php.c'; else $(CYGPATH_W) '$(srcdir)/php.c'; fi` + +test_spine_regressions-ping.o: ping.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-ping.o -MD -MP -MF $(DEPDIR)/test_spine_regressions-ping.Tpo -c -o test_spine_regressions-ping.o `test -f 'ping.c' || echo '$(srcdir)/'`ping.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-ping.Tpo $(DEPDIR)/test_spine_regressions-ping.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='ping.c' object='test_spine_regressions-ping.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-ping.o `test -f 'ping.c' || echo '$(srcdir)/'`ping.c + +test_spine_regressions-ping.obj: ping.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-ping.obj -MD -MP -MF $(DEPDIR)/test_spine_regressions-ping.Tpo -c -o test_spine_regressions-ping.obj `if test -f 'ping.c'; then $(CYGPATH_W) 'ping.c'; else $(CYGPATH_W) '$(srcdir)/ping.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-ping.Tpo $(DEPDIR)/test_spine_regressions-ping.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='ping.c' object='test_spine_regressions-ping.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-ping.obj `if test -f 'ping.c'; then $(CYGPATH_W) 'ping.c'; else $(CYGPATH_W) '$(srcdir)/ping.c'; fi` + +src/core/test_spine_regressions-keywords.o: src/core/keywords.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/test_spine_regressions-keywords.o -MD -MP -MF src/core/$(DEPDIR)/test_spine_regressions-keywords.Tpo -c -o src/core/test_spine_regressions-keywords.o `test -f 'src/core/keywords.c' || echo '$(srcdir)/'`src/core/keywords.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/test_spine_regressions-keywords.Tpo src/core/$(DEPDIR)/test_spine_regressions-keywords.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/keywords.c' object='src/core/test_spine_regressions-keywords.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/test_spine_regressions-keywords.o `test -f 'src/core/keywords.c' || echo '$(srcdir)/'`src/core/keywords.c + +src/core/test_spine_regressions-keywords.obj: src/core/keywords.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/test_spine_regressions-keywords.obj -MD -MP -MF src/core/$(DEPDIR)/test_spine_regressions-keywords.Tpo -c -o src/core/test_spine_regressions-keywords.obj `if test -f 'src/core/keywords.c'; then $(CYGPATH_W) 'src/core/keywords.c'; else $(CYGPATH_W) '$(srcdir)/src/core/keywords.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/test_spine_regressions-keywords.Tpo src/core/$(DEPDIR)/test_spine_regressions-keywords.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/keywords.c' object='src/core/test_spine_regressions-keywords.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/test_spine_regressions-keywords.obj `if test -f 'src/core/keywords.c'; then $(CYGPATH_W) 'src/core/keywords.c'; else $(CYGPATH_W) '$(srcdir)/src/core/keywords.c'; fi` + +test_spine_regressions-error.o: error.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-error.o -MD -MP -MF $(DEPDIR)/test_spine_regressions-error.Tpo -c -o test_spine_regressions-error.o `test -f 'error.c' || echo '$(srcdir)/'`error.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-error.Tpo $(DEPDIR)/test_spine_regressions-error.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='error.c' object='test_spine_regressions-error.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-error.o `test -f 'error.c' || echo '$(srcdir)/'`error.c + +test_spine_regressions-error.obj: error.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-error.obj -MD -MP -MF $(DEPDIR)/test_spine_regressions-error.Tpo -c -o test_spine_regressions-error.obj `if test -f 'error.c'; then $(CYGPATH_W) 'error.c'; else $(CYGPATH_W) '$(srcdir)/error.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-error.Tpo $(DEPDIR)/test_spine_regressions-error.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='error.c' object='test_spine_regressions-error.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-error.obj `if test -f 'error.c'; then $(CYGPATH_W) 'error.c'; else $(CYGPATH_W) '$(srcdir)/error.c'; fi` + +test_spine_regressions-spine.o: spine.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-spine.o -MD -MP -MF $(DEPDIR)/test_spine_regressions-spine.Tpo -c -o test_spine_regressions-spine.o `test -f 'spine.c' || echo '$(srcdir)/'`spine.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-spine.Tpo $(DEPDIR)/test_spine_regressions-spine.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='spine.c' object='test_spine_regressions-spine.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-spine.o `test -f 'spine.c' || echo '$(srcdir)/'`spine.c + +test_spine_regressions-spine.obj: spine.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT test_spine_regressions-spine.obj -MD -MP -MF $(DEPDIR)/test_spine_regressions-spine.Tpo -c -o test_spine_regressions-spine.obj `if test -f 'spine.c'; then $(CYGPATH_W) 'spine.c'; else $(CYGPATH_W) '$(srcdir)/spine.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/test_spine_regressions-spine.Tpo $(DEPDIR)/test_spine_regressions-spine.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='spine.c' object='test_spine_regressions-spine.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o test_spine_regressions-spine.obj `if test -f 'spine.c'; then $(CYGPATH_W) 'spine.c'; else $(CYGPATH_W) '$(srcdir)/spine.c'; fi` + +tests/test_spine_regressions-test_spine_regressions.o: tests/test_spine_regressions.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_regressions.o -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_regressions.Tpo -c -o tests/test_spine_regressions-test_spine_regressions.o `test -f 'tests/test_spine_regressions.c' || echo '$(srcdir)/'`tests/test_spine_regressions.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_regressions.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_regressions.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_regressions.c' object='tests/test_spine_regressions-test_spine_regressions.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_regressions.o `test -f 'tests/test_spine_regressions.c' || echo '$(srcdir)/'`tests/test_spine_regressions.c + +tests/test_spine_regressions-test_spine_regressions.obj: tests/test_spine_regressions.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_regressions.obj -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_regressions.Tpo -c -o tests/test_spine_regressions-test_spine_regressions.obj `if test -f 'tests/test_spine_regressions.c'; then $(CYGPATH_W) 'tests/test_spine_regressions.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_regressions.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_regressions.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_regressions.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_regressions.c' object='tests/test_spine_regressions-test_spine_regressions.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_regressions.obj `if test -f 'tests/test_spine_regressions.c'; then $(CYGPATH_W) 'tests/test_spine_regressions.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_regressions.c'; fi` + +tests/test_spine_regressions-test_spine_contracts.o: tests/test_spine_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_contracts.o -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_contracts.o `test -f 'tests/test_spine_contracts.c' || echo '$(srcdir)/'`tests/test_spine_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_contracts.c' object='tests/test_spine_regressions-test_spine_contracts.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_contracts.o `test -f 'tests/test_spine_contracts.c' || echo '$(srcdir)/'`tests/test_spine_contracts.c + +tests/test_spine_regressions-test_spine_contracts.obj: tests/test_spine_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_contracts.obj -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_contracts.obj `if test -f 'tests/test_spine_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_contracts.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_contracts.c' object='tests/test_spine_regressions-test_spine_contracts.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_contracts.obj `if test -f 'tests/test_spine_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_contracts.c'; fi` + +tests/test_spine_regressions-test_spine_reindex_contracts.o: tests/test_spine_reindex_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_reindex_contracts.o -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_reindex_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_reindex_contracts.o `test -f 'tests/test_spine_reindex_contracts.c' || echo '$(srcdir)/'`tests/test_spine_reindex_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_reindex_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_reindex_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_reindex_contracts.c' object='tests/test_spine_regressions-test_spine_reindex_contracts.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_reindex_contracts.o `test -f 'tests/test_spine_reindex_contracts.c' || echo '$(srcdir)/'`tests/test_spine_reindex_contracts.c + +tests/test_spine_regressions-test_spine_reindex_contracts.obj: tests/test_spine_reindex_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_reindex_contracts.obj -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_reindex_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_reindex_contracts.obj `if test -f 'tests/test_spine_reindex_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_reindex_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_reindex_contracts.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_reindex_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_reindex_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_reindex_contracts.c' object='tests/test_spine_regressions-test_spine_reindex_contracts.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_reindex_contracts.obj `if test -f 'tests/test_spine_reindex_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_reindex_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_reindex_contracts.c'; fi` + +tests/test_spine_regressions-test_spine_snmpv3_contracts.o: tests/test_spine_snmpv3_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_snmpv3_contracts.o -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_snmpv3_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_snmpv3_contracts.o `test -f 'tests/test_spine_snmpv3_contracts.c' || echo '$(srcdir)/'`tests/test_spine_snmpv3_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_snmpv3_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_snmpv3_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_snmpv3_contracts.c' object='tests/test_spine_regressions-test_spine_snmpv3_contracts.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_snmpv3_contracts.o `test -f 'tests/test_spine_snmpv3_contracts.c' || echo '$(srcdir)/'`tests/test_spine_snmpv3_contracts.c + +tests/test_spine_regressions-test_spine_snmpv3_contracts.obj: tests/test_spine_snmpv3_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_snmpv3_contracts.obj -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_snmpv3_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_snmpv3_contracts.obj `if test -f 'tests/test_spine_snmpv3_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_snmpv3_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_snmpv3_contracts.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_snmpv3_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_snmpv3_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_snmpv3_contracts.c' object='tests/test_spine_regressions-test_spine_snmpv3_contracts.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_snmpv3_contracts.obj `if test -f 'tests/test_spine_snmpv3_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_snmpv3_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_snmpv3_contracts.c'; fi` + +tests/test_spine_regressions-test_spine_settings_contracts.o: tests/test_spine_settings_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_settings_contracts.o -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_settings_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_settings_contracts.o `test -f 'tests/test_spine_settings_contracts.c' || echo '$(srcdir)/'`tests/test_spine_settings_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_settings_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_settings_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_settings_contracts.c' object='tests/test_spine_regressions-test_spine_settings_contracts.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_settings_contracts.o `test -f 'tests/test_spine_settings_contracts.c' || echo '$(srcdir)/'`tests/test_spine_settings_contracts.c + +tests/test_spine_regressions-test_spine_settings_contracts.obj: tests/test_spine_settings_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_settings_contracts.obj -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_settings_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_settings_contracts.obj `if test -f 'tests/test_spine_settings_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_settings_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_settings_contracts.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_settings_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_settings_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_settings_contracts.c' object='tests/test_spine_regressions-test_spine_settings_contracts.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_settings_contracts.obj `if test -f 'tests/test_spine_settings_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_settings_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_settings_contracts.c'; fi` + +tests/test_spine_regressions-test_spine_cli_contracts.o: tests/test_spine_cli_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_cli_contracts.o -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_cli_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_cli_contracts.o `test -f 'tests/test_spine_cli_contracts.c' || echo '$(srcdir)/'`tests/test_spine_cli_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_cli_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_cli_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_cli_contracts.c' object='tests/test_spine_regressions-test_spine_cli_contracts.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_cli_contracts.o `test -f 'tests/test_spine_cli_contracts.c' || echo '$(srcdir)/'`tests/test_spine_cli_contracts.c + +tests/test_spine_regressions-test_spine_cli_contracts.obj: tests/test_spine_cli_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_cli_contracts.obj -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_cli_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_cli_contracts.obj `if test -f 'tests/test_spine_cli_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_cli_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_cli_contracts.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_cli_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_cli_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_cli_contracts.c' object='tests/test_spine_regressions-test_spine_cli_contracts.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_cli_contracts.obj `if test -f 'tests/test_spine_cli_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_cli_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_cli_contracts.c'; fi` + +tests/test_spine_regressions-test_spine_stream_contracts.o: tests/test_spine_stream_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_stream_contracts.o -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_stream_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_stream_contracts.o `test -f 'tests/test_spine_stream_contracts.c' || echo '$(srcdir)/'`tests/test_spine_stream_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_stream_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_stream_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_stream_contracts.c' object='tests/test_spine_regressions-test_spine_stream_contracts.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_stream_contracts.o `test -f 'tests/test_spine_stream_contracts.c' || echo '$(srcdir)/'`tests/test_spine_stream_contracts.c + +tests/test_spine_regressions-test_spine_stream_contracts.obj: tests/test_spine_stream_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_stream_contracts.obj -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_stream_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_stream_contracts.obj `if test -f 'tests/test_spine_stream_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_stream_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_stream_contracts.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_stream_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_stream_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_stream_contracts.c' object='tests/test_spine_regressions-test_spine_stream_contracts.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_stream_contracts.obj `if test -f 'tests/test_spine_stream_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_stream_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_stream_contracts.c'; fi` + +tests/test_spine_regressions-test_spine_output_contracts.o: tests/test_spine_output_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_output_contracts.o -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_output_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_output_contracts.o `test -f 'tests/test_spine_output_contracts.c' || echo '$(srcdir)/'`tests/test_spine_output_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_output_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_output_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_output_contracts.c' object='tests/test_spine_regressions-test_spine_output_contracts.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_output_contracts.o `test -f 'tests/test_spine_output_contracts.c' || echo '$(srcdir)/'`tests/test_spine_output_contracts.c + +tests/test_spine_regressions-test_spine_output_contracts.obj: tests/test_spine_output_contracts.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/test_spine_regressions-test_spine_output_contracts.obj -MD -MP -MF tests/$(DEPDIR)/test_spine_regressions-test_spine_output_contracts.Tpo -c -o tests/test_spine_regressions-test_spine_output_contracts.obj `if test -f 'tests/test_spine_output_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_output_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_output_contracts.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/$(DEPDIR)/test_spine_regressions-test_spine_output_contracts.Tpo tests/$(DEPDIR)/test_spine_regressions-test_spine_output_contracts.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/test_spine_output_contracts.c' object='tests/test_spine_regressions-test_spine_output_contracts.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(test_spine_regressions_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/test_spine_regressions-test_spine_output_contracts.obj `if test -f 'tests/test_spine_output_contracts.c'; then $(CYGPATH_W) 'tests/test_spine_output_contracts.c'; else $(CYGPATH_W) '$(srcdir)/tests/test_spine_output_contracts.c'; fi` + +tests/unit/test_linked-test_linked.o: tests/unit/test_linked.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/unit/test_linked-test_linked.o -MD -MP -MF tests/unit/$(DEPDIR)/test_linked-test_linked.Tpo -c -o tests/unit/test_linked-test_linked.o `test -f 'tests/unit/test_linked.c' || echo '$(srcdir)/'`tests/unit/test_linked.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/unit/$(DEPDIR)/test_linked-test_linked.Tpo tests/unit/$(DEPDIR)/test_linked-test_linked.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/unit/test_linked.c' object='tests/unit/test_linked-test_linked.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/unit/test_linked-test_linked.o `test -f 'tests/unit/test_linked.c' || echo '$(srcdir)/'`tests/unit/test_linked.c + +tests/unit/test_linked-test_linked.obj: tests/unit/test_linked.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/unit/test_linked-test_linked.obj -MD -MP -MF tests/unit/$(DEPDIR)/test_linked-test_linked.Tpo -c -o tests/unit/test_linked-test_linked.obj `if test -f 'tests/unit/test_linked.c'; then $(CYGPATH_W) 'tests/unit/test_linked.c'; else $(CYGPATH_W) '$(srcdir)/tests/unit/test_linked.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/unit/$(DEPDIR)/test_linked-test_linked.Tpo tests/unit/$(DEPDIR)/test_linked-test_linked.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/unit/test_linked.c' object='tests/unit/test_linked-test_linked.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/unit/test_linked-test_linked.obj `if test -f 'tests/unit/test_linked.c'; then $(CYGPATH_W) 'tests/unit/test_linked.c'; else $(CYGPATH_W) '$(srcdir)/tests/unit/test_linked.c'; fi` + +tests/fuzz/unit_test_linked-stubs.o: tests/fuzz/stubs.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/fuzz/unit_test_linked-stubs.o -MD -MP -MF tests/fuzz/$(DEPDIR)/unit_test_linked-stubs.Tpo -c -o tests/fuzz/unit_test_linked-stubs.o `test -f 'tests/fuzz/stubs.c' || echo '$(srcdir)/'`tests/fuzz/stubs.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/fuzz/$(DEPDIR)/unit_test_linked-stubs.Tpo tests/fuzz/$(DEPDIR)/unit_test_linked-stubs.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/fuzz/stubs.c' object='tests/fuzz/unit_test_linked-stubs.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/fuzz/unit_test_linked-stubs.o `test -f 'tests/fuzz/stubs.c' || echo '$(srcdir)/'`tests/fuzz/stubs.c + +tests/fuzz/unit_test_linked-stubs.obj: tests/fuzz/stubs.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/fuzz/unit_test_linked-stubs.obj -MD -MP -MF tests/fuzz/$(DEPDIR)/unit_test_linked-stubs.Tpo -c -o tests/fuzz/unit_test_linked-stubs.obj `if test -f 'tests/fuzz/stubs.c'; then $(CYGPATH_W) 'tests/fuzz/stubs.c'; else $(CYGPATH_W) '$(srcdir)/tests/fuzz/stubs.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/fuzz/$(DEPDIR)/unit_test_linked-stubs.Tpo tests/fuzz/$(DEPDIR)/unit_test_linked-stubs.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='tests/fuzz/stubs.c' object='tests/fuzz/unit_test_linked-stubs.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests/fuzz/unit_test_linked-stubs.obj `if test -f 'tests/fuzz/stubs.c'; then $(CYGPATH_W) 'tests/fuzz/stubs.c'; else $(CYGPATH_W) '$(srcdir)/tests/fuzz/stubs.c'; fi` + +tests_unit_test_linked-sql.o: sql.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-sql.o -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-sql.Tpo -c -o tests_unit_test_linked-sql.o `test -f 'sql.c' || echo '$(srcdir)/'`sql.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-sql.Tpo $(DEPDIR)/tests_unit_test_linked-sql.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='sql.c' object='tests_unit_test_linked-sql.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-sql.o `test -f 'sql.c' || echo '$(srcdir)/'`sql.c + +tests_unit_test_linked-sql.obj: sql.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-sql.obj -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-sql.Tpo -c -o tests_unit_test_linked-sql.obj `if test -f 'sql.c'; then $(CYGPATH_W) 'sql.c'; else $(CYGPATH_W) '$(srcdir)/sql.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-sql.Tpo $(DEPDIR)/tests_unit_test_linked-sql.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='sql.c' object='tests_unit_test_linked-sql.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-sql.obj `if test -f 'sql.c'; then $(CYGPATH_W) 'sql.c'; else $(CYGPATH_W) '$(srcdir)/sql.c'; fi` + +tests_unit_test_linked-util.o: util.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-util.o -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-util.Tpo -c -o tests_unit_test_linked-util.o `test -f 'util.c' || echo '$(srcdir)/'`util.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-util.Tpo $(DEPDIR)/tests_unit_test_linked-util.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='util.c' object='tests_unit_test_linked-util.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-util.o `test -f 'util.c' || echo '$(srcdir)/'`util.c + +tests_unit_test_linked-util.obj: util.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-util.obj -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-util.Tpo -c -o tests_unit_test_linked-util.obj `if test -f 'util.c'; then $(CYGPATH_W) 'util.c'; else $(CYGPATH_W) '$(srcdir)/util.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-util.Tpo $(DEPDIR)/tests_unit_test_linked-util.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='util.c' object='tests_unit_test_linked-util.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-util.obj `if test -f 'util.c'; then $(CYGPATH_W) 'util.c'; else $(CYGPATH_W) '$(srcdir)/util.c'; fi` + +tests_unit_test_linked-snmp.o: snmp.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-snmp.o -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-snmp.Tpo -c -o tests_unit_test_linked-snmp.o `test -f 'snmp.c' || echo '$(srcdir)/'`snmp.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-snmp.Tpo $(DEPDIR)/tests_unit_test_linked-snmp.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='snmp.c' object='tests_unit_test_linked-snmp.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-snmp.o `test -f 'snmp.c' || echo '$(srcdir)/'`snmp.c + +tests_unit_test_linked-snmp.obj: snmp.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-snmp.obj -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-snmp.Tpo -c -o tests_unit_test_linked-snmp.obj `if test -f 'snmp.c'; then $(CYGPATH_W) 'snmp.c'; else $(CYGPATH_W) '$(srcdir)/snmp.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-snmp.Tpo $(DEPDIR)/tests_unit_test_linked-snmp.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='snmp.c' object='tests_unit_test_linked-snmp.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-snmp.obj `if test -f 'snmp.c'; then $(CYGPATH_W) 'snmp.c'; else $(CYGPATH_W) '$(srcdir)/snmp.c'; fi` + +src/core/tests_unit_test_linked-locks.o: src/core/locks.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/tests_unit_test_linked-locks.o -MD -MP -MF src/core/$(DEPDIR)/tests_unit_test_linked-locks.Tpo -c -o src/core/tests_unit_test_linked-locks.o `test -f 'src/core/locks.c' || echo '$(srcdir)/'`src/core/locks.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/tests_unit_test_linked-locks.Tpo src/core/$(DEPDIR)/tests_unit_test_linked-locks.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/locks.c' object='src/core/tests_unit_test_linked-locks.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/tests_unit_test_linked-locks.o `test -f 'src/core/locks.c' || echo '$(srcdir)/'`src/core/locks.c + +src/core/tests_unit_test_linked-locks.obj: src/core/locks.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/tests_unit_test_linked-locks.obj -MD -MP -MF src/core/$(DEPDIR)/tests_unit_test_linked-locks.Tpo -c -o src/core/tests_unit_test_linked-locks.obj `if test -f 'src/core/locks.c'; then $(CYGPATH_W) 'src/core/locks.c'; else $(CYGPATH_W) '$(srcdir)/src/core/locks.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/tests_unit_test_linked-locks.Tpo src/core/$(DEPDIR)/tests_unit_test_linked-locks.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/locks.c' object='src/core/tests_unit_test_linked-locks.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/tests_unit_test_linked-locks.obj `if test -f 'src/core/locks.c'; then $(CYGPATH_W) 'src/core/locks.c'; else $(CYGPATH_W) '$(srcdir)/src/core/locks.c'; fi` + +tests_unit_test_linked-poller.o: poller.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-poller.o -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-poller.Tpo -c -o tests_unit_test_linked-poller.o `test -f 'poller.c' || echo '$(srcdir)/'`poller.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-poller.Tpo $(DEPDIR)/tests_unit_test_linked-poller.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='poller.c' object='tests_unit_test_linked-poller.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-poller.o `test -f 'poller.c' || echo '$(srcdir)/'`poller.c + +tests_unit_test_linked-poller.obj: poller.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-poller.obj -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-poller.Tpo -c -o tests_unit_test_linked-poller.obj `if test -f 'poller.c'; then $(CYGPATH_W) 'poller.c'; else $(CYGPATH_W) '$(srcdir)/poller.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-poller.Tpo $(DEPDIR)/tests_unit_test_linked-poller.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='poller.c' object='tests_unit_test_linked-poller.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-poller.obj `if test -f 'poller.c'; then $(CYGPATH_W) 'poller.c'; else $(CYGPATH_W) '$(srcdir)/poller.c'; fi` + +tests_unit_test_linked-nft_popen.o: nft_popen.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-nft_popen.o -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-nft_popen.Tpo -c -o tests_unit_test_linked-nft_popen.o `test -f 'nft_popen.c' || echo '$(srcdir)/'`nft_popen.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-nft_popen.Tpo $(DEPDIR)/tests_unit_test_linked-nft_popen.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='nft_popen.c' object='tests_unit_test_linked-nft_popen.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-nft_popen.o `test -f 'nft_popen.c' || echo '$(srcdir)/'`nft_popen.c + +tests_unit_test_linked-nft_popen.obj: nft_popen.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-nft_popen.obj -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-nft_popen.Tpo -c -o tests_unit_test_linked-nft_popen.obj `if test -f 'nft_popen.c'; then $(CYGPATH_W) 'nft_popen.c'; else $(CYGPATH_W) '$(srcdir)/nft_popen.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-nft_popen.Tpo $(DEPDIR)/tests_unit_test_linked-nft_popen.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='nft_popen.c' object='tests_unit_test_linked-nft_popen.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-nft_popen.obj `if test -f 'nft_popen.c'; then $(CYGPATH_W) 'nft_popen.c'; else $(CYGPATH_W) '$(srcdir)/nft_popen.c'; fi` + +tests_unit_test_linked-php.o: php.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-php.o -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-php.Tpo -c -o tests_unit_test_linked-php.o `test -f 'php.c' || echo '$(srcdir)/'`php.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-php.Tpo $(DEPDIR)/tests_unit_test_linked-php.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='php.c' object='tests_unit_test_linked-php.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-php.o `test -f 'php.c' || echo '$(srcdir)/'`php.c + +tests_unit_test_linked-php.obj: php.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-php.obj -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-php.Tpo -c -o tests_unit_test_linked-php.obj `if test -f 'php.c'; then $(CYGPATH_W) 'php.c'; else $(CYGPATH_W) '$(srcdir)/php.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-php.Tpo $(DEPDIR)/tests_unit_test_linked-php.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='php.c' object='tests_unit_test_linked-php.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-php.obj `if test -f 'php.c'; then $(CYGPATH_W) 'php.c'; else $(CYGPATH_W) '$(srcdir)/php.c'; fi` + +tests_unit_test_linked-ping.o: ping.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-ping.o -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-ping.Tpo -c -o tests_unit_test_linked-ping.o `test -f 'ping.c' || echo '$(srcdir)/'`ping.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-ping.Tpo $(DEPDIR)/tests_unit_test_linked-ping.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='ping.c' object='tests_unit_test_linked-ping.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-ping.o `test -f 'ping.c' || echo '$(srcdir)/'`ping.c + +tests_unit_test_linked-ping.obj: ping.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-ping.obj -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-ping.Tpo -c -o tests_unit_test_linked-ping.obj `if test -f 'ping.c'; then $(CYGPATH_W) 'ping.c'; else $(CYGPATH_W) '$(srcdir)/ping.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-ping.Tpo $(DEPDIR)/tests_unit_test_linked-ping.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='ping.c' object='tests_unit_test_linked-ping.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-ping.obj `if test -f 'ping.c'; then $(CYGPATH_W) 'ping.c'; else $(CYGPATH_W) '$(srcdir)/ping.c'; fi` + +src/core/tests_unit_test_linked-keywords.o: src/core/keywords.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/tests_unit_test_linked-keywords.o -MD -MP -MF src/core/$(DEPDIR)/tests_unit_test_linked-keywords.Tpo -c -o src/core/tests_unit_test_linked-keywords.o `test -f 'src/core/keywords.c' || echo '$(srcdir)/'`src/core/keywords.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/tests_unit_test_linked-keywords.Tpo src/core/$(DEPDIR)/tests_unit_test_linked-keywords.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/keywords.c' object='src/core/tests_unit_test_linked-keywords.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/tests_unit_test_linked-keywords.o `test -f 'src/core/keywords.c' || echo '$(srcdir)/'`src/core/keywords.c + +src/core/tests_unit_test_linked-keywords.obj: src/core/keywords.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT src/core/tests_unit_test_linked-keywords.obj -MD -MP -MF src/core/$(DEPDIR)/tests_unit_test_linked-keywords.Tpo -c -o src/core/tests_unit_test_linked-keywords.obj `if test -f 'src/core/keywords.c'; then $(CYGPATH_W) 'src/core/keywords.c'; else $(CYGPATH_W) '$(srcdir)/src/core/keywords.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) src/core/$(DEPDIR)/tests_unit_test_linked-keywords.Tpo src/core/$(DEPDIR)/tests_unit_test_linked-keywords.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='src/core/keywords.c' object='src/core/tests_unit_test_linked-keywords.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o src/core/tests_unit_test_linked-keywords.obj `if test -f 'src/core/keywords.c'; then $(CYGPATH_W) 'src/core/keywords.c'; else $(CYGPATH_W) '$(srcdir)/src/core/keywords.c'; fi` + +tests_unit_test_linked-error.o: error.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-error.o -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-error.Tpo -c -o tests_unit_test_linked-error.o `test -f 'error.c' || echo '$(srcdir)/'`error.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-error.Tpo $(DEPDIR)/tests_unit_test_linked-error.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='error.c' object='tests_unit_test_linked-error.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-error.o `test -f 'error.c' || echo '$(srcdir)/'`error.c + +tests_unit_test_linked-error.obj: error.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests_unit_test_linked-error.obj -MD -MP -MF $(DEPDIR)/tests_unit_test_linked-error.Tpo -c -o tests_unit_test_linked-error.obj `if test -f 'error.c'; then $(CYGPATH_W) 'error.c'; else $(CYGPATH_W) '$(srcdir)/error.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/tests_unit_test_linked-error.Tpo $(DEPDIR)/tests_unit_test_linked-error.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='error.c' object='tests_unit_test_linked-error.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_linked_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o tests_unit_test_linked-error.obj `if test -f 'error.c'; then $(CYGPATH_W) 'error.c'; else $(CYGPATH_W) '$(srcdir)/error.c'; fi` + tests/unit/test_php_runtime-test_php_runtime.o: tests/unit/test_php_runtime.c @am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(tests_unit_test_php_runtime_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT tests/unit/test_php_runtime-test_php_runtime.o -MD -MP -MF tests/unit/$(DEPDIR)/test_php_runtime-test_php_runtime.Tpo -c -o tests/unit/test_php_runtime-test_php_runtime.o `test -f 'tests/unit/test_php_runtime.c' || echo '$(srcdir)/'`tests/unit/test_php_runtime.c @am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) tests/unit/$(DEPDIR)/test_php_runtime-test_php_runtime.Tpo tests/unit/$(DEPDIR)/test_php_runtime-test_php_runtime.Po @@ -1273,9 +2099,9 @@ uninstall-man1: } | sed -e 's,.*/,,;h;s,.*\.,,;s,^[^1][0-9a-z]*$$,1,;x' \ -e 's,\.[0-9a-z]*$$,,;$(transform);G;s,\n,.,'`; \ dir='$(DESTDIR)$(man1dir)'; $(am__uninstall_files_from_dir) -install-configDATA: $(config_DATA) +install-dist_configDATA: $(dist_config_DATA) @$(NORMAL_INSTALL) - @list='$(config_DATA)'; test -n "$(configdir)" || list=; \ + @list='$(dist_config_DATA)'; test -n "$(configdir)" || list=; \ if test -n "$$list"; then \ echo " $(MKDIR_P) '$(DESTDIR)$(configdir)'"; \ $(MKDIR_P) "$(DESTDIR)$(configdir)" || exit 1; \ @@ -1289,9 +2115,9 @@ install-configDATA: $(config_DATA) $(INSTALL_DATA) $$files "$(DESTDIR)$(configdir)" || exit $$?; \ done -uninstall-configDATA: +uninstall-dist_configDATA: @$(NORMAL_UNINSTALL) - @list='$(config_DATA)'; test -n "$(configdir)" || list=; \ + @list='$(dist_config_DATA)'; test -n "$(configdir)" || list=; \ files=`for p in $$list; do echo $$p; done | sed -e 's|^.*/||'`; \ dir='$(DESTDIR)$(configdir)'; $(am__uninstall_files_from_dir) @@ -1461,7 +2287,7 @@ $(TEST_SUITE_LOG): $(TEST_LOGS) test x"$$VERBOSE" = x || cat $(TEST_SUITE_LOG); \ fi; \ echo "$${col}$$br$${std}"; \ - echo "$${col}Testsuite summary for $(PACKAGE_STRING)$${std}"; \ + echo "$${col}Testsuite summary"$(AM_TESTSUITE_SUMMARY_HEADER)"$${std}"; \ echo "$${col}$$br$${std}"; \ create_testsuite_report --maybe-color; \ echo "$$col$$br$$std"; \ @@ -1495,6 +2321,20 @@ recheck: all $(check_PROGRAMS) am__force_recheck=am--force-recheck \ TEST_LOGS="$$log_list"; \ exit $$? +test_spine_regressions.log: test_spine_regressions$(EXEEXT) + @p='test_spine_regressions$(EXEEXT)'; \ + b='test_spine_regressions'; \ + $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ + --log-file $$b.log --trs-file $$b.trs \ + $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ + "$$tst" $(AM_TESTS_FD_REDIRECT) +test_spine_faults.log: test_spine_faults$(EXEEXT) + @p='test_spine_faults$(EXEEXT)'; \ + b='test_spine_faults'; \ + $(am__check_pre) $(LOG_DRIVER) --test-name "$$f" \ + --log-file $$b.log --trs-file $$b.trs \ + $(am__common_driver_flags) $(AM_LOG_DRIVER_FLAGS) $(LOG_DRIVER_FLAGS) -- $(LOG_COMPILE) \ + "$$tst" $(AM_TESTS_FD_REDIRECT) tests/unit/test_util_strings.log: tests/unit/test_util_strings$(EXEEXT) @p='tests/unit/test_util_strings$(EXEEXT)'; \ b='tests/unit/test_util_strings'; \ @@ -1565,7 +2405,6 @@ tests/unit/test_php_runtime.log: tests/unit/test_php_runtime$(EXEEXT) @am__EXEEXT_TRUE@ --log-file $$b.log --trs-file $$b.trs \ @am__EXEEXT_TRUE@ $(am__common_driver_flags) $(AM_SH_LOG_DRIVER_FLAGS) $(SH_LOG_DRIVER_FLAGS) -- $(SH_LOG_COMPILE) \ @am__EXEEXT_TRUE@ "$$tst" $(AM_TESTS_FD_REDIRECT) - distdir: $(BUILT_SOURCES) $(MAKE) $(AM_MAKEFLAGS) distdir-am @@ -1687,7 +2526,7 @@ distcheck: dist $(DISTCHECK_CONFIGURE_FLAGS) \ --srcdir=../.. --prefix="$$dc_install_base" \ && $(MAKE) $(AM_MAKEFLAGS) \ - && $(MAKE) $(AM_MAKEFLAGS) dvi \ + && $(MAKE) $(AM_MAKEFLAGS) $(AM_DISTCHECK_DVI_TARGET) \ && $(MAKE) $(AM_MAKEFLAGS) check \ && $(MAKE) $(AM_MAKEFLAGS) install \ && $(MAKE) $(AM_MAKEFLAGS) installcheck \ @@ -1778,10 +2617,13 @@ distclean-generic: -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) -rm -f src/core/$(DEPDIR)/$(am__dirstamp) -rm -f src/core/$(am__dirstamp) + -rm -f tests/$(DEPDIR)/$(am__dirstamp) + -rm -f tests/$(am__dirstamp) -rm -f tests/fuzz/$(DEPDIR)/$(am__dirstamp) -rm -f tests/fuzz/$(am__dirstamp) -rm -f tests/unit/$(DEPDIR)/$(am__dirstamp) -rm -f tests/unit/$(am__dirstamp) + -test -z "$(DISTCLEANFILES)" || rm -f $(DISTCLEANFILES) maintainer-clean-generic: @echo "This command is intended for maintainers to use" @@ -1801,6 +2643,31 @@ distclean: distclean-am -rm -f ./$(DEPDIR)/snmp.Po -rm -f ./$(DEPDIR)/spine.Po -rm -f ./$(DEPDIR)/sql.Po + -rm -f ./$(DEPDIR)/test_spine_faults-error.Po + -rm -f ./$(DEPDIR)/test_spine_faults-nft_popen.Po + -rm -f ./$(DEPDIR)/test_spine_faults-php.Po + -rm -f ./$(DEPDIR)/test_spine_faults-ping.Po + -rm -f ./$(DEPDIR)/test_spine_faults-poller.Po + -rm -f ./$(DEPDIR)/test_spine_faults-snmp.Po + -rm -f ./$(DEPDIR)/test_spine_faults-sql.Po + -rm -f ./$(DEPDIR)/test_spine_faults-util.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-error.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-nft_popen.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-php.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-ping.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-poller.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-snmp.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-spine.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-sql.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-util.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-error.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-nft_popen.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-php.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-ping.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-poller.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-snmp.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-sql.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-util.Po -rm -f ./$(DEPDIR)/tests_unit_test_php_runtime-error.Po -rm -f ./$(DEPDIR)/tests_unit_test_php_runtime-nft_popen.Po -rm -f ./$(DEPDIR)/tests_unit_test_php_runtime-php.Po @@ -1812,13 +2679,29 @@ distclean: distclean-am -rm -f ./$(DEPDIR)/util.Po -rm -f src/core/$(DEPDIR)/keywords.Po -rm -f src/core/$(DEPDIR)/locks.Po + -rm -f src/core/$(DEPDIR)/test_spine_faults-keywords.Po + -rm -f src/core/$(DEPDIR)/test_spine_faults-locks.Po + -rm -f src/core/$(DEPDIR)/test_spine_regressions-keywords.Po + -rm -f src/core/$(DEPDIR)/test_spine_regressions-locks.Po + -rm -f src/core/$(DEPDIR)/tests_unit_test_linked-keywords.Po + -rm -f src/core/$(DEPDIR)/tests_unit_test_linked-locks.Po -rm -f src/core/$(DEPDIR)/tests_unit_test_php_runtime-keywords.Po -rm -f src/core/$(DEPDIR)/tests_unit_test_php_runtime-locks.Po + -rm -f tests/$(DEPDIR)/test_spine_faults-test_spine_faults.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_cli_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_output_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_regressions.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_reindex_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_settings_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_snmpv3_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_stream_contracts.Po -rm -f tests/fuzz/$(DEPDIR)/stubs.Po + -rm -f tests/fuzz/$(DEPDIR)/unit_test_linked-stubs.Po -rm -f tests/fuzz/$(DEPDIR)/unit_test_php_runtime-stubs.Po -rm -f tests/unit/$(DEPDIR)/php_test_server.Po -rm -f tests/unit/$(DEPDIR)/test_build_fixes.Po - -rm -f tests/unit/$(DEPDIR)/test_linked.Po + -rm -f tests/unit/$(DEPDIR)/test_linked-test_linked.Po -rm -f tests/unit/$(DEPDIR)/test_php_runtime-test_php_runtime.Po -rm -f tests/unit/$(DEPDIR)/test_ping_icmp.Po -rm -f tests/unit/$(DEPDIR)/test_safety_fixes.Po @@ -1841,7 +2724,7 @@ info: info-am info-am: -install-data-am: install-configDATA install-man +install-data-am: install-dist_configDATA install-man @$(NORMAL_INSTALL) $(MAKE) $(AM_MAKEFLAGS) install-data-hook install-dvi: install-dvi-am @@ -1881,6 +2764,31 @@ maintainer-clean: maintainer-clean-am -rm -f ./$(DEPDIR)/snmp.Po -rm -f ./$(DEPDIR)/spine.Po -rm -f ./$(DEPDIR)/sql.Po + -rm -f ./$(DEPDIR)/test_spine_faults-error.Po + -rm -f ./$(DEPDIR)/test_spine_faults-nft_popen.Po + -rm -f ./$(DEPDIR)/test_spine_faults-php.Po + -rm -f ./$(DEPDIR)/test_spine_faults-ping.Po + -rm -f ./$(DEPDIR)/test_spine_faults-poller.Po + -rm -f ./$(DEPDIR)/test_spine_faults-snmp.Po + -rm -f ./$(DEPDIR)/test_spine_faults-sql.Po + -rm -f ./$(DEPDIR)/test_spine_faults-util.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-error.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-nft_popen.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-php.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-ping.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-poller.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-snmp.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-spine.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-sql.Po + -rm -f ./$(DEPDIR)/test_spine_regressions-util.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-error.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-nft_popen.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-php.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-ping.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-poller.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-snmp.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-sql.Po + -rm -f ./$(DEPDIR)/tests_unit_test_linked-util.Po -rm -f ./$(DEPDIR)/tests_unit_test_php_runtime-error.Po -rm -f ./$(DEPDIR)/tests_unit_test_php_runtime-nft_popen.Po -rm -f ./$(DEPDIR)/tests_unit_test_php_runtime-php.Po @@ -1892,13 +2800,29 @@ maintainer-clean: maintainer-clean-am -rm -f ./$(DEPDIR)/util.Po -rm -f src/core/$(DEPDIR)/keywords.Po -rm -f src/core/$(DEPDIR)/locks.Po + -rm -f src/core/$(DEPDIR)/test_spine_faults-keywords.Po + -rm -f src/core/$(DEPDIR)/test_spine_faults-locks.Po + -rm -f src/core/$(DEPDIR)/test_spine_regressions-keywords.Po + -rm -f src/core/$(DEPDIR)/test_spine_regressions-locks.Po + -rm -f src/core/$(DEPDIR)/tests_unit_test_linked-keywords.Po + -rm -f src/core/$(DEPDIR)/tests_unit_test_linked-locks.Po -rm -f src/core/$(DEPDIR)/tests_unit_test_php_runtime-keywords.Po -rm -f src/core/$(DEPDIR)/tests_unit_test_php_runtime-locks.Po + -rm -f tests/$(DEPDIR)/test_spine_faults-test_spine_faults.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_cli_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_output_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_regressions.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_reindex_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_settings_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_snmpv3_contracts.Po + -rm -f tests/$(DEPDIR)/test_spine_regressions-test_spine_stream_contracts.Po -rm -f tests/fuzz/$(DEPDIR)/stubs.Po + -rm -f tests/fuzz/$(DEPDIR)/unit_test_linked-stubs.Po -rm -f tests/fuzz/$(DEPDIR)/unit_test_php_runtime-stubs.Po -rm -f tests/unit/$(DEPDIR)/php_test_server.Po -rm -f tests/unit/$(DEPDIR)/test_build_fixes.Po - -rm -f tests/unit/$(DEPDIR)/test_linked.Po + -rm -f tests/unit/$(DEPDIR)/test_linked-test_linked.Po -rm -f tests/unit/$(DEPDIR)/test_php_runtime-test_php_runtime.Po -rm -f tests/unit/$(DEPDIR)/test_ping_icmp.Po -rm -f tests/unit/$(DEPDIR)/test_safety_fixes.Po @@ -1921,7 +2845,8 @@ ps: ps-am ps-am: -uninstall-am: uninstall-binPROGRAMS uninstall-configDATA uninstall-man +uninstall-am: uninstall-binPROGRAMS uninstall-dist_configDATA \ + uninstall-man @$(NORMAL_INSTALL) $(MAKE) $(AM_MAKEFLAGS) uninstall-hook uninstall-man: uninstall-man1 @@ -1937,17 +2862,17 @@ uninstall-man: uninstall-man1 distclean-compile distclean-generic distclean-hdr \ distclean-libtool distclean-tags distcleancheck distdir \ distuninstallcheck dvi dvi-am html html-am info info-am \ - install install-am install-binPROGRAMS install-configDATA \ - install-data install-data-am install-data-hook install-dvi \ - install-dvi-am install-exec install-exec-am install-html \ - install-html-am install-info install-info-am install-man \ - install-man1 install-pdf install-pdf-am install-ps \ + install install-am install-binPROGRAMS install-data \ + install-data-am install-data-hook install-dist_configDATA \ + install-dvi install-dvi-am install-exec install-exec-am \ + install-html install-html-am install-info install-info-am \ + install-man install-man1 install-pdf install-pdf-am install-ps \ install-ps-am install-strip installcheck installcheck-am \ installdirs maintainer-clean maintainer-clean-generic \ mostlyclean mostlyclean-compile mostlyclean-generic \ mostlyclean-libtool pdf pdf-am ps ps-am recheck tags tags-am \ uninstall uninstall-am uninstall-binPROGRAMS \ - uninstall-configDATA uninstall-hook uninstall-man \ + uninstall-dist_configDATA uninstall-hook uninstall-man \ uninstall-man1 .PRECIOUS: Makefile @@ -1967,7 +2892,16 @@ uninstall-hook: if cmp -s "$(DESTDIR)$(configdir)/spine.conf" "$(srcdir)/spine.conf.dist" 2>/dev/null; then \ rm -f "$(DESTDIR)$(configdir)/spine.conf"; \ fi +.PHONY: coverage-source-list coverage-test-source-list +coverage-source-list: + @printf '%s\n' $(spine_SOURCES) +coverage-test-source-list: + @printf '%s\n' $(test_spine_regressions_SOURCES) + +.PHONY: coverage-fault-source-list +coverage-fault-source-list: + @printf '%s\n' $(test_spine_faults_SOURCES) # Docker targets — require Dockerfile and Dockerfile.dev (from PR #401) .PHONY: docker docker-dev verify cppcheck diff --git a/common.h b/common.h index 82ba90e..ad3142d 100644 --- a/common.h +++ b/common.h @@ -134,11 +134,6 @@ # include #endif -#undef PACKAGE_NAME -#undef PACKAGE_VERSION -#undef PACKAGE_BUGREPORT -#undef PACKAGE_STRING -#undef PACKAGE_TARNAME #include #include #include @@ -146,7 +141,6 @@ #include #include #include -//#include #include #include diff --git a/configure.ac b/configure.ac index 390ea9f..9cb25ec 100644 --- a/configure.ac +++ b/configure.ac @@ -34,6 +34,22 @@ AM_INIT_AUTOMAKE([foreign subdir-objects]) AM_SILENT_RULES([yes]) AC_CONFIG_HEADERS(config/config.h) +# These generic package macros belong to the generated configuration header. +# Leave them available for Net-SNMP to define; snmp_spine_init uses its +# PACKAGE_VERSION when header/library version verification is enabled. +# Continue each directive so config.status preserves the literal cleanup +# instead of substituting another package definition into the header footer. +AH_BOTTOM([#undef \ +PACKAGE_NAME +#undef \ +PACKAGE_VERSION +#undef \ +PACKAGE_BUGREPORT +#undef \ +PACKAGE_STRING +#undef \ +PACKAGE_TARNAME]) + # static libraries AC_ARG_WITH(static, AS_HELP_STRING([--with-static],[Build using static libraries @@ -499,6 +515,23 @@ AC_LINK_IFELSE([AC_LANG_PROGRAM([[ ],[AC_MSG_RESULT(no) ]) +spine_saved_ldflags=$LDFLAGS +spine_linker_wrap=no +case "$host_os" in + linux*) + LDFLAGS="$LDFLAGS -Wl,--wrap=strftime" + AC_LINK_IFELSE([AC_LANG_PROGRAM([[ +#include +extern size_t __real_strftime(char *, size_t, const char *, const struct tm *); +size_t __wrap_strftime(char *out, size_t size, const char *format, const struct tm *time) { + return __real_strftime(out, size, format, time); +} +]], [[return 0;]])], [spine_linker_wrap=yes], []) + ;; + *) ;; +esac +LDFLAGS=$spine_saved_ldflags +AM_CONDITIONAL([HAVE_LINKER_WRAP], [test "$spine_linker_wrap" = yes]) # cmocka is only used by the unit tests under "make check". Absence is not # fatal; the tests are dropped from check_PROGRAMS instead. have_cmocka=no diff --git a/copyright_year.sh b/copyright_year.sh index 1c4d6cf..d551383 100644 --- a/copyright_year.sh +++ b/copyright_year.sh @@ -21,7 +21,8 @@ # +-------------------------------------------------------------------------+ update_copyright() { - local file=$1 + local original_file=$1 + local file=$original_file file=${file/$SCRIPT_BASE/} printf -v line "%60s" "$file" if [[ -z "$ERRORS_ONLY" ]]; then @@ -30,19 +31,19 @@ update_copyright() { fi old_reg="20[0-9][0-9][ ]*-[ ]*20[0-9][0-9]" - old_data=$(grep -c -e "$old_reg" "$1" 2>/dev/null) + old_data=$(grep -c -e "$old_reg" "$original_file" 2>/dev/null) new_reg="2004-$YEAR" - result=$? + if [[ $old_data -eq 0 ]]; then old_reg="(Copyright.*) 20[0-9][0-9] " - old_data=$(grep -c -e "$old_reg" "$1" 2>/dev/null) + old_data=$(grep -c -e "$old_reg" "$original_file" 2>/dev/null) new_reg="\1 2004-$YEAR" - result=$? + fi if [[ $old_data -gt 0 ]]; then - old_data=$(grep -e "$old_reg" "$1" 2>/dev/null) + old_data=$(grep -e "$old_reg" "$original_file" 2>/dev/null) new_data=$(echo "$old_data" | sed -r s/"$old_reg"/"$new_reg"/g) if [[ "$old_data" == "$new_data" ]]; then if [[ -z "$ERRORS_ONLY" ]]; then @@ -50,16 +51,17 @@ update_copyright() { fi else echo "$line Updating Copyright Data" - printf "%60s %s\n" "==============================" "====================" - printf "%60s %s\n" "$old_data" "=>" - printf "%60s %s\n" "$new_data" "" - sed -i -r s/"$old_reg"/"$new_reg"/g "$1" - printf "%60s %s\n" "==============================" "====================" + printf '%60s %s\n' "==============================" "====================" + printf '%60s %s\n' "$old_data" "=>" + printf '%60s %s\n' "$new_data" "" + sed -i -r s/"$old_reg"/"$new_reg"/g "$original_file" + printf '%60s %s\n' "==============================" "====================" fi else echo "$line Copyright not found!" - SCRIPT_ERR=1 + fi + return 0 } SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd) @@ -71,11 +73,11 @@ for f in $BAD_FOLDERS; do SCRIPT_EXCLUSION="$SCRIPT_EXCLUSION -not -path ${SCRIPT_BASE}$f/\* " done -SCRIPT_ERR=0 + YEAR=$(date +"%Y") EXT="" # "sh sql php js md conf c h ac dist" ERRORS_ONLY=1 -while [ -n "$1" ]; do +while [[ -n "$1" ]]; do case $1 in "--help") echo "NOTE: Checks all Cacti pages for this years copyright" @@ -103,7 +105,7 @@ done SCRIPT_INCLUSION= SCRIPT_SEPARATOR= for ext in $EXT; do - if [ -n "$SCRIPT_INCLUSION" ]; then + if [[ -n "$SCRIPT_INCLUSION" ]]; then SCRIPT_SEPARATOR="-o " fi SCRIPT_INCLUSION="$SCRIPT_INCLUSION $SCRIPT_SEPARATOR-name \*.$ext" diff --git a/error.c b/error.c index af272d4..3f6a587 100644 --- a/error.c +++ b/error.c @@ -108,7 +108,7 @@ static void spine_signal_handler(int spine_signal) { signal(spine_signal, SIG_DFL); } - set.exit_code = spine_signal; + set.exit.exit_code = spine_signal; switch (spine_signal) { case SIGABRT: diff --git a/nft_popen.c b/nft_popen.c index 6c83249..67898a4 100644 --- a/nft_popen.c +++ b/nft_popen.c @@ -361,13 +361,21 @@ int nft_popen(const char * command, const char * type) { int cancel_state; extern char **environ; int retry_count = 0; + /* nft_popen() reports why process creation failed through errno, as + * popen() does; cleanup below must not overwrite it. */ + int failure_errno = 0; + + if (command == NULL || type == NULL) { + errno = EINVAL; + return -1; + } /* On platforms where pipe() is bidirectional, - * "r+" gives two-way communication. + * "r+" gives two-way communication. Only the documented modes are + * accepted; any other "+" mode is a caller error, not a duplex request. */ - if (strchr(type, '+')) { + if (strcmp(type, "r+") == 0) { twoway = 1; - type = "r+"; }else { twoway = 0; if ((*type != 'r' && *type != 'w') || type[1]) { @@ -376,8 +384,20 @@ int nft_popen(const char * command, const char * type) { } } - if (!spine_open_pipe_cloexec(pdes)) + /* An ordinary pipe is one-way on Linux and macOS, so the duplex mode + * needs a socket pair to be readable and writable at the parent end. */ + if (twoway) { + if (socketpair(AF_UNIX, SOCK_STREAM, 0, pdes) < 0) + return -1; + + if (spine_set_cloexec(pdes[0]) != 0 || spine_set_cloexec(pdes[1]) != 0) { + (void)close(pdes[0]); + (void)close(pdes[1]); + return -1; + } + } else if (!spine_open_pipe_cloexec(pdes)) { return -1; + } /* Disable thread cancellation from this point forward. */ pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &cancel_state); @@ -449,6 +469,7 @@ int nft_popen(const char * command, const char * type) { inherit_fd = spine_dup_cloexec(pdes[1]); if (inherit_fd < 0) { + failure_errno = errno; SPINE_LOG(("ERROR: Unable to duplicate the pipe for the child: %s", strerror(errno))); goto spawn_failed; } @@ -467,6 +488,7 @@ int nft_popen(const char * command, const char * type) { inherit_fd = spine_dup_cloexec(pdes[0]); if (inherit_fd < 0) { + failure_errno = errno; SPINE_LOG(("ERROR: Unable to duplicate the pipe for the child: %s", strerror(errno))); goto spawn_failed; } @@ -508,6 +530,7 @@ int nft_popen(const char * command, const char * type) { goto retry; } + failure_errno = spawn_err; SPINE_LOG(("ERROR: SCRIPT: posix_spawn failed: %s", strerror(spawn_err))); spawn_failed: @@ -529,6 +552,9 @@ spawn_failed: free(command_copy); free(cur); pthread_setcancelstate(cancel_state, NULL); + if (failure_errno != 0) { + errno = failure_errno; + } return -1; } diff --git a/php.c b/php.c index d31379e..fe54198 100644 --- a/php.c +++ b/php.c @@ -170,7 +170,7 @@ char *php_cmd(const char *php_command, int php_process) { int retries = 0; assert(php_command != 0); - if (php_processes == NULL || php_process < 0 || php_process >= set.php_servers || + if (php_processes == NULL || php_process < 0 || php_process >= set.php.php_servers || php_process >= MAX_PHP_SERVERS) { SPINE_LOG(("ERROR: SS[%i] PHP Script Server slot is unavailable", php_process)); return php_undefined_result(); @@ -245,15 +245,15 @@ int php_get_process(void) { int start_candidate; int server_count; - if (php_processes == NULL || set.php_servers <= 0) return -1; - server_count = set.php_servers > MAX_PHP_SERVERS ? MAX_PHP_SERVERS : set.php_servers; + if (php_processes == NULL || set.php.php_servers <= 0) return -1; + server_count = set.php.php_servers > MAX_PHP_SERVERS ? MAX_PHP_SERVERS : set.php.php_servers; /* LOCK_PHP protects only the round-robin cursor. A startup handshake can * wait for script_timeout, so it must never run under this process-global * lock. */ thread_mutex_lock(LOCK_PHP); - if (set.php_current_server >= server_count) set.php_current_server = 0; - start_candidate = set.php_current_server++; + if (set.php.php_current_server >= server_count) set.php.php_current_server = 0; + start_candidate = set.php.php_current_server++; thread_mutex_unlock(LOCK_PHP); /* Prefer any ready slot. Each snapshot uses the same per-slot mutex as @@ -352,6 +352,7 @@ static char *php_read_result(int php_process, char *command, int allow_restart) double remaining_usec = 0; char *result_string; int response_timeout; + double read_deadline; ssize_t i; char *cp; @@ -377,10 +378,14 @@ static char *php_read_result(int php_process, char *command, int allow_restart) /* Selection-path recovery runs this handshake while holding the slot lock. * Bound startup independently so a bad PHP configuration cannot pin every * selector for the full per-command timeout. */ - response_timeout = allow_restart || set.script_timeout < 2 ? set.script_timeout : 2; + response_timeout = allow_restart || set.php.script_timeout < 2 ? set.php.script_timeout : 2; timeout.tv_sec = response_timeout; timeout.tv_usec = 0; + /* select() only says the first bytes arrived. A server that writes part of + * a line and stalls must not hold this thread past the same deadline. */ + read_deadline = spine_monotonic_time() + response_timeout; + /* check to see which pipe talked and take action * should only be the READ pipe */ retry: @@ -476,8 +481,24 @@ static char *php_read_result(int php_process, char *command, int allow_restart) } size_t space = (size_t)RESULTS_BUFFER - 1 - used; + + if (used > 0) { + int ready = spine_wait_readable(php_processes[php_process].php_read_fd, read_deadline); + + if (ready <= 0) { + SPINE_LOG(("WARNING: SS[%i] The PHP Script Server sent a partial response and %s, Command[%s]", php_process, ready == 0 ? "timed out" : "failed", command)); + SET_UNDEFINED(result_string); + read_ok = FALSE; + break; + } + } + i = read(php_processes[php_process].php_read_fd, bptr, space); + if (i < 0 && errno == EINTR) { + continue; + } + if (i <= 0) { SET_UNDEFINED(result_string); read_ok = FALSE; @@ -570,9 +591,16 @@ int php_init(int php_process) { return FALSE; } + /* An out-of-range slot would index past php_processes. */ + if (php_processes == NULL || set.php.php_servers < 0 || set.php.php_servers > MAX_PHP_SERVERS || + (php_process != PHP_INIT && (php_process < 0 || php_process >= set.php.php_servers))) { + SPINE_LOG(("ERROR: SS[%i] PHP Script Server slot is unavailable", php_process)); + goto cleanup; + } + /* special code to start all PHP Servers */ if (php_process == PHP_INIT) { - num_processes = set.php_servers; + num_processes = set.php.php_servers; } else { num_processes = 1; } @@ -606,23 +634,23 @@ int php_init(int php_process) { /* establish arguments for script server execution */ if (set.cacti_version <= 1222) { - argv[0] = set.path_php; + argv[0] = set.php.path_php; argv[1] = arg_q; - argv[2] = set.path_php_server; + argv[2] = set.php.path_php_server; argv[3] = arg_spine; - snprintf(poller_id, SMALL_BUFSIZE, "%d", set.poller_id); + snprintf(poller_id, SMALL_BUFSIZE, "%d", set.poller.poller_id); argv[4] = poller_id; argv[5] = NULL; - } else if (set.poller_id > 1) { - argv[0] = set.path_php; + } else if (set.poller.poller_id > 1) { + argv[0] = set.php.path_php; argv[1] = arg_q; - argv[2] = set.path_php_server; + argv[2] = set.php.path_php_server; argv[3] = arg_environ_spine; - snprintf(poller_id, SMALL_BUFSIZE, "--poller=%d", set.poller_id); + snprintf(poller_id, SMALL_BUFSIZE, "--poller=%d", set.poller.poller_id); argv[4] = poller_id; - if (set.mode == REMOTE_ONLINE) { + if (set.poller.mode == REMOTE_ONLINE) { argv[5] = arg_mode_online; } else { argv[5] = arg_mode_offline; @@ -630,11 +658,11 @@ int php_init(int php_process) { argv[6] = NULL; } else { - argv[0] = set.path_php; + argv[0] = set.php.path_php; argv[1] = arg_q; - argv[2] = set.path_php_server; + argv[2] = set.php.path_php_server; argv[3] = arg_environ_spine; - snprintf(poller_id, SMALL_BUFSIZE, "--poller=%d", set.poller_id); + snprintf(poller_id, SMALL_BUFSIZE, "--poller=%d", set.poller.poller_id); argv[4] = poller_id; argv[5] = NULL; @@ -854,8 +882,14 @@ void php_close(int php_process) { int num_processes; int len; + /* An out-of-range slot would index past php_processes. */ + if (php_processes == NULL || set.php.php_servers < 0 || set.php.php_servers > MAX_PHP_SERVERS || + (php_process != PHP_INIT && (php_process < 0 || php_process >= set.php.php_servers))) { + return; + } + if (php_process == PHP_INIT) { - num_processes = set.php_servers; + num_processes = set.php.php_servers; } else { num_processes = 1; } @@ -881,6 +915,13 @@ void php_close(int php_process) { */ if (phpp->php_write_fd >= 0) { static const char quit[] = "quit\r\n"; + /* A hung server can leave the request pipe full; the quit message + * is best effort and must not block shutdown. */ + int flags = fcntl(phpp->php_write_fd, F_GETFL); + + if (flags >= 0) { + (void)fcntl(phpp->php_write_fd, F_SETFL, flags | O_NONBLOCK); + } len = php_write_no_sigpipe(phpp->php_write_fd, quit, strlen(quit)); diff --git a/ping.c b/ping.c index bd306af..613839b 100644 --- a/ping.c +++ b/ping.c @@ -33,9 +33,10 @@ #include "common.h" #include "spine.h" +#include #ifdef SPINE_HAVE_ICMPV6 -static int ping_icmp_ipv6(host_t *host, ping_t *ping); +static int ping_icmp_ipv6(const host_t *host, ping_t *ping); #endif /*! \fn int ping_host(host_t *host, ping_t *ping) @@ -48,109 +49,68 @@ static int ping_icmp_ipv6(host_t *host, ping_t *ping); * * \return HOST_UP if the host is reachable, HOST_DOWN otherwise. */ +static int ping_network(host_t *host, ping_t *ping) { + if (host->availability.ping_method == PING_ICMP && !set.availability.icmp_avail) { + SPINE_LOG(("Device[%i] DEBUG Falling back to UDP Ping Due to SetUID Issues", host->id)); + host->availability.ping_method = PING_UDP; + } + if (strstr(host->hostname, "localhost")) { + STRNCOPY(ping->ping_status, "0.000"); + STRNCOPY(ping->ping_response, "PING: Device does not require ping."); + return HOST_UP; + } + int address_type = get_address_type(host); + #ifdef SPINE_HAVE_ICMPV6 + if (address_type == SPINE_IPV6 && host->availability.ping_method == PING_ICMP) { + return ping_icmp_ipv6(host, ping); + } + #endif + if (address_type != SPINE_IPV4) { + if (host->availability.method == AVAIL_PING) { + STRNCOPY(ping->ping_status, "0.000"); + STRNCOPY(ping->ping_response, "PING: Device is Unknown or is IPV6. Please use the SNMP ping options only."); + } + return HOST_DOWN; + } + switch (host->availability.ping_method) { + case PING_ICMP: return ping_icmp(host, ping); + case PING_UDP: return ping_udp(host, ping); + case PING_TCP: + case PING_TCP_CLOSED: return ping_tcp(host, ping); + default: return HOST_DOWN; + } +} + +static int ping_snmp_availability(host_t *host, ping_t *ping, int ping_result) { + if (host->availability.method == AVAIL_SNMP_AND_PING && ping_result != HOST_UP) return HOST_DOWN; + if (host->availability.method == AVAIL_SNMP_OR_PING && ping_result == HOST_UP) return HOST_UP; + /* Preserve the configured no-SNMP contract for v1/v2 without a community. */ + if (host->snmp.profile.community[0] == '\0' && host->snmp.profile.version < 3) return HOST_UP; + double begin = get_time_as_double(); + int result = ping_snmp(host, ping); + double elapsed = (get_time_as_double() - begin) * 1000.0; + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, + ("Device[%i] INFO: SNMP Device %s, Time:%.4f ms", host->id, result == HOST_UP ? "Alive" : "Down", elapsed)); + return result; +} + int ping_host(host_t *host, ping_t *ping) { - int ping_result; - int snmp_result; - int address_type; - double snmp_start_time; - double snmp_end_time; - - /* snmp pinging has been selected at a minimum */ - ping_result = 0; - snmp_result = 0; - - /* icmp/tcp/udp ping test */ - if ((host->availability_method == AVAIL_SNMP_AND_PING) || - (host->availability_method == AVAIL_PING) || - (host->availability_method == AVAIL_SNMP_OR_PING)) { - - if (host->ping_method == PING_ICMP) { - if (set.icmp_avail == FALSE) { - SPINE_LOG(("Device[%i] DEBUG Falling back to UDP Ping Due to SetUID Issues", host->id)); - host->ping_method = PING_UDP; - } - } - - if (!strstr(host->hostname, "localhost")) { - address_type = get_address_type(host); - - if (address_type == SPINE_IPV4) { - if (host->ping_method == PING_ICMP) { - ping_result = ping_icmp(host, ping); - } else if (host->ping_method == PING_UDP) { - ping_result = ping_udp(host, ping); - } else if (host->ping_method == PING_TCP || host->ping_method == PING_TCP_CLOSED) { - ping_result = ping_tcp(host, ping); - } - #ifdef SPINE_HAVE_ICMPV6 - } else if (address_type == SPINE_IPV6 && host->ping_method == PING_ICMP) { - ping_result = ping_icmp_ipv6(host, ping); - #endif - } else if (host->availability_method == AVAIL_PING) { - snprintf(ping->ping_status, 50, "0.000"); - snprintf(ping->ping_response, SMALL_BUFSIZE, "PING: Device is Unknown or is IPV6. Please use the SNMP ping options only."); - ping_result = HOST_DOWN; - } - } else { - snprintf(ping->ping_status, 50, "0.000"); - snprintf(ping->ping_response, SMALL_BUFSIZE, "PING: Device does not require ping."); - ping_result = HOST_UP; - } + int network_result = HOST_DOWN; + if (host->availability.method == AVAIL_SNMP_AND_PING || + host->availability.method == AVAIL_PING || + host->availability.method == AVAIL_SNMP_OR_PING) { + network_result = ping_network(host, ping); } - - /* snmp test */ - if ((host->availability_method == AVAIL_SNMP) || - (host->availability_method == AVAIL_SNMP_GET_SYSDESC) || - (host->availability_method == AVAIL_SNMP_GET_NEXT) || - (host->availability_method == AVAIL_SNMP_AND_PING) || - (host->availability_method == AVAIL_SNMP_OR_PING)) { - - /* If we are in AND mode and already have a failed ping result, we don't need SNMP */ - if ((ping_result == HOST_DOWN) && (host->availability_method == AVAIL_SNMP_AND_PING)) { - snmp_result = ping_result; - } else { - /* Lets assume the host is up because if we are in OR mode then we have already - * pinged the host successfully, or some when silly people have not entered an - * snmp_community under v1/2, we assume that this was successfully anyway */ - snmp_result = HOST_UP; - if ((host->availability_method != AVAIL_SNMP_OR_PING) && - ((strlen(host->snmp_community) > 0) || (host->snmp_version >= 3))) { - snmp_start_time = get_time_as_double(); - snmp_result = ping_snmp(host, ping); - snmp_end_time = get_time_as_double(); - - if (snmp_result == HOST_UP) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] INFO: SNMP Device Alive, Time:%.4f ms", host->id, snmp_end_time - snmp_start_time)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] INFO: SNMP Device Alive, Time:%.4f ms", host->id, snmp_end_time - snmp_start_time)); - } - } else { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] INFO: SNMP Device Down, Time:%.4f ms", host->id, snmp_end_time - snmp_start_time)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] INFO: SNMP Device Down, Time:%.4f ms", host->id, snmp_end_time - snmp_start_time)); - } - } - } - } - } - - switch (host->availability_method) { + switch (host->availability.method) { case AVAIL_SNMP_AND_PING: - return ((ping_result == HOST_UP) && (snmp_result == HOST_UP)) ? HOST_UP : HOST_DOWN; case AVAIL_SNMP_OR_PING: - return ((ping_result == HOST_UP) || (snmp_result == HOST_UP)) ? HOST_UP : HOST_DOWN; case AVAIL_SNMP: case AVAIL_SNMP_GET_NEXT: case AVAIL_SNMP_GET_SYSDESC: - return (snmp_result == HOST_UP) ? HOST_UP : HOST_DOWN; - case AVAIL_PING: - return (ping_result == HOST_UP) ? HOST_UP : HOST_DOWN; - case AVAIL_NONE: - return HOST_UP; - default: - return HOST_DOWN; + return ping_snmp_availability(host, ping, network_result); + case AVAIL_PING: return network_result; + case AVAIL_NONE: return HOST_UP; + default: return HOST_DOWN; } } @@ -166,95 +126,40 @@ int ping_host(host_t *host, ping_t *ping) { * */ int ping_snmp(host_t *host, ping_t *ping) { - char *poll_result = NULL; - char *oid; - double begin_time, end_time, total_time; - double one_thousand = 1000.00; - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] DEBUG: Entering SNMP Ping", host->id)); - } else { - SPINE_LOG_DEBUG(("Device[%i] DEBUG: Entering SNMP Ping", host->id)); - } - - if (host->snmp_session) { - if (strlen(host->snmp_community) != 0 || host->snmp_version == 3) { - /* by default, we look at sysUptime */ - if (host->availability_method == AVAIL_SNMP_GET_NEXT) { - oid = strdup(".1.3"); - } else if (host->availability_method == AVAIL_SNMP_GET_SYSDESC) { - oid = strdup(".1.3.6.1.2.1.1.1.0"); - } else { - oid = strdup(".1.3.6.1.2.1.1.3.0"); - } - - if (oid == NULL) die("ERROR: malloc(): strdup() oid ping.c failed"); - - /* record start time */ - begin_time = get_time_as_double(); - - if (host->availability_method == AVAIL_SNMP_GET_NEXT) { - poll_result = snmp_getnext(host, oid); - } else { - poll_result = snmp_get(host, oid); - } - - /* record end time */ - end_time = get_time_as_double(); - - SPINE_FREE(oid); - - total_time = (end_time - begin_time) * one_thousand; - - /* do positive test cases first */ - if (host->snmp_status == SNMPERR_UNKNOWN_OBJID) { - snprintf(ping->snmp_response, SMALL_BUFSIZE, "Device responded to SNMP"); - snprintf(ping->snmp_status, 50, "%.5f", total_time); - - SPINE_FREE(poll_result); - - return HOST_UP; - } else if (host->snmp_status != SNMPERR_SUCCESS) { - if (is_debug_device(host->id)) { - if (host->snmp_status == STAT_TIMEOUT) { - SPINE_LOG(("Device[%i] SNMP Ping Timeout", host->id)); - } else { - SPINE_LOG(("Device[%i] SNMP Ping Unknown Error", host->id)); - } - } else { - if (host->snmp_status == STAT_TIMEOUT) { - SPINE_LOG_HIGH(("Device[%i] SNMP Ping Timeout", host->id)); - } else { - SPINE_LOG_HIGH(("Device[%i] SNMP Ping Unknown Error", host->id)); - } - } - - snprintf(ping->snmp_response, SMALL_BUFSIZE, "Device did not respond to SNMP"); - - SPINE_FREE(poll_result); - - return HOST_DOWN; - } else { - snprintf(ping->snmp_response, SMALL_BUFSIZE, "Device responded to SNMP"); - snprintf(ping->snmp_status, 50, "%.5f", total_time); - - SPINE_FREE(poll_result); - - return HOST_UP; - } - } else { - snprintf(ping->snmp_status, 50, "0.00"); - snprintf(ping->snmp_response, SMALL_BUFSIZE, "Device does not require SNMP"); - - return HOST_UP; - } - } else { - snprintf(ping->snmp_status, 50, "0.00"); - snprintf(ping->snmp_response, SMALL_BUFSIZE, "Invalid SNMP Session"); + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_DEBUG, ("Device[%i] DEBUG: Entering SNMP Ping", host->id)); + if (!host->snmp.session) { + STRNCOPY(ping->snmp_status, "0.00"); + STRNCOPY(ping->snmp_response, "Invalid SNMP Session"); return HOST_DOWN; } + if (host->snmp.profile.community[0] == '\0' && host->snmp.profile.version != 3) { + STRNCOPY(ping->snmp_status, "0.00"); + STRNCOPY(ping->snmp_response, "Device does not require SNMP"); + return HOST_UP; + } + char oid[32]; + switch (host->availability.method) { + case AVAIL_SNMP_GET_NEXT: STRNCOPY(oid, ".1.3"); break; + case AVAIL_SNMP_GET_SYSDESC: STRNCOPY(oid, ".1.3.6.1.2.1.1.1.0"); break; + default: STRNCOPY(oid, ".1.3.6.1.2.1.1.3.0"); break; + } + double begin = get_time_as_double(); + char *result = host->availability.method == AVAIL_SNMP_GET_NEXT ? snmp_getnext(host, oid) : snmp_get(host, oid); + double elapsed = (get_time_as_double() - begin) * 1000.0; + SPINE_FREE(result); + if (host->snmp.status == SNMPERR_SUCCESS || host->snmp.status == SNMPERR_UNKNOWN_OBJID) { + STRNCOPY(ping->snmp_response, "Device responded to SNMP"); + spine_snprintf(ping->snmp_status, sizeof(ping->snmp_status), "%.5f", elapsed); + return HOST_UP; + } + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_HIGH, + ("Device[%i] SNMP Ping %s", host->id, host->snmp.status == STAT_TIMEOUT ? "Timeout" : "Unknown Error")); + STRNCOPY(ping->snmp_response, "Device did not respond to SNMP"); + return HOST_DOWN; } +static int ping_down(ping_t *ping, const char *message); + #ifdef __CYGWIN__ /*! \fn static void icmp_discard_reply(int icmp_socket, char *buffer) * \brief consumes a datagram that ping_icmp() has only peeked at @@ -288,9 +193,9 @@ static void icmp_discard_reply(int icmp_socket, char *buffer) { */ spine_icmp_reply_t spine_icmp_classify_reply(const unsigned char *reply, ssize_t len, uint16_t want_id, uint16_t want_seq, const struct icmp **out_pkt) { - const struct ip *iph; - const struct icmp *pkt; - int ihl; + size_t ihl; + uint16_t received_id; + uint16_t received_seq; if (out_pkt != NULL) { *out_pkt = NULL; @@ -300,25 +205,29 @@ spine_icmp_reply_t spine_icmp_classify_reply(const unsigned char *reply, ssize_t return SPINE_ICMP_REPLY_TOO_SHORT; } - iph = (const struct ip *) reply; - ihl = iph->ip_hl << 2; + /* Read header fields as bytes: the receive buffer has no alignment + * guarantee, and a raw socket can deliver a non-IPv4 or non-ICMP datagram. */ + ihl = (size_t) (reply[0] & 0x0F) << 2; - if (ihl < (int) sizeof(struct ip) || len < (ssize_t) (ihl + ICMP_HDR_SIZE)) { + if ((reply[0] >> 4) != 4 || reply[9] != IPPROTO_ICMP || + ihl < sizeof(struct ip) || len < (ssize_t) (ihl + ICMP_HDR_SIZE)) { return SPINE_ICMP_REPLY_BAD_HEADER; } - pkt = (const struct icmp *) (reply + ihl); - - if (pkt->icmp_type != ICMP_ECHOREPLY) { + /* An echo reply always carries code 0. */ + if (reply[ihl] != ICMP_ECHOREPLY || reply[ihl + 1] != 0) { return SPINE_ICMP_REPLY_NOT_ECHO; } - if (pkt->icmp_id != want_id || pkt->icmp_seq != want_seq) { + memcpy(&received_id, reply + ihl + 4, sizeof(received_id)); + memcpy(&received_seq, reply + ihl + 6, sizeof(received_seq)); + + if (received_id != want_id || received_seq != want_seq) { return SPINE_ICMP_REPLY_NOT_OURS; } if (out_pkt != NULL) { - *out_pkt = pkt; + *out_pkt = (const struct icmp *) (reply + ihl); } return SPINE_ICMP_REPLY_OK; @@ -335,6 +244,7 @@ spine_icmp_reply_t spine_icmp_classify_reply(const unsigned char *reply, ssize_t spine_icmp_reply_t spine_icmp_classify_dgram_reply(const unsigned char *reply, ssize_t len, uint16_t want_seq, const struct icmp **out_pkt) { const struct icmp *pkt; + uint16_t received_seq; if (out_pkt != NULL) { *out_pkt = NULL; @@ -346,11 +256,14 @@ spine_icmp_reply_t spine_icmp_classify_dgram_reply(const unsigned char *reply, s pkt = (const struct icmp *) reply; - if (pkt->icmp_type != ICMP_ECHOREPLY) { + /* Fields are read as bytes; the receive buffer has no alignment guarantee. */ + if (reply[0] != ICMP_ECHOREPLY || reply[1] != 0) { return SPINE_ICMP_REPLY_NOT_ECHO; } - if (pkt->icmp_seq != want_seq) { + memcpy(&received_seq, reply + 6, sizeof(received_seq)); + + if (received_seq != want_seq) { return SPINE_ICMP_REPLY_NOT_OURS; } @@ -373,13 +286,13 @@ spine_icmp_reply_t spine_icmp_classify_dgram_reply(const unsigned char *reply, s * \return HOST_UP if the host is reachable, HOST_DOWN otherwise. * */ -int ping_icmp(host_t *host, ping_t *ping) { +int ping_icmp(const host_t *host, ping_t *ping) { int icmp_socket = -1; int icmp_dgram; int rc = HOST_DOWN; int socket_errno = 0; #if !(defined(__CYGWIN__) && !defined(SOLAR_PRIV)) - const int needs_seteuid = (set.icmp_uses_caps != TRUE); + const int needs_seteuid = (set.availability.icmp_uses_caps != TRUE); #endif double begin_time, end_time, total_time; @@ -471,7 +384,7 @@ int ping_icmp(host_t *host, ping_t *ping) { } /* convert the host timeout to a double precision number in seconds */ - host_timeout = host->ping_timeout; + host_timeout = host->availability.timeout; /* allocate the packet in memory */ packet_len = ICMP_HDR_SIZE + strlen(cacti_msg); @@ -513,7 +426,7 @@ int ping_icmp(host_t *host, ping_t *ping) { begin_time = get_time_as_double(); while (1) { - if (retry_count > host->ping_retries) { + if (retry_count > host->availability.retries) { snprintf(ping->ping_response, SMALL_BUFSIZE, "ICMP: Ping timed out"); snprintf(ping->ping_status, 50, "down"); rc = HOST_DOWN; @@ -521,9 +434,9 @@ int ping_icmp(host_t *host, ping_t *ping) { } if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] DEBUG: Attempting to ping %s, seq %d (Retry %d of %d)", host->id, host->hostname, icmp->icmp_seq, retry_count, host->ping_retries)); + SPINE_LOG(("Device[%i] DEBUG: Attempting to ping %s, seq %d (Retry %d of %d)", host->id, host->hostname, icmp->icmp_seq, retry_count, host->availability.retries)); } else { - SPINE_LOG_DEBUG(("Device[%i] DEBUG: Attempting to ping %s, seq %d (Retry %d of %d)", host->id, host->hostname, icmp->icmp_seq, retry_count, host->ping_retries)); + SPINE_LOG_DEBUG(("Device[%i] DEBUG: Attempting to ping %s, seq %d (Retry %d of %d)", host->id, host->hostname, icmp->icmp_seq, retry_count, host->availability.retries)); } /* decrement the timeout value by the total time */ @@ -803,10 +716,10 @@ static void apply_ipv6_scope_id(struct sockaddr_in6 *name) { * \return HOST_UP if the host is reachable, HOST_DOWN otherwise. * */ -static int ping_icmp_ipv6(host_t *host, ping_t *ping) { +static int ping_icmp_ipv6(const host_t *host, ping_t *ping) { int icmp_socket; int socket_errno = 0; - const int needs_seteuid = (set.icmp_uses_caps != TRUE); + const int needs_seteuid = (set.availability.icmp_uses_caps != TRUE); double begin_time, end_time, total_time; double host_timeout; @@ -920,7 +833,7 @@ static int ping_icmp_ipv6(host_t *host, ping_t *ping) { #endif /* convert the host timeout to a double precision number in seconds */ - host_timeout = host->ping_timeout; + host_timeout = host->availability.timeout; /* allocate the packet in memory */ packet_len = (int) (sizeof(struct icmp6_hdr) + msg_len); @@ -978,16 +891,16 @@ static int ping_icmp_ipv6(host_t *host, ping_t *ping) { begin_time = get_time_as_double(); while (1) { - if (retry_count > host->ping_retries) { + if (retry_count > host->availability.retries) { snprintf(ping->ping_response, SMALL_BUFSIZE, "ICMPv6: Ping timed out"); snprintf(ping->ping_status, 50, "down"); goto cleanup; } if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] DEBUG: Attempting to ping %s, seq %d (Retry %d of %d)", host->id, host->hostname, our_seq, retry_count, host->ping_retries)); + SPINE_LOG(("Device[%i] DEBUG: Attempting to ping %s, seq %d (Retry %d of %d)", host->id, host->hostname, our_seq, retry_count, host->availability.retries)); } else { - SPINE_LOG_DEBUG(("DEBUG: Device[%i] Attempting to ping %s, seq %d (Retry %d of %d)", host->id, host->hostname, our_seq, retry_count, host->ping_retries)); + SPINE_LOG_DEBUG(("DEBUG: Device[%i] Attempting to ping %s, seq %d (Retry %d of %d)", host->id, host->hostname, our_seq, retry_count, host->availability.retries)); } /* decrement the timeout value by the total time */ @@ -1113,7 +1026,7 @@ cleanup: #endif /* SPINE_HAVE_ICMPV6 */ -/*! \fn int ping_udp(host_t *host, ping_t *ping) +/*! \fn int ping_udp(const host_t *host, ping_t *ping) * \brief ping a host using an UDP datagram * \param host a pointer to the current host structure * \param ping a pointer to the current hosts ping structure @@ -1125,161 +1038,67 @@ cleanup: * \return HOST_UP if the host is reachable, HOST_DOWN otherwise. * */ -int ping_udp(host_t *host, ping_t *ping) { - double begin_time, end_time, total_time; - double host_timeout; - double one_thousand = 1000.00; - struct timeval timeout; - int udp_socket; - struct sockaddr_in servername; - char socket_reply[BUFSIZE]; - int retry_count; - char request[BUFSIZE]; - int request_len; - int return_code; - fd_set socket_fds; +static int ping_down(ping_t *ping, const char *message) { + snprintf(ping->ping_status, 50, "down"); + strncopy(ping->ping_response, message, SMALL_BUFSIZE); + return HOST_DOWN; +} - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] DEBUG: Entering UDP Ping", host->id)); - } else { - SPINE_LOG_DEBUG(("Device[%i] DEBUG: Entering UDP Ping", host->id)); +/* UDP reachability is established by the existing ICMP port-error contract; + * receiving an application datagram alone does not establish that result. */ +static int ping_udp_response(int fd, double deadline) { + char response[BUFSIZE]; + for (;;) { + int ready = spine_wait_readable(fd, deadline); + if (ready <= 0) return ready; + ssize_t received = recv(fd, response, sizeof(response), 0); + if (received >= 0) continue; + if (errno == EHOSTUNREACH || errno == ECONNRESET || errno == ECONNREFUSED) return 1; + if (errno != EINTR && errno != EAGAIN && errno != EWOULDBLOCK) return -1; } +} - /* set total time */ - total_time = 0; - - begin_time = get_time_as_double(); - - /* convert the host timeout to a double precision number in seconds */ - host_timeout = host->ping_timeout; - - /* initialize the socket */ - udp_socket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP); - - /* hostname must be nonblank */ - if ((strlen(host->hostname) != 0) && (udp_socket != -1)) { - /* initialize variables */ - snprintf(ping->ping_status, 50, "down"); - snprintf(ping->ping_response, SMALL_BUFSIZE, "default"); - - /* get address of hostname */ - if (init_sockaddr(&servername, host->hostname, host->ping_port)) { - if (connect(udp_socket, (struct sockaddr *) &servername, sizeof(servername)) < 0) { - snprintf(ping->ping_status, 50, "down"); - snprintf(ping->ping_response, SMALL_BUFSIZE, "UDP: Cannot connect to host"); - close(udp_socket); - return HOST_DOWN; - } - - /* format packet */ - snprintf(request, BUFSIZE, "cacti-monitoring-system"); /* the actual test data */ - request_len = strlen(request); - - retry_count = 0; - - /* initialize file descriptor to review for input/output */ - FD_ZERO(&socket_fds); - FD_SET(udp_socket,&socket_fds); - - while (1) { - if (retry_count > host->ping_retries) { - snprintf(ping->ping_response, SMALL_BUFSIZE, "UDP: Ping timed out"); - snprintf(ping->ping_status, 50, "down"); - close(udp_socket); - return HOST_DOWN; - } - - /* record start time */ - if (total_time == 0) { - /* establish timeout value */ - timeout.tv_sec = rint(host_timeout / 1000); - timeout.tv_usec = rint((int) host_timeout % 1000) * 1000; - - /* set the socket send and receive timeout */ - setsockopt(udp_socket, SOL_SOCKET, SO_RCVTIMEO, (char*)&timeout, sizeof(timeout)); - setsockopt(udp_socket, SOL_SOCKET, SO_SNDTIMEO, (char*)&timeout, sizeof(timeout)); - } else { - /* decrement the timeout value by the total time */ - timeout.tv_sec = rint((host_timeout - total_time) / 1000); - timeout.tv_usec = ((int) (host_timeout - total_time) % 1000) * 1000; - - /* set the socket send and receive timeout */ - setsockopt(udp_socket, SOL_SOCKET, SO_RCVTIMEO, (char*)&timeout, sizeof(timeout)); - setsockopt(udp_socket, SOL_SOCKET, SO_SNDTIMEO, (char*)&timeout, sizeof(timeout)); - } - - /* send packet to destination */ - send(udp_socket, request, request_len, 0); - - /* wait for a response on the socket */ - wait_more: - return_code = select(FD_SETSIZE, &socket_fds, NULL, NULL, &timeout); - - /* record end time */ - end_time = get_time_as_double(); - - /* calculate total time */ - total_time = (end_time - begin_time) * one_thousand; - - /* check to see which socket talked */ - if (return_code > 0) { - if (FD_ISSET(udp_socket, &socket_fds)) { - return_code = read(udp_socket, socket_reply, BUFSIZE); - - if (return_code == -1 && (errno == EHOSTUNREACH || errno == ECONNRESET || errno == ECONNREFUSED)) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] INFO: UDP Device Alive, Try Count:%i, Time:%.4f ms", host->id, retry_count+1, (total_time))); - } else { - SPINE_LOG_MEDIUM(("Device[%i] INFO: UDP Device Alive, Try Count:%i, Time:%.4f ms", host->id, retry_count+1, (total_time))); - } - snprintf(ping->ping_response, SMALL_BUFSIZE, "UDP: Device is Alive"); - snprintf(ping->ping_status, 50, "%.5f", total_time); - close(udp_socket); - return HOST_UP; - } - } - } else if (return_code == -1) { - if (errno == EINTR) { - /* interrupted, try again */ - usleep(10000); - goto wait_more; - } else { - snprintf(ping->ping_response, SMALL_BUFSIZE, "UDP: Device is Down"); - snprintf(ping->ping_status, 50, "%.5f", total_time); - close(udp_socket); - return HOST_DOWN; - } - } else { - /* timeout */ - } - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] DEBUG: UDP Timeout, Try Count:%i, Time:%.4f ms", host->id, retry_count+1, (total_time))); - } else { - SPINE_LOG_DEBUG(("Device[%i] DEBUG: UDP Timeout, Try Count:%i, Time:%.4f ms", host->id, retry_count+1, (total_time))); - } - - retry_count++; - #ifndef SOLAR_THREAD - usleep(1000); - #endif - } - } else { - snprintf(ping->ping_response, SMALL_BUFSIZE, "UDP: Destination hostname invalid"); - snprintf(ping->ping_status, 50, "down"); - close(udp_socket); - return HOST_DOWN; +int ping_udp(const host_t *host, ping_t *ping) { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_DEBUG, ("Device[%i] DEBUG: Entering UDP Ping", host->id)); + if (host->hostname[0] == '\0') return ping_down(ping, "UDP: Destination address invalid or unable to create socket"); + struct sockaddr_in servername = {0}; + if (!init_sockaddr(&servername, host->hostname, host->availability.port)) return ping_down(ping, "UDP: Destination hostname invalid"); + if (host->availability.timeout <= 0 || host->availability.retries < 0) return ping_down(ping, "UDP: Ping timed out"); + int fd = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP); + if (fd < 0) return ping_down(ping, "UDP: Destination address invalid or unable to create socket"); + int flags = fcntl(fd, F_GETFL, 0); + if (flags < 0 || fcntl(fd, F_SETFL, flags | O_NONBLOCK) < 0 || + connect(fd, (struct sockaddr *)&servername, sizeof(servername)) < 0) { + close(fd); + return ping_down(ping, "UDP: Cannot connect to host"); + } + static const char request[] = "cacti-monitoring-system"; + double begin = spine_monotonic_time(); + for (unsigned int attempt = 0; ; attempt++) { + double deadline = spine_monotonic_time() + (double)host->availability.timeout / 1000; + ssize_t sent; + do { + sent = send(fd, request, sizeof(request) - 1, 0); + } while (sent < 0 && errno == EINTR && spine_monotonic_time() < deadline); + int result = sent == (ssize_t)(sizeof(request) - 1) ? ping_udp_response(fd, deadline) : -1; + double elapsed = (spine_monotonic_time() - begin) * 1000; + if (result > 0) { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] INFO: UDP Device Alive, Try Count:%u, Time:%.4f ms", host->id, attempt + 1, elapsed)); + strncopy(ping->ping_response, "UDP: Device is Alive", SMALL_BUFSIZE); + snprintf(ping->ping_status, 50, "%.5f", elapsed); + close(fd); + return HOST_UP; } - } else { - snprintf(ping->ping_response, SMALL_BUFSIZE, "UDP: Destination address invalid or unable to create socket"); - snprintf(ping->ping_status, 50, "down"); - if (udp_socket != -1) close(udp_socket); - return HOST_DOWN; + if (result < 0 || attempt >= (unsigned int)host->availability.retries) { + close(fd); + return ping_down(ping, result < 0 ? "UDP: Device is Down" : "UDP: Ping timed out"); + } + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_DEBUG, ("Device[%i] DEBUG: UDP Timeout, Try Count:%u, Time:%.4f ms", host->id, attempt + 1, elapsed)); } } -/*! \fn int ping_tcp(host_t *host, ping_t *ping) +/*! \fn int ping_tcp(const host_t *host, ping_t *ping) * \brief ping a host using an TCP syn * \param host a pointer to the current host structure * \param ping a pointer to the current hosts ping structure @@ -1291,102 +1110,57 @@ int ping_udp(host_t *host, ping_t *ping) { * \return HOST_UP if the host is reachable, HOST_DOWN otherwise. * */ -int ping_tcp(host_t *host, ping_t *ping) { - double begin_time, end_time, total_time; - double host_timeout; - double one_thousand = 1000.00; - struct timeval timeout; - int tcp_socket; - struct sockaddr_in servername; - int retry_count; - int return_code; +/* Each retry owns a new nonblocking socket; a failed connect socket must not + * be reused. SO_ERROR distinguishes completion from refused/timed-out peers. */ +static int ping_tcp_complete(int fd, double deadline) { + int ready = spine_wait_writable(fd, deadline); + if (ready <= 0) return ready == 0 ? ETIMEDOUT : errno; + int error = 0; + socklen_t length = sizeof(error); + if (getsockopt(fd, SOL_SOCKET, SO_ERROR, &error, &length) < 0) return errno; + return error; +} - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] DEBUG: Entering TCP Ping", host->id)); - } else { - SPINE_LOG_DEBUG(("Device[%i] DEBUG: Entering TCP Ping", host->id)); - } +static int ping_tcp_attempt(int fd, const struct sockaddr_in *address, double deadline) { + int flags = fcntl(fd, F_GETFL, 0); + if (flags < 0 || fcntl(fd, F_SETFL, flags | O_NONBLOCK) < 0) return errno; + if (connect(fd, (const struct sockaddr *)address, sizeof(*address)) == 0) return 0; + int error = errno; + if (error != EINPROGRESS && error != EINTR) return error; + return ping_tcp_complete(fd, deadline); +} - /* convert the host timeout to a double precision number in seconds */ - host_timeout = host->ping_timeout; +static int ping_tcp_connect(const struct sockaddr_in *address, double deadline) { + int fd = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); + if (fd < 0) return errno; + int error = ping_tcp_attempt(fd, address, deadline); + close(fd); + return error; +} - /* initialize the socket */ - tcp_socket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); - - /* initialize total time */ - total_time = 0; - - /* initialize begin time */ - begin_time = get_time_as_double(); - - /* hostname must be nonblank */ - if ((strlen(host->hostname) != 0) && (tcp_socket != -1)) { - /* initialize variables */ - snprintf(ping->ping_status, 50, "down"); - snprintf(ping->ping_response, SMALL_BUFSIZE, "default"); - - /* get address of hostname */ - if (init_sockaddr(&servername, host->hostname, host->ping_port)) { - /* first attempt a connect */ - retry_count = 0; - - while (1) { - /* establish timeout value */ - timeout.tv_sec = rint(host_timeout / 1000); - timeout.tv_usec = ((int) host_timeout % 1000) * 1000; - - /* set the socket send and receive timeout */ - setsockopt(tcp_socket, SOL_SOCKET, SO_RCVTIMEO, (char*)&timeout, sizeof(timeout)); - setsockopt(tcp_socket, SOL_SOCKET, SO_SNDTIMEO, (char*)&timeout, sizeof(timeout)); - - /* make the connection */ - return_code = connect(tcp_socket, (struct sockaddr *) &servername, sizeof(servername)); - - /* record end time */ - end_time = get_time_as_double(); - - /* calculate total time */ - total_time = (end_time - begin_time) * one_thousand; - - if ((return_code == -1 && errno == ECONNREFUSED && host->ping_method == PING_TCP_CLOSED) || return_code == 0) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] INFO: TCP Device Alive, Try Count:%i, Time:%.4f ms", host->id, retry_count+1, (total_time))); - } else { - SPINE_LOG_MEDIUM(("Device[%i] INFO: TCP Device Alive, Try Count:%i, Time:%.4f ms", host->id, retry_count+1, (total_time))); - } - snprintf(ping->ping_response, SMALL_BUFSIZE, "TCP: Device is Alive"); - snprintf(ping->ping_status, 50, "%.5f", total_time); - close(tcp_socket); - return HOST_UP; - } else { - #if defined(__CYGWIN__) - snprintf(ping->ping_status, 50, "down"); - snprintf(ping->ping_response, SMALL_BUFSIZE, "TCP: Cannot connect to host"); - close(tcp_socket); - return HOST_DOWN; - #else - if (retry_count > host->ping_retries) { - snprintf(ping->ping_status, 50, "down"); - snprintf(ping->ping_response, SMALL_BUFSIZE, "TCP: Cannot connect to host"); - close(tcp_socket); - return HOST_DOWN; - } else { - retry_count++; - } - #endif - } - } - } else { - snprintf(ping->ping_response, SMALL_BUFSIZE, "TCP: Destination hostname invalid"); - snprintf(ping->ping_status, 50, "down"); - close(tcp_socket); - return HOST_DOWN; +int ping_tcp(const host_t *host, ping_t *ping) { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_DEBUG, ("Device[%i] DEBUG: Entering TCP Ping", host->id)); + if (host->hostname[0] == '\0') return ping_down(ping, "TCP: Destination address invalid or unable to create socket"); + struct sockaddr_in address = {0}; + if (!init_sockaddr(&address, host->hostname, host->availability.port)) return ping_down(ping, "TCP: Destination hostname invalid"); + if (host->availability.timeout <= 0 || host->availability.retries < 0) return ping_down(ping, "TCP: Cannot connect to host"); + double begin = spine_monotonic_time(); + for (unsigned int attempt = 0; ; attempt++) { + double deadline = spine_monotonic_time() + (double)host->availability.timeout / 1000; + int error = ping_tcp_connect(&address, deadline); + if (error == 0 || (error == ECONNREFUSED && host->availability.ping_method == PING_TCP_CLOSED)) { + double elapsed = (spine_monotonic_time() - begin) * 1000; + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] INFO: TCP Device Alive, Try Count:%u, Time:%.4f ms", host->id, attempt + 1, elapsed)); + strncopy(ping->ping_response, "TCP: Device is Alive", SMALL_BUFSIZE); + snprintf(ping->ping_status, 50, "%.5f", elapsed); + return HOST_UP; } - } else { - snprintf(ping->ping_response, SMALL_BUFSIZE, "TCP: Destination address invalid or unable to create socket"); - snprintf(ping->ping_status, 50, "down"); - if (tcp_socket != -1) close(tcp_socket); - return HOST_DOWN; + /* Cygwin historically makes one attempt only. */ + #if defined(__CYGWIN__) + return ping_down(ping, "TCP: Cannot connect to host"); + #else + if (attempt >= (unsigned int)host->availability.retries) return ping_down(ping, "TCP: Cannot connect to host"); + #endif } } @@ -1396,9 +1170,10 @@ int ping_tcp(host_t *host, ping_t *ping) { * \return 1 - IPv4, 2 - IPv6, 0 - Unknown */ int get_address_type(host_t *host) { - struct addrinfo hints, *res, *res_list; + struct addrinfo hints; + struct addrinfo *res_list; char addrstr[255]; - void *ptr = NULL; + const void *ptr = NULL; int addr_found = FALSE; memset(&hints, 0, sizeof(hints)); @@ -1413,7 +1188,7 @@ int get_address_type(host_t *host) { return SPINE_NONE; } - for (res = res_list; res != NULL; res = res->ai_next) { + for (struct addrinfo *res = res_list; res != NULL; res = res->ai_next) { inet_ntop(res->ai_family, res->ai_addr->sa_data, addrstr, 100); switch(res->ai_family) { @@ -1425,6 +1200,8 @@ int get_address_type(host_t *host) { ptr = &((struct sockaddr_in6 *) res->ai_addr)->sin6_addr; addr_found = TRUE; break; + default: + continue; } inet_ntop(res->ai_family, ptr, addrstr, 100); @@ -1447,177 +1224,69 @@ int get_address_type(host_t *host) { } } -/*! \fn int init_sockaddr(struct sockaddr_in *name, const char *hostname, unsigned short int port) +/*! \fn int init_sockaddr(struct sockaddr_in *name, const char *hostname, int port) * \brief converts a hostname to an internet address * * \return TRUE if successful, FALSE otherwise. * */ -int init_sockaddr(struct sockaddr_in *name, const char *hostname, unsigned short int port) { - struct addrinfo hints, *hostinfo = NULL; - int rv, retry_count; - - /* Initialize the hints structure */ - memset(&hints, 0, sizeof hints); - +int init_sockaddr(struct sockaddr_in *name, const char *hostname, int port) { + if (port < 0 || port > 65535) return FALSE; + struct addrinfo hints = {0}; + struct addrinfo *hostinfo = NULL; hints.ai_family = AF_INET; hints.ai_flags = AI_CANONNAME | AI_ADDRCONFIG; - retry_count = 0; - rv = 0; - - while (TRUE) { - rv = getaddrinfo(hostname, NULL, &hints, &hostinfo); - - if (rv == 0) { - break; - } else { - switch (rv) { - case EAI_AGAIN: - if (retry_count < 3) { - SPINE_LOG(("WARNING: EAGAIN received resolving after 3 retryies for host %s (%s)", hostname, gai_strerror(rv))); - - retry_count++; - usleep(50000); - continue; - } else { - SPINE_LOG(("WARNING: Error resolving after 3 retryies for host %s (%s)", hostname, gai_strerror(rv))); - return FALSE; - } - - break; - case EAI_FAIL: - SPINE_LOG(("WARNING: DNS Server reported permanent error for host %s (%s)", hostname, gai_strerror(rv))); - return FALSE; - - break; - case EAI_MEMORY: - SPINE_LOG(("WARNING: Out of memory trying to resolve host %s (%s)", hostname, gai_strerror(rv))); - return FALSE; - - break; - default: - SPINE_LOG(("WARNING: Unknown error while resolving host %s (%s)", hostname, gai_strerror(rv))); - return FALSE; - - break; - } + for (int attempt = 0; ; attempt++) { + int status = getaddrinfo(hostname, NULL, &hints, &hostinfo); + if (status == 0) break; + if (status == EAI_AGAIN && attempt < 3) { + SPINE_LOG(("WARNING: Temporary DNS error for host %s (%s), retrying", hostname, gai_strerror(status))); + spine_sleep_usec(50000); + continue; } + SPINE_LOG(("WARNING: Error resolving host %s (%s)", hostname, gai_strerror(status))); + return FALSE; } - if (hostinfo == NULL) { SPINE_LOG(("WARNING: Unknown host %s", hostname)); return FALSE; - } else { - /* Copy socket details */ - name->sin_family = hostinfo->ai_family; - name->sin_addr = ((struct sockaddr_in *)hostinfo->ai_addr)->sin_addr; - name->sin_port = htons(port); - - /* Free results var */ - freeaddrinfo(hostinfo); - return TRUE; } + name->sin_family = AF_INET; + name->sin_addr = ((struct sockaddr_in *)hostinfo->ai_addr)->sin_addr; + name->sin_port = htons((unsigned short)port); + freeaddrinfo(hostinfo); + return TRUE; } -/*! \fn name_t *get_namebyhost(char *hostname, name_t *name) - * \brief splits the hostname into method, name and port - * - * \return name_t containing a trimmed hostname, port, and optional method - * - */ -name_t *get_namebyhost(char *hostname, name_t *name) { +/*! \brief Parse a device hostname with an optional transport and port. */ +name_t *get_namebyhost(const char *hostname, name_t *name) { if (name == NULL) { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - Allocating name_t", hostname)); - - if (!(name = (name_t *) malloc(sizeof(name_t)))) { - die("ERROR: Fatal malloc error: ping.c get_namebyhost->name"); - } - - memset(name, '\0', sizeof(name_t)); + name = calloc(1, sizeof(*name)); + if (name == NULL) die("ERROR: Fatal malloc error: ping.c get_namebyhost->name"); } - - int tokens = 0; - char *stack = NULL; - char *token = NULL; + /* IPv6 transport addresses are passed intact to the address resolver. */ + if (strchr(hostname, '[') != NULL || strstr(hostname, "::") != NULL || char_count(hostname, ':') > 2) { + strncopy(name->hostname, hostname, sizeof(name->hostname)); + return name; + } + char *copy = strdup(hostname); + if (copy == NULL) die("ERROR: Fatal malloc error: ping.c get_namebyhost->stack"); char *saveptr = NULL; - - if (!(stack = (char *) malloc(strlen(hostname)+1))) { - die("ERROR: Fatal malloc error: ping.c get_namebyhost->stack"); - } - - memset(stack, '\0', strlen(hostname)+1); - /* obuf includes the NUL, so pass the full buffer size to copy all chars */ - strncopy(stack, hostname, strlen(hostname)+1); - token = strtok_r(stack, ":", &saveptr); - - if (token == NULL) { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - No delimiter, assume full hostname", hostname)); - strncopy(name->hostname, hostname, SMALL_BUFSIZE); - } - - while (token != NULL && tokens <= 3) { - tokens++; - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - Token #%i - %s", hostname, tokens, token)); - if (tokens == 1) { - if (strlen(token) && token[0] == '[') { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - Have TCPv6 method", hostname)); - strncopy(name->hostname, hostname, sizeof(name->hostname)); + const char *token = strtok_r(copy, ":", &saveptr); + static const char *const methods[] = {"TCP", "UDP", "TCP6", "UDP6"}; + if (token != NULL) { + for (size_t i = 0; i < sizeof(methods) / sizeof(methods[0]); i++) { + if (strcasecmp(token, methods[i]) == 0) { + name->method = (int)i + 1; + token = strtok_r(NULL, ":", &saveptr); break; - } else if (strlen(token) == 3) { - if (strncasecmp(token, "TCP", 3) == 0) { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - Have TCPv4 method", hostname)); - name->method = 1; - } else if (strncasecmp(token, "UDP", 3) == 0) { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - Have UDPv4 method", hostname)); - name->method = 2; - } else { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - No matching method for 3 chars: %s", hostname, token)); - // assume we have had a method - tokens++; - } - } else if (strlen(token) == 4) { - if (strncasecmp(token, "TCP6", 4) == 0) { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - Have TCPv6 method", hostname)); - name->method = 3; - } else if (strncasecmp(token, "UDP6", 4) == 0) { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - Have UDPv6 method", hostname)); - name->method = 4; - } else { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - No matching method for 4 chars: %s", hostname, token)); - - // assume we have had a method - tokens++; - } - } else { - SPINE_LOG_DEBUG(("DEBUG: get_hostbyname(%s) - No matching method for %li chars: %s", hostname, strlen(token), token)); - - // assume we have had a method - tokens++; } } - - if (tokens == 2) { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - Setting hostname: %s", hostname, token)); - strncpy(name->hostname, token, sizeof(name->hostname) - 1); - name->hostname[sizeof(name->hostname) - 1] = '\0'; - } - - if (tokens == 3 && strlen(token)) { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - Setting port: %s", hostname, token)); - name->port = atoi(token); - } - - if (tokens > 3) { - SPINE_LOG_DEBUG(("DEBUG: get_namebyhost(%s) - Unexpected token: %i", hostname, tokens)); - } - token = strtok_r(NULL, ":", &saveptr); } - - if (stack != NULL) { - free(stack); - stack = NULL; - } - + strncopy(name->hostname, token != NULL ? token : hostname, sizeof(name->hostname)); + token = strtok_r(NULL, ":", &saveptr); + if (token != NULL) name->port = atoi(token); + free(copy); return name; } @@ -1648,259 +1317,138 @@ unsigned short int get_checksum(void* buf, int len) { sum = (sum >> 16) + (sum & 0xffff); sum += (sum >> 16); - answer = ~sum; /* truncate to 16 bits */ + answer = (unsigned short int)((unsigned int)~sum & 0xffff); /* checksum truncation is intentional */ return answer; } -/*! \fn void update_host_status(int status, host_t *host, ping_t *ping, int availability_method) - * \brief update the host table in Cacti with the result of the ping of the host. - * \param status the current poll status of the host, either HOST_UP, or HOST_DOWN - * \param host a pointer to the current host structure - * \param ping a pointer to the current hosts ping structure - * \param availability_method the method that was used to poll the host - * - * This function will determine if the host is UP, DOWN, or RECOVERING based upon - * the ping result and it's current status. It will update the Cacti database - * with the calculated status. - * - */ -void update_host_status(int status, host_t *host, ping_t *ping, int availability_method) { - int issue_log_message = FALSE; - double ping_time; - double hundred_percent = 100.00; - char current_date[40]; +static bool host_requires_snmp(const host_t *host) { + return host->snmp.profile.community[0] != '\0' || host->snmp.profile.version >= 3; +} - snprintf(current_date, 40, "%lu", time(NULL)); - - /* host is down */ - if (status == HOST_DOWN) { - /* update total polls, failed polls and availability */ - host->failed_polls = host->failed_polls + 1; - host->total_polls = host->total_polls + 1; - host->availability = hundred_percent * (host->total_polls - host->failed_polls) / host->total_polls; - - /*determine the error message to display */ - switch (availability_method) { +static void host_failure_message(host_t *host, const ping_t *ping, int method) { + switch (method) { case AVAIL_SNMP_OR_PING: case AVAIL_SNMP_AND_PING: - if (strlen(host->snmp_community) == 0 && host->snmp_version < 3) { - snprintf(host->status_last_error, BUFSIZE * 2 + 1, "%s", ping->ping_response); + if (host_requires_snmp(host)) { + snprintf(host->state.status_last_error, BUFSIZE * 2 + 1, "%s, %s", ping->snmp_response, ping->ping_response); } else { - snprintf(host->status_last_error, BUFSIZE * 2 + 1, "%s, %s", ping->snmp_response, ping->ping_response); + snprintf(host->state.status_last_error, BUFSIZE * 2 + 1, "%s", ping->ping_response); } break; case AVAIL_SNMP: - if (strlen(host->snmp_community) == 0 && host->snmp_version < 3) { - snprintf(host->status_last_error, BUFSIZE * 2 + 1, "%s", "Device does not require SNMP"); - } else { - snprintf(host->status_last_error, BUFSIZE * 2 + 1, "%s", ping->snmp_response); - } + snprintf(host->state.status_last_error, BUFSIZE * 2 + 1, "%s", host_requires_snmp(host) ? ping->snmp_response : "Device does not require SNMP"); break; default: - snprintf(host->status_last_error, BUFSIZE * 2 + 1, "%s", ping->ping_response); - } + snprintf(host->state.status_last_error, BUFSIZE * 2 + 1, "%s", ping->ping_response); + } +} + +static bool host_failure_transition(host_t *host, const char *date) { + switch (host->state.status) { + case HOST_UP: + host->state.status_event_count++; + if (host->state.status_event_count >= set.availability.ping_failure_count) { + host->state.status = HOST_DOWN; + if (set.availability.ping_failure_count == 1) snprintf(host->state.status_fail_date, sizeof(host->state.status_fail_date), "%s", date); + return TRUE; + } + if (host->state.status_event_count == 1) snprintf(host->state.status_fail_date, sizeof(host->state.status_fail_date), "%s", date); + return FALSE; + case HOST_RECOVERING: + host->state.status_event_count = 1; + host->state.status = HOST_DOWN; + return FALSE; + case HOST_UNKNOWN: + host->state.status = HOST_DOWN; + host->state.status_event_count = 0; + return FALSE; + default: + host->state.status_event_count++; + return FALSE; + } +} - /* determine if to send an alert and update remainder of statistics */ - if (host->status == HOST_UP) { - /* increment the event failure count */ - host->status_event_count++; - - /* if it's time to issue an error message, indicate so */ - if (host->status_event_count >= set.ping_failure_count) { - /* host is now down, flag it that way */ - host->status = HOST_DOWN; - - issue_log_message = TRUE; - - /* update the failure date only if the failure count is 1 */ - if (set.ping_failure_count == 1) { - snprintf(host->status_fail_date, 40, "%s", current_date); - } - } else { - /* host down for the first time, set event date */ - if (host->status_event_count == 1) { - snprintf(host->status_fail_date, 40, "%s", current_date); - } - } - } else if (host->status == HOST_RECOVERING) { - /* host is recovering, put back in failed state */ - host->status_event_count = 1; - host->status = HOST_DOWN; - } else if (host->status == HOST_UNKNOWN) { - /* host was unknown and now is down */ - host->status = HOST_DOWN; - host->status_event_count = 0; - } else { - host->status_event_count++; - } +static bool host_recovery_transition(host_t *host, const char *date) { + if (host->state.status != HOST_DOWN && host->state.status != HOST_RECOVERING) { + host->state.status = HOST_UP; + host->state.status_event_count = 0; + return FALSE; + } + if (host->state.status == HOST_DOWN) { + host->state.status = HOST_RECOVERING; + host->state.status_event_count = 1; + } else { + host->state.status_event_count++; + } + if (host->state.status_event_count >= set.availability.ping_recovery_count) { + host->state.status = HOST_UP; + if (set.availability.ping_recovery_count == 1) snprintf(host->state.status_rec_date, sizeof(host->state.status_rec_date), "%s", date); + host->state.status_event_count = 0; + return TRUE; + } + if (host->state.status_event_count == 1) snprintf(host->state.status_rec_date, sizeof(host->state.status_rec_date), "%s", date); + return FALSE; +} + +static double host_response_time(const host_t *host, const ping_t *ping, int method) { + switch (method) { + case AVAIL_SNMP_AND_PING: + return host_requires_snmp(host) ? (atof(ping->snmp_status) + atof(ping->ping_status)) / 2 : atof(ping->ping_status); + case AVAIL_SNMP: + return host_requires_snmp(host) ? atof(ping->snmp_status) : 0.0; + case AVAIL_NONE: + return 0.0; + default: + return atof(ping->ping_status); + } +} + +static void host_response_statistics(host_t *host, double ping_time) { + host->statistics.cur_time = ping_time; + if (ping_time > host->statistics.max_time) host->statistics.max_time = ping_time; + if (ping_time < host->statistics.min_time) host->statistics.min_time = ping_time; + host->statistics.avg_time = (((host->statistics.total_polls - 1 - host->statistics.failed_polls) * host->statistics.avg_time) + ping_time) / (host->statistics.total_polls - host->statistics.failed_polls); +} + +static void log_host_availability(const host_t *host, const ping_t *ping, int method) { + if (set.logging.log_level < POLLER_VERBOSITY_HIGH) return; + bool up = host->state.status == HOST_UP || host->state.status == HOST_RECOVERING; + if (method == AVAIL_SNMP_AND_PING || (method == AVAIL_SNMP_OR_PING && up)) { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_HIGH, ("Device[%i] PING Result: %s", host->id, ping->ping_response)); + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_HIGH, ("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); + } else if (method == AVAIL_SNMP) { + const char *message = up && !host_requires_snmp(host) ? "Device does not require SNMP" : ping->snmp_response; + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_HIGH, ("Device[%i] SNMP Result: %s", host->id, message)); + } else if (method == AVAIL_NONE) { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_HIGH, ("Device[%i] No Device Availability Method Selected", host->id)); + } else if (up) { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_HIGH, ("Device[%i] PING: Result %s", host->id, ping->ping_response)); } else { - /* host is up!! */ - - /* update total polls and availability */ - host->total_polls = host->total_polls + 1; - host->availability = hundred_percent * (host->total_polls - host->failed_polls) / host->total_polls; - - /* determine the ping statistic to set and do so */ - if (availability_method == AVAIL_SNMP_AND_PING) { - if (strlen(host->snmp_community) == 0 && host->snmp_version < 3) { - ping_time = atof(ping->ping_status); - } else { - /* calculate the average of the two times */ - ping_time = (atof(ping->snmp_status) + atof(ping->ping_status)) / 2; - } - } else if (availability_method == AVAIL_SNMP) { - if (strlen(host->snmp_community) == 0 && host->snmp_version < 3) { - ping_time = 0.000; - } else { - ping_time = atof(ping->snmp_status); - } - } else if (availability_method == AVAIL_NONE) { - ping_time = 0.000; - } else { - ping_time = atof(ping->ping_status); - } - - /* update times as required */ - host->cur_time = ping_time; - - /* maximum time */ - if (ping_time > host->max_time) - host->max_time = ping_time; - - /* minimum time */ - if (ping_time < host->min_time) - host->min_time = ping_time; - - /* average time */ - host->avg_time = (((host->total_polls-1-host->failed_polls) - * host->avg_time) + ping_time) / (host->total_polls-host->failed_polls); - - /* the host was down, now it's recovering */ - if ((host->status == HOST_DOWN) || (host->status == HOST_RECOVERING)) { - /* just up, change to recovering */ - if (host->status == HOST_DOWN) { - host->status = HOST_RECOVERING; - host->status_event_count = 1; - } else { - host->status_event_count++; - } - - /* if it's time to issue a recovery message, indicate so */ - if (host->status_event_count >= set.ping_recovery_count) { - /* host is up, flag it that way */ - host->status = HOST_UP; - - issue_log_message = TRUE; - - /* update the recovery date only if the recovery count is 1 */ - if (set.ping_recovery_count == 1) { - snprintf(host->status_rec_date, 40, "%s", current_date); - } - - /* reset the event counter */ - host->status_event_count = 0; - } else { - /* host recovering for the first time, set event date */ - if (host->status_event_count == 1) { - snprintf(host->status_rec_date, 40, "%s", current_date); - } - } - } else if (host->status_event_count > 0) { - /* host was unknown and now is up */ - host->status = HOST_UP; - host->status_event_count = 0; - } else { - /* host was unknown and now is up */ - host->status = HOST_UP; - host->status_event_count = 0; - } + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_HIGH, ("Device[%i] PING Result: %s", host->id, ping->ping_response)); } - - /* if the user wants a flood of information then flood them */ - if (set.log_level >= POLLER_VERBOSITY_HIGH) { - if ((host->status == HOST_UP) || (host->status == HOST_RECOVERING)) { - /* log ping result if we are to use a ping for reachability testing */ - if (availability_method == AVAIL_SNMP_AND_PING) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] PING Result: %s", host->id, ping->ping_response)); - SPINE_LOG(("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); - } else { - SPINE_LOG_HIGH(("Device[%i] PING Result: %s", host->id, ping->ping_response)); - SPINE_LOG_HIGH(("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); - } - } else if (availability_method == AVAIL_SNMP_OR_PING) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] PING Result: %s", host->id, ping->ping_response)); - SPINE_LOG(("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); - } else { - SPINE_LOG_HIGH(("Device[%i] PING Result: %s", host->id, ping->ping_response)); - SPINE_LOG_HIGH(("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); - } - } else if (availability_method == AVAIL_SNMP) { - if ((strlen(host->snmp_community) == 0) && (host->snmp_version < 3)) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] SNMP Result: Device does not require SNMP", host->id)); - } else { - SPINE_LOG_HIGH(("Device[%i] SNMP Result: Device does not require SNMP", host->id)); - } - } else { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); - } else { - SPINE_LOG_HIGH(("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); - } - } - } else if (availability_method == AVAIL_NONE) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] No Device Availability Method Selected", host->id)); - } else { - SPINE_LOG_HIGH(("Device[%i] No Device Availability Method Selected", host->id)); - } - } else { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] PING: Result %s", host->id, ping->ping_response)); - } else { - SPINE_LOG_HIGH(("Device[%i] PING: Result %s", host->id, ping->ping_response)); - } - } - } else { - if (availability_method == AVAIL_SNMP_AND_PING) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] PING Result: %s", host->id, ping->ping_response)); - SPINE_LOG(("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); - } else { - SPINE_LOG_HIGH(("Device[%i] PING Result: %s", host->id, ping->ping_response)); - SPINE_LOG_HIGH(("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); - } - } else if (availability_method == AVAIL_SNMP) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); - } else { - SPINE_LOG_HIGH(("Device[%i] SNMP Result: %s", host->id, ping->snmp_response)); - } - } else if (availability_method == AVAIL_NONE) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] No Device Availability Method Selected", host->id)); - } else { - SPINE_LOG_HIGH(("Device[%i] No Device Availability Method Selected", host->id)); - } - } else { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] PING Result: %s", host->id, ping->ping_response)); - } else { - SPINE_LOG_HIGH(("Device[%i] PING Result: %s", host->id, ping->ping_response)); - } - } - } +} + +/*! Update availability statistics and apply failure/recovery thresholds. */ +void update_host_status(int status, host_t *host, const ping_t *ping, int availability_method) { + char current_date[40]; + snprintf(current_date, sizeof(current_date), "%lu", time(NULL)); + bool issue_log_message; + host->statistics.total_polls++; + if (status == HOST_DOWN) { + host->statistics.failed_polls++; + host_failure_message(host, ping, availability_method); + issue_log_message = host_failure_transition(host, current_date); + } else { + host_response_statistics(host, host_response_time(host, ping, availability_method)); + issue_log_message = host_recovery_transition(host, current_date); } - - /* if there is supposed to be an event generated, do it */ - if (issue_log_message) { - if (host->status == HOST_DOWN) { - SPINE_LOG(("Device[%i] Hostname[%s] ERROR: HOST EVENT: Device is DOWN Message: %s", host->id, host->hostname, host->status_last_error)); - } else { - SPINE_LOG(("Device[%i] Hostname[%s] NOTICE: HOST EVENT: Device Returned from DOWN State", host->id, host->hostname)); - } + host->statistics.availability = 100.0 * (host->statistics.total_polls - host->statistics.failed_polls) / host->statistics.total_polls; + log_host_availability(host, ping, availability_method); + if (!issue_log_message) return; + if (host->state.status == HOST_DOWN) { + SPINE_LOG(("Device[%i] Hostname[%s] ERROR: HOST EVENT: Device is DOWN Message: %s", host->id, host->hostname, host->state.status_last_error)); + } else { + SPINE_LOG(("Device[%i] Hostname[%s] NOTICE: HOST EVENT: Device Returned from DOWN State", host->id, host->hostname)); } } diff --git a/ping.h b/ping.h index 85ac0b9..1e3a71c 100644 --- a/ping.h +++ b/ping.h @@ -163,12 +163,12 @@ extern spine_icmp_reply_t spine_icmp_classify_dgram_reply(const unsigned char *r extern int ping_host(host_t *host, ping_t *ping); extern int ping_snmp(host_t *host, ping_t *ping); -extern int ping_icmp(host_t *host, ping_t *ping); -extern int ping_udp(host_t *host, ping_t *ping); -extern int ping_tcp(host_t *host, ping_t *ping); -extern name_t *get_namebyhost(char *hostname, name_t *name); -extern void update_host_status(int status, host_t *host, ping_t *ping, int availability_method); -extern int init_sockaddr(struct sockaddr_in *name, const char *hostname, unsigned short int port); +extern int ping_icmp(const host_t *host, ping_t *ping); +extern int ping_udp(const host_t *host, ping_t *ping); +extern int ping_tcp(const host_t *host, ping_t *ping); +extern name_t *get_namebyhost(const char *hostname, name_t *name); +extern void update_host_status(int status, host_t *host, const ping_t *ping, int availability_method); +extern int init_sockaddr(struct sockaddr_in *name, const char *hostname, int port); extern int get_address_type(host_t *host); extern unsigned short int get_checksum(void* buf, int len); diff --git a/poller.c b/poller.c index 7e32281..23d994e 100644 --- a/poller.c +++ b/poller.c @@ -77,25 +77,26 @@ void child_cleanup(void *arg) { SPINE_LOG_DEVDBG(("Device[%i] HT[%i] DEBUG: The Device Thread has cleaned up.", poller_details.host_id, poller_details.host_thread)); - child_cleanup_thread(arg); + free(arg); + child_cleanup_thread(NULL); } void child_cleanup_thread(void *arg) { - UNUSED_PARAMETER(arg); - spine_sem_post(&available_threads); - + (void)arg; int a_threads_value; - spine_sem_getvalue(&available_threads, &a_threads_value); + a_threads_value = spine_permits_available(&available_threads) + 1; - SPINE_LOG_DEVDBG(("DEBUG: Available Threads is %i (%i outstanding)", a_threads_value, set.threads - a_threads_value)); + SPINE_LOG_DEVDBG(("DEBUG: Available Threads is %i (%i outstanding)", a_threads_value, set.poller.threads - a_threads_value)); + /* Releasing the permit publishes completion: no shared state is used after it. */ + spine_permits_release(&available_threads); } void child_cleanup_script(void *arg) { - UNUSED_PARAMETER(arg); - spine_sem_post(&available_scripts); + (void)arg; + spine_permits_release(&available_scripts); int a_scripts_value; - spine_sem_getvalue(&available_scripts, &a_scripts_value); + a_scripts_value = spine_permits_available(&available_scripts); SPINE_LOG_DEVDBG(("DEBUG: Available Scripts is %i (%i outstanding)", a_scripts_value, MAX_SIMULTANEOUS_SCRIPTS - a_scripts_value)); } @@ -112,49 +113,28 @@ void child_cleanup_script(void *arg) { void *child(void *arg) { pthread_cleanup_push(child_cleanup, arg); - int device_counter; - int host_id; - int host_thread; - int host_threads; - int host_data_ids; - int host_errors; - double host_time_double; - char host_time[SMALL_BUFSIZE]; - - host_errors = 0; - - poller_thread_t poller_details = *(poller_thread_t*) arg; - - device_counter = poller_details.device_counter; - host_id = poller_details.host_id; - host_thread = poller_details.host_thread; - host_threads = poller_details.host_threads; - host_data_ids = poller_details.host_data_ids; - host_time_double = poller_details.host_time_double; - - snprintf(host_time, SMALL_BUFSIZE, "%s", poller_details.host_time); - - thread_mutex_unlock(LOCK_HOST_TIME); - - /* Allows main thread to proceed with creation of other threads */ - spine_sem_post(poller_details.thread_init_sem); - - if (is_debug_device(host_id)) { - SPINE_LOG(("Device[%i] HT[%i] DEBUG: In Poller, About to Start Polling", host_id, host_thread)); - } else { - SPINE_LOG_DEBUG(("Device[%i] HT[%i] DEBUG: In Poller, About to Start Polling", host_id, host_thread)); - } - - poll_host(device_counter, host_id, host_thread, host_threads, host_data_ids, host_time, &host_errors, host_time_double); + int host_errors = 0; + poller_thread_t poller_details = *(poller_thread_t*)arg; + /* Allows main thread to proceed with creation of other threads. */ + spine_permits_release(poller_details.thread_init_sem); + SPINE_LOG_DEVICE(poller_details.host_id, POLLER_VERBOSITY_DEBUG, ("Device[%i] HT[%i] DEBUG: In Poller, About to Start Polling", poller_details.host_id, poller_details.host_thread)); + if (mysql_thread_init() != 0) die("ERROR: Unable to initialize polling thread database state"); + poll_host(&poller_details, &host_errors); pthread_cleanup_pop(1); /* end the thread */ pthread_exit(0); - - exit(0); } +/*! \fn void poll_host(const poller_thread_t *work, int *host_errors) + * \brief core Spine function that polls a host + * \param work borrowed polling instructions, valid until the synchronous call returns + * \param host_errors receives the count of invalid polling results + * + * This function is core to Spine. It takes a host_id and polls it, first + * checking reachability and any required data-query reindexing. + */ /*! \fn int poller_store_hex_result(char *result, size_t result_size, const char *hex, int *errors) * \brief convert a hexadecimal poll result and account for rejected values * @@ -174,9 +154,7 @@ int poller_store_hex_result(char *result, size_t result_size, const char *hex, i } if (!hex2dec(hex, &value)) { - /* An over-wide or malformed OctetString is unusable poll data. Count it - * like every other undefined result so host_errors reflects the failed - * collection; callers also retain the affected local_data_id. */ + /* An over-wide or malformed OctetString is unusable poll data. */ SET_UNDEFINED(result); if (errors != NULL) { (*errors)++; @@ -196,82 +174,1199 @@ int poller_store_hex_result(char *result, size_t result_size, const char *hex, i return TRUE; } -/*! \fn void poll_host(int device_counter, int host_id, int host_thread, int host_threads, int host_data_ids, const char *host_time, int *host_errors, double host_time_double) - * \brief core Spine function that polls a host - * \param host_id integer value for the host_id from the hosts table in Cacti - * - * This function is core to Spine. It takes a host_id and polls it, first - * checking reachability and any required data-query reindexing. - */ -void poll_host(int device_counter, int host_id, int host_thread, int host_threads, int host_data_ids, const char *host_time, int *host_errors, double host_time_double) { - char query1[BUFSIZE]; - char query2[BIG_BUFSIZE]; +enum poll_result_status normalize_poll_result(char *result, bool snmp) { + if (IS_UNDEFINED(result)) return POLL_RESULT_UNDEFINED; + if (is_numeric(result) || is_multipart_output(snmp ? result : trim(result))) { + return POLL_RESULT_VALID; + } + if (is_hexadecimal(result, TRUE)) { + /* Values wider than 64 bits become U and count as collection errors. */ + return poller_store_hex_result(result, RESULTS_BUFFER, result, NULL) ? POLL_RESULT_VALID : POLL_RESULT_INVALID; + } + if (snmp && (STRIMATCH(result, "U") || STRIMATCH(result, "Nan"))) { + return POLL_RESULT_UNDEFINED; + } + /* trim a non-numeric prefix or suffix, then validate below */ + char normalized[RESULTS_BUFFER]; + snprintf(normalized, sizeof(normalized), "%s", strip_alpha(result)); + strncopy(result, normalized, RESULTS_BUFFER); + return validate_result(result) ? POLL_RESULT_VALID : POLL_RESULT_INVALID; +} + +typedef struct poll_error_context { + char *buffer; + int *size; + int *count; + int *errors; + int host_id; + int thread_id; +} poll_error_context_t; + +static void record_result_error(const poll_error_context_t *context, const host_t *host, + const target_t *item, const char *result, bool snmp) { + buffer_output_errors(context->buffer, context->size, context->count, + context->host_id, context->thread_id, item->local_data_id, false); + (*context->errors)++; + if (set.logging.spine_log_level != 2) return; + if (snmp) { + SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", + context->host_id, context->thread_id, item->local_data_id, + host->snmp.profile.version, host->hostname, item->rrd_name, item->arg1, result)); + } else { + SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SCRIPT: %s, output: %s", + context->host_id, context->thread_id, item->local_data_id, item->arg1, result)); + } +} + +static void normalize_snmp_item(const host_t *host, const target_t *item, + char *result, const poll_error_context_t *errors) { + if (host->ignore_host) { + SPINE_LOG(("Device[%i] HT[%i] DS[%i] WARNING: SNMP timeout detected [%i ms], ignoring host '%s'", + errors->host_id, errors->thread_id, item->local_data_id, host->snmp.profile.timeout, host->hostname)); + SET_UNDEFINED(result); + return; + } + enum poll_result_status status = normalize_poll_result(result, true); + if (status == POLL_RESULT_VALID) return; + record_result_error(errors, host, item, result, true); + if (status == POLL_RESULT_INVALID) SET_UNDEFINED(result); +} + + +static void store_snmp_results(const host_t *host, target_t *poller_items, snmp_oids_t *snmp_oids, + int num_oids, const poll_error_context_t *errors, double thread_start, bool spike_kill) { + for (int j = 0; j < num_oids; j++) { + const target_t *item = &poller_items[snmp_oids[j].array_position]; + normalize_snmp_item(host, item, snmp_oids[j].result, errors); + + if (strlen(item->output_regex)) { + char temp_result[RESULTS_BUFFER]; + snprintf(temp_result, sizeof(temp_result), "%s", regex_replace(item->output_regex, snmp_oids[j].result)); + snprintf(snmp_oids[j].result, RESULTS_BUFFER, "%s", temp_result); + } + + snprintf(poller_items[snmp_oids[j].array_position].result, RESULTS_BUFFER, "%s", snmp_oids[j].result); + + double thread_end = get_time_as_double(); + + SPINE_LOG_DEVICE(errors->host_id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DS[%i] TT[%.2f] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", errors->host_id, errors->thread_id, poller_items[snmp_oids[j].array_position].local_data_id, (float) ((thread_end - thread_start) * 1000), host->snmp.profile.version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, poller_items[snmp_oids[j].array_position].arg1, poller_items[snmp_oids[j].array_position].result)); + + if ((!IS_UNDEFINED(poller_items[snmp_oids[j].array_position].result)) && (spike_kill && (!strstr(poller_items[snmp_oids[j].array_position].result,":")))) { + SET_UNDEFINED(poller_items[snmp_oids[j].array_position].result); + } + } +} + +typedef struct { + int host_id; + const char *limits; + bool due_only; + bool group_ports; +} poller_query_filter_t; + +static void poller_item_query(char *buffer, size_t capacity, const char *columns, const poller_query_filter_t *filter) { + char *cursor = buffer; + cursor += spine_snprintf(cursor, capacity, "SELECT SQL_NO_CACHE %s FROM poller_item WHERE host_id = %i", columns, filter->host_id); + if (set.poller.poller_id != 0) cursor += spine_snprintf(cursor, capacity - (size_t)(cursor - buffer), " AND poller_id = %i", set.poller.poller_id); + if (filter->due_only) cursor += spine_snprintf(cursor, capacity - (size_t)(cursor - buffer), " AND rrd_next_step <= 0"); + if (filter->group_ports) { + cursor += spine_snprintf(cursor, capacity - (size_t)(cursor - buffer), " GROUP BY snmp_port"); + } else if (set.snmp.total_snmp_ports != 1) { + cursor += spine_snprintf(cursor, capacity - (size_t)(cursor - buffer), " ORDER BY snmp_port"); + } + spine_snprintf(cursor, capacity - (size_t)(cursor - buffer), " %s", filter->limits); +} + +void poller_prepare_queries(poller_queries_t *queries, int host_id, int host_thread, int host_data_ids) { + char limits[SMALL_BUFSIZE] = ""; + if (host_data_ids > 0) { + if (host_thread < 1) die("ERROR: Invalid device thread for poller item selection"); + long long offset = (long long)host_data_ids * ((long long)host_thread - 1); + spine_snprintf(limits, sizeof(limits), "LIMIT %lld, %i", offset, host_data_ids); + } + char columns[BUFSIZE]; + /* optional output_regex column (added in Cacti 1.3.1) */ + spine_snprintf(columns, sizeof(columns), "%s%s", + "action, hostname, snmp_community, snmp_version, snmp_username, snmp_password, " + "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, rrd_num, snmp_port, snmp_timeout, " + "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id", + set.hosts.has_output_regex ? ", output_regex" : ""); + poller_query_filter_t filter = {host_id, limits, FALSE, FALSE}; + poller_item_query(queries->items, sizeof(queries->items), columns, &filter); + filter.due_only = set.poller.active_profiles != 1; + poller_item_query(queries->due_items, sizeof(queries->due_items), columns, &filter); + filter.group_ports = TRUE; + filter.due_only = FALSE; + poller_item_query(queries->agents, sizeof(queries->agents), "snmp_port, count(snmp_port)", &filter); + filter.due_only = set.poller.active_profiles != 1; + poller_item_query(queries->due_agents, sizeof(queries->due_agents), "snmp_port, count(snmp_port)", &filter); + spine_snprintf(queries->host, sizeof(queries->host), + "SELECT SQL_NO_CACHE id, hostname, snmp_community, snmp_version, snmp_username, snmp_password, snmp_auth_protocol, " + "snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id, snmp_port, snmp_timeout, max_oids, " + "availability_method, ping_method, ping_port, ping_timeout, ping_retries, status, status_event_count, " + "UNIX_TIMESTAMP(status_fail_date), UNIX_TIMESTAMP(status_rec_date), status_last_error, min_time, max_time, " + "cur_time, avg_time, total_polls, failed_polls, availability, snmp_sysUpTimeInstance, snmp_sysDescr, snmp_sysObjectID, " + "snmp_sysContact, snmp_sysName, snmp_sysLocation FROM host WHERE id = %i AND deleted = ''", host_id); + spine_snprintf(queries->reindex, sizeof(queries->reindex), + "SELECT SQL_NO_CACHE data_query_id, action, op, assert_value, arg1 FROM poller_reindex WHERE host_id = %i", host_id); + spine_snprintf(queries->schedule, sizeof(queries->schedule), + "UPDATE poller_item SET rrd_next_step = IF(rrd_step = %i, 0, IF(rrd_next_step - %i < 0, rrd_step - %i, rrd_next_step - %i)) WHERE host_id = %i", + set.poller.poller_interval, set.poller.poller_interval, set.poller.poller_interval, set.poller.poller_interval, host_id); + if (set.poller.poller_id != 0) { + size_t length = strlen(queries->schedule); + spine_snprintf(queries->schedule + length, sizeof(queries->schedule) - length, " AND poller_id = %i", set.poller.poller_id); + } + strncopy(queries->output, "INSERT INTO poller_output (local_data_id, rrd_name, time, output) VALUES", sizeof(queries->output)); + strncopy(queries->boost_output, "INSERT INTO poller_output_boost (local_data_id, rrd_name, time, output) VALUES", sizeof(queries->boost_output)); + /* The cached upsert capability describes the local connection, while output can + * go to the remote one. VALUES() is accepted by both MySQL and MariaDB. */ + strncopy(queries->suffix, " ON DUPLICATE KEY UPDATE output=VALUES(output)", sizeof(queries->suffix)); +} + +static void host_metadata_defaults(host_t *host) { + /* initialize variables first */ + host->id = 0; // 0 + host->hostname[0] = '\0'; // 1 + host->snmp.session = NULL; // - + host->snmp.profile.community[0] = '\0'; // 2 + host->snmp.profile.version = 1; // 3 + host->snmp.profile.username[0] = '\0'; // 4 + host->snmp.profile.password[0] = '\0'; // 5 + host->snmp.profile.auth_protocol[0] = '\0'; // 6 + host->snmp.profile.priv_passphrase[0] = '\0'; // 7 + host->snmp.profile.priv_protocol[0] = '\0'; // 8 + host->snmp.profile.context[0] = '\0'; // 9 + host->snmp.profile.engine_id[0] = '\0'; // 10 + host->snmp.profile.port = 161; // 11 + host->snmp.profile.timeout = 500; // 12 + host->snmp.retries = set.snmp.snmp_retries; // - + host->snmp.max_oids = 10; // 13 + host->availability.method = 0; // 14 + host->availability.ping_method = 0; // 15 + host->availability.port = 23; // 16 + host->availability.timeout = 500; // 17 + host->availability.retries = 2; // 18 + host->state.status = HOST_UP; // 19 + host->state.status_event_count = 0; // 20 + host->state.status_fail_date[0] = '\0'; // 21 + host->state.status_rec_date[0] = '\0'; // 22 + host->state.status_last_error[0] = '\0'; // 23 + host->statistics.min_time = 0; // 24 + host->statistics.max_time = 0; // 25 + host->statistics.cur_time = 0; // 26 + host->statistics.avg_time = 0; // 27 + host->statistics.total_polls = 0; // 28 + host->statistics.failed_polls = 0; // 29 + host->statistics.availability = 100; // 30 + host->system.snmp_sysUpTimeInstance = 0; // 31 + host->system.snmp_sysDescr[0] = '\0'; // 32 + host->system.snmp_sysObjectID[0] = '\0'; // 33 + host->system.snmp_sysContact[0] = '\0'; // 34 + host->system.snmp_sysName[0] = '\0'; // 35 + host->system.snmp_sysLocation[0] = '\0'; // 36 +} + +static void host_metadata_connection(host_t *host, MYSQL_ROW row) { + /* populate host structure */ + host->ignore_host = FALSE; + if (row[0] != NULL) host->id = atoi(row[0]); + + if (row[1] != NULL) { + name_t *name = get_namebyhost(row[1], NULL); + STRNCOPY(host->hostname, name->hostname); + host->availability.port = name->port; + SPINE_FREE(name); + } + + if (row[2] != NULL) STRNCOPY(host->snmp.profile.community, row[2]); + + if (row[3] != NULL) host->snmp.profile.version = atoi(row[3]); + + if (row[4] != NULL) STRNCOPY(host->snmp.profile.username, row[4]); + if (row[5] != NULL) STRNCOPY(host->snmp.profile.password, row[5]); + if (row[6] != NULL) STRNCOPY(host->snmp.profile.auth_protocol, row[6]); + if (row[7] != NULL) STRNCOPY(host->snmp.profile.priv_passphrase, row[7]); + if (row[8] != NULL) STRNCOPY(host->snmp.profile.priv_protocol, row[8]); + if (row[9] != NULL) STRNCOPY(host->snmp.profile.context, row[9]); + if (row[10] != NULL) STRNCOPY(host->snmp.profile.engine_id, row[10]); + + if (row[11] != NULL) host->snmp.profile.port = atoi(row[11]); + if (row[12] != NULL) host->snmp.profile.timeout = atoi(row[12]); + if (row[13] != NULL) host->snmp.max_oids = atoi(row[13]); +} + +static void host_metadata_status(host_t *host, MYSQL_ROW row) { + if (row[14] != NULL) host->availability.method = atoi(row[14]); + if (row[15] != NULL) host->availability.ping_method = atoi(row[15]); + if (row[16] != NULL) host->availability.port = atoi(row[16]); + if (row[17] != NULL) host->availability.timeout = atoi(row[17]); + if (row[18] != NULL) host->availability.retries = atoi(row[18]); + + if (row[19] != NULL) host->state.status = atoi(row[19]); + if (row[20] != NULL) host->state.status_event_count = atoi(row[20]); + + if (row[21] != NULL) STRNCOPY(host->state.status_fail_date, row[21]); + if (row[22] != NULL) STRNCOPY(host->state.status_rec_date, row[22]); + + if (row[23] != NULL) STRNCOPY(host->state.status_last_error, row[23]); +} + +static void host_metadata_statistics(host_t *host, MYSQL_ROW row) { + if (row[24] != NULL) host->statistics.min_time = atof(row[24]); + if (row[25] != NULL) host->statistics.max_time = atof(row[25]); + if (row[26] != NULL) host->statistics.cur_time = atof(row[26]); + if (row[27] != NULL) host->statistics.avg_time = atof(row[27]); + if (row[28] != NULL) host->statistics.total_polls = atoi(row[28]); + if (row[29] != NULL) host->statistics.failed_polls = atoi(row[29]); + if (row[30] != NULL) host->statistics.availability = atof(row[30]); +} + +static void host_metadata_system(host_t *host, MYSQL_ROW row, MYSQL *mysql) { + if (row[31] != NULL) host->system.snmp_sysUpTimeInstance=atoll(row[31]); + if (row[32] != NULL) db_escape(mysql, host->system.snmp_sysDescr, sizeof(host->system.snmp_sysDescr), row[32]); + if (row[33] != NULL) db_escape(mysql, host->system.snmp_sysObjectID, sizeof(host->system.snmp_sysObjectID), row[33]); + if (row[34] != NULL) db_escape(mysql, host->system.snmp_sysContact, sizeof(host->system.snmp_sysContact), row[34]); + if (row[35] != NULL) db_escape(mysql, host->system.snmp_sysName, sizeof(host->system.snmp_sysName), row[35]); + if (row[36] != NULL) db_escape(mysql, host->system.snmp_sysLocation, sizeof(host->system.snmp_sysLocation), row[36]); +} + +static void persist_host_status(MYSQL *mysql, const host_t *host, bool include_system_information) { + char update_sql[BIG_BUFSIZE]; + char escaped_last_error[BUFSIZE]; + db_escape(mysql, escaped_last_error, sizeof(escaped_last_error), host->state.status_last_error); + if (include_system_information) { + snprintf(update_sql, BIG_BUFSIZE, "UPDATE host " + "SET status='%i', status_event_count='%i', status_fail_date=FROM_UNIXTIME(%s)," + " status_rec_date=FROM_UNIXTIME(%s), status_last_error='%s', min_time='%f'," + " max_time='%f', cur_time='%f', avg_time='%f', total_polls='%i'," + " failed_polls='%i', availability='%.4f', snmp_sysDescr='%s', " + " snmp_sysObjectID='%s', snmp_sysUpTimeInstance='%llu', " + " snmp_sysContact='%s', snmp_sysName='%s', snmp_sysLocation='%s' " + "WHERE id='%i'", + host->state.status, + host->state.status_event_count, + host->state.status_fail_date, + host->state.status_rec_date, + escaped_last_error, + host->statistics.min_time, + host->statistics.max_time, + host->statistics.cur_time, + host->statistics.avg_time, + host->statistics.total_polls, + host->statistics.failed_polls, + host->statistics.availability, + host->system.snmp_sysDescr, + host->system.snmp_sysObjectID, + host->system.snmp_sysUpTimeInstance, + host->system.snmp_sysContact, + host->system.snmp_sysName, + host->system.snmp_sysLocation, + host->id); + } else { + snprintf(update_sql, BIG_BUFSIZE, "UPDATE host " + "SET status='%i', status_event_count='%i', status_fail_date=FROM_UNIXTIME(%s)," + " status_rec_date=FROM_UNIXTIME(%s), status_last_error='%s', min_time='%f'," + " max_time='%f', cur_time='%f', avg_time='%f', total_polls='%i'," + " failed_polls='%i', availability='%.4f' " + "WHERE id='%i'", + host->state.status, + host->state.status_event_count, + host->state.status_fail_date, + host->state.status_rec_date, + escaped_last_error, + host->statistics.min_time, + host->statistics.max_time, + host->statistics.cur_time, + host->statistics.avg_time, + host->statistics.total_polls, + host->statistics.failed_polls, + host->statistics.availability, + host->id); + } + db_insert(mysql, LOCAL, update_sql); +} + +static void load_host_metadata(MYSQL *mysql, MYSQL_ROW row, host_t *host, const poller_thread_t *work) { + host_metadata_defaults(host); + host_metadata_connection(host, row); + host_metadata_status(host, row); + host_metadata_statistics(host, row); + host_metadata_system(host, row, mysql); + + /* correct max_oid bounds issues */ + if ((host->snmp.max_oids == 0) || (host->snmp.max_oids > 100)) { + SPINE_LOG(("Device[%i] HT[%i] WARNING: Max OIDS is out of range with value of '%i'. Resetting to default of 5", work->host_id, work->host_thread, host->snmp.max_oids)); + host->snmp.max_oids = 5; + } +} + +static void initialize_host_snmp(host_t *host) { + if (((host->snmp.profile.version >= 1) && (host->snmp.profile.version <= 2) && + (strlen(host->snmp.profile.community) > 0)) || + (host->snmp.profile.version == 3)) { + host->snmp.session = snmp_host_init(&(snmp_connection_t){ + .host_id = host->id, + .hostname = host->hostname, + .snmp_version = host->snmp.profile.version, + .snmp_community = host->snmp.profile.community, + .snmp_username = host->snmp.profile.username, + .snmp_password = host->snmp.profile.password, + .snmp_auth_protocol = host->snmp.profile.auth_protocol, + .snmp_priv_passphrase = host->snmp.profile.priv_passphrase, + .snmp_priv_protocol = host->snmp.profile.priv_protocol, + .snmp_context = host->snmp.profile.context, + .snmp_engine_id = host->snmp.profile.engine_id, + .snmp_port = host->snmp.profile.port, + .snmp_timeout = host->snmp.profile.timeout, + }); + } else { + host->snmp.session = NULL; + } +} + +static bool refresh_host_availability(MYSQL *mysql, host_t *host, ping_t *ping, int host_thread) { + if (host->availability.method == AVAIL_SNMP && strlen(host->snmp.profile.community) == 0 && host->snmp.profile.version < 3) { + host->ignore_host = FALSE; + update_host_status(HOST_UP, host, ping, host->availability.method); + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] No host availability check possible for '%s'", host->id, host_thread, host->hostname)); + return FALSE; + } + if (host->availability.method == AVAIL_STREAM) { + update_host_status(HOST_UP, host, ping, host->availability.method); + return FALSE; + } + bool alive = ping_host(host, ping) == HOST_UP; + host->ignore_host = !alive; + if (host_thread != 1) return FALSE; + update_host_status(alive ? HOST_UP : HOST_DOWN, host, ping, host->availability.method); + if (!alive || host->availability.method == AVAIL_PING || host->availability.method == AVAIL_NONE || host->snmp.session == NULL || !set.snmp.mibs) return FALSE; + get_system_information(host, mysql, 1); + return TRUE; +} + +typedef struct { + MYSQL *local; + MYSQL *remote; + const host_t *host; + const poller_thread_t *work; + int *errors; + int *spike_kill; +} reindex_evaluation_t; + +static bool reindex_assertion_failed(const reindex_t *reindex, const char *value) { + if (value == NULL || IS_UNDEFINED(value) || STRIMATCH(value, "No Such Instance")) return FALSE; + if (STRMATCH(reindex->op, "=")) return strcmp(reindex->assert_value, value) != 0; + if (STRMATCH(reindex->op, ">")) return atoll(reindex->assert_value) < atoll(value); + if (STRMATCH(reindex->op, "<")) return !STRMATCH(reindex->assert_value, "0") && atoll(reindex->assert_value) > atoll(value); + return FALSE; +} + +static void log_reindex_assertion(const reindex_evaluation_t *evaluation, const reindex_t *reindex, const char *value, bool failed) { + bool highlighted = is_debug_device(evaluation->host->id) || set.logging.spine_log_level == 2; + if (!failed && !highlighted) return; + if (failed && !highlighted && set.logging.spine_log_level == 1) (*evaluation->errors)++; + SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE ASSERT FAILED: '%s%s%s'", evaluation->host->id, evaluation->work->host_thread, reindex->data_query_id, reindex->assert_value, failed ? reindex->op : "=", value != NULL ? value : "(null)")); +} + +static void queue_reindex(const reindex_evaluation_t *evaluation, const reindex_t *reindex) { + if (evaluation->work->host_thread != 1) return; + char query[LRG_BUFSIZE]; + snprintf(query, sizeof(query), "REPLACE INTO poller_command (poller_id, time, action, command) VALUES (%i, NOW(), %i, '%i:%i')", set.poller.poller_id, POLLER_COMMAND_REINDEX, evaluation->host->id, reindex->data_query_id); + if (set.poller.poller_id > 1 && set.poller.mode == REMOTE_ONLINE) db_insert(evaluation->remote, REMOTE, query); + else db_insert(evaluation->local, LOCAL, query); +} + +static void update_reindex_value(const reindex_evaluation_t *evaluation, const reindex_t *reindex, const char *value) { + /* An unavailable result cannot replace the stored assertion with uninitialized bytes. */ + if (evaluation->work->host_thread != 1 || value == NULL) return; + char escaped_value[BUFSIZE]; + char escaped_argument[BUFSIZE]; + char query[LRG_BUFSIZE]; + db_escape(evaluation->local, escaped_value, sizeof(escaped_value), value); + db_escape(evaluation->local, escaped_argument, sizeof(escaped_argument), reindex->arg1); + snprintf(query, sizeof(query), "UPDATE poller_reindex SET assert_value='%s' WHERE host_id='%i' AND data_query_id='%i' AND arg1='%s'", escaped_value, evaluation->work->host_id, reindex->data_query_id, escaped_argument); + db_insert(evaluation->local, LOCAL, query); +} + +static void discard_reindex_spike(const reindex_evaluation_t *evaluation) { + *evaluation->spike_kill = TRUE; + if (is_debug_device(evaluation->host->id) || set.logging.spine_log_level == 2) { + SPINE_LOG(("Device[%i] HT[%i] NOTICE: Spike Kill in Effect for '%s'", evaluation->work->host_id, evaluation->work->host_thread, evaluation->host->hostname)); + } else { + if (set.logging.spine_log_level == 1) (*evaluation->errors)++; + SPINE_LOG_MEDIUM(("Device[%i] HT[%i] NOTICE: Spike Kill in Effect for '%s'", evaluation->work->host_id, evaluation->work->host_thread, evaluation->host->hostname)); + } +} + +static bool evaluate_reindex_assertion(const reindex_evaluation_t *evaluation, const reindex_t *reindex, const char *value) { + bool failed = reindex_assertion_failed(reindex, value); + bool unavailable = value == NULL || IS_UNDEFINED(value) || STRIMATCH(value, "No Such Instance"); + if (failed || unavailable) log_reindex_assertion(evaluation, reindex, value, failed); + if (failed) queue_reindex(evaluation, reindex); + if (failed || STRMATCH(reindex->op, ">") || STRMATCH(reindex->op, "<")) update_reindex_value(evaluation, reindex, value); + /* A failed uptime assertion means the counters reset, so the sample is a spike. */ + if (failed && (STRMATCH(reindex->op, "<") || + STRMATCH(reindex->arg1, ".1.3.6.1.2.1.1.3.0") || + STRMATCH(reindex->arg1, ".1.3.6.1.6.3.10.2.1.3.0"))) { + discard_reindex_spike(evaluation); + } + return failed; +} + +static char *poll_reindex_snmp(host_t *host, reindex_t *reindex, int host_thread, char *sysUptime, bool *unavailable) { + if (host->snmp.session == NULL) { + *unavailable = TRUE; + SPINE_LOG(("WARNING: Device[%i] HT[%i] DQ[%i] Reindex Check FAILED: No SNMP Session. If not an SNMP host, don't use Uptime Goes Backwards!", host->id, host_thread, reindex->data_query_id)); + return NULL; + } + char *poll_result = NULL; + if ((strstr(reindex->arg1, ".1.3.6.1.2.1.1.3.0") || + strstr(reindex->arg1, ".1.3.6.1.6.3.10.2.1.3.0")) && strlen(sysUptime) > 0) { + + if (!(poll_result = (char *) malloc(BUFSIZE))) { + die("ERROR: Fatal malloc error: poller.c poll_result"); + } + + poll_result[0] = '\0'; + + snprintf(poll_result, BUFSIZE, "%s", sysUptime); + } else if (strstr(reindex->arg1, ".1.3.6.1.2.1.1.3.0") || + strstr(reindex->arg1, ".1.3.6.1.6.3.10.2.1.3.0")) { + // Ensure uptime is empty to start with + sysUptime[0] = '\0'; + + /* Pin the uptime-goes-backward calculation to a single OID for this poll. + The legacy (centisecond) and modern (second) OIDs are different counters, + and letting either supply the value on different rows or cycles caused + false "uptime went backward" detections and constant reindexing. Prefer + the modern engine OID and fall back to the legacy OID only when the + engine OID is not present with numeric data. */ + poll_result = snmp_get_base(host, ".1.3.6.1.6.3.10.2.1.3.0", false); + + bool uptime_use_engine_oid = (poll_result != NULL && is_numeric(poll_result)); + + if (uptime_use_engine_oid) { + snprintf(sysUptime, BUFSIZE, "%lld", atoll(poll_result) * 100); + } + + SPINE_FREE(poll_result); + + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DQ[%i] Engine Uptime OID Present: %d", host->id, host_thread, reindex->data_query_id, uptime_use_engine_oid)); + + if (!uptime_use_engine_oid) { + // Engine OID unavailable, fall back to the legacy sysUpTime OID + poll_result = snmp_get(host, ".1.3.6.1.2.1.1.3.0"); + + if (poll_result && is_numeric(poll_result)) { + snprintf(sysUptime, BUFSIZE, "%s", poll_result); + } + + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DQ[%i] Legacy Uptime Result: %s, Is Numeric: %d", host->id, host_thread, reindex->data_query_id, poll_result != NULL ? poll_result : "U", is_numeric(poll_result) )); + + SPINE_FREE(poll_result); + } + + // Use the primed uptime to repopulate the poll_result + // This ensures whichever response was valid gets used + poll_result = strdup(sysUptime); + if (poll_result == NULL) { + die("ERROR: Fatal malloc error: poller.c uptime result"); + } + + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DQ[%i] Extended Uptime Result: %s, Is Numeric: %d", host->id, host_thread, reindex->data_query_id, poll_result, is_numeric(poll_result) )); + } else { + poll_result = snmp_get(host, reindex->arg1); + } + + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DQ[%i] RECACHE OID: %s, (assert: %s %s output: %s)", host->id, host_thread, reindex->data_query_id, reindex->arg1, reindex->assert_value, reindex->op, poll_result)); + return poll_result; +} + +static char *poll_reindex_action(host_t *host, reindex_t *reindex, int host_thread, char *sysUptime, bool *unavailable) { + char *poll_result = NULL; + int php_process; + switch(reindex->action) { + case POLLER_ACTION_SNMP: + poll_result = poll_reindex_snmp(host, reindex, host_thread, sysUptime, unavailable); + break; + case POLLER_ACTION_SCRIPT: /* script (popen) */ + /* Reject empty script commands that could cause unexpected behavior */ + if (strlen(reindex->arg1) == 0) { + SPINE_LOG(("WARNING: Device[%i] HT[%i] DQ[%i] empty script command, skipping", + host->id, host_thread, reindex->data_query_id)); + break; + } + + poll_result = trim(exec_poll(host, reindex->arg1, reindex->data_query_id, "DQ")); + + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DQ[%i] RECACHE CMD: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); + + break; + case POLLER_ACTION_PHP_SCRIPT_SERVER: /* script (php script server) */ + php_process = php_get_process(); + + poll_result = trim(php_cmd(reindex->arg1, php_process)); + + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DQ[%i] RECACHE SERVER: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); + + break; + case POLLER_ACTION_SNMP_COUNT: { /* snmp; count items */ + if (!(poll_result = (char *) malloc(BUFSIZE))) { + die("ERROR: Fatal malloc error: poller.c poll_result"); + } + poll_result[0] = '\0'; + + int snmp_items = snmp_count(host, reindex->arg1); + if (snmp_items < 0) { + SET_UNDEFINED(poll_result); + } else { + snprintf(poll_result, BUFSIZE, "%d", snmp_items); + } + + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DQ[%i] RECACHE OID COUNT: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); + + break; + } + case POLLER_ACTION_SCRIPT_COUNT: { /* script (popen); count line feeds */ + if (!(poll_result = (char *) malloc(BUFSIZE))) { + die("ERROR: Fatal malloc error: poller.c poll_result"); + } + poll_result[0] = '\0'; + + char *count_result = exec_poll(host, reindex->arg1, reindex->data_query_id, "DQ"); + snprintf(poll_result, BUFSIZE, "%d", char_count(count_result, '\n')); + SPINE_FREE(count_result); + + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DQ[%i] RECACHE CMD COUNT: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); + + break; + } + case POLLER_ACTION_PHP_SCRIPT_SERVER_COUNT: { /* script (php script server); count number of lines */ + if (!(poll_result = (char *) malloc(BUFSIZE))) { + die("ERROR: Fatal malloc error: poller.c poll_result"); + } + poll_result[0] = '\0'; + + php_process = php_get_process(); + + char *count_result = php_cmd(reindex->arg1, php_process); + spine_snprintf(poll_result, BUFSIZE, "%d", char_count(count_result, '\n')); + SPINE_FREE(count_result); + + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DQ[%i] RECACHE SERVER COUNT: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); + + break; + } + default: + SPINE_LOG(("Device[%i] HT[%i] ERROR: Unknown Assert Action!", host->id, host_thread)); + } + return poll_result; +} + +static void load_reindex_item(reindex_t *reindex, MYSQL_ROW row) { + /* initialize the reindex struction */ + reindex->data_query_id = 0; + reindex->action = -1; + reindex->op[0] = '\0'; + reindex->assert_value[0] = '\0'; + reindex->arg1[0] = '\0'; + + if (row[0] != NULL) reindex->data_query_id = atoi(row[0]); + if (row[1] != NULL) reindex->action = atoi(row[1]); + + if (row[2] != NULL) snprintf(reindex->op, sizeof(reindex->op), "%s", row[2]); + + if (row[3] != NULL) snprintf(reindex->assert_value, sizeof(reindex->assert_value), "%s", row[3]); + + if (row[4] != NULL) snprintf(reindex->arg1, sizeof(reindex->arg1), "%s", row[4]); +} + +static void poll_host_reindex(host_t *host, reindex_t *reindex, const char *query, + const reindex_evaluation_t *evaluation) { + if (host->ignore_host || evaluation->work->host_id == 0) return; + int thread = evaluation->work->host_thread; + MYSQL_RES *result = db_query(evaluation->local, LOCAL, query); + if (result == NULL) { + SPINE_LOG(("Device[%i] HT[%i] ERROR: RECACHE Query Returned Null Result!", host->id, thread)); + } else { + int count = spine_count_to_int(mysql_num_rows(result)); + if (count == 0) { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_HIGH, ("Device[%i] HT[%i] Device has no information for recache.", host->id, thread)); + } else { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_DEBUG, ("Device[%i] HT[%i] DEBUG: RECACHE: Processing %i items in the auto reindex cache for '%s'", host->id, thread, count, host->hostname)); + } + char sysUptime[BUFSIZE] = ""; + int last_data_query_id = 0; + bool previous_assert_failure = FALSE; + MYSQL_ROW row; + while ((row = mysql_fetch_row(result))) { + load_reindex_item(reindex, row); + if (last_data_query_id != reindex->data_query_id) { + last_data_query_id = reindex->data_query_id; + previous_assert_failure = FALSE; + } + if (previous_assert_failure) continue; + bool unavailable = FALSE; + char *value = poll_reindex_action(host, reindex, thread, sysUptime, &unavailable); + if (unavailable) continue; + if (evaluate_reindex_assertion(evaluation, reindex, value)) previous_assert_failure = TRUE; + SPINE_FREE(value); + } + db_free_result(result); + } + /* Item polling opens a new session after reindex work has finished. */ + if (host->snmp.session != NULL) { + snmp_host_cleanup(host->snmp.session); + host->snmp.session = NULL; + } +} + +static void load_poll_item(target_t *item, MYSQL_ROW row) { + /* initialize monitored object */ + item->target_id = 0; + item->action = -1; + item->hostname[0] = '\0'; + item->snmp.community[0] = '\0'; + item->snmp.version = 1; + item->snmp.username[0] = '\0'; + item->snmp.password[0] = '\0'; + item->snmp.auth_protocol[0] = '\0'; + item->snmp.priv_passphrase[0] = '\0'; + item->snmp.priv_protocol[0] = '\0'; + item->snmp.context[0] = '\0'; + item->snmp.engine_id[0] = '\0'; + item->snmp.port = 161; + item->snmp.timeout = 500; + item->rrd_name[0] = '\0'; + item->rrd_path[0] = '\0'; + item->arg1[0] = '\0'; + item->arg2[0] = '\0'; + item->arg3[0] = '\0'; + item->local_data_id = 0; + item->rrd_num = 0; + item->output_regex[0] = '\0'; + + if (row[0] != NULL) item->action = atoi(row[0]); + + if (row[1] != NULL) snprintf(item->hostname, sizeof(item->hostname), "%s", row[1]); + if (row[2] != NULL) snprintf(item->snmp.community, sizeof(item->snmp.community), "%s", row[2]); + + if (row[3] != NULL) item->snmp.version = atoi(row[3]); + + if (row[4] != NULL) snprintf(item->snmp.username, sizeof(item->snmp.username), "%s", row[4]); + if (row[5] != NULL) snprintf(item->snmp.password, sizeof(item->snmp.password), "%s", row[5]); + + if (row[6] != NULL) snprintf(item->rrd_name, sizeof(item->rrd_name), "%s", row[6]); + if (row[7] != NULL) snprintf(item->rrd_path, sizeof(item->rrd_path), "%s", row[7]); + if (row[8] != NULL) snprintf(item->arg1, sizeof(item->arg1), "%s", row[8]); + if (row[9] != NULL) snprintf(item->arg2, sizeof(item->arg2), "%s", row[9]); + if (row[10] != NULL) snprintf(item->arg3, sizeof(item->arg3), "%s", row[10]); + + if (row[11] != NULL) item->local_data_id = atoi(row[11]); + + if (row[12] != NULL) item->rrd_num = atoi(row[12]); + if (row[13] != NULL) item->snmp.port = atoi(row[13]); + if (row[14] != NULL) item->snmp.timeout = atoi(row[14]); + + if (row[15] != NULL) snprintf(item->snmp.auth_protocol, + sizeof(item->snmp.auth_protocol), "%s", row[15]); + if (row[16] != NULL) snprintf(item->snmp.priv_passphrase, + sizeof(item->snmp.priv_passphrase), "%s", row[16]); + if (row[17] != NULL) snprintf(item->snmp.priv_protocol, + sizeof(item->snmp.priv_protocol), "%s", row[17]); + if (row[18] != NULL) snprintf(item->snmp.context, + sizeof(item->snmp.context), "%s", row[18]); + if (row[19] != NULL) snprintf(item->snmp.engine_id, + sizeof(item->snmp.engine_id), "%s", row[19]); + + if (set.hosts.has_output_regex && row[20] != NULL) + snprintf(item->output_regex, sizeof(item->output_regex), "%s", row[20]); + + SET_UNDEFINED(item->result); +} + +static void poll_script_item(host_t *host, target_t *item, + const poll_error_context_t *errors, double thread_start, bool spike_kill, + bool script_server) { + int php_process = -1; + char *poll_result; + /* Reject empty script commands that could cause unexpected behavior */ + if (strlen(item->arg1) == 0) { + SPINE_LOG(("WARNING: Device[%i] HT[%i] DS[%i] empty script%s command, skipping", + errors->host_id, errors->thread_id, item->local_data_id, script_server ? " server" : "")); + SET_UNDEFINED(item->result); + return; + } + if (script_server) { + php_process = php_get_process(); + poll_result = php_cmd(item->arg1, php_process); + } else { + poll_result = exec_poll(host, item->arg1, item->local_data_id, "DS"); + } + strncopy(item->result, poll_result, sizeof(item->result)); + enum poll_result_status status = normalize_poll_result(item->result, false); + if (status != POLL_RESULT_VALID) { + record_result_error(errors, host, item, item->result, false); + if (status == POLL_RESULT_INVALID) SET_UNDEFINED(item->result); + } + SPINE_FREE(poll_result); + double thread_end = get_time_as_double(); + if (script_server) { + SPINE_LOG_DEVICE(errors->host_id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DS[%i] TT[%.2f] SS[%i] SERVER: %s, output: %s", errors->host_id, errors->thread_id, item->local_data_id, (float) ((thread_end - thread_start) * 1000), php_process, item->arg1, item->result)); + } else { + SPINE_LOG_DEVICE(errors->host_id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] DS[%i] TT[%.2f] SCRIPT: %s, output: %s", errors->host_id, errors->thread_id, item->local_data_id, (float) ((thread_end - thread_start) * 1000), item->arg1, item->result)); + } + /* insert a NaN in place of the actual value if the snmp agent restarts */ + if (!IS_UNDEFINED(item->result) && spike_kill && !strstr(item->result, ":")) SET_UNDEFINED(item->result); +} + +typedef struct { + char *output; + char *boost; + bool failed; +} poll_output_buffers_t; + +static char *allocate_output_query(const char *prefix, const char *failure_message) { + char *query = malloc(MAX_MYSQL_BUF_SIZE + RESULTS_BUFFER); + if (query == NULL) die("%s", failure_message); + memset(query, 0, MAX_MYSQL_BUF_SIZE + RESULTS_BUFFER); + strncat(query, prefix, spine_count_to_int(strlen(prefix))); + return query; +} + +/* Attempt each destination independently. Reset only early-flush buffers; + * a failed write must not suppress the other destination or later batches. */ +static bool flush_output_query(MYSQL *mysql, int mode, char *query, + const char *suffix, const char *prefix) { + if (query == NULL) return TRUE; + strncat(query, suffix, spine_count_to_int(strlen(suffix))); + bool success = db_insert(mysql, mode, query); + if (prefix != NULL) { + memset(query, 0, MAX_MYSQL_BUF_SIZE + RESULTS_BUFFER); + strncat(query, prefix, spine_count_to_int(strlen(prefix))); + } + return success; +} + +static poll_output_buffers_t write_poll_results(MYSQL *mysql, MYSQL *mysqlr, + const poller_queries_t *queries, const target_t *poller_items, + int rows_processed, const char *host_time) { + char *query3 = NULL; + char *query12 = NULL; + bool failed = FALSE; + /* Escaping can double the result and the data source name. */ + char result_string[RESULTS_BUFFER * 2 + DBL_BUFSIZE + SMALL_BUFSIZE]; + int result_length; + int new_buffer = TRUE; + size_t out_buffer; + MYSQL *mysqlt; + int i; + + /* insert the query results into the database */ + query3 = allocate_output_query(queries->output, "ERROR: Fatal malloc error: poller.c query3 output buffer!"); + + out_buffer = strlen(query3); + + if (set.boost.boost_redirect && set.boost.boost_enabled) { + query12 = allocate_output_query(queries->boost_output, "ERROR: Fatal malloc error: poller.c query12 boost output buffer!"); + } + + int mode; + if (set.poller.poller_id > 1 && set.poller.mode == REMOTE_ONLINE) { + SPINE_LOG_DEBUG(("DEBUG: Setting up writes to remote database")); + mysqlt = mysqlr; + mode = REMOTE; + } else { + SPINE_LOG_DEBUG(("DEBUG: Setting up writes to local database")); + mysqlt = mysql; + mode = LOCAL; + } + + i = 0; + while (i < rows_processed) { + char escaped_result[RESULTS_BUFFER * 2 + 1]; + char escaped_rrd_name[DBL_BUFSIZE]; + + db_escape(mysqlt, escaped_result, sizeof(escaped_result), poller_items[i].result); + db_escape(mysqlt, escaped_rrd_name, sizeof(escaped_rrd_name), poller_items[i].rrd_name); + + if (!format_poller_output_row(result_string, sizeof(result_string), + poller_items[i].local_data_id, + escaped_rrd_name, + host_time, + escaped_result)) { + SPINE_LOG(("ERROR: Poller output for DS[%i] exceeds the configured result buffer and was skipped", + poller_items[i].local_data_id)); + i++; + continue; + } + + result_length = spine_count_to_int(strlen(result_string)); + + /* if the next element to the buffer will overflow it, write to the database */ + if ((out_buffer + result_length) >= MAX_MYSQL_BUF_SIZE) { + if (!flush_output_query(mysqlt, mode, query3, queries->suffix, queries->output)) failed = TRUE; + if (!flush_output_query(mysqlt, mode, query12, queries->suffix, queries->boost_output)) failed = TRUE; + + /* reset the output buffer length */ + out_buffer = strlen(query3); + + /* set binary, let the system know we are a new buffer */ + new_buffer = TRUE; + } + + /* if this is our first pass, or we just outputted to the database, need to change the delimiter */ + result_string[0] = new_buffer ? ' ' : ','; + + strncat(query3, result_string, result_length); + + if (query12 != NULL) { + strncat(query12, result_string, result_length); + } + + out_buffer = out_buffer + strlen(result_string); + new_buffer = FALSE; + i++; + } + + /* perform the last insert if there is data to process */ + if (out_buffer > strlen(queries->output)) { + if (!flush_output_query(mysqlt, mode, query3, queries->suffix, NULL)) failed = TRUE; + if (!flush_output_query(mysqlt, mode, query12, queries->suffix, NULL)) failed = TRUE; + } + /* MEMORY output tables can retain earlier rows after a rejected write. + * Confirmed partial output remains; keep due items eligible for recollection. */ + return (poll_output_buffers_t){query3, query12, failed}; +} + +typedef struct { + int initialized; + int version; + int port; + char community[50]; + char username[50]; + char password[50]; + char auth_protocol[7]; + char priv_passphrase[200]; + char priv_protocol[8]; + char context[65]; + char engine_id[30]; +} snmp_item_key_t; + +typedef struct { + target_t *items; + snmp_oids_t *oids; + int count; + snmp_item_key_t key; + const poll_error_context_t *errors; +} snmp_poll_batch_t; + +static void remember_snmp_item(snmp_item_key_t *key, const target_t *item) { + key->port = item->snmp.port; + key->version = item->snmp.version; + + STRNCOPY(key->community, item->snmp.community); + STRNCOPY(key->username, item->snmp.username); + STRNCOPY(key->password, item->snmp.password); + STRNCOPY(key->auth_protocol, item->snmp.auth_protocol); + STRNCOPY(key->priv_passphrase, item->snmp.priv_passphrase); + STRNCOPY(key->priv_protocol, item->snmp.priv_protocol); + STRNCOPY(key->context, item->snmp.context); + STRNCOPY(key->engine_id, item->snmp.engine_id); +} + +static void *open_snmp_item(const host_t *host, target_t *item) { + return snmp_host_init(&(snmp_connection_t){ + .host_id = host->id, + .hostname = item->hostname, + .snmp_version = item->snmp.version, + .snmp_community = item->snmp.community, + .snmp_username = item->snmp.username, + .snmp_password = item->snmp.password, + .snmp_auth_protocol = item->snmp.auth_protocol, + .snmp_priv_passphrase = item->snmp.priv_passphrase, + .snmp_priv_protocol = item->snmp.priv_protocol, + .snmp_context = item->snmp.context, + .snmp_engine_id = item->snmp.engine_id, + .snmp_port = item->snmp.port, + .snmp_timeout = item->snmp.timeout, +}); +} + +static bool snmp_item_changed(const snmp_item_key_t *key, const target_t *item) { + return (key->port != item->snmp.port) || + (key->version != item->snmp.version) || + (item->snmp.version < 3 && + (!STRMATCH(key->community, item->snmp.community))) || + (item->snmp.version > 2 && + ((!STRMATCH(key->username, item->snmp.username)) || + (!STRMATCH(key->password, item->snmp.password)) || + (!STRMATCH(key->auth_protocol, item->snmp.auth_protocol)) || + (!STRMATCH(key->priv_passphrase, item->snmp.priv_passphrase)) || + (!STRMATCH(key->priv_protocol, item->snmp.priv_protocol)) || + (!STRMATCH(key->context, item->snmp.context)) || + (!STRMATCH(key->engine_id, item->snmp.engine_id)))); +} + +static void flush_snmp_batch(host_t *host, snmp_poll_batch_t *batch, + double thread_start, bool spike_kill) { + if (batch->count <= 0) return; + snmp_get_multi(host, batch->items, batch->oids, batch->count); + store_snmp_results(host, batch->items, batch->oids, batch->count, batch->errors, thread_start, spike_kill); + batch->count = 0; + memset(batch->oids, 0, sizeof(snmp_oids_t) * host->snmp.max_oids); +} + +static void poll_snmp_item(host_t *host, snmp_poll_batch_t *batch, int position, + double thread_start, bool spike_kill) { + target_t *item = &batch->items[position]; + if (batch->key.initialized == 0) { + remember_snmp_item(&batch->key, item); + host->snmp.session = open_snmp_item(host, item); + batch->key.initialized++; + } + if (host->snmp.session == NULL) { + host->ignore_host = TRUE; + return; + } + if (snmp_item_changed(&batch->key, item)) { + /* Credential-switch flushes preserve the legacy no-discard policy. */ + flush_snmp_batch(host, batch, thread_start, FALSE); + if (host->snmp.session != NULL) { + snmp_host_cleanup(host->snmp.session); + host->snmp.session = NULL; + } + host->snmp.session = open_snmp_item(host, item); + remember_snmp_item(&batch->key, item); + } + if (batch->count >= host->snmp.max_oids) flush_snmp_batch(host, batch, thread_start, spike_kill); + snprintf(batch->oids[batch->count].oid, sizeof(batch->oids[batch->count].oid), "%s", item->arg1); + batch->oids[batch->count].array_position = position; + batch->count++; +} + + +typedef struct { + target_t *items; + snmp_oids_t *oids; +} poll_item_storage_t; + +static MYSQL_RES *select_poll_items(MYSQL *mysql, const poller_queries_t *queries, + const host_t *host, const poller_thread_t *work, int *num_rows) { + const char *query = set.poller.poller_interval == 0 ? queries->items : queries->due_items; + MYSQL_RES *result = db_query(mysql, LOCAL, query); + *num_rows = 0; + if (result != NULL) { + *num_rows = spine_count_to_int(mysql_num_rows(result)); + } else { + SPINE_LOG(("Device[%i] HT[%i] ERROR: Unable to Retrieve Rows due to Null Result!", host->id, work->host_thread)); + } + return result; +} + +static poll_item_storage_t load_poll_items(MYSQL_RES *result, host_t *host, int num_rows) { + poll_item_storage_t storage; + /* retrieve each hosts polling items from poller cache and load into array */ + storage.items = (target_t *) calloc(num_rows, sizeof(target_t)); + if (storage.items == NULL) die("ERROR: Fatal calloc error: poller.c poller_items"); + + int i = 0; + MYSQL_ROW row; + while ((row = mysql_fetch_row(result))) { + load_poll_item(&storage.items[i], row); + + i++; + } + + /* free the mysql result */ + db_free_result(result); + + /* create an array for snmp oids */ + if (host->snmp.max_oids <= 0) { + host->snmp.max_oids = 1; + } + storage.oids = (snmp_oids_t *) calloc(host->snmp.max_oids, sizeof(snmp_oids_t)); + if (storage.oids == NULL) { + die("ERROR: Fatal calloc error: poller.c snmp_oids"); + } + + return storage; +} + +static int collect_poll_items(host_t *host, snmp_poll_batch_t *batch, int num_rows, bool spike_kill) { + int i = 0; + int rows_processed = 0; + double thread_start = 0; + while ((i < num_rows) && (!host->ignore_host)) { + thread_start = get_time_as_double(); + + switch(batch->items[i].action) { + case POLLER_ACTION_SNMP: + poll_snmp_item(host, batch, i, thread_start, spike_kill); + break; + case POLLER_ACTION_SCRIPT: + poll_script_item(host, &batch->items[i], batch->errors, thread_start, spike_kill, FALSE); + break; + case POLLER_ACTION_PHP_SCRIPT_SERVER: + poll_script_item(host, &batch->items[i], batch->errors, thread_start, spike_kill, TRUE); + break; + default: /* unknown action, generate error */ + SPINE_LOG(("Device[%i] HT[%i] DS[%i] ERROR: Unknown Poller Action: %s", batch->errors->host_id, batch->errors->thread_id, batch->items[i].local_data_id, batch->items[i].arg1)); + + break; + } + + i++; + rows_processed++; + } + + /* process last multi-get request if applicable */ + if (batch->count > 0) { + snmp_get_multi(host, batch->items, batch->oids, batch->count); + + store_snmp_results(host, batch->items, batch->oids, batch->count, batch->errors, thread_start, spike_kill); + } + + return rows_processed; +} + +static void release_poll_connections(const poller_thread_t *work, + const pool_t *local, const pool_t *remote) { + if (local != NULL) { + db_release_connection(LOCAL, local->id); + } else { + SPINE_LOG(("WARNING: Device[%i] HT[%i] Trying to close uninitialized local connection.", work->host_id, work->host_thread)); + } + if (set.poller.poller_id > 1 && set.poller.mode == REMOTE_ONLINE) { + if (remote != NULL) { + db_release_connection(REMOTE, remote->id); + } else { + SPINE_LOG(("WARNING: Device[%i] HT[%i] Trying to close uninitialized remote connection.", work->host_id, work->host_thread)); + } + } +} + +/* FALSE requests the existing missing-device early return. Other database + * failures leave the host ignored and continue through the normal caller. */ +static bool load_poll_host(MYSQL *mysql, const poller_queries_t *queries, + host_t *host, ping_t *ping, const poller_thread_t *work) { + /* host_id=0 denotes a data source without a device. */ + if (!work->host_id) { + host->id = 0; + host->snmp.max_oids = 1; + host->snmp.session = NULL; + host->ignore_host = FALSE; + return TRUE; + } + MYSQL_RES *result = db_query(mysql, LOCAL, queries->host); + if (result == NULL) { + host->ignore_host = TRUE; + return TRUE; + } + if (spine_count_to_int(mysql_num_rows(result)) != 1) { + db_free_result(result); + return FALSE; + } + MYSQL_ROW row = mysql_fetch_row(result); + if (row == NULL) { + SPINE_LOG(("Device[%i] HT[%i] ERROR: MySQL Returned a Null Device Result", host->id, work->host_thread)); + host->ignore_host = TRUE; + return TRUE; + } + load_host_metadata(mysql, row, host, work); + db_free_result(result); + initialize_host_snmp(host); + bool include_system_information = refresh_host_availability(mysql, host, ping, work->host_thread); + if (work->host_thread == 1) { + persist_host_status(mysql, host, include_system_information && host->ignore_host != TRUE); + } + return TRUE; +} + +static void complete_poll_host(MYSQL *mysql, poller_queries_t *queries, + const poller_thread_t *work, const poll_error_context_t *error_context, + bool output_failed, double poll_start) { + extern poller_thread_t **details; + int host_id = work->host_id; + int host_thread = work->host_thread; + double host_time_double = work->host_time_double; + int errors = *error_context->errors; + const char *error_string = error_context->buffer; + double poll_time; + /* record the polling time for the device */ + poll_time = get_time_as_double() - poll_start; + SPINE_LOG_DEVICE(host_id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] Total Time: %0.2g Seconds", host_id, host_thread, poll_time)); + + /* record the total time for the host */ + thread_mutex_lock(LOCK_THDET); + poller_thread_t *device = details[work->device_counter]; + if (output_failed) { + device->output_failed = TRUE; + set.exit.exit_code = EXIT_FAILURE; + SPINE_LOG(("ERROR: Device[%i] HT[%i] output write failed; partial writes remain and due items stay eligible for recollection", host_id, host_thread)); + } + device->threads_complete++; + if (device->threads_complete == device->host_threads) { + /* Keep the due-item set stable until every device partition has finished. */ + if (set.poller.active_profiles != 1 && !device->output_failed) { + SPINE_LOG_MEDIUM(("Device[%i] HT[%i] Updating Poller Items for Next Poll", host_id, host_thread)); + db_query(mysql, LOCAL, queries->schedule); + } + device->complete = !device->output_failed; + + poll_time = get_time_as_double(); + queries->items[0] = '\0'; + snprintf(queries->items, BUFSIZE, "UPDATE host SET polling_time = %.3f - %.3f WHERE id = %i", poll_time, host_time_double, host_id); + db_query(mysql, LOCAL, queries->items); + + } + + if (errors > 0) { + int error_query_len = spine_count_to_int(strlen(error_string) + BUFSIZE); + char *error_query = (char *)malloc(error_query_len); + if (error_query == NULL) die("ERROR: Fatal malloc error: poller.c error_query!"); + + snprintf(error_query, error_query_len, "INSERT INTO host_errors (host_id, poller_id, errors, local_data_ids)" + " VALUES(%i, %i, %i, '%s')" + " ON DUPLICATE KEY UPDATE" + " errors = errors + VALUES(errors)," + " local_data_ids = CONCAT(local_data_ids, ', ', VALUES(local_data_ids))", + host_id, set.poller.poller_id, errors, error_string); + + db_query(mysql, LOCAL, error_query); + + free(error_query); + } + + thread_mutex_unlock(LOCK_THDET); +} + +void poll_host(const poller_thread_t *work, int *host_errors) { + assert(work != NULL && host_errors != NULL); + int host_id = work->host_id; + int host_thread = work->host_thread; + int host_data_ids = work->host_data_ids; + const char *host_time = work->host_time; + poller_queries_t queries; char *query3 = NULL; - char query4[BUFSIZE]; - char query5[BUFSIZE]; - char query6[BUFSIZE]; - char query8[BUFSIZE]; - char query9[BUFSIZE]; - char query10[BUFSIZE]; - char query11[BUFSIZE]; char *query12 = NULL; - char posuffix[BUFSIZE]; - int query8_len = 0; - int query11_len = 0; - int posuffix_len = 0; - char sysUptime[BUFSIZE]; - int uptime_use_engine_oid = FALSE; /* pins uptime-goes-backward checks to the seconds granularity engine OID once found */ - char result_string[RESULTS_BUFFER+SMALL_BUFSIZE]; - int result_length; - char temp_result[RESULTS_BUFFER]; int errors = 0; int *buf_errors; int *buf_size; char *error_string; int num_rows; - int assert_fail = FALSE; - int reindex_err = FALSE; int spike_kill = FALSE; int rows_processed = 0; - int i = 0; - int j = 0; - int k = 0; - int num_oids = 0; - size_t out_buffer; - int php_process; + bool output_failed = FALSE; - char *poll_result = NULL; - char update_sql[BIG_BUFSIZE]; - char temp_poll_result[BUFSIZE]; - char temp_arg1[BUFSIZE]; - char limits[SMALL_BUFSIZE]; - int last_snmp_version = 0; - int last_snmp_port = 0; - char last_snmp_community[50]; - char last_snmp_username[50]; - char last_snmp_password[50]; - char last_snmp_auth_protocol[7]; - char last_snmp_priv_passphrase[200]; - char last_snmp_priv_protocol[8]; - char last_snmp_context[65]; - char last_snmp_engine_id[30]; double poll_time = get_time_as_double(); - double thread_start = 0; - double thread_end = 0; /* reindex shortcuts to speed polling */ - int previous_assert_failure = FALSE; - int last_data_query_id = 0; - int perform_assert = TRUE; - int new_buffer = TRUE; - int ignore_sysinfo = TRUE; - int buf_length = 0; - extern poller_thread_t** details; pool_t *local_cnn = NULL; pool_t *remote_cnn = NULL; @@ -279,7 +1374,6 @@ void poll_host(int device_counter, int host_id, int host_thread, int host_thread reindex_t *reindex = NULL; host_t *host = NULL; ping_t *ping = NULL; - name_t *name = NULL; target_t *poller_items = NULL; snmp_oids_t *snmp_oids = NULL; @@ -293,28 +1387,27 @@ void poll_host(int device_counter, int host_id, int host_thread, int host_thread die("ERROR: Fatal malloc error: poller.c buf_errors!"); } - *buf_size = 0; - *buf_errors = 0; + if (error_string == NULL || buf_size == NULL || buf_errors == NULL) { + die("ERROR: Fatal malloc error: poller error buffer!"); + } + *buf_size = 0; + *buf_errors = 0; + const poll_error_context_t error_context = { + error_string, buf_size, buf_errors, &errors, host_id, host_thread + }; - MYSQL mysql; - MYSQL mysqlr; - MYSQL mysqlt; + MYSQL *mysql; + MYSQL *mysqlr = NULL; MYSQL_RES *result; - MYSQL_ROW row; - //db_connect(LOCAL, &mysql); local_cnn = db_get_connection(LOCAL); - if (local_cnn == NULL) { // GCOVR_EXCL_START: pool exhaustion is not reachable in tests - die("ERROR: Fatal error: poller.c could not obtain a local database connection!"); - } // GCOVR_EXCL_STOP - mysql = local_cnn->mysql; + if (local_cnn == NULL) die("ERROR: No local database connection available for polling"); + mysql = &local_cnn->mysql; - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { + if (set.poller.poller_id > 1 && set.poller.mode == REMOTE_ONLINE) { remote_cnn = db_get_connection(REMOTE); - if (remote_cnn == NULL) { // GCOVR_EXCL_START: pool exhaustion is not reachable in tests - die("ERROR: Fatal error: poller.c could not obtain a remote database connection!"); - } // GCOVR_EXCL_STOP - mysqlr = remote_cnn->mysql; + if (remote_cnn == NULL) die("ERROR: No remote database connection available for polling"); + mysqlr = &remote_cnn->mysql; } /* allocate host and ping structures with appropriate values */ @@ -337,298 +1430,7 @@ void poll_host(int device_counter, int host_id, int host_thread, int host_thread } memset(reindex, 0, sizeof(reindex_t)); - /* determine the SQL limits using the poller instructions */ - if (host_data_ids > 0) { - snprintf(limits, SMALL_BUFSIZE, "LIMIT %i, %i", host_data_ids * (host_thread - 1), host_data_ids); - } else { - limits[0] = '\0'; - } - - /* optional output_regex column (added in Cacti 1.3.1) */ - const char *regex_col = set.has_output_regex ? ", output_regex" : ""; - - /* single polling interval query for items */ - if (set.poller_id == 0) { - if (set.total_snmp_ports == 1) { - snprintf(query1, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " AND deleted = '' %s", regex_col, host_id, limits); - } else { - snprintf(query1, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " AND deleted = ''" - " ORDER BY snmp_port %s", regex_col, host_id, limits); - } - - /* host structure for uptime checks */ - snprintf(query2, BIG_BUFSIZE, - "SELECT SQL_NO_CACHE id, hostname, snmp_community, snmp_version, " - "snmp_username, snmp_password, snmp_auth_protocol, " - "snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id, snmp_port, snmp_timeout, max_oids, " - "availability_method, ping_method, ping_port, ping_timeout, ping_retries, " - "status, status_event_count, UNIX_TIMESTAMP(status_fail_date), " - "UNIX_TIMESTAMP(status_rec_date), status_last_error, " - "min_time, max_time, cur_time, avg_time, " - "total_polls, failed_polls, availability, snmp_sysUpTimeInstance, snmp_sysDescr, snmp_sysObjectID, " - "snmp_sysContact, snmp_sysName, snmp_sysLocation" - " FROM host" - " WHERE id = %i" - " AND deleted = ''", host_id); - - /* data query structure for reindex detection */ - snprintf(query4, BUFSIZE, - "SELECT SQL_NO_CACHE data_query_id, action, op, assert_value, arg1" - " FROM poller_reindex" - " WHERE host_id = %i", host_id); - - /* multiple polling interval query for items */ - if (set.active_profiles != 1) { - if (set.total_snmp_ports == 1) { - snprintf(query5, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " AND rrd_next_step <= 0" - " %s", regex_col, host_id, limits); - } else { - snprintf(query5, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " AND rrd_next_step <= 0" - " ORDER BY snmp_port %s", regex_col, host_id, limits); - } - } else { - if (set.total_snmp_ports == 1) { - snprintf(query5, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " %s", regex_col, host_id, limits); - } else { - snprintf(query5, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " ORDER BY snmp_port %s", regex_col, host_id, limits); - } - } - - /* query to setup the next polling interval in cacti */ - snprintf(query6, BUFSIZE, - "UPDATE poller_item" - " SET rrd_next_step = IF(rrd_step = %i, 0, IF(rrd_next_step - %i < 0, rrd_step - %i, rrd_next_step - %i))" - " WHERE host_id = %i", set.poller_interval, set.poller_interval, set.poller_interval, set.poller_interval, host_id); - - /* query to add output records to the poller output table */ - snprintf(query8, BUFSIZE, - "INSERT INTO poller_output" - " (local_data_id, rrd_name, time, output) VALUES"); - - /* number of agent's count for single polling interval */ - snprintf(query9, BUFSIZE, - "SELECT SQL_NO_CACHE snmp_port, count(snmp_port)" - " FROM poller_item" - " WHERE host_id = %i" - " GROUP BY snmp_port %s", host_id, limits); - - /* number of agent's count for multiple polling intervals */ - if (set.active_profiles != 1) { - snprintf(query10, BUFSIZE, - "SELECT SQL_NO_CACHE snmp_port, count(snmp_port)" - " FROM poller_item" - " WHERE host_id = %i" - " AND rrd_next_step <= 0" - " GROUP BY snmp_port %s", host_id, limits); - } else { - snprintf(query10, BUFSIZE, - "SELECT SQL_NO_CACHE snmp_port, count(snmp_port)" - " FROM poller_item" - " WHERE host_id = %i" - " GROUP BY snmp_port %s", host_id, limits); - } - } else { - if (set.total_snmp_ports == 1) { - snprintf(query1, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " AND poller_id=%i %s", regex_col, host_id, set.poller_id, limits); - } else { - snprintf(query1, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " AND poller_id=%i" - " ORDER BY snmp_port %s", regex_col, host_id, set.poller_id, limits); - } - - /* host structure for uptime checks */ - snprintf(query2, BIG_BUFSIZE, - "SELECT SQL_NO_CACHE id, hostname, snmp_community, snmp_version, " - "snmp_username, snmp_password, snmp_auth_protocol, " - "snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id, snmp_port, snmp_timeout, max_oids, " - "availability_method, ping_method, ping_port, ping_timeout, ping_retries, " - "status, status_event_count, UNIX_TIMESTAMP(status_fail_date), " - "UNIX_TIMESTAMP(status_rec_date), status_last_error, " - "min_time, max_time, cur_time, avg_time, " - "total_polls, failed_polls, availability, snmp_sysUpTimeInstance, snmp_sysDescr, snmp_sysObjectID, " - "snmp_sysContact, snmp_sysName, snmp_sysLocation" - " FROM host" - " WHERE id = %i" - " AND deleted = ''", host_id); - - /* data query structure for reindex detection */ - snprintf(query4, BUFSIZE, - "SELECT SQL_NO_CACHE data_query_id, action, op, assert_value, arg1" - " FROM poller_reindex" - " WHERE host_id = %i", host_id); - - /* multiple polling interval query for items */ - if (set.active_profiles != 1) { - if (set.total_snmp_ports == 1) { - snprintf(query5, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " AND rrd_next_step <= 0" - " AND poller_id = %i %s", regex_col, host_id, set.poller_id, limits); - } else { - snprintf(query5, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " AND rrd_next_step <= 0" - " AND poller_id = %i" - " ORDER BY snmp_port %s", regex_col, host_id, set.poller_id, limits); - } - } else { - if (set.total_snmp_ports == 1) { - snprintf(query5, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " AND poller_id = %i %s", regex_col, host_id, set.poller_id, limits); - } else { - snprintf(query5, BUFSIZE, - "SELECT SQL_NO_CACHE action, hostname, snmp_community, " - "snmp_version, snmp_username, snmp_password, " - "rrd_name, rrd_path, arg1, arg2, arg3, local_data_id, " - "rrd_num, snmp_port, snmp_timeout, " - "snmp_auth_protocol, snmp_priv_passphrase, snmp_priv_protocol, snmp_context, snmp_engine_id" - "%s" - " FROM poller_item" - " WHERE host_id = %i" - " AND poller_id = %i" - " ORDER BY snmp_port %s", regex_col, host_id, set.poller_id, limits); - } - } - - /* query to setup the next polling interval in cacti */ - snprintf(query6, BUFSIZE, - "UPDATE poller_item" - " SET rrd_next_step = IF(rrd_step = %i, 0, IF(rrd_next_step - %i < 0, rrd_step - %i, rrd_next_step - %i))" - " WHERE host_id = %i" - " AND poller_id = %i", set.poller_interval, set.poller_interval, set.poller_interval, set.poller_interval, host_id, set.poller_id); - - /* query to add output records to the poller output table */ - snprintf(query8, BUFSIZE, - "INSERT INTO poller_output" - " (local_data_id, rrd_name, time, output) VALUES"); - - /* number of agent's count for single polling interval */ - snprintf(query9, BUFSIZE, - "SELECT SQL_NO_CACHE snmp_port, count(snmp_port)" - " FROM poller_item" - " WHERE host_id = %i" - " AND poller_id = %i" - " GROUP BY snmp_port %s", host_id, set.poller_id, limits); - - /* number of agent's count for multiple polling intervals */ - if (set.active_profiles != 1) { - snprintf(query10, BUFSIZE, - "SELECT SQL_NO_CACHE snmp_port, count(snmp_port)" - " FROM poller_item" - " WHERE host_id = %i" - " AND rrd_next_step <= 0" - " AND poller_id = %i" - " GROUP BY snmp_port %s", host_id, set.poller_id, limits); - } else { - snprintf(query10, BUFSIZE, - "SELECT SQL_NO_CACHE snmp_port, count(snmp_port)" - " FROM poller_item" - " WHERE host_id = %i" - " AND poller_id = %i" - " GROUP BY snmp_port %s", host_id, set.poller_id, limits); - } - } - - /* query to add output records to the poller output table */ - snprintf(query11, BUFSIZE, - "INSERT INTO poller_output_boost" - " (local_data_id, rrd_name, time, output) VALUES"); - - query8_len = strlen(query8); - query11_len = strlen(query11); + poller_prepare_queries(&queries, host_id, host_thread, host_data_ids); /* initialize the ping structure variables */ snprintf(ping->ping_status, 50, "down"); @@ -636,304 +1438,24 @@ void poll_host(int device_counter, int host_id, int host_thread, int host_thread snprintf(ping->snmp_status, 50, "down"); snprintf(ping->snmp_response, SMALL_BUFSIZE, "SNMP not performed due to setting or ping result"); - /* if the host is a real host. Note host_id=0 is not host based data source */ - if (host_id) { - /* get data about this host */ - if ((result = db_query(&mysql, LOCAL, query2)) != 0) { - num_rows = mysql_num_rows(result); + if (!load_poll_host(mysql, &queries, host, ping, work)) { + release_poll_connections(work, local_cnn, remote_cnn); + SPINE_FREE(host); + SPINE_FREE(reindex); + SPINE_FREE(ping); + SPINE_FREE(error_string); + SPINE_FREE(buf_size); + SPINE_FREE(buf_errors); + mysql_thread_end(); - if (num_rows != 1) { - db_free_result(result); - - if (local_cnn != NULL) { - db_release_connection(LOCAL, local_cnn->id); - } else { - SPINE_LOG(("WARNING: Device[%i] HT[%i] Trying to close uninitialized local connection.", host_id, host_thread)); - } - - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { - if (remote_cnn != NULL) { - db_release_connection(REMOTE, remote_cnn->id); - } else { - SPINE_LOG(("WARNING: Device[%i] HT[%i] Trying to close uninitialized remote connection.", host_id, host_thread)); - } - } - - SPINE_FREE(host); - SPINE_FREE(reindex); - SPINE_FREE(ping); - SPINE_FREE(error_string); - SPINE_FREE(buf_size); - SPINE_FREE(buf_errors); - - mysql_thread_end(); - - return; - } - - /* fetch the result */ - row = mysql_fetch_row(result); - - if (row) { - /* initialize variables first */ - host->id = 0; // 0 - host->hostname[0] = '\0'; // 1 - host->snmp_session = NULL; // - - host->snmp_community[0] = '\0'; // 2 - host->snmp_version = 1; // 3 - host->snmp_username[0] = '\0'; // 4 - host->snmp_password[0] = '\0'; // 5 - host->snmp_auth_protocol[0] = '\0'; // 6 - host->snmp_priv_passphrase[0] = '\0'; // 7 - host->snmp_priv_protocol[0] = '\0'; // 8 - host->snmp_context[0] = '\0'; // 9 - host->snmp_engine_id[0] = '\0'; // 10 - host->snmp_port = 161; // 11 - host->snmp_timeout = 500; // 12 - host->snmp_retries = set.snmp_retries; // - - host->max_oids = 10; // 13 - host->availability_method = 0; // 14 - host->ping_method = 0; // 15 - host->ping_port = 23; // 16 - host->ping_timeout = 500; // 17 - host->ping_retries = 2; // 18 - host->status = HOST_UP; // 19 - host->status_event_count = 0; // 20 - host->status_fail_date[0] = '\0'; // 21 - host->status_rec_date[0] = '\0'; // 22 - host->status_last_error[0] = '\0'; // 23 - host->min_time = 0; // 24 - host->max_time = 0; // 25 - host->cur_time = 0; // 26 - host->avg_time = 0; // 27 - host->total_polls = 0; // 28 - host->failed_polls = 0; // 29 - host->availability = 100; // 30 - host->snmp_sysUpTimeInstance = 0; // 31 - host->snmp_sysDescr[0] = '\0'; // 32 - host->snmp_sysObjectID[0] = '\0'; // 33 - host->snmp_sysContact[0] = '\0'; // 34 - host->snmp_sysName[0] = '\0'; // 35 - host->snmp_sysLocation[0] = '\0'; // 36 - - /* populate host structure */ - host->ignore_host = FALSE; - if (row[0] != NULL) host->id = atoi(row[0]); - - if (row[1] != NULL) { - name = get_namebyhost(row[1], NULL); - STRNCOPY(host->hostname, name->hostname); - host->ping_port = name->port; - SPINE_FREE(name); - } - - if (row[2] != NULL) STRNCOPY(host->snmp_community, row[2]); - - if (row[3] != NULL) host->snmp_version = atoi(row[3]); - - if (row[4] != NULL) STRNCOPY(host->snmp_username, row[4]); - if (row[5] != NULL) STRNCOPY(host->snmp_password, row[5]); - if (row[6] != NULL) STRNCOPY(host->snmp_auth_protocol, row[6]); - if (row[7] != NULL) STRNCOPY(host->snmp_priv_passphrase, row[7]); - if (row[8] != NULL) STRNCOPY(host->snmp_priv_protocol, row[8]); - if (row[9] != NULL) STRNCOPY(host->snmp_context, row[9]); - if (row[10] != NULL) STRNCOPY(host->snmp_engine_id, row[10]); - - if (row[11] != NULL) host->snmp_port = atoi(row[11]); - if (row[12] != NULL) host->snmp_timeout = atoi(row[12]); - if (row[13] != NULL) host->max_oids = atoi(row[13]); - - if (row[14] != NULL) host->availability_method = atoi(row[14]); - if (row[15] != NULL) host->ping_method = atoi(row[15]); - if (row[16] != NULL) host->ping_port = atoi(row[16]); - if (row[17] != NULL) host->ping_timeout = atoi(row[17]); - if (row[18] != NULL) host->ping_retries = atoi(row[18]); - - if (row[19] != NULL) host->status = atoi(row[19]); - if (row[20] != NULL) host->status_event_count = atoi(row[20]); - - if (row[21] != NULL) STRNCOPY(host->status_fail_date, row[21]); - if (row[22] != NULL) STRNCOPY(host->status_rec_date, row[22]); - - if (row[23] != NULL) STRNCOPY(host->status_last_error, row[23]); - - if (row[24] != NULL) host->min_time = atof(row[24]); - if (row[25] != NULL) host->max_time = atof(row[25]); - if (row[26] != NULL) host->cur_time = atof(row[26]); - if (row[27] != NULL) host->avg_time = atof(row[27]); - if (row[28] != NULL) host->total_polls = atoi(row[28]); - if (row[29] != NULL) host->failed_polls = atoi(row[29]); - if (row[30] != NULL) host->availability = atof(row[30]); - - if (row[31] != NULL) host->snmp_sysUpTimeInstance=atoll(row[31]); - if (row[32] != NULL) db_escape(&mysql, host->snmp_sysDescr, sizeof(host->snmp_sysDescr), row[32]); - if (row[33] != NULL) db_escape(&mysql, host->snmp_sysObjectID, sizeof(host->snmp_sysObjectID), row[33]); - if (row[34] != NULL) db_escape(&mysql, host->snmp_sysContact, sizeof(host->snmp_sysContact), row[34]); - if (row[35] != NULL) db_escape(&mysql, host->snmp_sysName, sizeof(host->snmp_sysName), row[35]); - if (row[36] != NULL) db_escape(&mysql, host->snmp_sysLocation, sizeof(host->snmp_sysLocation), row[36]); - - /* correct max_oid bounds issues */ - if ((host->max_oids == 0) || (host->max_oids > 100)) { - SPINE_LOG(("Device[%i] HT[%i] WARNING: Max OIDS is out of range with value of '%i'. Resetting to default of 5", host_id, host_thread, host->max_oids)); - host->max_oids = 5; - } - - /* free the host result */ - db_free_result(result); - - if (((host->snmp_version >= 1) && (host->snmp_version <= 2) && - (strlen(host->snmp_community) > 0)) || - (host->snmp_version == 3)) { - host->snmp_session = snmp_host_init(host->id, - host->hostname, - host->snmp_version, - host->snmp_community, - host->snmp_username, - host->snmp_password, - host->snmp_auth_protocol, - host->snmp_priv_passphrase, - host->snmp_priv_protocol, - host->snmp_context, - host->snmp_engine_id, - host->snmp_port, - host->snmp_timeout); - } else { - host->snmp_session = NULL; - } - - /* perform a check to see if the host is alive by polling it's SysDesc - * if the host down from an snmp perspective, don't poll it. - * function sets the ignore_host bit */ - if ((host->availability_method == AVAIL_SNMP) && - (strlen(host->snmp_community) == 0) && - (host->snmp_version < 3)) { - host->ignore_host = FALSE; - update_host_status(HOST_UP, host, ping, host->availability_method); - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] No host availability check possible for '%s'", host->id, host_thread, host->hostname)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] No host availability check possible for '%s'", host->id, host_thread, host->hostname)); - } - } else if (host->availability_method == AVAIL_STREAM) { - update_host_status(HOST_UP, host, ping, host->availability_method); - } else { - if (ping_host(host, ping) == HOST_UP) { - host->ignore_host = FALSE; - if (host_thread == 1) { - update_host_status(HOST_UP, host, ping, host->availability_method); - - if ((host->availability_method != AVAIL_PING) && (host->availability_method != AVAIL_NONE)) { - if (host->snmp_session != NULL && set.mibs) { - get_system_information(host, &mysql, 1); - ignore_sysinfo = FALSE; - } - } - } - } else { - host->ignore_host = TRUE; - if (host_thread == 1) { - update_host_status(HOST_DOWN, host, ping, host->availability_method); - } - } - } - - /* update host table */ - if (host_thread == 1) { - char escaped_last_error[BUFSIZE]; - db_escape(&mysql, escaped_last_error, sizeof(escaped_last_error), host->status_last_error); - - if (!ignore_sysinfo) { - if (host->ignore_host != TRUE) { - snprintf(update_sql, BIG_BUFSIZE, "UPDATE host " - "SET status='%i', status_event_count='%i', status_fail_date=FROM_UNIXTIME(%s)," - " status_rec_date=FROM_UNIXTIME(%s), status_last_error='%s', min_time='%f'," - " max_time='%f', cur_time='%f', avg_time='%f', total_polls='%i'," - " failed_polls='%i', availability='%.4f', snmp_sysDescr='%s', " - " snmp_sysObjectID='%s', snmp_sysUpTimeInstance='%llu', " - " snmp_sysContact='%s', snmp_sysName='%s', snmp_sysLocation='%s' " - "WHERE id='%i'", - host->status, - host->status_event_count, - host->status_fail_date, - host->status_rec_date, - escaped_last_error, - host->min_time, - host->max_time, - host->cur_time, - host->avg_time, - host->total_polls, - host->failed_polls, - host->availability, - host->snmp_sysDescr, - host->snmp_sysObjectID, - host->snmp_sysUpTimeInstance, - host->snmp_sysContact, - host->snmp_sysName, - host->snmp_sysLocation, - host->id); - } else { - snprintf(update_sql, BIG_BUFSIZE, "UPDATE host " - "SET status='%i', status_event_count='%i', status_fail_date=FROM_UNIXTIME(%s)," - " status_rec_date=FROM_UNIXTIME(%s), status_last_error='%s', min_time='%f'," - " max_time='%f', cur_time='%f', avg_time='%f', total_polls='%i'," - " failed_polls='%i', availability='%.4f' " - "WHERE id='%i'", - host->status, - host->status_event_count, - host->status_fail_date, - host->status_rec_date, - escaped_last_error, - host->min_time, - host->max_time, - host->cur_time, - host->avg_time, - host->total_polls, - host->failed_polls, - host->availability, - host->id); - } - } else { - snprintf(update_sql, BIG_BUFSIZE, "UPDATE host " - "SET status='%i', status_event_count='%i', status_fail_date=FROM_UNIXTIME(%s)," - " status_rec_date=FROM_UNIXTIME(%s), status_last_error='%s', min_time='%f'," - " max_time='%f', cur_time='%f', avg_time='%f', total_polls='%i'," - " failed_polls='%i', availability='%.4f' " - "WHERE id='%i'", - host->status, - host->status_event_count, - host->status_fail_date, - host->status_rec_date, - escaped_last_error, - host->min_time, - host->max_time, - host->cur_time, - host->avg_time, - host->total_polls, - host->failed_polls, - host->availability, - host->id); - } - - db_insert(&mysql, LOCAL, update_sql); - } - } else { - SPINE_LOG(("Device[%i] HT[%i] ERROR: MySQL Returned a Null Device Result", host->id, host_thread)); - num_rows = 0; - host->ignore_host = TRUE; - } - } else { - num_rows = 0; - host->ignore_host = TRUE; - } - } else { - host->id = 0; - host->max_oids = 1; - host->snmp_session = NULL; - host->ignore_host = FALSE; + return; } - if (set.ping_only) { + if (set.availability.ping_only) { + if (host->snmp.session != NULL) { + snmp_host_cleanup(host->snmp.session); + host->snmp.session = NULL; + } SPINE_FREE(host); SPINE_FREE(reindex); SPINE_FREE(ping); @@ -941,1162 +1463,43 @@ void poll_host(int device_counter, int host_id, int host_thread, int host_thread SPINE_FREE(buf_size); SPINE_FREE(buf_errors); - if (local_cnn != NULL) { - db_release_connection(LOCAL, local_cnn->id); - } else { - SPINE_LOG(("WARNING: Device[%i] HT[%i] Trying to close uninitialized local connection.", host_id, host_thread)); - } - - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { - if (remote_cnn != NULL) { - db_release_connection(REMOTE, remote_cnn->id); - } else { - SPINE_LOG(("WARNING: Device[%i] HT[%i] Trying to close uninitialized remote connection.", host_id, host_thread)); - } - } + release_poll_connections(work, local_cnn, remote_cnn); mysql_thread_end(); return; } - /* do the reindex check for this host if not script based */ - if ((!host->ignore_host) && (host_id)) { - if ((result = db_query(&mysql, LOCAL, query4)) != 0) { - num_rows = mysql_num_rows(result); - - if (num_rows > 0) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] DEBUG: RECACHE: Processing %i items in the auto reindex cache for '%s'", host->id, host_thread, num_rows, host->hostname)); - } else { - SPINE_LOG_DEBUG(("Device[%i] HT[%i] DEBUG: RECACHE: Processing %i items in the auto reindex cache for '%s'", host->id, host_thread, num_rows, host->hostname)); - } - - // Cache uptime in case we need it again - sysUptime[0] = '\0'; - while ((row = mysql_fetch_row(result))) { - assert_fail = FALSE; - reindex_err = FALSE; - - /* initialize the reindex struction */ - reindex->data_query_id = 0; - reindex->action = -1; - reindex->op[0] = '\0'; - reindex->assert_value[0] = '\0'; - reindex->arg1[0] = '\0'; - - if (row[0] != NULL) reindex->data_query_id = atoi(row[0]); - if (row[1] != NULL) reindex->action = atoi(row[1]); - - if (row[2] != NULL) snprintf(reindex->op, sizeof(reindex->op), "%s", row[2]); - - if (row[3] != NULL) snprintf(reindex->assert_value, sizeof(reindex->assert_value), "%s", row[3]); - - if (row[4] != NULL) snprintf(reindex->arg1, sizeof(reindex->arg1), "%s", row[4]); - - /* shortcut assertion checks if a data query reindex has already been queued */ - if ((last_data_query_id == reindex->data_query_id) && - (!previous_assert_failure)) { - perform_assert = TRUE; - } else if (last_data_query_id != reindex->data_query_id) { - last_data_query_id = reindex->data_query_id; - perform_assert = TRUE; - previous_assert_failure = FALSE; - } else { - perform_assert = FALSE; - } - - poll_result = NULL; - - if (perform_assert) { - switch(reindex->action) { - case POLLER_ACTION_SNMP: /* snmp */ - /* if there is no snmp session, don't probe */ - if (host->snmp_session == NULL) { - reindex_err = TRUE; - } - - /* check to see if you are checking uptime */ - if (!reindex_err) { - if ((strstr(reindex->arg1, ".1.3.6.1.2.1.1.3.0") || - strstr(reindex->arg1, ".1.3.6.1.6.3.10.2.1.3.0")) && strlen(sysUptime) > 0) { - - if (!(poll_result = (char *) malloc(BUFSIZE))) { - die("ERROR: Fatal malloc error: poller.c poll_result"); - } - - poll_result[0] = '\0'; - - snprintf(poll_result, BUFSIZE, "%s", sysUptime); - } else if (strstr(reindex->arg1, ".1.3.6.1.2.1.1.3.0") || - strstr(reindex->arg1, ".1.3.6.1.6.3.10.2.1.3.0")) { - // Ensure uptime is empty to start with - sysUptime[0] = '\0'; - - /* Pin the uptime-goes-backward calculation to a single OID for this poll. - Previously the legacy (centisecond) and modern (second) OIDs could each - supply the value on different reindex rows or different poll cycles, - and a mismatch between the two caused false "uptime went backward" - detections and constant reindexing. Prefer the modern engine OID, since - it offers seconds granularity, and only fall back to the legacy OID - when the engine OID isn't present with numeric data. */ - poll_result = snmp_get_base(host, ".1.3.6.1.6.3.10.2.1.3.0", false); - - uptime_use_engine_oid = (poll_result != NULL && is_numeric(poll_result)); - - if (uptime_use_engine_oid) { - snprintf(sysUptime, BUFSIZE, "%lld", atoll(poll_result) * 100); - } - - SPINE_FREE(poll_result); - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] Engine Uptime OID Present: %d", host->id, host_thread, reindex->data_query_id, uptime_use_engine_oid)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DQ[%i] Engine Uptime OID Present: %d", host->id, host_thread, reindex->data_query_id, uptime_use_engine_oid)); - } - - if (!uptime_use_engine_oid) { - // Engine OID unavailable, fall back to the legacy sysUpTime OID - poll_result = snmp_get(host, ".1.3.6.1.2.1.1.3.0"); - - if (poll_result && is_numeric(poll_result)) { - snprintf(sysUptime, BUFSIZE, "%s", poll_result); - } - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] Legacy Uptime Result: %s, Is Numeric: %d", host->id, host_thread, reindex->data_query_id, poll_result, is_numeric(poll_result) )); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DQ[%i] Legacy Uptime Result: %s, Is Numeric: %d", host->id, host_thread, reindex->data_query_id, poll_result, is_numeric(poll_result) )); - } - - SPINE_FREE(poll_result); - } - - /* allocate and populate with whichever uptime was valid */ - if (!(poll_result = (char *) malloc(BUFSIZE))) { - die("ERROR: Fatal malloc error: poller.c poll_result"); - } - snprintf(poll_result, BUFSIZE, "%s", sysUptime); - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] Extended Uptime Result: %s, Is Numeric: %d", host->id, host_thread, reindex->data_query_id, poll_result, is_numeric(poll_result) )); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DQ[%i] Extended Uptime Result: %s, Is Numeric: %d", host->id, host_thread, reindex->data_query_id, poll_result, is_numeric(poll_result) )); - } - } else { - poll_result = snmp_get(host, reindex->arg1); - } - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE OID: %s, (assert: %s %s output: %s)", host->id, host_thread, reindex->data_query_id, reindex->arg1, reindex->assert_value, reindex->op, poll_result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DQ[%i] RECACHE OID: %s, (assert: %s %s output: %s)", host->id, host_thread, reindex->data_query_id, reindex->arg1, reindex->assert_value, reindex->op, poll_result)); - } - } else { - SPINE_LOG(("WARNING: Device[%i] HT[%i] DQ[%i] Reindex Check FAILED: No SNMP Session. If not an SNMP host, don't use Uptime Goes Backwards!", host->id, host_thread, reindex->data_query_id)); - } - - break; - case POLLER_ACTION_SCRIPT: /* script (popen) */ - /* Reject empty script commands that could cause unexpected behavior */ - if (strlen(reindex->arg1) == 0) { - SPINE_LOG(("WARNING: Device[%i] HT[%i] DQ[%i] empty script command, skipping", - host->id, host_thread, reindex->data_query_id)); - break; - } - - poll_result = trim(exec_poll(host, reindex->arg1, reindex->data_query_id, "DQ")); - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE CMD: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DQ[%i] RECACHE CMD: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); - } - - break; - case POLLER_ACTION_PHP_SCRIPT_SERVER: /* script (php script server) */ - php_process = php_get_process(); - - poll_result = trim(php_cmd(reindex->arg1, php_process)); - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE SERVER: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DQ[%i] RECACHE SERVER: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); - } - - break; - case POLLER_ACTION_SNMP_COUNT: { /* snmp; count items */ - int snmp_items; - - if (!(poll_result = (char *) malloc(BUFSIZE))) { - die("ERROR: Fatal malloc error: poller.c poll_result"); - } - poll_result[0] = '\0'; - - snmp_items = snmp_count(host, reindex->arg1); - if (snmp_items < 0) { - SET_UNDEFINED(poll_result); - } else { - snprintf(poll_result, BUFSIZE, "%d", snmp_items); - } - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE OID COUNT: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DQ[%i] RECACHE OID COUNT: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); - } - - break; - } - case POLLER_ACTION_SCRIPT_COUNT: /* script (popen); count items by counting line feeds */ - if (!(poll_result = (char *) malloc(BUFSIZE))) { - die("ERROR: Fatal malloc error: poller.c poll_result"); - } - poll_result[0] = '\0'; - - { - char *ep_result = exec_poll(host, reindex->arg1, reindex->data_query_id, "DQ"); - snprintf(poll_result, BUFSIZE, "%d", char_count(ep_result, '\n')); - free(ep_result); - } - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE CMD COUNT: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DQ[%i] RECACHE CMD COUNT: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); - } - - break; - case POLLER_ACTION_PHP_SCRIPT_SERVER_COUNT: /* script (php script server); count number of lines */ - if (!(poll_result = (char *) malloc(BUFSIZE))) { - die("ERROR: Fatal malloc error: poller.c poll_result"); - } - poll_result[0] = '\0'; - - php_process = php_get_process(); - - { - char *php_result = php_cmd(reindex->arg1, php_process); - snprintf(poll_result, BUFSIZE, "%d", char_count(php_result, '\n')); - free(php_result); - } - - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE SERVER COUNT: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DQ[%i] RECACHE SERVER COUNT: %s, output: %s", host->id, host_thread, reindex->data_query_id, reindex->arg1, poll_result)); - } - - break; - default: - SPINE_LOG(("Device[%i] HT[%i] ERROR: Unknown Assert Action!", host->id, host_thread)); - } - - if (!reindex_err) { - if (!(query3 = (char *)malloc(LRG_BUFSIZE))) { - die("ERROR: Fatal malloc error: poller.c reindex insert!"); - } - query3[0] = '\0'; - - /* assume ok if host is up and result wasn't obtained */ - if (poll_result == NULL || (IS_UNDEFINED(poll_result)) || (STRIMATCH(poll_result, "No Such Instance"))) { - if (is_debug_device(host->id) || set.spine_log_level == 2) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE ASSERT FAILED: '%s=%s'", host->id, host_thread, reindex->data_query_id, reindex->assert_value, poll_result)); - } - - assert_fail = FALSE; - } else if ((!strcmp(reindex->op, "=")) && (strcmp(reindex->assert_value, poll_result))) { - if (is_debug_device(host->id) || set.spine_log_level == 2) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE ASSERT FAILED: '%s=%s'", host->id, host_thread, reindex->data_query_id, reindex->assert_value, poll_result)); - } else { - if (set.spine_log_level == 1) { - errors++; - } - - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE ASSERT FAILED: '%s=%s'", host->id, host_thread, reindex->data_query_id, reindex->assert_value, poll_result)); - } - - if (host_thread == 1) { - snprintf(query3, LRG_BUFSIZE, "REPLACE INTO poller_command (poller_id, time, action,command) values (%i, NOW(), %i, '%i:%i')", set.poller_id, POLLER_COMMAND_REINDEX, host->id, reindex->data_query_id); - - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { - db_insert(&mysqlr, REMOTE, query3); - } else { - db_insert(&mysql, LOCAL, query3); - } - - /* set zeros */ - memset(query3, 0, LRG_BUFSIZE); - } - - assert_fail = TRUE; - previous_assert_failure = TRUE; - } else if ((!strcmp(reindex->op, ">")) && (atoll(reindex->assert_value) < atoll(poll_result))) { - if (is_debug_device(host->id) || set.spine_log_level == 2) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE ASSERT FAILED: '%s>%s'", host->id, host_thread, reindex->data_query_id, reindex->assert_value, poll_result)); - } else { - if (set.spine_log_level == 1) { - errors++; - } - - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE ASSERT FAILED: '%s>%s'", host->id, host_thread, reindex->data_query_id, reindex->assert_value, poll_result)); - } - - if (host_thread == 1) { - snprintf(query3, LRG_BUFSIZE, "REPLACE INTO poller_command (poller_id, time, action, command) ValueS (%i, NOW(), %i, '%i:%i')", set.poller_id, POLLER_COMMAND_REINDEX, host->id, reindex->data_query_id); - - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { - db_insert(&mysqlr, REMOTE, query3); - } else { - db_insert(&mysql, LOCAL, query3); - } - - /* set zeros */ - memset(query3, 0, LRG_BUFSIZE); - } - - assert_fail = TRUE; - previous_assert_failure = TRUE; - /* if uptime is set to '0' don't fail out */ - } else if (strcmp(reindex->assert_value, "0")) { - if ((!strcmp(reindex->op, "<")) && (atoll(reindex->assert_value) > atoll(poll_result))) { - if (is_debug_device(host->id) || set.spine_log_level == 2) { - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE ASSERT FAILED: '%s<%s'", host->id, host_thread, reindex->data_query_id, reindex->assert_value, poll_result)); - } else { - if (set.spine_log_level == 1) { - errors++; - } - - SPINE_LOG(("Device[%i] HT[%i] DQ[%i] RECACHE ASSERT FAILED: '%s<%s'", host->id, host_thread, reindex->data_query_id, reindex->assert_value, poll_result)); - } - - if (host_thread == 1) { - snprintf(query3, LRG_BUFSIZE, "REPLACE INTO poller_command (poller_id, time, action, command) VALUES (%i, NOW(), %i, '%i:%i')", set.poller_id, POLLER_COMMAND_REINDEX, host->id, reindex->data_query_id); - - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { - db_insert(&mysqlr, REMOTE, query3); - } else { - db_insert(&mysql, LOCAL, query3); - } - - /* set zeros */ - memset(query3, 0, LRG_BUFSIZE); - } - - assert_fail = TRUE; - previous_assert_failure = TRUE; - } - } - - /* update 'poller_reindex' with the correct information if: - * 1) the assert fails - * 2) the OP code is > or < meaning the current value could have changed without causing - * the assert to fail */ - if ((assert_fail) || (!strcmp(reindex->op, ">")) || (!strcmp(reindex->op, "<"))) { - if (host_thread == 1) { - db_escape(&mysql, temp_poll_result, sizeof(temp_poll_result), poll_result); - db_escape(&mysql, temp_arg1, sizeof(temp_arg1), reindex->arg1); - - snprintf(query3, LRG_BUFSIZE, "UPDATE poller_reindex SET assert_value='%s' WHERE host_id='%i' AND data_query_id='%i' AND arg1='%s'", temp_poll_result, host_id, reindex->data_query_id, temp_arg1); - - db_insert(&mysql, LOCAL, query3); - - /* set zeros */ - memset(query3, 0, LRG_BUFSIZE); - } - - if ((assert_fail) && - ((!strcmp(reindex->op, "<")) || (!strcmp(reindex->arg1,".1.3.6.1.2.1.1.3.0") || !strcmp(reindex->arg1, ".1.3.6.1.6.3.10.2.1.3.0")))) { - spike_kill = TRUE; - - if (is_debug_device(host->id) || set.spine_log_level == 2) { - SPINE_LOG(("Device[%i] HT[%i] NOTICE: Spike Kill in Effect for '%s'", host_id, host_thread, host->hostname)); - } else { - if (set.spine_log_level == 1) { - errors++; - } - - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] NOTICE: Spike Kill in Effect for '%s'", host_id, host_thread, host->hostname)); - } - } - } - - SPINE_FREE(query3); - SPINE_FREE(poll_result); - } - } - } - } else { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%i] HT[%i] Device has no information for recache.", host->id, host_thread)); - } else { - SPINE_LOG_HIGH(("Device[%i] HT[%i] Device has no information for recache.", host->id, host_thread)); - } - } - - /* free the host result */ - db_free_result(result); - } else { - SPINE_LOG(("Device[%i] HT[%i] ERROR: RECACHE Query Returned Null Result!", host->id, host_thread)); - } - - /* close the host snmp session, we will create again momentarily */ - if (host->snmp_session != NULL) { - snmp_host_cleanup(host->snmp_session); - host->snmp_session = NULL; - } - } + const reindex_evaluation_t evaluation = {mysql, mysqlr, host, work, &errors, &spike_kill}; + poll_host_reindex(host, reindex, queries.reindex, &evaluation); /* calculate the number of poller items to poll this cycle */ - num_rows = 0; - if (set.poller_interval == 0) { - /* get the poller items */ - if ((result = db_query(&mysql, LOCAL, query1)) != 0) { - num_rows = mysql_num_rows(result); - } else { - SPINE_LOG(("Device[%i] HT[%i] ERROR: Unable to Retrieve Rows due to Null Result!", host->id, host_thread)); - } - } else { - /* get the poller items */ - if ((result = db_query(&mysql, LOCAL, query5)) != 0) { - num_rows = mysql_num_rows(result); - } else { - SPINE_LOG(("Device[%i] HT[%i] ERROR: Unable to Retrieve Rows due to Null Result!", host->id, host_thread)); - } - } + result = select_poll_items(mysql, &queries, host, work, &num_rows); if (num_rows > 0) { - /* retrieve each hosts polling items from poller cache and load into array */ - if (!(poller_items = (target_t *) calloc(num_rows, sizeof(target_t)))) { - die("ERROR: Fatal calloc error: poller.c poller_items!"); - } - - i = 0; - while ((row = mysql_fetch_row(result))) { - /* initialize monitored object */ - poller_items[i].target_id = 0; - poller_items[i].action = -1; - poller_items[i].hostname[0] = '\0'; - poller_items[i].snmp_community[0] = '\0'; - poller_items[i].snmp_version = 1; - poller_items[i].snmp_username[0] = '\0'; - poller_items[i].snmp_password[0] = '\0'; - poller_items[i].snmp_auth_protocol[0] = '\0'; - poller_items[i].snmp_priv_passphrase[0] = '\0'; - poller_items[i].snmp_priv_protocol[0] = '\0'; - poller_items[i].snmp_context[0] = '\0'; - poller_items[i].snmp_engine_id[0] = '\0'; - poller_items[i].snmp_port = 161; - poller_items[i].snmp_timeout = 500; - poller_items[i].rrd_name[0] = '\0'; - poller_items[i].rrd_path[0] = '\0'; - poller_items[i].arg1[0] = '\0'; - poller_items[i].arg2[0] = '\0'; - poller_items[i].arg3[0] = '\0'; - poller_items[i].local_data_id = 0; - poller_items[i].rrd_num = 0; - poller_items[i].output_regex[0] = '\0'; - - if (row[0] != NULL) poller_items[i].action = atoi(row[0]); - - if (row[1] != NULL) snprintf(poller_items[i].hostname, sizeof(poller_items[i].hostname), "%s", row[1]); - if (row[2] != NULL) snprintf(poller_items[i].snmp_community, sizeof(poller_items[i].snmp_community), "%s", row[2]); - - if (row[3] != NULL) poller_items[i].snmp_version = atoi(row[3]); - - if (row[4] != NULL) snprintf(poller_items[i].snmp_username, sizeof(poller_items[i].snmp_username), "%s", row[4]); - if (row[5] != NULL) snprintf(poller_items[i].snmp_password, sizeof(poller_items[i].snmp_password), "%s", row[5]); - - if (row[6] != NULL) snprintf(poller_items[i].rrd_name, sizeof(poller_items[i].rrd_name), "%s", row[6]); - if (row[7] != NULL) snprintf(poller_items[i].rrd_path, sizeof(poller_items[i].rrd_path), "%s", row[7]); - if (row[8] != NULL) snprintf(poller_items[i].arg1, sizeof(poller_items[i].arg1), "%s", row[8]); - if (row[9] != NULL) snprintf(poller_items[i].arg2, sizeof(poller_items[i].arg2), "%s", row[9]); - if (row[10] != NULL) snprintf(poller_items[i].arg3, sizeof(poller_items[i].arg3), "%s", row[10]); - - if (row[11] != NULL) poller_items[i].local_data_id = atoi(row[11]); - - if (row[12] != NULL) poller_items[i].rrd_num = atoi(row[12]); - if (row[13] != NULL) poller_items[i].snmp_port = atoi(row[13]); - if (row[14] != NULL) poller_items[i].snmp_timeout = atoi(row[14]); - - if (row[15] != NULL) snprintf(poller_items[i].snmp_auth_protocol, - sizeof(poller_items[i].snmp_auth_protocol), "%s", row[15]); - if (row[16] != NULL) snprintf(poller_items[i].snmp_priv_passphrase, - sizeof(poller_items[i].snmp_priv_passphrase), "%s", row[16]); - if (row[17] != NULL) snprintf(poller_items[i].snmp_priv_protocol, - sizeof(poller_items[i].snmp_priv_protocol), "%s", row[17]); - if (row[18] != NULL) snprintf(poller_items[i].snmp_context, - sizeof(poller_items[i].snmp_context), "%s", row[18]); - if (row[19] != NULL) snprintf(poller_items[i].snmp_engine_id, - sizeof(poller_items[i].snmp_engine_id), "%s", row[19]); - - if (set.has_output_regex && row[20] != NULL) - snprintf(poller_items[i].output_regex, - sizeof(poller_items[i].output_regex), "%s", row[20]); - - SET_UNDEFINED(poller_items[i].result); - - i++; - } - - /* free the mysql result */ - db_free_result(result); - - /* create an array for snmp oids */ - if (!(snmp_oids = (snmp_oids_t *) calloc(host->max_oids, sizeof(snmp_oids_t)))) { - die("ERROR: Fatal calloc error: poller.c snmp_oids!"); - } - - /* initialize all the memory to insure we don't get issues */ - memset(snmp_oids, 0, sizeof(snmp_oids_t)*host->max_oids); + poll_item_storage_t storage = load_poll_items(result, host, num_rows); + poller_items = storage.items; + snmp_oids = storage.oids; /* log an informative message */ - if (is_debug_device(host_id)) { - SPINE_LOG(("Device[%i] HT[%i] NOTE: There are '%i' Polling Items for this Device", host_id, host_thread, num_rows)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] NOTE: There are '%i' Polling Items for this Device", host_id, host_thread, num_rows)); - } + SPINE_LOG_DEVICE(host_id, POLLER_VERBOSITY_MEDIUM, ("Device[%i] HT[%i] NOTE: There are '%i' Polling Items for this Device", host_id, host_thread, num_rows)); - i = 0; k = 0; - while ((i < num_rows) && (!host->ignore_host)) { - thread_start = get_time_as_double(); + snmp_poll_batch_t batch = {.items = poller_items, .oids = snmp_oids, .errors = &error_context}; + rows_processed = collect_poll_items(host, &batch, num_rows, spike_kill); - switch(poller_items[i].action) { - case POLLER_ACTION_SNMP: /* raw SNMP poll */ - /* initialize or reinitialize snmp as required */ - if (k == 0) { - last_snmp_port = poller_items[i].snmp_port; - last_snmp_version = poller_items[i].snmp_version; - - STRNCOPY(last_snmp_community, poller_items[i].snmp_community); - STRNCOPY(last_snmp_username, poller_items[i].snmp_username); - STRNCOPY(last_snmp_password, poller_items[i].snmp_password); - STRNCOPY(last_snmp_auth_protocol, poller_items[i].snmp_auth_protocol); - STRNCOPY(last_snmp_priv_passphrase, poller_items[i].snmp_priv_passphrase); - STRNCOPY(last_snmp_priv_protocol, poller_items[i].snmp_priv_protocol); - STRNCOPY(last_snmp_context, poller_items[i].snmp_context); - STRNCOPY(last_snmp_engine_id, poller_items[i].snmp_engine_id); - - host->snmp_session = snmp_host_init(host->id, poller_items[i].hostname, - poller_items[i].snmp_version, poller_items[i].snmp_community, - poller_items[i].snmp_username, poller_items[i].snmp_password, - poller_items[i].snmp_auth_protocol, poller_items[i].snmp_priv_passphrase, - poller_items[i].snmp_priv_protocol, poller_items[i].snmp_context, - poller_items[i].snmp_engine_id, - poller_items[i].snmp_port, poller_items[i].snmp_timeout); - - k++; - } - - /* catch snmp initialization issues */ - if (host->snmp_session == NULL) { - host->ignore_host = TRUE; - break; - } - - /* some snmp data changed from poller item to poller item. therefore, poll host and store data */ - if ((last_snmp_port != poller_items[i].snmp_port) || - (last_snmp_version != poller_items[i].snmp_version) || - (poller_items[i].snmp_version < 3 && - (!STRMATCH(last_snmp_community, poller_items[i].snmp_community))) || - (poller_items[i].snmp_version > 2 && - ((!STRMATCH(last_snmp_username, poller_items[i].snmp_username)) || - (!STRMATCH(last_snmp_password, poller_items[i].snmp_password)) || - (!STRMATCH(last_snmp_auth_protocol, poller_items[i].snmp_auth_protocol)) || - (!STRMATCH(last_snmp_priv_passphrase, poller_items[i].snmp_priv_passphrase)) || - (!STRMATCH(last_snmp_priv_protocol, poller_items[i].snmp_priv_protocol)) || - (!STRMATCH(last_snmp_context, poller_items[i].snmp_context)) || - (!STRMATCH(last_snmp_engine_id, poller_items[i].snmp_engine_id))))) { - - if (num_oids > 0) { - snmp_get_multi(host, poller_items, snmp_oids, num_oids); - - for (j = 0; j < num_oids; j++) { - if (host->ignore_host) { - SPINE_LOG(("Device[%i] HT[%i] DS[%i] WARNING: SNMP timeout detected [%i ms], ignoring host '%s'", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, host->snmp_timeout, host->hostname)); - SET_UNDEFINED(snmp_oids[j].result); - } else if (IS_UNDEFINED(snmp_oids[j].result)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", - host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, - host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, - poller_items[snmp_oids[j].array_position].arg1, snmp_oids[j].result)); - } - - /* continue */ - } else if ((is_numeric(snmp_oids[j].result)) || (is_multipart_output(snmp_oids[j].result))) { - /* continue */ - } else if (is_hexadecimal(snmp_oids[j].result, TRUE)) { - if (!poller_store_hex_result(snmp_oids[j].result, RESULTS_BUFFER, snmp_oids[j].result, &errors)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Hexadecimal Response Exceeds 64 Bits, Device[%i] HT[%i] DS[%i]", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id)); - } - } - } else if ((STRIMATCH(snmp_oids[j].result, "U")) || - (STRIMATCH(snmp_oids[j].result, "Nan"))) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", - host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, - host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, - poller_items[snmp_oids[j].array_position].arg1, snmp_oids[j].result)); - } - - /* is valid output, continue */ - } else { - /* trim a non-numeric prefix or suffix, then validate below */ - snprintf(temp_result, RESULTS_BUFFER, "%s", strip_alpha(snmp_oids[j].result)); - snprintf(snmp_oids[j].result , RESULTS_BUFFER, "%s", temp_result); - - /* detect erroneous non-numeric result */ - if (!validate_result(snmp_oids[j].result)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", - host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, - host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, - poller_items[snmp_oids[j].array_position].arg1, snmp_oids[j].result)); - } - - SET_UNDEFINED(snmp_oids[j].result); - } - } - - snprintf(poller_items[snmp_oids[j].array_position].result, RESULTS_BUFFER, "%s", snmp_oids[j].result); - - thread_end = get_time_as_double(); - - if (is_debug_device(host_id)) { - SPINE_LOG(("Device[%i] HT[%i] DS[%i] TT[%.2f] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, (float) ((thread_end - thread_start) * 1000), host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, poller_items[snmp_oids[j].array_position].arg1, poller_items[snmp_oids[j].array_position].result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DS[%i] TT[%.2f] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, (float) ((thread_end - thread_start) * 1000), host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, poller_items[snmp_oids[j].array_position].arg1, poller_items[snmp_oids[j].array_position].result)); - } - } - - /* reset num_snmps */ - num_oids = 0; - - /* initialize all the memory to insure we don't get issues */ - memset(snmp_oids, 0, sizeof(snmp_oids_t)*host->max_oids); - } - - if (host->snmp_session != NULL) { - snmp_host_cleanup(host->snmp_session); - host->snmp_session = NULL; - } - - host->snmp_session = snmp_host_init(host->id, poller_items[i].hostname, - poller_items[i].snmp_version, poller_items[i].snmp_community, - poller_items[i].snmp_username, poller_items[i].snmp_password, - poller_items[i].snmp_auth_protocol, poller_items[i].snmp_priv_passphrase, - poller_items[i].snmp_priv_protocol, poller_items[i].snmp_context, - poller_items[i].snmp_engine_id, - poller_items[i].snmp_port, poller_items[i].snmp_timeout); - - last_snmp_port = poller_items[i].snmp_port; - last_snmp_version = poller_items[i].snmp_version; - - STRNCOPY(last_snmp_community, poller_items[i].snmp_community); - STRNCOPY(last_snmp_username, poller_items[i].snmp_username); - STRNCOPY(last_snmp_password, poller_items[i].snmp_password); - STRNCOPY(last_snmp_auth_protocol, poller_items[i].snmp_auth_protocol); - STRNCOPY(last_snmp_priv_passphrase, poller_items[i].snmp_priv_passphrase); - STRNCOPY(last_snmp_priv_protocol, poller_items[i].snmp_priv_protocol); - STRNCOPY(last_snmp_context, poller_items[i].snmp_context); - STRNCOPY(last_snmp_engine_id, poller_items[i].snmp_engine_id); - } - - if (num_oids >= host->max_oids) { - snmp_get_multi(host, poller_items, snmp_oids, num_oids); - - for (j = 0; j < num_oids; j++) { - if (host->ignore_host) { - SPINE_LOG(("Device[%i] HT[%i] DS[%i] WARNING: SNMP timeout detected [%i ms], ignoring host '%s'", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, host->snmp_timeout, host->hostname)); - SET_UNDEFINED(snmp_oids[j].result); - } else if (IS_UNDEFINED(snmp_oids[j].result)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", - host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, - host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, - poller_items[snmp_oids[j].array_position].arg1, snmp_oids[j].result)); - } - - /* continue */ - } else if ((is_numeric(snmp_oids[j].result)) || (is_multipart_output(snmp_oids[j].result))) { - /* continue */ - } else if (is_hexadecimal(snmp_oids[j].result, TRUE)) { - if (!poller_store_hex_result(snmp_oids[j].result, RESULTS_BUFFER, snmp_oids[j].result, &errors)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Hexadecimal Response Exceeds 64 Bits, Device[%i] HT[%i] DS[%i]", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id)); - } - } - } else if ((STRIMATCH(snmp_oids[j].result, "U")) || - (STRIMATCH(snmp_oids[j].result, "Nan"))) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", - host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, - host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, - poller_items[snmp_oids[j].array_position].arg1, snmp_oids[j].result)); - } - - /* is valid output, continue */ - } else { - /* trim a non-numeric prefix or suffix, then validate below */ - snprintf(temp_result, RESULTS_BUFFER, "%s", strip_alpha(snmp_oids[j].result)); - snprintf(snmp_oids[j].result , RESULTS_BUFFER, "%s", temp_result); - - /* detect erroneous non-numeric result */ - if (!validate_result(snmp_oids[j].result)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", - host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, - host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, - poller_items[snmp_oids[j].array_position].arg1, snmp_oids[j].result)); - } - - SET_UNDEFINED(snmp_oids[j].result); - } - } - - if (strlen(poller_items[snmp_oids[j].array_position].output_regex)) { - snprintf(temp_result, RESULTS_BUFFER, "%s", regex_replace(poller_items[snmp_oids[j].array_position].output_regex, snmp_oids[j].result)); - snprintf(snmp_oids[j].result, RESULTS_BUFFER, "%s", temp_result); - } - - snprintf(poller_items[snmp_oids[j].array_position].result, RESULTS_BUFFER, "%s", snmp_oids[j].result); - - thread_end = get_time_as_double(); - - if (is_debug_device(host_id)) { - SPINE_LOG(("Device[%i] HT[%i] DS[%i] TT[%.2f] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, (float) ((thread_end - thread_start) * 1000), host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, poller_items[snmp_oids[j].array_position].arg1, poller_items[snmp_oids[j].array_position].result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DS[%i] TT[%.2f] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, (float) ((thread_end - thread_start) * 1000), host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, poller_items[snmp_oids[j].array_position].arg1, poller_items[snmp_oids[j].array_position].result)); - } - - if (!IS_UNDEFINED(poller_items[snmp_oids[j].array_position].result)) { - /* insert a NaN in place of the actual value if the snmp agent restarts */ - if ((spike_kill) && (!strstr(poller_items[snmp_oids[j].array_position].result,":"))) { - SET_UNDEFINED(poller_items[snmp_oids[j].array_position].result); - } - } - } - - /* reset num_snmps */ - num_oids = 0; - - /* initialize all the memory to insure we don't get issues */ - memset(snmp_oids, 0, sizeof(snmp_oids_t)*host->max_oids); - } - - snprintf(snmp_oids[num_oids].oid, sizeof(snmp_oids[num_oids].oid), "%s", poller_items[i].arg1); - snmp_oids[num_oids].array_position = i; - num_oids++; - - break; - case POLLER_ACTION_SCRIPT: /* execute script file */ - /* Reject empty script commands that could cause unexpected behavior */ - if (strlen(poller_items[i].arg1) == 0) { - SPINE_LOG(("WARNING: Device[%i] HT[%i] DS[%i] empty script command, skipping", - host_id, host_thread, poller_items[i].local_data_id)); - SET_UNDEFINED(poller_items[i].result); - break; - } - - poll_result = exec_poll(host, poller_items[i].arg1, poller_items[i].local_data_id, "DS"); - - /* process the result */ - if (IS_UNDEFINED(poll_result)) { - SET_UNDEFINED(poller_items[i].result); - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[i].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SCRIPT: %s, output: %s", - host_id, host_thread, poller_items[i].local_data_id, - poller_items[i].arg1, poller_items[i].result)); - } - } else if ((is_numeric(poll_result)) || (is_multipart_output(trim(poll_result)))) { - snprintf(poller_items[i].result, RESULTS_BUFFER, "%s", poll_result); - } else if (is_hexadecimal(poll_result, TRUE)) { - if (!poller_store_hex_result(poller_items[i].result, RESULTS_BUFFER, poll_result, &errors)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[i].local_data_id, false); - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Hexadecimal Response Exceeds 64 Bits, Device[%i] HT[%i] DS[%i] SCRIPT: %s", host_id, host_thread, poller_items[i].local_data_id, poller_items[i].arg1)); - } - } - } else { - /* trim a non-numeric prefix or suffix, then validate below */ - snprintf(temp_result, RESULTS_BUFFER, "%s", strip_alpha(poll_result)); - snprintf(poller_items[i].result , RESULTS_BUFFER, "%s", temp_result); - - /* detect erroneous result. can be non-numeric */ - if (!validate_result(poller_items[i].result)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[i].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SCRIPT: %s, output: %s", - host_id, host_thread, poller_items[i].local_data_id, - poller_items[i].arg1, poller_items[i].result)); - } - - SET_UNDEFINED(poller_items[i].result); - } - } - - SPINE_FREE(poll_result); - - thread_end = get_time_as_double(); - - if (is_debug_device(host_id)) { - SPINE_LOG(("Device[%i] HT[%i] DS[%i] TT[%.2f] SCRIPT: %s, output: %s", host_id, host_thread, poller_items[i].local_data_id, (float) ((thread_end - thread_start) * 1000), poller_items[i].arg1, poller_items[i].result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DS[%i] TT[%.2f] SCRIPT: %s, output: %s", host_id, host_thread, poller_items[i].local_data_id, (float) ((thread_end - thread_start) * 1000), poller_items[i].arg1, poller_items[i].result)); - } - - if (!IS_UNDEFINED(poller_items[i].result)) { - /* insert a NaN in place of the actual value if the snmp agent restarts */ - if ((spike_kill) && (!strstr(poller_items[i].result,":"))) { - SET_UNDEFINED(poller_items[i].result); - } - } - - break; - case POLLER_ACTION_PHP_SCRIPT_SERVER: /* execute script server */ - /* Reject empty script commands that could cause unexpected behavior */ - if (strlen(poller_items[i].arg1) == 0) { - SPINE_LOG(("WARNING: Device[%i] HT[%i] DS[%i] empty script server command, skipping", - host_id, host_thread, poller_items[i].local_data_id)); - SET_UNDEFINED(poller_items[i].result); - break; - } - - php_process = php_get_process(); - - poll_result = php_cmd(poller_items[i].arg1, php_process); - - /* process the output */ - if (IS_UNDEFINED(poll_result)) { - SET_UNDEFINED(poller_items[i].result); - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[i].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SCRIPT: %s, output: %s", - host_id, host_thread, poller_items[i].local_data_id, - poller_items[i].arg1, poller_items[i].result)); - } - } else if ((is_numeric(poll_result)) || (is_multipart_output(trim(poll_result)))) { - snprintf(poller_items[i].result, RESULTS_BUFFER, "%s", poll_result); - } else if (is_hexadecimal(poll_result, TRUE)) { - if (!poller_store_hex_result(poller_items[i].result, RESULTS_BUFFER, poll_result, &errors)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[i].local_data_id, false); - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Hexadecimal Response Exceeds 64 Bits, Device[%i] HT[%i] DS[%i] SCRIPT: %s", host_id, host_thread, poller_items[i].local_data_id, poller_items[i].arg1)); - } - } - } else { - /* trim a non-numeric prefix or suffix, then validate below */ - snprintf(temp_result, RESULTS_BUFFER, "%s", strip_alpha(poll_result)); - snprintf(poller_items[i].result , RESULTS_BUFFER, "%s", temp_result); - - /* detect erroneous result. can be non-numeric */ - if (!validate_result(poller_items[i].result)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[i].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SCRIPT: %s, output: %s", - host_id, host_thread, poller_items[i].local_data_id, - poller_items[i].arg1, poller_items[i].result)); - } - - SET_UNDEFINED(poller_items[i].result); - } - } - - SPINE_FREE(poll_result); - - thread_end = get_time_as_double(); - - if (is_debug_device(host_id)) { - SPINE_LOG(("Device[%i] HT[%i] DS[%i] TT[%.2f] SS[%i] SERVER: %s, output: %s", host_id, host_thread, poller_items[i].local_data_id, (float) ((thread_end - thread_start) * 1000), php_process, poller_items[i].arg1, poller_items[i].result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DS[%i] TT[%.2f] SS[%i] SERVER: %s, output: %s", host_id, host_thread, poller_items[i].local_data_id, (float) ((thread_end - thread_start) * 1000), php_process, poller_items[i].arg1, poller_items[i].result)); - } - - if (!IS_UNDEFINED(poller_items[i].result)) { - /* insert a NaN in place of the actual value if the snmp agent restarts */ - if ((spike_kill) && (!strstr(poller_items[i].result,":"))) { - SET_UNDEFINED(poller_items[i].result); - } - } - - break; - default: /* unknown action, generate error */ - SPINE_LOG(("Device[%i] HT[%i] DS[%i] ERROR: Unknown Poller Action: %s", host_id, host_thread, poller_items[i].local_data_id, poller_items[i].arg1)); - - break; - } - - i++; - rows_processed++; - } - - /* process last multi-get request if applicable */ - if (num_oids > 0) { - snmp_get_multi(host, poller_items, snmp_oids, num_oids); - - for (j = 0; j < num_oids; j++) { - if (host->ignore_host) { - SPINE_LOG(("Device[%i] HT[%i] DS[%i] WARNING: SNMP timeout detected [%i ms], ignoring host '%s'", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, host->snmp_timeout, host->hostname)); - SET_UNDEFINED(snmp_oids[j].result); - } else if (IS_UNDEFINED(snmp_oids[j].result)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", - host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, host->snmp_version, - host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, - poller_items[snmp_oids[j].array_position].arg1, snmp_oids[j].result)); - } - - /* continue */ - } else if ((is_numeric(snmp_oids[j].result)) || (is_multipart_output(snmp_oids[j].result))) { - /* continue */ - } else if (is_hexadecimal(snmp_oids[j].result, TRUE)) { - if (!poller_store_hex_result(snmp_oids[j].result, RESULTS_BUFFER, snmp_oids[j].result, &errors)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Hexadecimal Response Exceeds 64 Bits, Device[%i] HT[%i] DS[%i]", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id)); - } - } - } else if ((STRIMATCH(snmp_oids[j].result, "U")) || - (STRIMATCH(snmp_oids[j].result, "Nan"))) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", - host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, host->snmp_version, - host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, - poller_items[snmp_oids[j].array_position].arg1, snmp_oids[j].result)); - } - - /* is valid output, continue */ - } else { - /* trim a non-numeric prefix or suffix, then validate below */ - snprintf(temp_result, RESULTS_BUFFER, "%s", strip_alpha(snmp_oids[j].result)); - snprintf(snmp_oids[j].result , RESULTS_BUFFER, "%s", temp_result); - - /* detect erroneous non-numeric result */ - if (!validate_result(snmp_oids[j].result)) { - buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, false); - errors++; - - if (set.spine_log_level == 2) { - SPINE_LOG(("WARNING: Invalid Response, Device[%i] HT[%i] DS[%i] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", - host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, host->snmp_version, - host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, - poller_items[snmp_oids[j].array_position].arg1, snmp_oids[j].result)); - } - - SET_UNDEFINED(snmp_oids[j].result); - } - } - - if (strlen(poller_items[snmp_oids[j].array_position].output_regex)) { - snprintf(temp_result, RESULTS_BUFFER, "%s", regex_replace(poller_items[snmp_oids[j].array_position].output_regex, snmp_oids[j].result)); - snprintf(snmp_oids[j].result, RESULTS_BUFFER, "%s", temp_result); - } - - snprintf(poller_items[snmp_oids[j].array_position].result, RESULTS_BUFFER, "%s", snmp_oids[j].result); - - thread_end = get_time_as_double(); - - if (is_debug_device(host_id)) { - SPINE_LOG(("Device[%i] HT[%i] DS[%i] TT[%.2f] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, (float) ((thread_end - thread_start) * 1000), host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, poller_items[snmp_oids[j].array_position].arg1, poller_items[snmp_oids[j].array_position].result)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] DS[%i] TT[%.2f] SNMP: v%i: %s, dsname: %s, oid: %s, value: %s", host_id, host_thread, poller_items[snmp_oids[j].array_position].local_data_id, (float) ((thread_end - thread_start) * 1000), host->snmp_version, host->hostname, poller_items[snmp_oids[j].array_position].rrd_name, poller_items[snmp_oids[j].array_position].arg1, poller_items[snmp_oids[j].array_position].result)); - } - - if (!IS_UNDEFINED(poller_items[snmp_oids[j].array_position].result)) { - /* insert a NaN in place of the actual value if the snmp agent restarts */ - if ((spike_kill) && (!strstr(poller_items[snmp_oids[j].array_position].result,":"))) { - SET_UNDEFINED(poller_items[snmp_oids[j].array_position].result); - } - } - } - } - - buf_length = MAX_MYSQL_BUF_SIZE+RESULTS_BUFFER; - - /* insert the query results into the database */ - if (!(query3 = (char *)malloc(buf_length))) { - die("ERROR: Fatal malloc error: poller.c query3 output buffer!"); - } - - /* set zeros */ - memset(query3, 0, buf_length); - - /* append data */ - strncat(query3, query8, query8_len); - - out_buffer = strlen(query3); - - if (set.boost_redirect && set.boost_enabled) { - /* insert the query results into the database */ - if (!(query12 = (char *)malloc(buf_length))) { - die("ERROR: Fatal malloc error: poller.c query12 boost output buffer!"); - } - - /* set zeros */ - memset(query12, 0, buf_length); - - /* append data */ - strncat(query12, query11, query11_len); - } - - int mode; - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { - SPINE_LOG_DEBUG(("DEBUG: Setting up writes to remote database")); - mysqlt = mysqlr; - mode = REMOTE; - } else { - SPINE_LOG_DEBUG(("DEBUG: Setting up writes to local database")); - mysqlt = mysql; - mode = LOCAL; - } - - /* set.dbonupdate describes the local connection, while mysqlt can be - * remote. VALUES() is accepted by both MySQL and MariaDB, so use it - * until the remote server has its own version capability flag (#590). */ - snprintf(posuffix, sizeof(posuffix), - " ON DUPLICATE KEY UPDATE output=VALUES(output)"); - posuffix_len = strlen(posuffix); - - i = 0; - while (i < rows_processed) { - /* poller_items[i].result is RESULTS_BUFFER bytes and escaping can - * double it, so size the destination to match rather than the - * fixed DBL_BUFSIZE, which silently truncated any result of 1024 - * bytes or more regardless of --with-results-buffer. */ - char escaped_result[RESULTS_BUFFER * 2 + 1]; - char escaped_rrd_name[DBL_BUFSIZE]; - - db_escape(&mysqlt, escaped_result, sizeof(escaped_result), poller_items[i].result); - db_escape(&mysqlt, escaped_rrd_name, sizeof(escaped_rrd_name), poller_items[i].rrd_name); - - if (!format_poller_output_row(result_string, sizeof(result_string), - poller_items[i].local_data_id, - escaped_rrd_name, - host_time, - escaped_result)) { - SPINE_LOG(("Device[%i] HT[%i] ERROR: Poller output for DS[%i] " - "exceeds the configured result buffer and was skipped", - host_id, host_thread, poller_items[i].local_data_id)); - i++; - continue; - } - - result_length = strlen(result_string); - - /* if the next element to the buffer will overflow it, write to the database */ - if ((out_buffer + result_length) >= MAX_MYSQL_BUF_SIZE) { - /* append the suffix */ - strncat(query3, posuffix, posuffix_len); - - /* insert the record */ - db_insert(&mysqlt, mode, query3); - - /* re-initialize the query buffer */ - memset(query3, 0, MAX_MYSQL_BUF_SIZE+RESULTS_BUFFER); - - strncat(query3, query8, query8_len); - - /* insert the record for boost */ - if (set.boost_redirect && set.boost_enabled) { - /* append the suffix */ - strncat(query12, posuffix, posuffix_len); - - db_insert(&mysqlt, mode, query12); - - memset(query12, 0, MAX_MYSQL_BUF_SIZE+RESULTS_BUFFER); - - strncat(query12, query11, query11_len); - } - - /* reset the output buffer length */ - out_buffer = strlen(query3); - - /* set binary, let the system know we are a new buffer */ - new_buffer = TRUE; - } - - /* if this is our first pass, or we just outputted to the database, need to change the delimiter */ - if (new_buffer) { - result_string[0] = ' '; - } else { - result_string[0] = ','; - } - - strncat(query3, result_string, result_length); - - if (set.boost_redirect && set.boost_enabled) { - strncat(query12, result_string, result_length); - } - - out_buffer = out_buffer + strlen(result_string); - new_buffer = FALSE; - i++; - } - - /* perform the last insert if there is data to process */ - if (out_buffer > strlen(query8)) { - /* append the suffix */ - strncat(query3, posuffix, posuffix_len); - - /* insert records into database */ - db_insert(&mysqlt, mode, query3); - - /* insert the record for boost */ - if (set.boost_redirect && set.boost_enabled) { - /* append the suffix */ - strncat(query12, posuffix, posuffix_len); - - db_insert(&mysqlt, mode, query12); - } - } + poll_output_buffers_t output = write_poll_results(mysql, mysqlr, &queries, poller_items, rows_processed, host_time); + query3 = output.output; + query12 = output.boost; + output_failed = output.failed; /* cleanup memory and prepare for function exit */ - if (host->snmp_session != NULL) { - snmp_host_cleanup(host->snmp_session); - host->snmp_session = NULL; + if (host->snmp.session != NULL) { + snmp_host_cleanup(host->snmp.session); + host->snmp.session = NULL; } SPINE_FREE(query3); - if (set.boost_redirect && set.boost_enabled) { + if (query12 != NULL) { SPINE_FREE(query12); } @@ -2111,78 +1514,15 @@ void poll_host(int device_counter, int host_id, int host_thread, int host_thread SPINE_FREE(reindex); SPINE_FREE(ping); - /* update poller_items table for next polling interval */ - if (host_thread == host_threads && set.active_profiles != 1) { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] Updating Poller Items for Next Poll", host_id, host_thread)); + complete_poll_host(mysql, &queries, work, &error_context, output_failed, poll_time); - db_query(&mysql, LOCAL, query6); - } - - /* record the polling time for the device */ - poll_time = get_time_as_double() - poll_time; - if (is_debug_device(host_id)) { - SPINE_LOG(("Device[%i] HT[%i] Total Time: %0.2g Seconds", host_id, host_thread, poll_time)); - } else { - SPINE_LOG_MEDIUM(("Device[%i] HT[%i] Total Time: %0.2g Seconds", host_id, host_thread, poll_time)); - } - - /* record the total time for the host */ - thread_mutex_lock(LOCK_THDET); - details[device_counter]->threads_complete++; - if (details[device_counter]->threads_complete == details[device_counter]->host_threads) { - details[device_counter]->complete = TRUE; - - poll_time = get_time_as_double(); - query1[0] = '\0'; - snprintf(query1, BUFSIZE, "UPDATE host SET polling_time = %.3f - %.3f WHERE id = %i", poll_time, host_time_double, host_id); - db_query(&mysql, LOCAL, query1); - - } - - if (errors > 0) { - int error_query_len = strlen(error_string) + BUFSIZE; - char *error_query; - if (!(error_query = (char *)malloc(error_query_len))) { - die("ERROR: Fatal malloc error: poller.c error_query!"); - } - - snprintf(error_query, error_query_len, "INSERT INTO host_errors (host_id, poller_id, errors, local_data_ids)" - " VALUES(%i, %i, %i, '%s')" - " ON DUPLICATE KEY UPDATE" - " errors = errors + VALUES(errors)," - " local_data_ids = CONCAT(local_data_ids, ', ', VALUES(local_data_ids))", - host_id, set.poller_id, errors, error_string); - - db_query(&mysql, LOCAL, error_query); - - free(error_query); - } - - thread_mutex_unlock(LOCK_THDET); - - if (local_cnn != NULL) { - db_release_connection(LOCAL, local_cnn->id); - } else { - SPINE_LOG(("WARNING: Device[%i] HT[%i] Trying to close uninitialized local connection.", host_id, host_thread)); - } - - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { - if (remote_cnn != NULL) { - db_release_connection(REMOTE, remote_cnn->id); - } else { - SPINE_LOG(("WARNING: Device[%i] HT[%i] Trying to close uninitialized remote connection.", host_id, host_thread)); - } - } + release_poll_connections(work, local_cnn, remote_cnn); mysql_thread_end(); - if (is_debug_device(host_id)) { - SPINE_LOG(("Device[%i] HT[%i] DEBUG: HOST COMPLETE: About to Exit Device Polling Thread Function", host_id, host_thread)); - } else { - SPINE_LOG_DEBUG(("Device[%i] HT[%i] DEBUG: HOST COMPLETE: About to Exit Device Polling Thread Function", host_id, host_thread)); - } + SPINE_LOG_DEVICE(host_id, POLLER_VERBOSITY_DEBUG, ("Device[%i] HT[%i] DEBUG: HOST COMPLETE: About to Exit Device Polling Thread Function", host_id, host_thread)); - if (set.spine_log_level == 1) { + if (set.logging.spine_log_level == 1) { buffer_output_errors(error_string, buf_size, buf_errors, host_id, host_thread, 0, true); } @@ -2210,7 +1550,7 @@ void buffer_output_errors(char *error_string, int *buf_size, int *buf_errors, in SPINE_LOG(("WARNING: Invalid Response(s), Errors[%i] Device[%i] Thread[%i] DS[%s]", *buf_errors, device_id, thread_id, error_string)); } else if (!flush) { snprintf(tbuffer, SMALL_BUFSIZE, *buf_errors > 0 ? ", %i" : "%i", local_data_id); - error_len = strlen(tbuffer); + error_len = spine_count_to_int(strlen(tbuffer)); if (*buf_size + error_len >= DBL_BUFSIZE) { SPINE_LOG(("WARNING: Invalid Response(s), Errors[%i] Device[%i] Thread[%i] DS[%s]", *buf_errors, device_id, thread_id, error_string)); *buf_errors = 1; @@ -2223,7 +1563,7 @@ void buffer_output_errors(char *error_string, int *buf_size, int *buf_errors, in } } -/*! \fn int is_multipart_output(char *result) +/*! \fn int is_multipart_output(const char *result) * \brief validates the output syntax is a valid name value pair syntax * \param result the value to be checked for legality * @@ -2233,146 +1573,63 @@ void buffer_output_errors(char *error_string, int *buf_size, int *buf_errors, in * \return TRUE if the result is valid, otherwise FALSE. * */ -int is_multipart_output(char *result) { - int space_cnt = 0; - int delim_cnt = 0; - int i; +int is_multipart_output(const char *result) { + if (result == NULL) return FALSE; + if (strchr(result, ':') == NULL && strchr(result, '!') == NULL) return FALSE; + if (strchr(result, ' ') == NULL) return TRUE; - /* check the easy cases first */ - if (result) { - /* it must have delimiters */ - if ((strstr(result, ":")) || (strstr(result, "!"))) { - if (!strstr(result, " ")) { - return TRUE; - } else { - const int len = strlen(result); - - for (i=0; iid)); + poll_result = snmp_get_allow_fail(host, ".1.3.6.1.2.1.1.3.0"); + SPINE_LOG_DEVDBG(("DEVDGB: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.3.0'); [complete]", host->id)); + if (poll_result && is_numeric(poll_result)) { + host->system.snmp_sysUpTimeInstance = atoll(poll_result); + SPINE_FREE(poll_result); + + // Attempt to get the more modern version + SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.6.3.10.2.1.3.0');", host->id)); + poll_result = snmp_get_allow_fail(host, ".1.3.6.1.6.3.10.2.1.3.0"); + SPINE_LOG_DEVDBG(("DEVDGB: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.6.3.10.2.1.3.0'); [complete]", host->id)); + + if (poll_result && is_numeric(poll_result)) { + host->system.snmp_sysUpTimeInstance = atoll(poll_result) * 100; + } + } + SPINE_FREE(poll_result); +} + +static void poll_system_field(host_t *host, MYSQL *mysql, char *oid, char *destination, size_t capacity) { + SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] requesting system OID %s", host->id, oid)); + char *result = snmp_get_allow_fail(host, oid); + if (result != NULL) db_escape(mysql, destination, (int)capacity, result); + SPINE_FREE(result); +} + +void get_system_information(host_t *host, MYSQL *mysql, int system) { SPINE_LOG_MEDIUM(("Device[%d] Checking for System Information Update", host->id)); - - if (set.mibs || system) { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%d] Updating Full System Information Table", host->id)); - } else { - SPINE_LOG_MEDIUM(("Device[%d] Updating Full System Information Table", host->id)); - } - - SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.1.0');", host->id)); - poll_result = snmp_get_allow_fail(host, ".1.3.6.1.2.1.1.1.0"); - SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.1.0'); [complete]", host->id)); - - if (poll_result) { - db_escape(mysql, host->snmp_sysDescr, sizeof(host->snmp_sysDescr), poll_result); - SPINE_FREE(poll_result); - } - - SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.2.0');", host->id)); - poll_result = snmp_get_allow_fail(host, ".1.3.6.1.2.1.1.2.0"); - SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.2.0'); [complete]", host->id)); - - if (poll_result) { - db_escape(mysql, host->snmp_sysObjectID, sizeof(host->snmp_sysObjectID), poll_result); - SPINE_FREE(poll_result); - } - - // Get the legacy system uptime instance first - SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.3.0');", host->id)); - poll_result = snmp_get_allow_fail(host, ".1.3.6.1.2.1.1.3.0"); - SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.3.0'); [complete]", host->id)); - - if (poll_result && is_numeric(poll_result)) { - host->snmp_sysUpTimeInstance = atoll(poll_result); - SPINE_FREE(poll_result); - - // Attempt to get the more modern version - SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.6.3.10.2.1.3.0');", host->id)); - poll_result = snmp_get_allow_fail(host, ".1.3.6.1.6.3.10.2.1.3.0"); - SPINE_LOG_DEVDBG(("DEVDGB: Device[%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.6.3.10.2.1.3.0'); [complete]", host->id)); - - if (poll_result && is_numeric(poll_result)) { - host->snmp_sysUpTimeInstance = atoll(poll_result) * 100; - snprintf(poll_result, BUFSIZE, "%llu", host->snmp_sysUpTimeInstance); - } - - SPINE_FREE(poll_result); - } - - SPINE_LOG_DEVDBG(("DEVDBG: Device [%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.4.0');", host->id)); - poll_result = snmp_get_allow_fail(host, ".1.3.6.1.2.1.1.4.0"); - SPINE_LOG_DEVDBG(("DEVDBG: Device [%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.4.0'); [complete]", host->id)); - - if (poll_result) { - db_escape(mysql, host->snmp_sysContact, sizeof(host->snmp_sysContact), poll_result); - SPINE_FREE(poll_result); - } - - SPINE_LOG_DEVDBG(("DEVDBG: Device [%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.5.0');", host->id)); - poll_result = snmp_get_allow_fail(host, ".1.3.6.1.2.1.1.5.0"); - SPINE_LOG_DEVDBG(("DEVDBG: Device [%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.5.0'); [complete]", host->id)); - - if (poll_result) { - db_escape(mysql, host->snmp_sysName, sizeof(host->snmp_sysName), poll_result); - SPINE_FREE(poll_result); - } - - SPINE_LOG_DEVDBG(("DEVDBG: Device [%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.6.0');", host->id)); - poll_result = snmp_get_allow_fail(host, ".1.3.6.1.2.1.1.6.0"); - SPINE_LOG_DEVDBG(("DEVDBG: Device [%d] poll_result = snmp_get_allow_fail(host, '.1.3.6.1.2.1.1.6.0'); [complete]", host->id)); - - if (poll_result) { - db_escape(mysql, host->snmp_sysLocation, sizeof(host->snmp_sysLocation), poll_result); - SPINE_FREE(poll_result); - } - } else { - if (is_debug_device(host->id)) { - SPINE_LOG(("Device[%d] Updating Short System Information Table", host->id)); - } else { - SPINE_LOG_MEDIUM(("Device[%d] Updating Short System Information Table", host->id)); - } - - // Get the legacy system uptime instance first - SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] poll_result = snmp_get(host, '.1.3.6.1.2.1.1.3.0');", host->id)); - poll_result = snmp_get(host, ".1.3.6.1.2.1.1.3.0"); - SPINE_LOG_DEVDBG(("DEVDGB: Device[%d] poll_result = snmp_get(host, '.1.3.6.1.2.1.1.3.0'); [complete]", host->id)); - - if (poll_result && is_numeric(poll_result)) { - host->snmp_sysUpTimeInstance = atoll(poll_result); - SPINE_FREE(poll_result); - - // Attempt to get the more modern version - SPINE_LOG_DEVDBG(("DEVDBG: Device[%d] poll_result = snmp_get(host, '.1.3.6.1.6.3.10.2.1.3.0');", host->id)); - poll_result = snmp_get(host, ".1.3.6.1.6.3.10.2.1.3.0"); - SPINE_LOG_DEVDBG(("DEVDGB: Device[%d] poll_result = snmp_get(host, '.1.3.6.1.6.3.10.2.1.3.0'); [complete]", host->id)); - - if (poll_result && is_numeric(poll_result)) { - host->snmp_sysUpTimeInstance = atoll(poll_result) * 100; - snprintf(poll_result, BUFSIZE, "%llu", host->snmp_sysUpTimeInstance); - } - - SPINE_FREE(poll_result); - } + bool full = set.snmp.mibs || system; + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_MEDIUM, ("Device[%d] Updating %s System Information Table", host->id, full ? "Full" : "Short")); + if (full) { + poll_system_field(host, mysql, ".1.3.6.1.2.1.1.1.0", host->system.snmp_sysDescr, sizeof(host->system.snmp_sysDescr)); + poll_system_field(host, mysql, ".1.3.6.1.2.1.1.2.0", host->system.snmp_sysObjectID, sizeof(host->system.snmp_sysObjectID)); + } + poll_system_uptime(host); + if (full) { + poll_system_field(host, mysql, ".1.3.6.1.2.1.1.4.0", host->system.snmp_sysContact, sizeof(host->system.snmp_sysContact)); + poll_system_field(host, mysql, ".1.3.6.1.2.1.1.5.0", host->system.snmp_sysName, sizeof(host->system.snmp_sysName)); + poll_system_field(host, mysql, ".1.3.6.1.2.1.1.6.0", host->system.snmp_sysLocation, sizeof(host->system.snmp_sysLocation)); } } @@ -2403,7 +1660,77 @@ int validate_result(char *result) { return FALSE; } -/*! \fn char *exec_poll(host_t *current_host, char *command, int id, const char *type) +static int acquire_script_permit(const host_t *host) { + if (set.php.script_timeout <= 0) return EINVAL; + /* Preserve the existing retry budget without signed multiplication overflow. */ + uint64_t attempts = (uint64_t)set.php.script_timeout * 15; + int error = EAGAIN; + for (uint64_t retry = 1; retry < attempts; retry++) { + error = spine_permits_try_acquire(&available_scripts); + if (error == 0) return 0; + if (error == EAGAIN || error == EWOULDBLOCK) { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_DEVDBG, ("Device[%i] DEBUG: Pausing as unable to obtain a script execution lock", host->id)); + } else { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_DEVDBG, ("Device[%i] DEBUG: Pausing as error %d whilst obtaining a script execution lock", host->id, error)); + } + spine_sleep_usec(10000); + } + return error; +} + +typedef struct { + const host_t *host; + const char *command; + int id; + const char *type; +} script_result_context_t; + +/* Return whether the legacy popen backend may close without blocking on a + * timed-out command; nft_popen owns and reaps its registered child instead. */ +static bool read_script_result(const script_result_context_t *context, int fd, double deadline, char *result) { + int ready = spine_wait_readable(fd, deadline); + if (ready < 0) { + switch (errno) { + case EBADF: + SPINE_LOG(("Device[%i] ERROR: One or more of the file descriptor sets specified a file descriptor that is not a valid open file descriptor.", context->host->id)); + break; + case EINVAL: + SPINE_LOG(("Device[%i] ERROR: Possible invalid timeout specified in select() statement.", context->host->id)); + break; + default: + SPINE_LOG(("Device[%i] ERROR: The script/command select() failed", context->host->id)); + break; + } + SET_UNDEFINED(result); + return FALSE; + } + if (ready == 0) { + SPINE_LOG_MEDIUM(("Device[%i] ERROR: The NIFTY POPEN timed out", context->host->id)); + int pid = nft_pchild(fd); + if (pid > 1) { + kill(pid, SIGKILL); + } else { + SPINE_LOG(("Device[%i] ERROR: Unable to find the timed-out POPEN child", context->host->id)); + } + SET_UNDEFINED(result); + return FALSE; + } + /* Preserve the single-read response contract; later output is ignored. */ + ssize_t bytes = read(fd, result, RESULTS_BUFFER - 1); + if (bytes > 0 && bytes < RESULTS_BUFFER) { + result[bytes] = '\0'; + } else { + if (STRIMATCH(context->type, "DS")) { + SPINE_LOG(("Device[%i] DS[%i] ERROR: Empty result [%s]: '%s'", context->host->id, context->id, context->host->hostname, context->command)); + } else { + SPINE_LOG(("Device[%i] DQ[%i] ERROR: Empty result [%s]: '%s'", context->host->id, context->id, context->host->hostname, context->command)); + } + SET_UNDEFINED(result); + } + return TRUE; +} + +/*! \fn char *exec_poll(host_t *current_host, char *command, int id, char *type) * \brief polls a host using a script * \param current_host a pointer to the current host structure * \param command the command to be executed @@ -2420,15 +1747,7 @@ int validate_result(char *result) { * (the Cacti database). Do not pass user-controlled input directly. */ char *exec_poll(host_t *current_host, char *command, int id, const char *type) { int cmd_fd; - int pid; - - int bytes_read; - fd_set fds; - double begin_time = 0; - double end_time = 0; - double script_timeout; - double remaining_usec = 0; - struct timeval timeout; + double deadline; char *proc_command; char *result_string; @@ -2446,16 +1765,9 @@ char *exec_poll(host_t *current_host, char *command, int id, const char *type) { /* set zeros */ memset(result_string, 0, RESULTS_BUFFER); - /* set script timeout as double */ - script_timeout = set.script_timeout; - - /* establish timeout of 25 seconds for pipe response */ - timeout.tv_sec = set.script_timeout; - timeout.tv_usec = 0; /* don't run too many scripts, operating systems do not like that. */ - int retries = 0; - int sem_err = 0; + int sem_err; int needs_cleanup = 0; /* used for checking executable status */ @@ -2464,49 +1776,30 @@ char *exec_poll(host_t *current_host, char *command, int id, const char *type) { pthread_cleanup_push(child_cleanup_script, NULL); - // use the script server timeout value, allow for 50% leeway - while (++retries < (set.script_timeout * 15)) { - sem_err = spine_sem_trywait(&available_scripts); - if (sem_err == 0) { - break; - } else { - int sem_errno = errno; - - if (sem_errno == EAGAIN || sem_errno == EWOULDBLOCK) { - if (is_debug_device(current_host->id)) { - SPINE_LOG(("Device[%i] DEBUG: Pausing as unable to obtain a script execution lock", current_host->id)); - } else { - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: Pausing as unable to obtain a script execution lock", current_host->id)); - } - } else { - if (is_debug_device(current_host->id)) { - SPINE_LOG(("Device[%i] DEBUG: Pausing as error %d whilst obtaining a script execution lock", current_host->id, sem_errno)); - } else { - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: Pausing as error %d whilst obtaining a script execution lock", current_host->id, sem_errno)); - } - } - } - usleep(10000); - } + sem_err = acquire_script_permit(current_host); if (sem_err) { - SPINE_LOG(("ERROR: Device[%i]: Failed to obtain a script execution lock within 30 seconds", current_host->id)); + SPINE_LOG(("ERROR: Device[%i]: Failed to obtain a script execution lock (error %d)", current_host->id, sem_err)); + SET_UNDEFINED(result_string); + #if defined(__CYGWIN__) + SPINE_FREE(proc_command); + #endif } else { /* Mark for cleanup */ needs_cleanup = 1; /* record start time */ - begin_time = get_time_as_double(); + deadline = spine_monotonic_time() + set.php.script_timeout; /* peel the executable from the command */ saveptr = proc_command; - snprintf(executable, BUFSIZE, "%s", proc_command); + spine_snprintf(executable, sizeof(executable), "%s", proc_command); strtok_r(executable, " ", &saveptr); /* cheesy little hack to add /usr/bin/ if its not included */ if (strstr(executable, "/") == NULL) { saveptr = proc_command; - snprintf(executable, BUFSIZE, "/usr/bin/%s", proc_command); + spine_snprintf(executable, sizeof(executable), "/usr/bin/%s", proc_command); strtok_r(executable, " ", &saveptr); } @@ -2514,93 +1807,11 @@ char *exec_poll(host_t *current_host, char *command, int id, const char *type) { if (access(executable, X_OK | F_OK) != -1) { cmd_fd = nft_popen(proc_command, "r"); - if (is_debug_device(current_host->id)) { - SPINE_LOG(("Device[%i] DEBUG: The NIFTY POPEN returned the following File Descriptor %i", current_host->id, cmd_fd)); - } else { - SPINE_LOG_DEBUG(("Device[%i] DEBUG: The NIFTY POPEN returned the following File Descriptor %i", current_host->id, cmd_fd)); - } + SPINE_LOG_DEVICE(current_host->id, POLLER_VERBOSITY_DEBUG, ("Device[%i] DEBUG: The NIFTY POPEN returned the following File Descriptor %i", current_host->id, cmd_fd)); - if (cmd_fd > 0) { - retry: - - /* Initialize File Descriptors to Review for Input/Output */ - FD_ZERO(&fds); - FD_SET(cmd_fd, &fds); - - /* wait x seconds for pipe response */ - switch (select(FD_SETSIZE, &fds, NULL, NULL, &timeout)) { - case -1: - switch (errno) { - case EBADF: - SPINE_LOG(("Device[%i] ERROR: One or more of the file descriptor sets specified a file descriptor that is not a valid open file descriptor.", current_host->id)); - SET_UNDEFINED(result_string); - - break; - case EINTR: - #ifndef SOLAR_THREAD - /* take a moment */ - usleep(2000); - #endif - - /* record end time */ - end_time = get_time_as_double(); - - /* re-establish new timeout value */ - timeout.tv_sec = rint(floor(script_timeout-(end_time-begin_time))); - remaining_usec = set.script_timeout - timeout.tv_sec - (end_time - begin_time); - - if (remaining_usec > 0) { - timeout.tv_usec = rint(remaining_usec * 1000000); - } else { - timeout.tv_usec = 0; - } - timeout.tv_sec = rint(floor(script_timeout-(end_time-begin_time))); - timeout.tv_usec = rint((script_timeout-(end_time-begin_time)-timeout.tv_sec)*1000000); - - if (timeout.tv_sec + timeout.tv_usec > 0) { - goto retry; - } else { - SPINE_LOG(("WARNING: A script timed out while processing EINTR's.")); - SET_UNDEFINED(result_string); - } - break; - case EINVAL: - SPINE_LOG(("Device[%i] ERROR: Possible invalid timeout specified in select() statement.", current_host->id)); - SET_UNDEFINED(result_string); - break; - default: - SPINE_LOG(("Device[%i] ERROR: The script/command select() failed", current_host->id)); - SET_UNDEFINED(result_string); - break; - } - - break; - case 0: - SPINE_LOG_MEDIUM(("Device[%i] ERROR: The NIFTY POPEN timed out", current_host->id)); - - pid = nft_pchild(cmd_fd); - if (pid > 0) { - kill(pid, SIGKILL); - } else { - SPINE_LOG(("Device[%i] ERROR: Unable to find the timed-out POPEN child", current_host->id)); - } - - SET_UNDEFINED(result_string); - break; - default: - /* get only one line of output, we will ignore the rest */ - bytes_read = read(cmd_fd, result_string, RESULTS_BUFFER-1); - if (bytes_read > 0) { - result_string[bytes_read] = '\0'; - } else { - if (STRIMATCH(type,"DS")) { - SPINE_LOG(("Device[%i] DS[%i] ERROR: Empty result [%s]: '%s'", current_host->id, id, current_host->hostname, command)); - } else { - SPINE_LOG(("Device[%i] DQ[%i] ERROR: Empty result [%s]: '%s'", current_host->id, id, current_host->hostname, command)); - } - SET_UNDEFINED(result_string); - } - } + if (cmd_fd >= 0) { + const script_result_context_t context = {current_host, command, id, type}; + (void)read_script_result(&context, cmd_fd, deadline, result_string); /* close pipe */ nft_pclose(cmd_fd); diff --git a/poller.h b/poller.h index 4e479ba..6ee97b6 100644 --- a/poller.h +++ b/poller.h @@ -33,16 +33,37 @@ #ifndef SPINE_POLLER_H #define SPINE_POLLER_H +typedef struct { + char items[BUFSIZE]; + char host[BIG_BUFSIZE]; + char reindex[BUFSIZE]; + char due_items[BUFSIZE]; + char schedule[BUFSIZE]; + char output[BUFSIZE]; + char agents[BUFSIZE]; + char due_agents[BUFSIZE]; + char boost_output[BUFSIZE]; + char suffix[BUFSIZE]; +} poller_queries_t; + +extern void poller_prepare_queries(poller_queries_t *queries, int host_id, int host_thread, int host_data_ids); + extern void *child(void *arg); extern void child_cleanup(void *arg); extern void child_cleanup_thread(void *arg); extern void child_cleanup_script(void *arg); -extern void poll_host(int device_counter, int host_id, int host_thread, int host_threads, int host_data_ids, const char *host_time, int *host_errors, double host_time_double); extern int poller_store_hex_result(char *result, size_t result_size, const char *hex, int *errors); +extern void poll_host(const poller_thread_t *work, int *host_errors); extern char *exec_poll(host_t *current_host, char *command, int id, const char *type); extern void get_system_information(host_t *host, MYSQL *mysql, int system); -extern int is_multipart_output(char *result); +extern int is_multipart_output(const char *result); extern int validate_result(char *result); +enum poll_result_status { + POLL_RESULT_VALID, + POLL_RESULT_UNDEFINED, + POLL_RESULT_INVALID +}; +extern enum poll_result_status normalize_poll_result(char *result, bool snmp); extern int format_poller_output_row(char *output, size_t output_size, int local_data_id, const char *escaped_rrd_name, const char *host_time, const char *escaped_result); diff --git a/scripts/cppcheck/spine.cfg b/scripts/cppcheck/spine.cfg new file mode 100644 index 0000000..c2448fd --- /dev/null +++ b/scripts/cppcheck/spine.cfg @@ -0,0 +1,31 @@ + + + + + + true + + + + diff --git a/scripts/verify.sh b/scripts/verify.sh index 769e0b0..8b530e4 100755 --- a/scripts/verify.sh +++ b/scripts/verify.sh @@ -22,6 +22,7 @@ cppcheck --enable=all --std=c11 --error-exitcode=1 \ --suppress=unusedFunction \ --suppress=checkersReport \ --suppress=toomanyconfigs \ + --library=scripts/cppcheck/spine.cfg \ -- "${source_files[@]}" 2>&1 | tee /tmp/cppcheck.txt if [[ "${VERIFY_CPPCHECK_ONLY:-0}" == "1" ]]; then diff --git a/snmp.c b/snmp.c index af28617..80830fd 100644 --- a/snmp.c +++ b/snmp.c @@ -33,6 +33,7 @@ #include "common.h" #include "spine.h" +#include #include #include @@ -42,7 +43,6 @@ #define NETSNMP_DS_LIB_DONT_PERSIST_STATE 32 #endif -#define OIDSIZE(p) (sizeof(p)/sizeof(oid)) /*! \fn int spine_snmpv3_protocol_is_set(const char *value) * \brief Whether a Cacti-supplied SNMPv3 protocol field selects anything. @@ -109,37 +109,6 @@ int spine_snmpv3_security_level(const char *auth_protocol, const char *auth_pass return SNMP_SEC_LEVEL_NOAUTH; } -/*! \fn static void free_passphrase(char **psz) - * \brief Wipes a local passphrase copy, then releases it. - * - * Only the copies snmp_host_init() makes are wiped. The caller's - * snmp_password and snmp_priv_passphrase belong to the poller item and have - * to survive the call: poller.c compares them against last_snmp_password and - * last_snmp_priv_passphrase to decide whether the next item can keep the - * open session, so blanking them would tear down the SNMPv3 session and - * re-derive the USM keys for every remaining item on the device. - */ -static void free_passphrase(char **psz) { - volatile char *wipe; - size_t len; - - if (psz != NULL && *psz != NULL) { - /* Written through a volatile pointer on purpose. A plain memset() here - is a dead store into memory that is about to be freed, and gcc -O2 - removes it outright, which leaves the passphrase in the heap for - whatever allocates the block next. explicit_bzero() would say this - more clearly but is absent on the Solaris and Cygwin builds. */ - wipe = (volatile char *) *psz; - len = strlen(*psz); - - while (len-- > 0) { - *wipe++ = '\0'; - } - - SPINE_FREE(*psz); - } -} - /*! \fn void snmp_spine_init() * \brief wrapper function for init_snmp * @@ -203,39 +172,251 @@ void snmp_spine_close(void) { snmp_shutdown("spine"); } -/*! \fn void *snmp_host_init(int host_id, char *hostname, int snmp_version, - * char *snmp_community, char *snmp_username, char *snmp_password, - * char *snmp_auth_protocol, char *snmp_priv_passphrase, char *snmp_priv_protocol, - * char *snmp_context, char *snmp_engine_id, int snmp_port, int snmp_timeout) - * \brief initializes an snmp_session object for a Spine host - * - * This function will initialize NET-SNMP for the Spine host - * in question. - * +static bool snmp_set_security_keys(struct snmp_session *session, int host_id, + const char *auth_password, const char *priv_password) { + char *Apsz = NULL; + char *Xpsz = NULL; + // Auth Protocol Setup + + Apsz = strdup(auth_password); + if (Apsz == NULL) die("ERROR: Fatal malloc error: SNMP authentication passphrase"); + + // Privacy Protocol Setup + + Xpsz = strdup(priv_password); + if (Xpsz == NULL) { + spine_clear_sensitive(Apsz, strlen(Apsz)); + free(Apsz); + die("ERROR: Fatal malloc error: SNMP privacy passphrase"); + } + + session->securityAuthKeyLen = USM_AUTH_KU_LEN; + if (session->securityAuthProto == NULL) { + /* + * get .conf set default + */ + const oid *def = get_default_authtype(&session->securityAuthProtoLen); + session->securityAuthProto = snmp_duplicate_objid(def, session->securityAuthProtoLen); + } + + if (session->securityAuthProto == NULL) { + session->securityAuthProto = snmp_duplicate_objid(SNMP_DEFAULT_AUTH_PROTO, SNMP_DEFAULT_AUTH_PROTOLEN); + session->securityAuthProtoLen = SNMP_DEFAULT_AUTH_PROTOLEN; + } + + if (generate_Ku(session->securityAuthProto, + (u_int)spine_count_to_int(session->securityAuthProtoLen), + (u_char *) Apsz, strlen(Apsz), + session->securityAuthKey, + &session->securityAuthKeyLen) != SNMPERR_SUCCESS) { + SPINE_LOG(("SNMP: Device[%i] Error generating SNMPv3 Ku from authentication passphrase.", host_id)); + if (Apsz != NULL) spine_clear_sensitive(Apsz, strlen(Apsz)); + free(Apsz); + spine_clear_sensitive(Xpsz, strlen(Xpsz)); + free(Xpsz); + return FALSE; + } + + if (Apsz != NULL) spine_clear_sensitive(Apsz, strlen(Apsz)); + free(Apsz); + Apsz = NULL; + if (session->securityLevel == SNMP_SEC_LEVEL_AUTHNOPRIV) { + spine_clear_sensitive(Xpsz, strlen(Xpsz)); + free(Xpsz); + return TRUE; + } + + session->securityPrivKeyLen = USM_PRIV_KU_LEN; + if (session->securityPrivProto == NULL) { + /* + * get .conf set default + */ + const oid *def = get_default_privtype(&session->securityPrivProtoLen); + session->securityPrivProto = + snmp_duplicate_objid(def, session->securityPrivProtoLen); + } + + if (session->securityPrivProto == NULL) { + #ifdef HAVE_USM_DES_PRIV_PROTOCOL + session->securityPrivProto = snmp_duplicate_objid(SNMP_DEFAULT_PRIV_PROTO, SNMP_DEFAULT_PRIV_PROTOLEN); + session->securityPrivProtoLen = SNMP_DEFAULT_PRIV_PROTOLEN; + #else + /* The header's default macro expands to usmDESPrivProtocol, but some + * distributions (e.g. Fedora) ship a net-snmp-config.h that advertises it + * without libnetsnmp actually exporting the symbol, which fails to link. + * Fall back to AES, which configure confirmed libnetsnmp provides. */ + session->securityPrivProto = snmp_duplicate_objid(usmAESPrivProtocol, OID_LENGTH(usmAESPrivProtocol)); + session->securityPrivProtoLen = OID_LENGTH(usmAESPrivProtocol); + #endif + } + + if (generate_Ku(session->securityAuthProto, + (u_int)spine_count_to_int(session->securityAuthProtoLen), + (u_char *) Xpsz, strlen(Xpsz), + session->securityPrivKey, + &session->securityPrivKeyLen) != SNMPERR_SUCCESS) { + SPINE_LOG(("SNMP: Device[%i] Error generating SNMPv3 Ku from privacy pass phrase.", host_id)); + if (Apsz != NULL) spine_clear_sensitive(Apsz, strlen(Apsz)); + free(Apsz); + spine_clear_sensitive(Xpsz, strlen(Xpsz)); + free(Xpsz); + return FALSE; + } + + spine_clear_sensitive(Xpsz, strlen(Xpsz)); + free(Xpsz); + Xpsz = NULL; + return TRUE; +} + +static void snmp_host_init_release(struct snmp_session *session, char *auth, char *priv) { + if (auth != NULL) spine_clear_sensitive(auth, strlen(auth)); + if (priv != NULL) spine_clear_sensitive(priv, strlen(priv)); + free(auth); + free(priv); + #if SNMP_LOCALNAME == 1 + free(session->localname); + #endif + free(session->securityAuthProto); + free(session->securityPrivProto); + spine_clear_sensitive(session->securityAuthKey, sizeof(session->securityAuthKey)); + spine_clear_sensitive(session->securityPrivKey, sizeof(session->securityPrivKey)); +} + +static bool snmp_install_auth_protocol(struct snmp_session *session, const snmp_connection_t *options) { + /* A protocol that is set but unrecognised is a configuration error at any + * security level; validating only on the authenticated path would let a + * typo through as noAuthNoPriv. Install it whenever it is configured so a + * device never falls back to the library default. */ + int auth_type = usm_lookup_auth_type(options->snmp_auth_protocol); + if (auth_type <= 0) { + SPINE_LOG(("SNMP: Device[%i] Error auth protocol %s is invalid.", options->host_id, options->snmp_auth_protocol)); + return FALSE; + } + + const oid *auth_proto = sc_get_auth_oid(auth_type, &session->securityAuthProtoLen); + free(session->securityAuthProto); + session->securityAuthProto = auth_proto == NULL ? NULL : + snmp_duplicate_objid(auth_proto, session->securityAuthProtoLen); + if (session->securityAuthProto == NULL) { + SPINE_LOG(("SNMP: Device[%i] Error installing auth protocol %s.", options->host_id, options->snmp_auth_protocol)); + return FALSE; + } + return TRUE; +} + +static bool snmp_set_security_protocols(struct snmp_session *session, const snmp_connection_t *options) { + /* Cacti stores "[None]" when no protocol is selected. Match cmd.php's + * effective-level rules for incomplete pairs, but report the downgrade + * explicitly so an operator does not mistake it for auth or privacy. */ + int security_level = spine_snmpv3_security_level(options->snmp_auth_protocol, options->snmp_password, + options->snmp_priv_protocol, options->snmp_priv_passphrase); + + /* Refusals precede downgrade warnings so the log never promises that a + * device will be polled when this function is about to reject it. */ + if (spine_snmpv3_passphrase_is_set(options->snmp_password) && options->snmp_auth_protocol[0] == '\0') { + SPINE_LOG(("SNMP: Device[%i] Error authentication password is set but the authentication protocol is empty.", options->host_id)); + return FALSE; + } + + if (spine_snmpv3_passphrase_is_set(options->snmp_priv_passphrase) && options->snmp_priv_protocol[0] == '\0') { + SPINE_LOG(("SNMP: Device[%i] Error privacy passphrase is set but the privacy protocol is empty.", options->host_id)); + return FALSE; + } + + /* Complete privacy credentials with no usable authentication cannot be + * honoured by USM. */ + if (security_level == SNMP_SEC_LEVEL_NOAUTH && + spine_snmpv3_protocol_is_set(options->snmp_priv_protocol) && + spine_snmpv3_passphrase_is_set(options->snmp_priv_passphrase)) { + SPINE_LOG(("SNMP: Device[%i] Error privacy passphrase is configured but authentication is unavailable; set an auth protocol and password, or clear the privacy passphrase.", options->host_id)); + return FALSE; + } + + if (spine_snmpv3_protocol_is_set(options->snmp_auth_protocol) != + spine_snmpv3_passphrase_is_set(options->snmp_password)) { + SPINE_LOG_LOW(("SNMP: Device[%i] WARNING incomplete authentication settings; Cacti's effective security level is noAuthNoPriv.", options->host_id)); + } + + if (spine_snmpv3_protocol_is_set(options->snmp_priv_protocol) != + spine_snmpv3_passphrase_is_set(options->snmp_priv_passphrase)) { + SPINE_LOG_LOW(("SNMP: Device[%i] WARNING incomplete privacy settings; Cacti's effective security level does not include encryption.", options->host_id)); + } + + if (spine_snmpv3_protocol_is_set(options->snmp_auth_protocol) && !snmp_install_auth_protocol(session, options)) { + return FALSE; + } + + session->securityLevel = security_level; + + /* Privacy follows the computed level, which requires authentication + * before encryption. */ + if (security_level != SNMP_SEC_LEVEL_AUTHPRIV) { + session->securityPrivProto = snmp_duplicate_objid(usmNoPrivProtocol, OID_LENGTH(usmNoPrivProtocol)); + session->securityPrivProtoLen = OID_LENGTH(usmNoPrivProtocol); + session->securityPrivKeyLen = 0; + if (session->securityPrivProto == NULL) { + session->securityPrivProtoLen = 0; + SPINE_LOG(("SNMP: Device[%i] Error installing the no-privacy protocol.", options->host_id)); + return FALSE; + } + + /* authNoPriv needs its authentication key derived here too. */ + if (security_level == SNMP_SEC_LEVEL_AUTHNOPRIV) { + return snmp_set_security_keys(session, options->host_id, options->snmp_password, options->snmp_priv_passphrase); + } + return TRUE; + } + + int priv_type = usm_lookup_priv_type(options->snmp_priv_protocol); + if (priv_type < 0) { + SPINE_LOG(("SNMP: Device[%i] Error privacy protocol %s is invalid.", options->host_id, options->snmp_priv_protocol)); + return FALSE; + } + + const oid *priv_proto = sc_get_priv_oid(priv_type, &session->securityPrivProtoLen); + free(session->securityPrivProto); + session->securityPrivProto = priv_proto == NULL ? NULL : + snmp_duplicate_objid(priv_proto, session->securityPrivProtoLen); + if (session->securityPrivProto == NULL) { + SPINE_LOG(("SNMP: Device[%i] Error installing privacy protocol %s.", options->host_id, options->snmp_priv_protocol)); + return FALSE; + } + + return snmp_set_security_keys(session, options->host_id, options->snmp_password, options->snmp_priv_passphrase); +} + +/*! \fn void *snmp_host_init(const snmp_connection_t *options) + * \brief initializes an owned Net-SNMP session from borrowed connection inputs. */ -void *snmp_host_init(int host_id, char *hostname, int snmp_version, char *snmp_community, - char *snmp_username, const char *snmp_password, char *snmp_auth_protocol, - const char *snmp_priv_passphrase, char *snmp_priv_protocol, - char *snmp_context, char *snmp_engine_id, int snmp_port, int snmp_timeout) { +void *snmp_host_init(const snmp_connection_t *options) { + if (options->snmp_timeout <= 0) { + SPINE_LOG(("SNMP: Device[%i] Invalid nonpositive timeout.", options->host_id)); + return NULL; + } + #if LONG_MAX / 1000L < INT_MAX + if (options->snmp_timeout > LONG_MAX / 1000L) { + SPINE_LOG(("SNMP: Device[%i] Timeout exceeds Net-SNMP's microsecond range.", options->host_id)); + return NULL; + } + #endif void *sessp = NULL; struct snmp_session session; char hostnameport[BUFSIZE]; - size_t len; char *Apsz = NULL; char *Xpsz = NULL; char *Cpsz = NULL; - int priv_type = -1; /* initialize SNMP */ snmp_sess_init(&session); /* Bind to snmp_clientaddr if specified */ - len = strlen(set.snmp_clientaddr); + size_t len = strlen(set.snmp.snmp_clientaddr); if (len > 0 && len <= SMALL_BUFSIZE) { #if SNMP_LOCALNAME == 1 - session.localname = strdup(set.snmp_clientaddr); + session.localname = strdup(set.snmp.snmp_clientaddr); #endif } @@ -274,286 +455,54 @@ void *snmp_host_init(int host_id, char *hostname, int snmp_version, char *snmp_c session.contextEngineIDLen = 0; /* verify snmp version is accurate */ - if (snmp_version == 2) { + if (options->snmp_version == 2) { session.version = SNMP_VERSION_2c; session.securityModel = SNMP_SEC_MODEL_SNMPv2c; - } else if (snmp_version == 1) { + } else if (options->snmp_version == 1) { session.version = SNMP_VERSION_1; session.securityModel = SNMP_SEC_MODEL_SNMPv1; - } else if (snmp_version == 3) { + } else if (options->snmp_version == 3) { session.version = SNMP_VERSION_3; session.securityModel = USM_SEC_MODEL_NUMBER; } else { - SPINE_LOG(("Device[%i] ERROR: SNMP Version Error for Device '%s'", host_id, hostname)); - free(session.localname); + SPINE_LOG(("Device[%i] ERROR: SNMP Version Error for Device '%s'", options->host_id, options->hostname)); + snmp_host_init_release(&session, Apsz, Xpsz); return 0; } - snprintf(hostnameport, BUFSIZE, "%s:%i", hostname, snmp_port); - session.peername = strdup(hostnameport); - if (!session.peername) { - SPINE_LOG(("Device[%i] ERROR: Failed to allocate peername for '%s'", host_id, hostname)); - free(session.localname); - return 0; - } - session.retries = set.snmp_retries; - session.timeout = (snmp_timeout * 1000); /* net-snmp likes microseconds */ + snprintf(hostnameport, BUFSIZE, "%s:%i", options->hostname, options->snmp_port); + session.peername = hostnameport; + session.retries = set.snmp.snmp_retries; + session.timeout = ((long)options->snmp_timeout * 1000L); /* net-snmp likes microseconds */ - SPINE_LOG_HIGH(("Device[%i] INFO: SNMP Device '%s' has a timeout of %ld (%d), with %d retries", host_id, hostnameport, session.timeout, snmp_timeout, session.retries)); + SPINE_LOG_HIGH(("Device[%i] INFO: SNMP Device '%s' has a timeout of %ld (%d), with %d retries", options->host_id, hostnameport, session.timeout, options->snmp_timeout, session.retries)); - if ((snmp_version == 2) || (snmp_version == 1)) { - session.community = (unsigned char*) snmp_community; - session.community_len = strlen(snmp_community); + if ((options->snmp_version == 2) || (options->snmp_version == 1)) { + session.community = (unsigned char*) options->snmp_community; + session.community_len = strlen(options->snmp_community); } else { session.community = (unsigned char *) Cpsz; session.community_len = 0; - session.securityName = snmp_username; + session.securityName = options->snmp_username; session.securityNameLen = strlen(session.securityName); - if (snmp_context && strlen(snmp_context)) { - session.contextName = snmp_context; + if (options->snmp_context && strlen(options->snmp_context)) { + session.contextName = options->snmp_context; session.contextNameLen = strlen(session.contextName); } - if (snmp_engine_id && strlen(snmp_engine_id)) { - session.contextEngineID = (unsigned char*) snmp_engine_id; - session.contextEngineIDLen = strlen(snmp_engine_id); + if (options->snmp_engine_id && strlen(options->snmp_engine_id)) { + session.contextEngineID = (unsigned char*) options->snmp_engine_id; + session.contextEngineIDLen = strlen(options->snmp_engine_id); } - /* set the authentication protocol */ - { - int auth_type; - int security_level; - const oid *auth_proto; - - /* Cacti stores "[None]" when no protocol is selected. Match cmd.php's - * effective-level rules for incomplete pairs, but report the downgrade - * explicitly so an operator does not mistake it for auth or privacy. */ - security_level = spine_snmpv3_security_level(snmp_auth_protocol, snmp_password, - snmp_priv_protocol, snmp_priv_passphrase); - - /* Refusals precede downgrade warnings so the log never promises that a - * device will be polled when this function is about to reject it. */ - if (spine_snmpv3_passphrase_is_set(snmp_password) && - (snmp_auth_protocol == NULL || snmp_auth_protocol[0] == '\0')) { - SPINE_LOG(("SNMP: Device[%i] Error authentication password is set but the authentication protocol is empty.", host_id)); - free(session.peername); - free(session.localname); - return 0; + if (!snmp_set_security_protocols(&session, options)) { + snmp_host_init_release(&session, Apsz, Xpsz); + return NULL; } - if (spine_snmpv3_passphrase_is_set(snmp_priv_passphrase) && - (snmp_priv_protocol == NULL || snmp_priv_protocol[0] == '\0')) { - SPINE_LOG(("SNMP: Device[%i] Error privacy passphrase is set but the privacy protocol is empty.", host_id)); - free(session.peername); - free(session.localname); - return 0; - } - - /* Complete privacy credentials with no usable authentication cannot be - * honoured by USM. Refuse that case before describing any downgrade. */ - if (security_level == SNMP_SEC_LEVEL_NOAUTH && - spine_snmpv3_protocol_is_set(snmp_priv_protocol) && - spine_snmpv3_passphrase_is_set(snmp_priv_passphrase)) { - SPINE_LOG(("SNMP: Device[%i] Error privacy passphrase is configured but authentication is unavailable; set an auth protocol and password, or clear the privacy passphrase.", host_id)); - free(session.peername); - free(session.localname); - return 0; - } - - if (spine_snmpv3_protocol_is_set(snmp_auth_protocol) != - spine_snmpv3_passphrase_is_set(snmp_password)) { - SPINE_LOG_LOW(("SNMP: Device[%i] WARNING incomplete authentication settings; Cacti's effective security level is noAuthNoPriv.", host_id)); - } - - if (spine_snmpv3_protocol_is_set(snmp_priv_protocol) != - spine_snmpv3_passphrase_is_set(snmp_priv_passphrase)) { - SPINE_LOG_LOW(("SNMP: Device[%i] WARNING incomplete privacy settings; Cacti's effective security level does not include encryption.", host_id)); - } - - /* A protocol that is set but unrecognised is a configuration error at - * any security level. Deciding the level first and only validating on - * the authenticated path would let a typo through as noAuthNoPriv, - * because a device with no passphrase never reaches the check. */ - if (spine_snmpv3_protocol_is_set(snmp_auth_protocol)) { - auth_type = usm_lookup_auth_type(snmp_auth_protocol); - - if (auth_type <= 0) { - SPINE_LOG(("SNMP: Device[%i] Error auth protocol %s is invalid.", host_id, snmp_auth_protocol)); - free(session.peername); - free(session.localname); - return 0; - } - - /* Install it whenever it is configured, not only when the level - * says the session authenticates. The privacy branch below can - * still raise the level, and leaving this to that branch meant a - * device with a protocol but no passphrase got the library - * default instead of the one it was configured with. */ - auth_proto = sc_get_auth_oid(auth_type, &session.securityAuthProtoLen); - free(session.securityAuthProto); - session.securityAuthProto = auth_proto == NULL ? NULL : - snmp_duplicate_objid(auth_proto, session.securityAuthProtoLen); - if (session.securityAuthProto == NULL) { - SPINE_LOG(("SNMP: Device[%i] Error installing auth protocol %s.", host_id, snmp_auth_protocol)); - free(session.peername); - free(session.localname); - return 0; - } - } - - session.securityLevel = security_level; - - /* Privacy follows the computed level. Selecting it from the privacy - * fields alone disagreed with spine_snmpv3_security_level(), which - * requires authentication before encryption: a device with privacy - * configured but no auth passphrase took this branch's else and was - * built as authPriv with no authentication key. */ - if (security_level != SNMP_SEC_LEVEL_AUTHPRIV) { - session.securityPrivProto = snmp_duplicate_objid(usmNoPrivProtocol, OID_LENGTH(usmNoPrivProtocol)); - session.securityPrivProtoLen = OID_LENGTH(usmNoPrivProtocol); - session.securityPrivKeyLen = USM_PRIV_KU_LEN; - - if (session.securityPrivProto == NULL) { - session.securityPrivProtoLen = 0; - SPINE_LOG(("SNMP: Device[%i] Error installing the no-privacy protocol.", host_id)); - free(session.peername); - free(session.securityAuthProto); - free(session.localname); - return 0; - } - - /* The authentication key was only ever derived on the privacy path, - * so authNoPriv sessions authenticated with an empty key and every - * such device failed with a USM authentication error. */ - if (security_level == SNMP_SEC_LEVEL_AUTHNOPRIV) { - free_passphrase(&Apsz); - Apsz = strdup(snmp_password); - - session.securityAuthKeyLen = USM_AUTH_KU_LEN; - if (Apsz == NULL || session.securityAuthProto == NULL || - generate_Ku(session.securityAuthProto, - session.securityAuthProtoLen, - (u_char *) Apsz, strlen(Apsz), - session.securityAuthKey, - &session.securityAuthKeyLen) != SNMPERR_SUCCESS) { - SPINE_LOG(("SNMP: Device[%i] Error generating SNMPv3 Ku from authentication passphrase.", host_id)); - free(session.peername); - free(session.securityAuthProto); - free(session.securityPrivProto); - free_passphrase(&Apsz); - free_passphrase(&Xpsz); - free(session.localname); - return 0; - } - - /* The privacy path releases this after deriving its key; this - * one did not, so every authNoPriv session leaked the - * passphrase copy. */ - free_passphrase(&Apsz); - } - } else { - const oid *priv_proto; - - priv_type = usm_lookup_priv_type(snmp_priv_protocol); - - if (priv_type < 0) { - SPINE_LOG(("SNMP: Device[%i] Error privacy protocol %s is invalid.", host_id, snmp_priv_protocol)); - free(session.peername); - free(session.securityAuthProto); - free(session.localname); - return 0; - } - - priv_proto = sc_get_priv_oid(priv_type, &session.securityPrivProtoLen); - free(session.securityPrivProto); - session.securityPrivProto = priv_proto == NULL ? NULL : - snmp_duplicate_objid(priv_proto, session.securityPrivProtoLen); - if (session.securityPrivProto == NULL) { - SPINE_LOG(("SNMP: Device[%i] Error installing privacy protocol %s.", host_id, snmp_priv_protocol)); - free(session.peername); - free(session.securityAuthProto); - free(session.localname); - return 0; - } - /* security_level is AUTHPRIV here by construction: this branch is - * only reached when it is. Assigning the computed value keeps one - * predicate authoritative rather than two that can disagree. */ - session.securityLevel = security_level; - - // Auth Protocol Setup - free_passphrase(&Apsz); - Apsz = strdup(snmp_password); - - // Privacy Protocol Setup - free_passphrase(&Xpsz); - Xpsz = strdup(snmp_priv_passphrase); - - /* authPriv cannot be constructed safely without both local copies. - * Treat allocator failure like key-derivation failure instead of - * handing net-snmp an authPriv session with an empty key. */ - if (Apsz == NULL || Xpsz == NULL) { - SPINE_LOG(("SNMP: Device[%i] Error allocating SNMPv3 passphrase storage.", host_id)); - free(session.peername); - free(session.securityAuthProto); - free(session.securityPrivProto); - free_passphrase(&Apsz); - free_passphrase(&Xpsz); - free(session.localname); - return 0; - } - - { - session.securityAuthKeyLen = USM_AUTH_KU_LEN; - if (session.securityAuthProto == NULL || - generate_Ku(session.securityAuthProto, - session.securityAuthProtoLen, - (u_char *) Apsz, strlen(Apsz), - session.securityAuthKey, - &session.securityAuthKeyLen) != SNMPERR_SUCCESS) { - SPINE_LOG(("SNMP: Device[%i] Error generating SNMPv3 Ku from authentication passphrase.", host_id)); - free(session.peername); - free(session.securityAuthProto); - free(session.securityPrivProto); - free_passphrase(&Apsz); - free_passphrase(&Xpsz); - if (session.localname) { - free(session.localname); - session.localname = NULL; - } - return 0; - } - - free_passphrase(&Apsz); - } - - { - session.securityPrivKeyLen = USM_PRIV_KU_LEN; - if (session.securityPrivProto == NULL || - generate_Ku(session.securityAuthProto, - session.securityAuthProtoLen, - (u_char *) Xpsz, strlen(Xpsz), - session.securityPrivKey, - &session.securityPrivKeyLen) != SNMPERR_SUCCESS) { - SPINE_LOG(("SNMP: Device[%i] Error generating SNMPv3 Ku from privacy pass phrase.", host_id)); - free(session.peername); - free(session.securityAuthProto); - free(session.securityPrivProto); - free_passphrase(&Xpsz); - if (session.localname) { - free(session.localname); - session.localname = NULL; - } - return 0; - } - - free_passphrase(&Xpsz); - } - } - - SPINE_LOG_MEDIUM(("Device[%i] SNMPv3 Using AuthProto: %s, PrivProto: %s", host_id, snmp_auth_protocol, snmp_priv_protocol)); - } /* end auth/priv block */ + SPINE_LOG_MEDIUM(("Device[%i] SNMPv3 Using AuthProto: %s, PrivProto: %s", options->host_id, options->snmp_auth_protocol, options->snmp_priv_protocol)); } /* open SNMP Session */ @@ -561,19 +510,11 @@ void *snmp_host_init(int host_id, char *hostname, int snmp_version, char *snmp_c sessp = snmp_sess_open(&session); thread_mutex_unlock(LOCK_SNMP); - free(session.peername); - free(session.securityAuthProto); - free(session.securityPrivProto); - free(session.localname); - if (!sessp) { - if (is_debug_device(host_id)) { - SPINE_LOG(("ERROR: Device[%i] Problem initializing SNMP session '%s'", host_id, hostname)); - } else { - SPINE_LOG_MEDIUM(("ERROR: Device[%i] Problem initializing SNMP session '%s'", host_id, hostname)); - } + SPINE_LOG_DEVICE(options->host_id, POLLER_VERBOSITY_MEDIUM, ("ERROR: Device[%i] Problem initializing SNMP session '%s'", options->host_id, options->hostname)); } + snmp_host_init_release(&session, Apsz, Xpsz); return sessp; } @@ -592,6 +533,435 @@ void snmp_host_cleanup(void *snmp_session) { } } +typedef struct { + struct snmp_pdu *response; + int status; + bool valid_oid; +} snmp_reply_t; + +static snmp_reply_t snmp_request_parsed(host_t *host, const oid *name, size_t length, int command) { + snmp_reply_t reply = {NULL, STAT_DESCRIP_ERROR, TRUE}; + if (host->snmp.session == NULL) return reply; + struct snmp_pdu *request = snmp_pdu_create(command); + if (request == NULL) { + SPINE_LOG(("ERROR: Unable to create SNMP PDU")); + host->snmp.status = reply.status; + return reply; + } + if (snmp_add_null_var(request, name, length) == NULL) { + snmp_free_pdu(request); + reply.status = STAT_ERROR; + host->snmp.status = reply.status; + return reply; + } + /* Net-SNMP owns and frees request after the synchronous call, including + * a failed send; the caller owns only the returned response. */ + reply.status = snmp_sess_synch_response(host->snmp.session, request, &reply.response); + host->snmp.status = reply.status; + return reply; +} + +static snmp_reply_t snmp_single_request(host_t *host, char *text_oid, int command) { + snmp_reply_t reply = {NULL, STAT_DESCRIP_ERROR, TRUE}; + if (host->snmp.session == NULL) return reply; + oid parsed[MAX_OID_LEN]; + size_t length = MAX_OID_LEN; + if (!snmp_parse_oid(text_oid, parsed, &length)) { + SPINE_LOG(("Device[%i] ERROR: Problems parsing SNMP OID %s", host->id, text_oid)); + reply.status = STAT_ERROR; + reply.valid_oid = FALSE; + host->snmp.status = reply.status; + return reply; + } + return snmp_request_parsed(host, parsed, length, command); +} + +static int snmp_format_scalar(char *output, const struct variable_list *variable, bool ascii) { + char temporary[RESULTS_BUFFER] = {0}; + if (variable->name == NULL || snprint_value(temporary, sizeof(temporary), variable->name, variable->name_length, variable) < 0) { + SET_UNDEFINED(output); + return STAT_ERROR; + } + if (ascii) { + if (snprint_asciistring(output, RESULTS_BUFFER, (unsigned char *)temporary, strlen(temporary)) < 0) { + SET_UNDEFINED(output); + return STAT_ERROR; + } + } else { + strncopy(output, trim(temporary), RESULTS_BUFFER); + } + return STAT_SUCCESS; +} + +static int snmp_get_variable(const host_t *host, const char *text_oid, const struct variable_list *variable, char *output) { + switch (variable->type) { + case SNMP_NOSUCHOBJECT: + if (strstr(text_oid, ".1.3.6.1.2.1.1.1.0") || strstr(text_oid, ".1.3.6.1.2.1.1.3.0")) { + SPINE_LOG_HIGH(("DEBUG: OID '%s' for Device[%i], SNMP_NOSUCHOBJECT for sysDesc or sysUptime", text_oid, host->id)); + return snmp_format_scalar(output, variable, FALSE); + } + SPINE_LOG_DEBUG(("DEBUG: OID '%s' for Device[%i], SNMP_NOSUCHOBJECT not sysDesc or sysUptime", text_oid, host->id)); + break; + case SNMP_NOSUCHINSTANCE: + if (strstr(text_oid, ".1.3.6.1.6.3.10.2.1.3.0")) { + SPINE_LOG_DEBUG(("NOTE: Legacy SNMP agent found! No per second Uptime oid '%s' for Device[%i]", text_oid, host->id)); + } else { + SPINE_LOG_HIGH(("WARNING: No such Instance for oid '%s' for Device[%i]", text_oid, host->id)); + } + break; + case SNMP_ENDOFMIBVIEW: + SPINE_LOG_HIGH(("ERROR: End of Mib for oid '%s' for Device[%i]", text_oid, host->id)); + break; + default: return snmp_format_scalar(output, variable, FALSE); + } + SET_UNDEFINED(output); + return STAT_ERROR; +} + +/* Decode a USM report instead of logging a generic error. New constant names + * (SNMPERR_NOT_IN_TIME_WINDOW etc.) arrived in Net-SNMP 5.8 and the old ones + * (SNMPERR_USM_NOTINTIMEWINDOW etc.) in 5.7; on older versions every USM error + * falls through to the default case. notInTimeWindow is engine time drift and + * recovers on the next request, so it does not mark the host down. */ +static int snmp_decode_session_error(host_t *host, const char *text_oid, const char *operation, int status, char *output) { + int liberr = 0; + int syserr = 0; + char *errstr = NULL; + + if (status != STAT_ERROR || host->snmp.session == NULL) { + SPINE_LOG_HIGH(("ERROR: Unknown error getting oid '%s' for Device[%i] with Status[%d]", text_oid, host->id, status)); + return status; + } + + snmp_sess_error(host->snmp.session, &liberr, &syserr, &errstr); + SPINE_LOG_DEBUG(("Device[%i] DEBUG: SNMP %ssession error for oid '%s': %s (liberr=%d)", + host->id, operation, text_oid, errstr ? errstr : "unknown", liberr)); + SNMP_FREE(errstr); + + switch (liberr) { +#if defined(SNMPERR_NOT_IN_TIME_WINDOW) || defined(SNMPERR_USM_NOTINTIMEWINDOW) +#if defined(SNMPERR_NOT_IN_TIME_WINDOW) + case SNMPERR_NOT_IN_TIME_WINDOW: +#else + case SNMPERR_USM_NOTINTIMEWINDOW: +#endif + SPINE_LOG_MEDIUM(("WARNING: Device[%i] USM notInTimeWindow for %soid '%s' -- engine time drift (recoverable)", + host->id, operation, text_oid)); + SET_UNDEFINED(output); + host->snmp.status = STAT_SUCCESS; + return STAT_SUCCESS; +#endif +#if defined(SNMPERR_UNKNOWN_ENG_ID) || defined(SNMPERR_USM_UNKNOWNENGINEID) +#if defined(SNMPERR_UNKNOWN_ENG_ID) + case SNMPERR_UNKNOWN_ENG_ID: +#else + case SNMPERR_USM_UNKNOWNENGINEID: +#endif + SPINE_LOG_HIGH(("ERROR: Device[%i] USM unknownEngineID for %soid '%s'", host->id, operation, text_oid)); + break; +#endif +#if defined(SNMPERR_UNKNOWN_USER_NAME) || defined(SNMPERR_USM_UNKNOWNSECURITYNAME) +#if defined(SNMPERR_UNKNOWN_USER_NAME) + case SNMPERR_UNKNOWN_USER_NAME: +#else + case SNMPERR_USM_UNKNOWNSECURITYNAME: +#endif + SPINE_LOG_HIGH(("ERROR: Device[%i] USM unknownSecurityName for %soid '%s'", host->id, operation, text_oid)); + break; +#endif +#if defined(SNMPERR_AUTHENTICATION_FAILURE) || defined(SNMPERR_USM_AUTHENTICATIONFAILURE) +#if defined(SNMPERR_AUTHENTICATION_FAILURE) + case SNMPERR_AUTHENTICATION_FAILURE: +#else + case SNMPERR_USM_AUTHENTICATIONFAILURE: +#endif + SPINE_LOG_HIGH(("ERROR: Device[%i] USM authenticationFailure for %soid '%s'", host->id, operation, text_oid)); + break; +#endif +#if defined(SNMPERR_DECRYPTION_ERR) || defined(SNMPERR_USM_DECRYPTIONERROR) +#if defined(SNMPERR_DECRYPTION_ERR) + case SNMPERR_DECRYPTION_ERR: +#else + case SNMPERR_USM_DECRYPTIONERROR: +#endif + SPINE_LOG_HIGH(("ERROR: Device[%i] USM decryptionError for %soid '%s'", host->id, operation, text_oid)); + break; +#endif +#if defined(SNMPERR_UNSUPPORTED_SEC_LEVEL) || defined(SNMPERR_USM_UNSUPPORTEDSECURITYLEVEL) +#if defined(SNMPERR_UNSUPPORTED_SEC_LEVEL) + case SNMPERR_UNSUPPORTED_SEC_LEVEL: +#else + case SNMPERR_USM_UNSUPPORTEDSECURITYLEVEL: +#endif + SPINE_LOG_HIGH(("ERROR: Device[%i] USM unsupportedSecurityLevel for %soid '%s'", host->id, operation, text_oid)); + break; +#endif + default: + SPINE_LOG_HIGH(("ERROR: Unknown error getting oid '%s' for Device[%i] with Status[%d] Errno[%d]", + text_oid, host->id, status, liberr)); + break; + } + + return status; +} + +static int snmp_get_response(host_t *host, const char *text_oid, const snmp_reply_t *reply, char *output) { + if (reply->status == STAT_DESCRIP_ERROR) { + SET_UNDEFINED(output); + return STAT_ERROR; + } + if (reply->status == STAT_SUCCESS) { + if (reply->response == NULL) { + SPINE_LOG(("ERROR: An internal Net-Snmp error condition detected in Cacti snmp_get")); + SET_UNDEFINED(output); + return STAT_ERROR; + } + if (reply->response->errstat == SNMP_ERR_NOERROR && reply->response->variables != NULL && reply->response->variables->name != NULL) { + return snmp_get_variable(host, text_oid, reply->response->variables, output); + } + /* Preserve the legacy transport-success outcome for an agent error: + * availability observes host->snmp_status, while data remains empty. */ + SPINE_LOG_HIGH(("ERROR: Failed to get oid '%s' for Device[%i] with Response[%ld]", text_oid, host->id, reply->response->errstat)); + return STAT_SUCCESS; + } + if (reply->response != NULL && reply->response->variables != NULL) { + SET_UNDEFINED(output); + SPINE_LOG_HIGH(("ERROR: Agent error getting oid '%s' for Device[%i] with Status[%d]", text_oid, host->id, reply->status)); + return STAT_ERROR; + } + if (reply->status == STAT_TIMEOUT) { + SPINE_LOG_HIGH(("ERROR: Timeout getting oid '%s' for Device[%i] with Status[%d]", text_oid, host->id, reply->status)); + return reply->status; + } + return snmp_decode_session_error(host, text_oid, "", reply->status, output); +} + +/*! \fn char *snmp_get_base(host_t *current_host, char *snmp_oid, bool should_fail) + * \brief performs a single snmp_get for a specific snmp OID + * + * This function will poll a specific snmp OID for a host. The host snmp + * session must already be established. + * + * \return returns the character representation of the snmp OID, or "U" if + * unsuccessful. + * + */ +char *snmp_get_base(host_t *host, char *text_oid, bool should_fail) { + char *output = calloc(RESULTS_BUFFER, 1); + if (output == NULL) die("ERROR: Fatal malloc error: snmp.c snmp_get!"); + if (host->ignore_host) { + SPINE_LOG_HIGH(("WARNING: Skipped oid '%s' for Device[%i] as host ignore flag is active", text_oid, host->id)); + SET_UNDEFINED(output); + return output; + } + snmp_reply_t reply = snmp_single_request(host, text_oid, SNMP_MSG_GET); + if (!reply.valid_oid) { + SET_UNDEFINED(output); + return output; + } + int status = reply.status; + if (host->snmp.session != NULL) status = snmp_get_response(host, text_oid, &reply, output); + if (reply.response != NULL) snmp_free_pdu(reply.response); + host->snmp.status = status; + if (status != STAT_SUCCESS && should_fail) { + host->ignore_host = TRUE; + SET_UNDEFINED(output); + } + return output; +} + +char *snmp_get(host_t *host, char *text_oid) { + return snmp_get_base(host, text_oid, TRUE); +} + +/* System OIDs may legitimately be absent; a NoSuchObject there must not + * mark the host ignored and suppress the rest of its polling. */ +char *snmp_get_allow_fail(host_t *host, char *text_oid) { + return snmp_get_base(host, text_oid, FALSE); +} + +char *snmp_getnext(host_t *host, char *text_oid) { + char *output = calloc(RESULTS_BUFFER, 1); + if (output == NULL) die("ERROR: Fatal malloc error: snmp.c snmp_getnext!"); + snmp_reply_t reply = snmp_single_request(host, text_oid, SNMP_MSG_GETNEXT); + if (!reply.valid_oid) { + SET_UNDEFINED(output); + return output; + } + int status = reply.status; + if (status == STAT_SUCCESS) { + if (reply.response == NULL) { + SPINE_LOG(("ERROR: An internal Net-Snmp error condition detected in Cacti snmp_getnext")); + status = STAT_ERROR; + } else if (reply.response->errstat == SNMP_ERR_NOERROR) { + const struct variable_list *variable = reply.response->variables; + status = variable == NULL || snmp_varbind_is_exception(variable) ? STAT_ERROR : snmp_format_scalar(output, variable, TRUE); + } + } else if (status == STAT_TIMEOUT) { + SPINE_LOG_HIGH(("ERROR: Timeout getting oid '%s' for Device[%i] with Status[%d]", text_oid, host->id, status)); + } else if (status == STAT_ERROR) { + status = snmp_decode_session_error(host, text_oid, "getnext ", status, output); + } + if (reply.response != NULL) snmp_free_pdu(reply.response); + host->snmp.status = status; + if (status != STAT_SUCCESS) { + host->ignore_host = TRUE; + SET_UNDEFINED(output); + } + return output; +} + +typedef struct { + oid root[MAX_OID_LEN]; + size_t root_length; + oid current[MAX_OID_LEN]; + size_t current_length; + int count; + bool failed; +} snmp_walk_t; + +static bool snmp_count_advance(snmp_walk_t *walk, const struct variable_list *variable) { + if (variable->name == NULL || variable->name_length > MAX_OID_LEN) { + walk->failed = TRUE; + return FALSE; + } + if (variable->name_length < walk->root_length || memcmp(walk->root, variable->name, walk->root_length * sizeof(oid)) != 0) return FALSE; + if (walk->count == INT_MAX) { + SPINE_LOG(("ERROR: SNMP table count exceeds supported integer range")); + walk->failed = TRUE; + return FALSE; + } + /* The walk terminator is not a table entry, so it does not count. */ + if (variable->type == SNMP_ENDOFMIBVIEW || variable->type == SNMP_NOSUCHOBJECT || variable->type == SNMP_NOSUCHINSTANCE) return FALSE; + walk->count++; + if (snmp_oid_compare(walk->current, walk->current_length, variable->name, variable->name_length) >= 0) { + SPINE_LOG(("ERROR: OID not increasing")); + walk->failed = TRUE; + return FALSE; + } + memcpy(walk->current, variable->name, variable->name_length * sizeof(oid)); + walk->current_length = variable->name_length; + return TRUE; +} + +static bool snmp_count_response(snmp_walk_t *walk, const snmp_reply_t *reply) { + /* SNMPv1 has no endOfMibView variable type; it reports the normal end of a + * GETNEXT walk as a PDU-level noSuchName instead. */ + if (reply->status == STAT_SUCCESS && reply->response != NULL && reply->response->errstat == SNMP_ERR_NOSUCHNAME) return FALSE; + if (reply->status != STAT_SUCCESS || reply->response == NULL || reply->response->errstat != SNMP_ERR_NOERROR || reply->response->variables == NULL) { + SPINE_LOG(("ERROR: %s detected in Cacti snmp_count", reply->status == STAT_TIMEOUT ? "Timeout" : "Invalid SNMP response")); + walk->failed = TRUE; + return FALSE; + } + for (const struct variable_list *variable = reply->response->variables; variable != NULL; variable = variable->next_variable) { + if (!snmp_count_advance(walk, variable)) return FALSE; + } + return TRUE; +} + +/*! \fn char *snmp_count(host_t *current_host, char *snmp_oid) + * \brief counts entries of snmp table specified by a specific snmp OID + * + * This function will poll a specific snmp OID for a host. The host snmp + * session must already be established. + * + * \return returns count of table entries + * + */ +int snmp_count(host_t *host, char *text_oid) { + SPINE_LOG_DEVICE(host->id, POLLER_VERBOSITY_DEBUG, ("DEBUG: walk starts at OID %s", text_oid)); + if (host->snmp.session == NULL) { + host->ignore_host = TRUE; + return -1; + } + snmp_walk_t walk = {0}; + walk.root_length = MAX_OID_LEN; + if (!snmp_parse_oid(text_oid, walk.root, &walk.root_length)) { + SPINE_LOG(("Device[%i] ERROR: SNMP Count Problems parsing SNMP OID %s", host->id, text_oid)); + return -1; + } + memcpy(walk.current, walk.root, walk.root_length * sizeof(oid)); + walk.current_length = walk.root_length; + bool more = TRUE; + while (more) { + snmp_reply_t reply = snmp_request_parsed(host, walk.current, walk.current_length, SNMP_MSG_GETNEXT); + more = snmp_count_response(&walk, &reply); + if (reply.response != NULL) snmp_free_pdu(reply.response); + } + if (walk.failed) { + host->ignore_host = TRUE; + if (host->snmp.status == STAT_SUCCESS) host->snmp.status = STAT_ERROR; + } + /* A negative count tells the caller this walk never produced a usable + * result, so it is not mistaken for a legitimate zero-item count. */ + return walk.failed ? -1 : walk.count; +} + +/*! \fn void snmp_snprint_value(char *obuf, size_t buf_len, const oid *objid, size_t objidlen, const struct variable_list *variable) + * + * \brief replacement for the buggy net-snmp.org snprint_value function + * + * This function format an output buffer with the correct string representation + * of an snmp OID result fetched with snmp_get_multi. The buffer pointed to by + * the function is modified. + * + */ +void snmp_snprint_value(char *obuf, size_t buf_len, const oid *objid, size_t objidlen, const struct variable_list *variable) { + (void)objid; + (void)objidlen; + if (obuf == NULL || buf_len == 0) return; + u_char *buf = calloc(buf_len, 1); + if (buf == NULL) { + snprintf(obuf, buf_len, "%s", "U"); + return; + } + size_t scratch_capacity = buf_len; + size_t out_len = 0; + if (sprint_realloc_by_type(&buf, &scratch_capacity, &out_len, 0, variable, NULL, NULL, NULL)) { + snprintf(obuf, buf_len, "%s", buf); + } else { + snprintf(obuf, buf_len, "%s", "U"); + } + free(buf); +} + +static void snmp_multi_undefined(snmp_oids_t *oids, int count) { + for (int index = 0; index < count; index++) SET_UNDEFINED(oids[index].result); +} + +static struct snmp_pdu *snmp_multi_request(const host_t *host, const target_t *items, snmp_oids_t *oids, int count) { + struct snmp_pdu *request = snmp_pdu_create(SNMP_MSG_GET); + if (request == NULL) return NULL; + for (int index = 0; index < count; index++) { + if (IS_UNDEFINED(oids[index].result)) continue; + oid name[MAX_OID_LEN]; + size_t length = MAX_OID_LEN; + if (!snmp_parse_oid(oids[index].oid, name, &length)) { + SPINE_LOG(("Device[%i] DS[%i] ERROR: Problems parsing Multi SNMP OID! (oid: %s), Set MAX_OIDS to 1 for this host to isolate bad OID", host->id, items[oids[index].array_position].local_data_id, oids[index].oid)); + SET_UNDEFINED(oids[index].result); + } else if (snmp_add_null_var(request, name, length) == NULL) { + SET_UNDEFINED(oids[index].result); + } + } + if (request->variables == NULL) { + snmp_free_pdu(request); + return NULL; + } + return request; +} + +static bool snmp_multi_error_index(snmp_oids_t *oids, int count, long error_index) { + long current = 0; + for (int index = 0; index < count; index++) { + if (IS_UNDEFINED(oids[index].result)) continue; + if (++current == error_index) { + SET_UNDEFINED(oids[index].result); + return TRUE; + } + } + return FALSE; +} + /*! \fn int snmp_varbind_is_exception(const struct variable_list *vars) * \brief report whether a varbind carries a per-OID exception rather than data * @@ -612,590 +982,24 @@ int snmp_varbind_is_exception(const struct variable_list *vars) { vars->type == SNMP_ENDOFMIBVIEW); } -/*! \fn char *snmp_get_base(host_t *current_host, const char *snmp_oid, bool should_fail) - * \brief performs a single snmp_get for a specific snmp OID - * - * This function will poll a specific snmp OID for a host. The host snmp - * session must already be established. - * - * \return returns the character representation of the snmp OID, or "U" if - * unsuccessful. - * - */ -char *snmp_get_base(host_t *current_host, const char *snmp_oid, bool should_fail) { - struct snmp_pdu *pdu = NULL; - struct snmp_pdu *response = NULL; - struct variable_list *vars = NULL; - size_t anOID_len = MAX_OID_LEN; - oid anOID[MAX_OID_LEN]; - int status; - int sess_liberr = SNMPERR_SUCCESS; - char *result_string; - char temp_result[RESULTS_BUFFER]; - - if (!(result_string = (char *) malloc(RESULTS_BUFFER))) { - die("ERROR: Fatal malloc error: snmp.c snmp_get!"); - } - result_string[0] = '\0'; - - if (current_host->ignore_host) { - SPINE_LOG_HIGH(("WARNING: Skipped oid '%s' for Device[%i] as host ignore flag is active", snmp_oid, current_host->id)); - SET_UNDEFINED(result_string); - return result_string; - } - - status = STAT_DESCRIP_ERROR; - - if (current_host->snmp_session != NULL) { - anOID_len = MAX_OID_LEN; - - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: snmp_pdu_create(%s)", current_host->id, snmp_oid)); - pdu = snmp_pdu_create(SNMP_MSG_GET); - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: snmp_pdu_create(%s) [complete]", current_host->id, snmp_oid)); - - if (pdu != NULL) { - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: snmp_parse_oid(%s)", current_host->id, snmp_oid)); - - if (!snmp_parse_oid(snmp_oid, anOID, &anOID_len)) { - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: snmp_parse_oid(%s) [complete]", current_host->id, snmp_oid)); - SPINE_LOG(("Device[%i] ERROR: SNMP Get Problems parsing SNMP OID %s", current_host->id, snmp_oid)); - snmp_free_pdu(pdu); - SET_UNDEFINED(result_string); - return result_string; - } else { - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: snmp_parse_oid(%s) [complete]", current_host->id, snmp_oid)); - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: snmp_add_null_var(%s)", current_host->id, snmp_oid)); - snmp_add_null_var(pdu, anOID, anOID_len); - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: snmp_add_null_var(%s) [complete]", current_host->id, snmp_oid)); - } - - /* poll host */ - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: snmp_sess_sync_response(%s)", current_host->id, snmp_oid)); - status = snmp_sess_synch_response(current_host->snmp_session, pdu, &response); - SPINE_LOG_DEVDBG(("Device[%i] DEBUG: snmp_sess_sync_response(%s) [complete]", current_host->id, snmp_oid)); +static void snmp_multi_values(const host_t *host, snmp_oids_t *oids, int count, const struct variable_list *variable) { + for (int index = 0; index < count; index++) { + if (IS_UNDEFINED(oids[index].result)) continue; + if (variable == NULL) { + SET_UNDEFINED(oids[index].result); + continue; } - - /* retrieve session-level error code for non-success responses */ - if (status == STAT_ERROR && current_host->snmp_session != NULL) { - int liberr = 0, syserr = 0; - char *errstr = NULL; - - snmp_sess_error(current_host->snmp_session, &liberr, &syserr, &errstr); - sess_liberr = liberr; - SPINE_LOG_DEBUG(("Device[%i] DEBUG: SNMP session error for oid '%s': %s (liberr=%d)", - current_host->id, snmp_oid, errstr ? errstr : "unknown", liberr)); - SNMP_FREE(errstr); - } - - /* liftoff, successful poll, process it!! */ - if (status == STAT_DESCRIP_ERROR) { - SPINE_LOG(("ERROR: Unable to create SNMP PDU")); - - SET_UNDEFINED(result_string); - status = STAT_ERROR; - response = NULL; - } else if (status == STAT_SUCCESS) { - if (response == NULL) { - SPINE_LOG(("ERROR: An internal Net-Snmp error condition detected in Cacti snmp_get")); - - SET_UNDEFINED(result_string); - status = STAT_ERROR; - } else if (response->errstat == SNMP_ERR_NOERROR && - response->variables != NULL && - response->variables->name != NULL) { - - vars = response->variables; - - if (vars->type == SNMP_NOSUCHOBJECT) { - if (!strstr(snmp_oid, ".1.3.6.1.2.1.1.1.0") && !strstr(snmp_oid, ".1.3.6.1.2.1.1.3.0")) { - SET_UNDEFINED(result_string); - status = STAT_ERROR; - SPINE_LOG_DEBUG(("DEBUG: OID '%s' for Device[%i], SNMP_NOSUCHOBJECT not sysDesc or sysUptime", snmp_oid, current_host->id)); - } else { - SPINE_LOG_HIGH(("DEBUG: OID '%s' for Device[%i], SNMP_NOSUCHOBJECT for sysDesc or sysUptime", snmp_oid, current_host->id)); - snprint_value(temp_result, RESULTS_BUFFER, vars->name, vars->name_length, vars); - snprintf(result_string, RESULTS_BUFFER, "%s", trim(temp_result)); - } - } else if (vars->type == SNMP_NOSUCHINSTANCE) { - SET_UNDEFINED(result_string); - status = STAT_ERROR; - - // We will ignore the new OID error - if (!strstr(snmp_oid, ".1.3.6.1.6.3.10.2.1.3.0")) { - SPINE_LOG_HIGH(("WARNING: No such Instance for oid '%s' for Device[%i] with Status[%d]", snmp_oid, current_host->id, status)); - } else { - SPINE_LOG_DEBUG(("NOTE: Legacy SNMP agent found! No per second Uptime oid found '%s' for Device[%i] with Status[%d]", snmp_oid, current_host->id, status)); - } - } else if (vars->type == SNMP_ENDOFMIBVIEW) { - SET_UNDEFINED(result_string); - status = STAT_ERROR; - SPINE_LOG_HIGH(("ERROR: End of Mib for oid '%s' for Device[%i] with Status[%d]", snmp_oid, current_host->id, status)); - } else { - snprint_value(temp_result, RESULTS_BUFFER, vars->name, vars->name_length, vars); - - snprintf(result_string, RESULTS_BUFFER, "%s", trim(temp_result)); - } - } else { - SPINE_LOG_HIGH(("ERROR: Failed to get oid '%s' for Device[%i] with Response[%ld]", snmp_oid, current_host->id, response->errstat)); - } - } else if (response != NULL && response->variables != NULL) { - vars = response->variables; - - if (vars->type == SNMP_NOSUCHOBJECT) { - SET_UNDEFINED(result_string); - status = STAT_ERROR; - SPINE_LOG_HIGH(("ERROR: No such Object for oid '%s' for Device[%i] with Status[%d]", snmp_oid, current_host->id, status)); - } else if (vars->type == SNMP_NOSUCHINSTANCE) { - SET_UNDEFINED(result_string); - status = STAT_ERROR; - - // We will ignore the new OID error - if (!strstr(snmp_oid, ".1.3.6.1.6.3.10.2.1.3.0")) { - SPINE_LOG_HIGH(("WARNING: No such Instance for oid '%s' for Device[%i] with Status[%d]", snmp_oid, current_host->id, status)); - } else { - SPINE_LOG_DEBUG(("NOTE: Per second level uptime oid missing oid '%s' for Device[%i] with Status[%d]", snmp_oid, current_host->id, status)); - } - } else if (vars->type == SNMP_ENDOFMIBVIEW) { - SET_UNDEFINED(result_string); - status = STAT_ERROR; - SPINE_LOG_HIGH(("ERROR: End of Mib for oid '%s' for Device[%i] with Status[%d]", snmp_oid, current_host->id, status)); - } else { - SET_UNDEFINED(result_string); - status = STAT_ERROR; - SPINE_LOG_HIGH(("ERROR: Unknown error getting oid '%s' for Device[%i] with Status[%d]", snmp_oid, current_host->id, status)); - } - } else if (status == STAT_TIMEOUT) { - SPINE_LOG_HIGH(("ERROR: Timeout getting oid '%s' for Device[%i] with Status[%d]", snmp_oid, current_host->id, status)); + if (snmp_varbind_is_exception(variable)) { + SPINE_LOG_HIGH(("Device[%i] WARNING: No SNMP data returned for OID '%s'", host->id, oids[index].oid)); + SET_UNDEFINED(oids[index].result); } else { - /* decode USM-specific errors instead of generic "Unknown error" - * New constant names (SNMPERR_NOT_IN_TIME_WINDOW etc.) added in Net-SNMP 5.8. - * Old names (SNMPERR_USM_NOTINTIMEWINDOW etc.) added in Net-SNMP 5.7. - * On older versions, all USM errors fall through to the default case. - */ - switch (sess_liberr) { -#if defined(SNMPERR_NOT_IN_TIME_WINDOW) || defined(SNMPERR_USM_NOTINTIMEWINDOW) -#if defined(SNMPERR_NOT_IN_TIME_WINDOW) - case SNMPERR_NOT_IN_TIME_WINDOW: -#else - case SNMPERR_USM_NOTINTIMEWINDOW: -#endif - SPINE_LOG_MEDIUM(("WARNING: Device[%i] USM notInTimeWindow for oid '%s' -- engine time drift (recoverable)", - current_host->id, snmp_oid)); - /* treat as transient, do not mark host down */ - status = STAT_SUCCESS; - SET_UNDEFINED(result_string); - break; -#endif -#if defined(SNMPERR_UNKNOWN_ENG_ID) || defined(SNMPERR_USM_UNKNOWNENGINEID) -#if defined(SNMPERR_UNKNOWN_ENG_ID) - case SNMPERR_UNKNOWN_ENG_ID: -#else - case SNMPERR_USM_UNKNOWNENGINEID: -#endif - SPINE_LOG_HIGH(("ERROR: Device[%i] USM unknownEngineID for oid '%s'", - current_host->id, snmp_oid)); - break; -#endif -#if defined(SNMPERR_UNKNOWN_USER_NAME) || defined(SNMPERR_USM_UNKNOWNSECURITYNAME) -#if defined(SNMPERR_UNKNOWN_USER_NAME) - case SNMPERR_UNKNOWN_USER_NAME: -#else - case SNMPERR_USM_UNKNOWNSECURITYNAME: -#endif - SPINE_LOG_HIGH(("ERROR: Device[%i] USM unknownSecurityName for oid '%s'", - current_host->id, snmp_oid)); - break; -#endif -#if defined(SNMPERR_AUTHENTICATION_FAILURE) || defined(SNMPERR_USM_AUTHENTICATIONFAILURE) -#if defined(SNMPERR_AUTHENTICATION_FAILURE) - case SNMPERR_AUTHENTICATION_FAILURE: -#else - case SNMPERR_USM_AUTHENTICATIONFAILURE: -#endif - SPINE_LOG_HIGH(("ERROR: Device[%i] USM authenticationFailure for oid '%s'", - current_host->id, snmp_oid)); - break; -#endif -#if defined(SNMPERR_DECRYPTION_ERR) || defined(SNMPERR_USM_DECRYPTIONERROR) -#if defined(SNMPERR_DECRYPTION_ERR) - case SNMPERR_DECRYPTION_ERR: -#else - case SNMPERR_USM_DECRYPTIONERROR: -#endif - SPINE_LOG_HIGH(("ERROR: Device[%i] USM decryptionError for oid '%s'", - current_host->id, snmp_oid)); - break; -#endif -#if defined(SNMPERR_UNSUPPORTED_SEC_LEVEL) || defined(SNMPERR_USM_UNSUPPORTEDSECURITYLEVEL) -#if defined(SNMPERR_UNSUPPORTED_SEC_LEVEL) - case SNMPERR_UNSUPPORTED_SEC_LEVEL: -#else - case SNMPERR_USM_UNSUPPORTEDSECURITYLEVEL: -#endif - SPINE_LOG_HIGH(("ERROR: Device[%i] USM unsupportedSecurityLevel for oid '%s'", - current_host->id, snmp_oid)); - break; -#endif - default: - SPINE_LOG_HIGH(("ERROR: Unknown error getting oid '%s' for Device[%i] with Status[%d] Errno[%d]", - snmp_oid, current_host->id, status, sess_liberr)); - break; - } + snmp_format_scalar(oids[index].result, variable, FALSE); } - - /* update host status after all adjustments */ - current_host->snmp_status = status; - - if (response != NULL && status != STAT_DESCRIP_ERROR) { - snmp_free_pdu(response); - response = NULL; - } - } - - if (status != STAT_SUCCESS && should_fail) { - current_host->ignore_host = TRUE; - - SET_UNDEFINED(result_string); - } - - return result_string; -} - -char *snmp_get(host_t *current_host, const char *snmp_oid) { - return snmp_get_base(current_host, snmp_oid, true); -} - -char *snmp_get_allow_fail(host_t *current_host, const char *snmp_oid) { - return snmp_get_base(current_host, snmp_oid, false); -} - -/*! \fn char *snmp_getnext(host_t *current_host, const char *snmp_oid) - * \brief performs a single snmp_getnext for a specific snmp OID - * - * This function will poll a specific snmp OID for a host. The host snmp - * session must already be established. - * - * \return returns the character representation of the snmp OID, or "U" if - * unsuccessful. - * - */ -char *snmp_getnext(host_t *current_host, const char *snmp_oid) { - struct snmp_pdu *pdu = NULL; - struct snmp_pdu *response = NULL; - struct variable_list *vars = NULL; - size_t anOID_len = MAX_OID_LEN; - oid anOID[MAX_OID_LEN]; - int status; - int sess_liberr = SNMPERR_SUCCESS; - char *result_string; - char temp_result[RESULTS_BUFFER]; - - if (!(result_string = (char *) malloc(RESULTS_BUFFER))) { - die("ERROR: Fatal malloc error: snmp.c snmp_get!"); - } - result_string[0] = '\0'; - - status = STAT_DESCRIP_ERROR; - - if (current_host->snmp_session != NULL) { - anOID_len = MAX_OID_LEN; - pdu = snmp_pdu_create(SNMP_MSG_GETNEXT); - - if (!snmp_parse_oid(snmp_oid, anOID, &anOID_len)) { - SPINE_LOG(("Device[%i] ERROR: SNMP Getnext Problems parsing SNMP OID %s", current_host->id, snmp_oid)); - snmp_free_pdu(pdu); - SET_UNDEFINED(result_string); - return result_string; - } else { - snmp_add_null_var(pdu, anOID, anOID_len); - } - - /* poll host */ - status = snmp_sess_synch_response(current_host->snmp_session, pdu, &response); - - /* retrieve session-level error code for non-success responses */ - if (status == STAT_ERROR && current_host->snmp_session != NULL) { - int liberr = 0, syserr = 0; - char *errstr = NULL; - - snmp_sess_error(current_host->snmp_session, &liberr, &syserr, &errstr); - sess_liberr = liberr; - SPINE_LOG_DEBUG(("Device[%i] DEBUG: SNMP getnext session error for oid '%s': %s (liberr=%d)", - current_host->id, snmp_oid, errstr ? errstr : "unknown", liberr)); - SNMP_FREE(errstr); - } - - /* liftoff, successful poll, process it!! */ - if (status == STAT_SUCCESS) { - if (response == NULL) { - SPINE_LOG(("ERROR: An internal Net-Snmp error condition detected in Cacti snmp_getnext")); - - SET_UNDEFINED(result_string); - status = STAT_ERROR; - } else { - if (response->errstat == SNMP_ERR_NOERROR) { - vars = response->variables; - - if (vars != NULL && !snmp_varbind_is_exception(vars)) { - snprint_value(temp_result, RESULTS_BUFFER, vars->name, vars->name_length, vars); - - snprint_asciistring(result_string, RESULTS_BUFFER, (unsigned char *)temp_result, strlen(temp_result)); - } else { - SET_UNDEFINED(result_string); - status = STAT_ERROR; - } - } - } - } else if (status == STAT_TIMEOUT) { - SPINE_LOG_HIGH(("ERROR: Timeout getting oid '%s' for Device[%i] with Status[%d]", snmp_oid, current_host->id, status)); - } else if (status == STAT_ERROR) { - /* decode USM-specific errors instead of generic "Unknown error" - * See version notes in snmp_get_base above. - */ - switch (sess_liberr) { -#if defined(SNMPERR_NOT_IN_TIME_WINDOW) || defined(SNMPERR_USM_NOTINTIMEWINDOW) -#if defined(SNMPERR_NOT_IN_TIME_WINDOW) - case SNMPERR_NOT_IN_TIME_WINDOW: -#else - case SNMPERR_USM_NOTINTIMEWINDOW: -#endif - SPINE_LOG_MEDIUM(("WARNING: Device[%i] USM notInTimeWindow for getnext oid '%s' -- engine time drift (recoverable)", - current_host->id, snmp_oid)); - /* treat as transient, do not mark host down */ - status = STAT_SUCCESS; - SET_UNDEFINED(result_string); - break; -#endif -#if defined(SNMPERR_UNKNOWN_ENG_ID) || defined(SNMPERR_USM_UNKNOWNENGINEID) -#if defined(SNMPERR_UNKNOWN_ENG_ID) - case SNMPERR_UNKNOWN_ENG_ID: -#else - case SNMPERR_USM_UNKNOWNENGINEID: -#endif - SPINE_LOG_HIGH(("ERROR: Device[%i] USM unknownEngineID for getnext oid '%s'", - current_host->id, snmp_oid)); - break; -#endif -#if defined(SNMPERR_UNKNOWN_USER_NAME) || defined(SNMPERR_USM_UNKNOWNSECURITYNAME) -#if defined(SNMPERR_UNKNOWN_USER_NAME) - case SNMPERR_UNKNOWN_USER_NAME: -#else - case SNMPERR_USM_UNKNOWNSECURITYNAME: -#endif - SPINE_LOG_HIGH(("ERROR: Device[%i] USM unknownSecurityName for getnext oid '%s'", - current_host->id, snmp_oid)); - break; -#endif -#if defined(SNMPERR_AUTHENTICATION_FAILURE) || defined(SNMPERR_USM_AUTHENTICATIONFAILURE) -#if defined(SNMPERR_AUTHENTICATION_FAILURE) - case SNMPERR_AUTHENTICATION_FAILURE: -#else - case SNMPERR_USM_AUTHENTICATIONFAILURE: -#endif - SPINE_LOG_HIGH(("ERROR: Device[%i] USM authenticationFailure for getnext oid '%s'", - current_host->id, snmp_oid)); - break; -#endif -#if defined(SNMPERR_DECRYPTION_ERR) || defined(SNMPERR_USM_DECRYPTIONERROR) -#if defined(SNMPERR_DECRYPTION_ERR) - case SNMPERR_DECRYPTION_ERR: -#else - case SNMPERR_USM_DECRYPTIONERROR: -#endif - SPINE_LOG_HIGH(("ERROR: Device[%i] USM decryptionError for getnext oid '%s'", - current_host->id, snmp_oid)); - break; -#endif -#if defined(SNMPERR_UNSUPPORTED_SEC_LEVEL) || defined(SNMPERR_USM_UNSUPPORTEDSECURITYLEVEL) -#if defined(SNMPERR_UNSUPPORTED_SEC_LEVEL) - case SNMPERR_UNSUPPORTED_SEC_LEVEL: -#else - case SNMPERR_USM_UNSUPPORTEDSECURITYLEVEL: -#endif - SPINE_LOG_HIGH(("ERROR: Device[%i] USM unsupportedSecurityLevel for getnext oid '%s'", - current_host->id, snmp_oid)); - break; -#endif - default: - SPINE_LOG_HIGH(("ERROR: Unknown error getting oid '%s' for Device[%i] with Status[%d] Errno[%d]", - snmp_oid, current_host->id, status, sess_liberr)); - break; - } - } - - /* update host status after all adjustments */ - current_host->snmp_status = status; - - if (response) { - snmp_free_pdu(response); - response = NULL; - } - } else { - status = STAT_DESCRIP_ERROR; - } - - if (status != STAT_SUCCESS) { - current_host->ignore_host = TRUE; - - SET_UNDEFINED(result_string); - } - - return result_string; -} - -/*! \fn char *snmp_count(host_t *current_host, char *snmp_oid) - * \brief counts entries of snmp table specified by a specific snmp OID - * - * This function will poll a specific snmp OID for a host. The host snmp - * session must already be established. - * - * \return returns count of table entries - * - */ -int snmp_count(host_t *current_host, const char *snmp_oid) { - struct snmp_pdu *pdu = NULL; - struct snmp_pdu *response = NULL; - struct variable_list *vars = NULL; - size_t anOID_len = MAX_OID_LEN; - size_t rootlen = MAX_OID_LEN; - oid anOID[MAX_OID_LEN]; - oid root[MAX_OID_LEN]; - int status; - int ok = 1; - int error_occurred = 0; - int count = 0; - - status = STAT_DESCRIP_ERROR; - - if (is_debug_device(current_host->id)) { - SPINE_LOG(("DEBUG: walk starts at OID %s", snmp_oid)); - } else { - SPINE_LOG_DEBUG(("DEBUG: walk starts at OID %s", snmp_oid)); - } - - if (current_host->snmp_session != NULL) { - rootlen = MAX_OID_LEN; - /* parse input parm to an array for use with snmp functions */ - if (!snmp_parse_oid(snmp_oid, root, &rootlen)) { - SPINE_LOG(("Device[%i] ERROR: SNMP Count Problems parsing SNMP OID %s", current_host->id, snmp_oid)); - return -1; - } - memmove(anOID, root, rootlen * sizeof(oid)); - anOID_len = rootlen; - - while (ok && !error_occurred) { - /* create PDU for GETNEXT request */ - pdu = snmp_pdu_create(SNMP_MSG_GETNEXT); - snmp_add_null_var(pdu, anOID, anOID_len); - - /* do the request, use thread safe call */ - status = snmp_sess_synch_response(current_host->snmp_session, pdu, &response); - - /* add status to host structure */ - current_host->snmp_status = status; - - //SPINE_LOG_DEBUG(("TRACE: Status %i Response %i", status, response->errstat)); - - if (status == STAT_SUCCESS) { - if (response == NULL) { - SPINE_LOG(("ERROR: Device[%i] internal Net-SNMP error in snmp_count for OID %s", current_host->id, snmp_oid)); - ok = 0; - error_occurred = 1; - } else if (response->errstat == SNMP_ERR_NOERROR) { - /* check resulting variables */ - for (vars = response->variables; vars; vars = vars->next_variable) { - if ((vars->name_length < rootlen) || (memcmp(root, vars->name, rootlen * sizeof(oid)) != 0)) { - /* next OID is not part of snmptable */ - ok = 0; - continue; - } - /* END OF MIB or NO SUCH OBJECT or NO SUCH INSTANCE */ - if ((vars->type != SNMP_ENDOFMIBVIEW) && - (vars->type != SNMP_NOSUCHOBJECT) && - (vars->type != SNMP_NOSUCHINSTANCE)) { - /* count only real entries, not the walk terminator */ - count++; - /* valid data, so perform a compare */ - if (snmp_oid_compare(anOID, anOID_len, vars->name, vars->name_length) >= 0) { - SPINE_LOG(("ERROR: OID not increasing")); - ok = 0; - error_occurred = 1; - } - /* prepare next turn */ - memmove((char *) anOID, (char *) vars->name, vars->name_length * sizeof(oid)); - anOID_len = vars->name_length; - } else { - /* abnormal end of loop */ - ok = 0; - } - } - } else if (response->errstat == SNMP_ERR_NOSUCHNAME) { - /* SNMPv1 has no endOfMibView variable type; it reports the normal - * end of a GETNEXT walk as a PDU-level noSuchName instead. */ - ok = 0; - } else { - SPINE_LOG(("ERROR: Device[%i] internal Net-SNMP error %ld in snmp_count for OID %s", current_host->id, response->errstat, snmp_oid)); - ok = 0; - error_occurred = 1; - } - } else if (status == STAT_TIMEOUT) { - SPINE_LOG(("ERROR: Device[%i] timeout in snmp_count for OID %s", current_host->id, snmp_oid)); - ok = 0; - error_occurred = 1; - } else { /* status == STAT_ERROR */ - SPINE_LOG(("ERROR: Device[%i] internal Net-SNMP error in snmp_count for OID %s (STAT_ERROR)", current_host->id, snmp_oid)); - ok = 0; - error_occurred = 1; - } - - if (response) { - snmp_free_pdu(response); - } - } - } else { - status = STAT_DESCRIP_ERROR; - error_occurred = 1; - } - - if (status != STAT_SUCCESS) { - current_host->ignore_host = TRUE; - } - - /* A negative count tells the caller this walk never produced a usable - * result, so it is not mistaken for a legitimate zero-item count. */ - return error_occurred ? -1 : count; -} - -/*! \fn void snmp_snprint_value(char *obuf, size_t buf_len, const oid *objid, size_t objidlen, struct variable_list *variable) - * - * \brief replacement for the buggy net-snmp.org snprint_value function - * - * This function format an output buffer with the correct string representation - * of an snmp OID result fetched with snmp_get_multi. The buffer pointed to by - * the function is modified. - * - */ -void snmp_snprint_value(char *obuf, size_t buf_len, const oid *objid, size_t objidlen, struct variable_list *variable) { - u_char *buf = NULL; - size_t out_len = 0; - UNUSED_PARAMETER(objid); - UNUSED_PARAMETER(objidlen); - - if (buf_len > 0) { - if ((buf = (u_char *) calloc(buf_len, 1)) != 0) { - sprint_realloc_by_type(&buf, &buf_len, &out_len, 0, variable, NULL, NULL, NULL); - snprintf(obuf, buf_len, "%s", buf); - } else { - SET_UNDEFINED(obuf); - } - - free(buf); - } else { - SET_UNDEFINED(obuf); + variable = variable->next_variable; } } -/*! \fn char *snmp_get_multi(host_t *current_host, target_t *poller_items, snmp_oids_t *snmp_oids, int num_oids) +/*! \fn char *snmp_get_multi(host_t *current_host, const target_t *poller_items, snmp_oids_t *snmp_oids, int num_oids) * \brief performs multiple OID snmp_get's in a single network call * * This function will a group of snmp OID's for a host. The host snmp @@ -1203,155 +1007,60 @@ void snmp_snprint_value(char *obuf, size_t buf_len, const oid *objid, size_t obj * the snmp_oids array with the results from the snmp api call. * */ -void snmp_get_multi(host_t *current_host, target_t *poller_items, snmp_oids_t *snmp_oids, int num_oids) { - struct snmp_pdu *pdu = NULL; - struct snmp_pdu *response = NULL; - struct variable_list *vars = NULL; - int status; - int i; - int array_count; - int index_count; - char temp_result[RESULTS_BUFFER]; - - struct nameStruct { - oid name[MAX_OID_LEN]; - size_t name_len; - } *name, *namep; - - /* A per-item credential change can fail to rebuild the session after the - * caller's earlier NULL check. Fail the pending group as one host error - * instead of passing NULL into net-snmp. */ - if (current_host == NULL || current_host->snmp_session == NULL) { - if (current_host != NULL) { - current_host->ignore_host = TRUE; - } - if (snmp_oids != NULL) { - for (i = 0; i < num_oids; i++) { - SET_UNDEFINED(snmp_oids[i].result); - } - } +void snmp_get_multi(host_t *host, const target_t *items, snmp_oids_t *oids, int count) { + if (count <= 0) return; + if (host == NULL || items == NULL || oids == NULL) die("ERROR: Invalid multi-SNMP request storage"); + if (host->snmp.session == NULL) { + /* A failed mid-loop session rebuild fails the group as one host error. */ + host->ignore_host = TRUE; + snmp_multi_undefined(oids, count); + host->snmp.status = STAT_DESCRIP_ERROR; return; } - - /* load up oids */ - namep = name = (struct nameStruct *) calloc(num_oids, sizeof(*name)); - if (name == NULL) { - SPINE_LOG(("ERROR: Failed to allocate memory for SNMP OID name array")); + struct snmp_pdu *request = snmp_multi_request(host, items, oids, count); + if (request == NULL) { + snmp_multi_undefined(oids, count); + host->snmp.status = STAT_ERROR; return; } - pdu = snmp_pdu_create(SNMP_MSG_GET); - for (i = 0; i < num_oids; i++) { - namep->name_len = MAX_OID_LEN; - - if (!snmp_parse_oid(snmp_oids[i].oid, namep->name, &namep->name_len)) { - SPINE_LOG(("Device[%i] DS[%i] ERROR: Problems parsing Multi SNMP OID! (oid: %s), Set MAX_OIDS to 1 for this host to isolate bad OID", current_host->id, poller_items[snmp_oids[i].array_position].local_data_id, snmp_oids[i].oid)); - - /* Mark this OID as "bad" */ - SET_UNDEFINED(snmp_oids[i].result); - } else { - snmp_add_null_var(pdu, namep->name, namep->name_len); - } - - namep++; - } - - status = STAT_DESCRIP_ERROR; - - /* execute the multi-get request */ - retry: - status = snmp_sess_synch_response(current_host->snmp_session, pdu, &response); - - /* add status to host structure */ - current_host->snmp_status = status; - - /* liftoff, successful poll, process it!! */ - if (status == STAT_SUCCESS) { - if (response == NULL) { - SPINE_LOG(("ERROR: An internal Net-Snmp error condition detected in Cacti snmp_get_multi")); - status = STAT_ERROR; - } else { - if (response->errstat == SNMP_ERR_NOERROR) { - vars = response->variables; - - for (i = 0; i < num_oids && vars; i++) { - if (!IS_UNDEFINED(snmp_oids[i].result)) { - /* Under v2c an agent reports a per-OID failure as an - * exception varbind while the PDU errstat stays - * NOERROR. Without this check snprint_value() renders - * the exception as text and it is stored as a value. */ - if (snmp_varbind_is_exception(vars)) { - SPINE_LOG_HIGH(("Device[%i] WARNING: No SNMP data returned for OID '%s'", current_host->id, snmp_oids[i].oid)); - - SET_UNDEFINED(snmp_oids[i].result); - } else { - snprint_value(temp_result, RESULTS_BUFFER, vars->name, vars->name_length, vars); - - snprintf(snmp_oids[i].result, RESULTS_BUFFER, "%s", trim(temp_result)); - } - - vars = vars->next_variable; - } - } + int status = STAT_DESCRIP_ERROR; + /* Every v1 retry removes one valid OID, so at most count requests are + * possible. The synchronous call consumes each submitted request. */ + int attempt = 0; + while (request != NULL && attempt < count) { + attempt++; + struct snmp_pdu *response = NULL; + status = snmp_sess_synch_response(host->snmp.session, request, &response); + request = NULL; + host->snmp.status = status; + if (status == STAT_SUCCESS) { + if (response == NULL) { + SPINE_LOG(("ERROR: An internal Net-Snmp error condition detected in Cacti snmp_get_multi")); + status = STAT_ERROR; + snmp_multi_undefined(oids, count); + } else if (response->errstat == SNMP_ERR_NOERROR) { + snmp_multi_values(host, oids, count, response->variables); + } else if (snmp_multi_error_index(oids, count, response->errindex)) { + request = snmp_fix_pdu(response, SNMP_MSG_GET); } else { - if (response->errindex != 0) { - index_count = 1; - array_count = 0; - - /* Find our index against errindex */ - while (array_count < num_oids) { - if (IS_UNDEFINED(snmp_oids[array_count].result) ) { - array_count++; - } else { - /* if we have found our error, exit */ - if (index_count == response->errindex) { - SET_UNDEFINED(snmp_oids[array_count].result); - - break; - } - array_count++; - index_count++; - } - - } - - /* remove the invalid OID from the PDU */ - pdu = snmp_fix_pdu(response, SNMP_MSG_GET); - - /* free the previous response */ - snmp_free_pdu(response); - - response = NULL; - if (pdu != NULL) { - /* retry the request */ - goto retry; - } else { - /* all OID's errored out so exit cleanly */ - status = STAT_SUCCESS; - } - } else { - /* errindex==0: agent reported a PDU-level error with no specific - * failing OID (RFC 3416 s.4.2). Mark all pending results UNDEFINED - * so callers write "U" to RRD rather than an empty string. */ + if (response->errindex == 0) { + /* RFC 3416 4.2: a PDU-level error names no failing OID. */ SPINE_LOG_HIGH(("Device[%i] WARNING: snmp_get_multi PDU error (errstat=%ld, errindex=0), all OIDs set undefined", - current_host->id, (long)response->errstat)); - for (i = 0; i < num_oids; i++) { - SET_UNDEFINED(snmp_oids[i].result); - } + host->id, (long)response->errstat)); } + status = STAT_ERROR; + snmp_multi_undefined(oids, count); } } + if (response != NULL) snmp_free_pdu(response); + } + if (request != NULL) { + snmp_free_pdu(request); + status = STAT_ERROR; + snmp_multi_undefined(oids, count); } - if (status == STAT_TIMEOUT) { - current_host->ignore_host = 1; - for (i = 0; i < num_oids; i++) { - SET_UNDEFINED(snmp_oids[i].result); - } + host->ignore_host = TRUE; + snmp_multi_undefined(oids, count); } - - if (response != NULL) { - snmp_free_pdu(response); - } - - free(name); } diff --git a/snmp.h b/snmp.h index f15cd97..d8c0033 100644 --- a/snmp.h +++ b/snmp.h @@ -37,15 +37,36 @@ extern void snmp_spine_init(void); extern void snmp_spine_close(void); -extern void *snmp_host_init(int host_id, char *hostname, int snmp_version, char *snmp_community, char *snmp_username, const char *snmp_password, char *snmp_auth_protocol, const char *snmp_priv_passphrase, char *snmp_priv_protocol, char *snmp_context, char *snmp_engine_id, int snmp_port, int snmp_timeout); +/* Borrowed connection inputs; Net-SNMP clones the session before return. */ +typedef struct { + int host_id; + char *hostname; + int snmp_version; + char *snmp_community; + char *snmp_username; + const char *snmp_password; + char *snmp_auth_protocol; + const char *snmp_priv_passphrase; + char *snmp_priv_protocol; + char *snmp_context; + char *snmp_engine_id; + int snmp_port; + int snmp_timeout; +} snmp_connection_t; + +extern void *snmp_host_init(const snmp_connection_t *options); extern void snmp_host_cleanup(void *snmp_session); -extern char *snmp_get_base(host_t *current_host, const char *snmp_oid, bool should_fail); +extern char *snmp_get_base(host_t *current_host, char *snmp_oid, bool should_fail); +extern char *snmp_get(host_t *current_host, char *snmp_oid); +extern char *snmp_get_allow_fail(host_t *current_host, char *snmp_oid); +extern char *snmp_getnext(host_t *current_host, char *snmp_oid); +extern int spine_snmpv3_protocol_is_set(const char *value); +extern int spine_snmpv3_passphrase_is_set(const char *value); +extern int spine_snmpv3_security_level(const char *auth_protocol, const char *auth_password, + const char *priv_protocol, const char *priv_passphrase); extern int snmp_varbind_is_exception(const struct variable_list *vars); -extern char *snmp_get(host_t *current_host, const char *snmp_oid); -extern char *snmp_get_allow_fail(host_t *current_host, const char *snmp_oid); -extern char *snmp_getnext(host_t *current_host, const char *snmp_oid); -extern int snmp_count(host_t *current_host, const char *snmp_oid); -extern void snmp_get_multi(host_t *current_host, target_t *poller_items, snmp_oids_t *snmp_oids, int num_oids); -extern void snmp_snprint_value(char *obuf, size_t buf_len, const oid *objid, size_t objidlen, struct variable_list *variable); +extern int snmp_count(host_t *current_host, char *snmp_oid); +extern void snmp_get_multi(host_t *current_host, const target_t *poller_items, snmp_oids_t *snmp_oids, int num_oids); +extern void snmp_snprint_value(char *obuf, size_t buf_len, const oid *objid, size_t objidlen, const struct variable_list *variable); #endif /* SPINE_SNMP_H */ diff --git a/spine.c b/spine.c index 29f9886..3d88adf 100644 --- a/spine.c +++ b/spine.c @@ -97,12 +97,17 @@ #include "common.h" #include "spine.h" +#include + +#ifdef SPINE_TEST_PROGRAM_ENTRY +#define main spine_program_main +#endif /* Global Variables */ int entries = 0; int num_hosts = 0; -spine_sem_t available_threads; -spine_sem_t available_scripts; +spine_permits_t available_threads; +spine_permits_t available_scripts; double start_time; double total_time; @@ -118,7 +123,6 @@ poller_thread_t** details = NULL; static char *getarg(char *opt, char ***pargv); static void display_help(int only_version); -void poller_push_data_to_main(void); #ifdef HAVE_LCAP /* This patch is adapted (copied) patch for ntpd from Jarno Huuskonen and @@ -191,42 +195,878 @@ void drop_root(uid_t server_uid, gid_t server_gid) { * \return 0 if SUCCESS, or -1 if FAILED * */ +typedef enum { + CLI_FIRST, + CLI_LAST, + CLI_POLLER, + CLI_THREADS, + CLI_PINGONLY, + CLI_MODE, + CLI_HOSTLIST, + CLI_MIBS, + CLI_HELP, + CLI_VERSION, + CLI_OPTION, + CLI_READONLY, + CLI_CONF, + CLI_STDOUT, + CLI_LOG, + CLI_VERBOSITY, + CLI_UNKNOWN +} cli_option_t; + +typedef struct { + const char *name; + cli_option_t option; + bool ignore_case; +} cli_alias_t; + +static cli_option_t lookup_cli_option(const char *arg) { + static const cli_alias_t aliases[] = { + {"-f", CLI_FIRST, TRUE}, + {"--first", CLI_FIRST, FALSE}, + {"-l", CLI_LAST, TRUE}, + {"--last", CLI_LAST, TRUE}, + {"-p", CLI_POLLER, FALSE}, + {"--poller", CLI_POLLER, TRUE}, + {"-t", CLI_THREADS, FALSE}, + {"--threads", CLI_THREADS, TRUE}, + {"-P", CLI_PINGONLY, FALSE}, + {"--pingonly", CLI_PINGONLY, TRUE}, + {"-N", CLI_MODE, FALSE}, + {"--mode", CLI_MODE, TRUE}, + {"-H", CLI_HOSTLIST, FALSE}, + {"--hostlist", CLI_HOSTLIST, TRUE}, + {"-M", CLI_MIBS, TRUE}, + {"--mibs", CLI_MIBS, FALSE}, + {"-h", CLI_HELP, TRUE}, + {"--help", CLI_HELP, FALSE}, + {"-v", CLI_VERSION, FALSE}, + {"--version", CLI_VERSION, FALSE}, + {"-O", CLI_OPTION, TRUE}, + {"--option", CLI_OPTION, TRUE}, + {"-R", CLI_READONLY, TRUE}, + {"--readonly", CLI_READONLY, FALSE}, + {"--read-only", CLI_READONLY, FALSE}, + {"-C", CLI_CONF, TRUE}, + {"--conf", CLI_CONF, FALSE}, + {"-S", CLI_STDOUT, TRUE}, + {"--stdout", CLI_STDOUT, FALSE}, + {"-D", CLI_LOG, TRUE}, + {"--log", CLI_LOG, FALSE}, + {"-V", CLI_VERBOSITY, FALSE}, + {"--verbosity", CLI_VERBOSITY, FALSE}, + }; + for (size_t index = 0; index < sizeof(aliases) / sizeof(aliases[0]); index++) { + bool matches = aliases[index].ignore_case ? STRIMATCH(arg, aliases[index].name) : STRMATCH(arg, aliases[index].name); + if (matches) return aliases[index].option; + } + return CLI_UNKNOWN; +} + +static void parse_polling_mode(const char *requested_mode) { + if (STRIMATCH(requested_mode, "online")) { + set.poller.mode = REMOTE_ONLINE; + } else if (STRIMATCH(requested_mode, "offline")) { + set.poller.mode = REMOTE_OFFLINE; + } else if (STRIMATCH(requested_mode, "recovery")) { + set.poller.mode = REMOTE_RECOVERY; + } else { + die("ERROR: invalid polling mode '%s' specified", requested_mode); + } +} + +static void parse_host_list(const char *input) { + if (strnlen(input, sizeof(set.hosts.host_id_list)) == sizeof(set.hosts.host_id_list)) { + die("ERROR: invalid or oversized host list"); + } + size_t used = 0; + const char *cursor = input; + for (;;) { + while (isspace((unsigned char)*cursor)) cursor++; + if (*cursor < '0' || *cursor > '9') die("ERROR: invalid or oversized host list"); + errno = 0; + char *end; + unsigned long id = strtoul(cursor, &end, 10); + if (errno == ERANGE || id > INT_MAX) die("ERROR: invalid or oversized host list"); + while (isspace((unsigned char)*end)) end++; + if (*end != '\0' && *end != ',') die("ERROR: invalid or oversized host list"); + /* SQL receives only converted integer values and owned separators. */ + used += (size_t)spine_snprintf(set.hosts.host_id_list + used, + sizeof(set.hosts.host_id_list) - used, "%s%lu", used == 0 ? "" : ",", id); + if (*end == '\0') return; + cursor = end + 1; + } +} + +static void parse_cli_argument(const char *arg, char *opt, char ***argv, char **conf_file) { + switch (lookup_cli_option(arg)) { + case CLI_FIRST: { + if (HOSTID_DEFINED(set.hosts.start_host_id)) { + die("ERROR: %s can only be used once", arg); + } + + opt = getarg(opt, argv); + set.hosts.start_host_id = atoi(opt); + + if (!HOSTID_DEFINED(set.hosts.start_host_id)) { + die("ERROR: '%s=%s' is invalid first-host ID", arg, opt); + } + break; + } + case CLI_LAST: { + if (HOSTID_DEFINED(set.hosts.end_host_id)) { + die("ERROR: %s can only be used once", arg); + } + + opt = getarg(opt, argv); + set.hosts.end_host_id = atoi(opt); + + if (!HOSTID_DEFINED(set.hosts.end_host_id)) { + die("ERROR: '%s=%s' is invalid last-host ID", arg, opt); + } + break; + } + case CLI_POLLER: { + set.poller.poller_id = atoi(getarg(opt, argv)); + break; + } + case CLI_THREADS: { + set.poller.threads = atoi(getarg(opt, argv)); + set.poller.threads_set = TRUE; + break; + } + case CLI_PINGONLY: { + set.availability.ping_only = TRUE; + break; + } + case CLI_MODE: { + parse_polling_mode(getarg(opt, argv)); + break; + } + case CLI_HOSTLIST: { + parse_host_list(getarg(opt, argv)); + break; + } + case CLI_MIBS: { + set.snmp.mibs = 1; + break; + } + case CLI_HELP: { + display_help(FALSE); + + exit(EXIT_SUCCESS); + } + case CLI_VERSION: { + display_help(TRUE); + + exit(EXIT_SUCCESS); + } + case CLI_OPTION: { + const char *setting = getarg(opt, argv); + char *value = strchr(setting, ':'); + + if (value != NULL && value != setting) { + *value++ = '\0'; + } else { + die("ERROR: -O requires setting:value"); + } + + set_option(setting, value); + break; + } + case CLI_READONLY: { + set.poller.SQL_readonly = TRUE; + break; + } + case CLI_CONF: { + char *replacement = strdup(getarg(opt, argv)); + if (replacement == NULL) die("ERROR: Fatal malloc error: spine.c conf_file!"); + SPINE_FREE(*conf_file); + *conf_file = replacement; + break; + } + case CLI_STDOUT: { + set_option("log_destination", "STDOUT"); + break; + } + case CLI_LOG: { + set_option("log_destination", getarg(opt, argv)); + break; + } + case CLI_VERBOSITY: { + set_option("log_verbosity", getarg(opt, argv)); + break; + } + default: + if (!HOSTID_DEFINED(set.hosts.start_host_id) && all_digits(arg)) { + set.hosts.start_host_id = atoi(arg); + } + + else if (!HOSTID_DEFINED(set.hosts.end_host_id) && all_digits(arg)) { + set.hosts.end_host_id = atoi(arg); + } + + else { + die("ERROR: %s is an unknown command-line parameter", arg); + } + break; + } +} + +static void parse_command_line(char **argv, char **conf_file) { + argv++; + while (*argv) { + const char *arg = *argv; + char *opt = strchr(arg, '='); + if (opt) *opt++ = '\0'; + parse_cli_argument(arg, opt, &argv, conf_file); + argv++; + } +} + +static bool wait_for_worker_permit(spine_permits_t *permit, int host_id, int host_thread, const char *label) { + int retries = 0; + for (;;) { + int error = spine_permits_try_acquire(permit); + if (error == 0) return TRUE; + if (error == EDEADLK) { + SPINE_LOG_DEVDBG(("WARNING: Device[%i] HT[%i] would have deadlocked acquiring %s", host_id, host_thread, label)); + } else if (error != EINTR && error != EAGAIN) { + SPINE_LOG_DEVDBG(("WARNING: Device[%i] HT[%i] errored with %d while acquiring %s", host_id, host_thread, error, label)); + } + if (++retries == 10) { + if (get_time_as_double() - start_time + 1 > set.poller.poller_interval) { + SPINE_LOG(("ERROR: Device[%i] HT[%i] polling timed out while acquiring %s", host_id, host_thread, label)); + return FALSE; + } + retries = 0; + } + spine_sleep_usec(10000); + total_time = get_time_as_double(); + if (total_time - start_time > set.poller.poller_interval) { + SPINE_LOG(("ERROR: Device[%i] HT[%i] Spine Timed Out While Processing Devices (%s)", host_id, host_thread, label)); + return FALSE; + } + } +} + +static bool acquire_worker_permits(spine_permits_t *startup, int host_id, int host_thread) { + if (!wait_for_worker_permit(&available_threads, host_id, host_thread, "Available Thread Lock")) return FALSE; + if (wait_for_worker_permit(startup, host_id, host_thread, "Thread Initialization Lock")) return TRUE; + spine_permits_release(&available_threads); + return FALSE; +} + +static char *load_startup_configuration(char *conf_file) { + int valid_conf_file = FALSE; + /* read configuration file to establish local environment */ + if (conf_file) { + if ((read_spine_config(conf_file)) < 0) { + die("ERROR: Could not read config file: %s", conf_file); + } else { + valid_conf_file = TRUE; + } + } else { + if (!(conf_file = calloc(CONFIG_PATHS, DBL_BUFSIZE))) { + die("ERROR: Fatal malloc error: spine.c conf_file!"); + } + + for (int i=0; i= 0) { + valid_conf_file = TRUE; + break; + } + + if (i == CONFIG_PATHS-1) { + snprintf(conf_file, DBL_BUFSIZE, "%.*s%s", (int) sizeof(config_paths[0]) - 1, config_paths[0], DEFAULT_CONF_FILE); + } + } + } + + if (valid_conf_file) { + /* read settings table from the database to further establish environment */ + read_config_options(); + } else { + die("FATAL: Unable to read configuration file!"); + } + + return conf_file; +} + +static MYSQL_RES *select_poll_hosts(MYSQL *mysql) { + char querybuf[MEGA_BUFSIZE]; + char *qp = querybuf; + /* obtain the list of hosts to poll */ + qp += spine_snprintf(qp, sizeof(querybuf) - (size_t)(qp - querybuf), "SELECT SQL_NO_CACHE id, device_threads, picount, picount/device_threads AS tppi FROM host AS h LEFT JOIN (SELECT host_id, COUNT(*) AS picount FROM poller_item GROUP BY host_id) AS pi ON h.id = pi.host_id"); + qp += spine_snprintf(qp, sizeof(querybuf) - (size_t)(qp - querybuf), " WHERE disabled = ''"); + + qp += spine_snprintf(qp, sizeof(querybuf) - (size_t)(qp - querybuf), " AND availability_method != %d", AVAIL_STREAM); + + if (!strlen(set.hosts.host_id_list)) { + qp += append_hostrange(qp, sizeof(querybuf) - (size_t)(qp - querybuf), "h.id"); /* AND id BETWEEN a AND b */ + } else { + qp += spine_snprintf(qp, sizeof(querybuf) - (size_t)(qp - querybuf), " AND h.id IN(%s)", set.hosts.host_id_list); + } + + qp += spine_snprintf(qp, sizeof(querybuf) - (size_t)(qp - querybuf), " AND h.poller_id = %i", set.poller.poller_id); + spine_snprintf(qp, sizeof(querybuf) - (size_t)(qp - querybuf), " ORDER BY picount DESC"); + + SPINE_LOG_DEVDBG(("DEVDBG: Host SQL:%s", querybuf)); + return db_query(mysql, LOCAL, querybuf); + +} + +static void report_startup_version(int mode) { + if (set.logging.log_level == POLLER_VERBOSITY_DEBUG) { + SPINE_LOG_DEBUG(("DEBUG: Version %s starting", VERSION)); + if (set.poller.poller_id > 1) { + if (mode == REMOTE) { + SPINE_LOG_DEBUG(("DEBUG: Sending entries to remote database in 'online' mode")); + } else { + SPINE_LOG_DEBUG(("DEBUG: Sending entries to local database in 'offline', or 'recovery' mode")); + } + } + return; + } + if (set.console.stdout_notty) return; + printf("Version %s starting\n", VERSION); + if (set.poller.poller_id <= 1) return; + if (mode == REMOTE) { + printf("Sending entries to remote database in 'online' mode\n"); + } else { + printf("Sending entries to local database in 'offline', or 'recovery' mode\n"); + } +} + +static void report_startup(int mode) { + report_startup_version(mode); + if (set.hosts.has_device_0) { + SPINE_LOG_MEDIUM(("Device 0 Poller Items found. Ensure that these entries are accurate")); + } else { + SPINE_LOG_MEDIUM(("No Device 0 Poller Items found.")); + } + + /* see if mysql is thread safe */ + if (mysql_thread_safe()) { + if (set.logging.log_level == POLLER_VERBOSITY_DEBUG) { + SPINE_LOG(("DEBUG: MySQL is Thread Safe!")); + } + } else { + SPINE_LOG(("WARNING: MySQL is NOT Thread Safe!")); + } + +} + +typedef struct { + int threads; + int items; + char *timestamp; + double time; +} poller_partition_t; + +static int count_partition_items(MYSQL *mysql, int host_id, int divisor, bool due_only) { + char query[BIG_BUFSIZE]; + const char *due = due_only ? " AND rrd_next_step <=0" : ""; + if (divisor > 1) { + snprintf(query, sizeof(query), "SELECT SQL_NO_CACHE CEIL(COUNT(local_data_id)/%i) FROM poller_item WHERE host_id=%i%s", divisor, host_id, due); + } else { + snprintf(query, sizeof(query), "SELECT SQL_NO_CACHE COUNT(local_data_id) FROM poller_item WHERE host_id=%i%s", host_id, due); + } + MYSQL_RES *result = db_query(mysql, LOCAL, query); + if (result == NULL) die("ERROR: Unable to count polling items"); + MYSQL_ROW row = mysql_fetch_row(result); + if (row == NULL || row[0] == NULL) { + db_free_result(result); + die("ERROR: Missing polling item count"); + } + int count = atoi(row[0]); + db_free_result(result); + return count; +} + +static void update_partition_time(poller_partition_t *partition) { + spine_snprintf(partition->timestamp, SMALL_BUFSIZE, "%lu", (unsigned long)time(NULL)); + partition->time = get_time_as_double(); +} + +static void prepare_device_partition(MYSQL *mysql, int host_id, int current_thread, poller_partition_t *partition) { + if (set.availability.ping_only) { + partition->threads = 1; + } else { + int total_items = count_partition_items(mysql, host_id, 1, set.poller.active_profiles != 1); + if (total_items && total_items < partition->threads) partition->threads = total_items; + } + if (partition->threads > 1) { + if (current_thread == 1) { + partition->items = count_partition_items(mysql, host_id, partition->threads, set.poller.active_profiles != 1); + update_partition_time(partition); + } else if (partition->time == 0 || partition->timestamp == NULL) { + update_partition_time(partition); + } + } else { + partition->items = count_partition_items(mysql, host_id, 1, TRUE); + update_partition_time(partition); + } +} + +static bool start_poll_worker(const poller_thread_t *device, int current_thread, spine_permits_t *startup, + const pthread_attr_t *attributes, pthread_t *thread) { + if (!acquire_worker_permits(startup, device->host_id, current_thread)) { + thread_mutex_lock(LOCK_THDET); + set.exit.exit_code = EXIT_FAILURE; + thread_mutex_unlock(LOCK_THDET); + return FALSE; + } + poller_thread_t *worker = malloc(sizeof(*worker)); + if (worker == NULL) die("ERROR: Fatal malloc error: polling worker instructions"); + thread_mutex_lock(LOCK_THDET); + *worker = *device; + thread_mutex_unlock(LOCK_THDET); + worker->host_thread = current_thread; + int status; + do { + status = pthread_create(thread, attributes, child, worker); + if (status == EAGAIN) spine_sleep_usec(10000); + } while (status == EAGAIN && get_time_as_double() - start_time < set.poller.poller_interval); + if (status == 0) { + SPINE_LOG_DEBUG(("Device[%i] DEBUG: Valid Thread to be Created (%ld)", device->host_id, (unsigned long int)*thread)); + return TRUE; + } + SPINE_LOG(("ERROR: Device[%i] HT[%i] unable to create polling thread (error %d)", device->host_id, current_thread, status)); + free(worker); + spine_permits_release(startup); + spine_permits_release(&available_threads); + thread_mutex_lock(LOCK_THDET); + set.exit.exit_code = EXIT_FAILURE; + thread_mutex_unlock(LOCK_THDET); + return FALSE; +} + +static void report_worker_launch(const poller_thread_t *device, int device_counter) { + int available = spine_permits_available(&available_threads); + SPINE_LOG_HIGH(("Device[%i] DEBUG: Available Threads is %i (%i outstanding)", device->host_id, available, set.poller.threads - available)); + thread_mutex_lock(LOCK_THDET); + SPINE_LOG_DEVDBG(("DEBUG: DTS: device = %d, host_id = %d, host_thread = %d," + " host_threads = %d, host_data_ids = %d, complete = %d", + device_counter-1, device->host_id, device->host_thread, device->host_threads, + device->host_data_ids, device->complete)); + thread_mutex_unlock(LOCK_THDET); +} + +static int wait_for_workers(double begin_time) { + int a_threads_value; + double cur_time; + a_threads_value = spine_permits_available(&available_threads); + + /* wait for all threads to 'complete' + * using the mutex here as the semaphore will + * show zero before the children are done */ + while (a_threads_value < set.poller.threads) { + cur_time = get_time_as_double(); + + if (cur_time - begin_time > set.poller.poller_interval) { + SPINE_LOG(("ERROR: Polling timed out while waiting for %d Threads to End", set.poller.threads - a_threads_value)); + /* Active workers still own pool entries and completion state. Exit the + * process before normal cleanup can invalidate those borrowed objects. */ + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Polling deadline expired with active workers; polling is incomplete"); + } + + SPINE_LOG_HIGH(("NOTE: Polling sleeping while waiting for %d Threads to End", set.poller.threads - a_threads_value)); + spine_sleep_usec(500000); + a_threads_value = spine_permits_available(&available_threads); + } + + return a_threads_value; +} + +static void report_worker_completion(int num_rows) { + int threads_missing = -1; + if (set.availability.ping_only) return; + thread_mutex_lock(LOCK_THDET); + + for (int threads_count = 0; threads_count < num_rows; threads_count++) { + const poller_thread_t *det = details[threads_count]; + + if (threads_missing == -1 && det == NULL) { + threads_missing = threads_count; + } + + if (det != NULL) { + if (det->output_failed) { + SPINE_LOG(("ERROR: Device[%i] output persistence failed; due items remain scheduled for retry", det->host_id)); + } + SPINE_LOG_HIGH(("INFO: Device[%i] Thread %scomplete and %d to %d sources", + det->host_id, + det->complete ? "":"in", + det->host_data_ids * (det->host_thread - 1), + det->host_data_ids * (det->host_thread))); + + SPINE_LOG_DEVDBG(("DEBUG: DTF: device = %d, host_id = %d, host_thread = %d," + " host_threads = %d, host_data_ids = %d, complete = %d", + threads_count, + det->host_id, + det->host_thread, + det->host_threads, + det->host_data_ids, + det->complete)); + } + } + + thread_mutex_unlock(LOCK_THDET); + + if (threads_missing > -1) { + SPINE_LOG(("WARNING: There were %d threads which did not run", num_rows - threads_missing)); + } + +} + +static void launch_poll_workers(MYSQL *mysql, MYSQL_RES *result, int num_rows, + pthread_t *threads, spine_permits_t *thread_init_sem, + const pthread_attr_t *attr, char *host_time) { + poller_partition_t partition = {.timestamp = host_time}; + poller_thread_t *poller_details = NULL; + MYSQL_ROW mysql_row; + int device_counter = 0; + int current_thread; + int host_id = 0; + int change_host = TRUE; + int canexit = FALSE; + struct snmp_session session; + /* initialize the threading code */ + partition.threads = 1; + current_thread = 0; + + /* poller 1 always polls host 0 but only if it exists */ + if (set.poller.poller_id == 1 && set.hosts.has_device_0 == TRUE) { + host_id = 0; + change_host = FALSE; + } else { + change_host = TRUE; + } + + /** + * We must initialize the first snmp session + * in the main thread to initialize the mib files + * and other structures. After which it's snmp + * is thread safe in threads + */ + snmp_sess_init(&session); + + /* loop through devices until done */ + while (canexit == FALSE && device_counter < num_rows) { + if (change_host) { + mysql_row = mysql_fetch_row(result); + host_id = atoi(mysql_row[0]); + partition.threads = atoi(mysql_row[1]); + current_thread = 1; + + if (partition.threads < 1) { + partition.threads = 1; + } + } else { + current_thread++; + } + + prepare_device_partition(mysql, host_id, current_thread, &partition); + change_host = (current_thread >= partition.threads) ? TRUE : FALSE; + + if (current_thread == 1) { + /* populate the thread structure */ + if (!(poller_details = (poller_thread_t *)malloc(sizeof(poller_thread_t)))) { + die("ERROR: Fatal malloc error: spine.c poller_details!"); + } + + poller_details->device_counter = device_counter; + poller_details->host_id = host_id; + poller_details->host_thread = partition.threads; + poller_details->host_threads = partition.threads; + poller_details->host_data_ids = partition.items; + + snprintf(poller_details->host_time, 40, "%s", host_time); + + poller_details->host_time_double = partition.time; + poller_details->thread_init_sem = thread_init_sem; + poller_details->complete = FALSE; + poller_details->threads_complete = 0; + poller_details->output_failed = FALSE; + + thread_mutex_lock(LOCK_THDET); + details[device_counter] = poller_details; + thread_mutex_unlock(LOCK_THDET); + } else { + poller_details = details[device_counter]; + } + + + if (start_poll_worker(poller_details, current_thread, thread_init_sem, attr, &threads[device_counter])) { + if (change_host) device_counter++; + report_worker_launch(poller_details, device_counter); + } else { + canexit = TRUE; + } + } + +} + +static void prepare_worker_storage(MYSQL_RES *result, int *rows, + pthread_t **worker_threads, int **host_ids, char **timestamp) { + int num_rows; + pthread_t *threads = NULL; + int *ids = NULL; + char *host_time = NULL; + if (set.poller.poller_id == 1) { + if (set.hosts.has_device_0) { + num_rows = spine_count_to_int(mysql_num_rows(result) + 1); /* pollerid 1 takes care of non host based data sources */ + } else { + num_rows = spine_count_to_int(mysql_num_rows(result)); /* pollerid 1 takes care of non host based data sources */ + } + } else { + num_rows = spine_count_to_int(mysql_num_rows(result)); + } + + if (num_rows > 0) { + if (!(threads = (pthread_t *)malloc(num_rows * sizeof(pthread_t)))) { + die("ERROR: Fatal malloc error: spine.c threads!"); + } + + if (!(details = (poller_thread_t **)calloc((size_t)num_rows, sizeof(poller_thread_t*)))) { + die("ERROR: Fatal malloc error: spine.c details!"); + } + + if (!(ids = (int *)malloc(num_rows * sizeof(int)))) { + die("ERROR: Fatal malloc error: spine.c host id's!"); + } + + if (!(host_time = (char *) malloc(SMALL_BUFSIZE))) { + die("ERROR: Fatal malloc error: util.c host_time"); + } + + memset(host_time, 0, SMALL_BUFSIZE); + } + + + *rows = num_rows; + *worker_threads = threads; + *host_ids = ids; + *timestamp = host_time; + +} + +static double initialize_process_defaults(void) { + double begin_time; + /* establish php processes and initialize space */ + php_processes = (php_t*) calloc(MAX_PHP_SERVERS, sizeof(php_t)); + if (php_processes == NULL) die("ERROR: Fatal calloc error: spine.c php_processes!"); + php_processes_initialize(php_processes, MAX_PHP_SERVERS); + + /* create the array of debug devices */ + debug_devices = calloc(MAX_DEBUG_DEVICES, sizeof(int)); + if (debug_devices == NULL) die("ERROR: Fatal calloc error: spine.c debug_devices!"); + + /* initialize icmp_avail */ + set.availability.icmp_avail = TRUE; + set.availability.icmp_uses_caps = FALSE; + + /* initialize number of threads */ + set.poller.threads = 1; + set.poller.threads_set = FALSE; + + /* detect and compensate for stdin/stderr ttys */ + if (!isatty(fileno(stdout))) { + set.console.stdout_notty = TRUE; + } else { + set.console.stdout_notty = FALSE; + } + + if (!isatty(fileno(stderr))) { + set.console.stderr_notty = TRUE; + } else { + set.console.stderr_notty = FALSE; + } + + /* set start time for cacti */ + begin_time = get_time_as_double(); + + /* set default verbosity */ + set.logging.log_level = POLLER_VERBOSITY_LOW; + + /* set default log separator */ + set.logging.log_datetime_separator = GDC_DEFAULT; + + /* set default log format */ + set.logging.log_datetime_format = GD_DEFAULT; + + /* set the default exit code */ + set.exit.exit_code = 0; + set.exit.exit_size = 0; + + /* get static defaults for system */ + config_defaults(); + + return begin_time; + +} + +static int initialize_main_database(MYSQL *mysql, MYSQL *mysqlr) { + MYSQL_RES *result; + int mode; + /* initialize mysql objects for threads */ + mysql_library_init(0, NULL, NULL); + + /* connect for main loop */ + db_connect(LOCAL, mysql); + + /* setup local connection pool for hosts */ + db_pool_local = (pool_t *) calloc(set.poller.threads, sizeof(pool_t)); + if (db_pool_local == NULL) die("ERROR: Fatal calloc error: spine.c db_pool_local!"); + db_create_connection_pool(LOCAL); + + if (set.poller.poller_id > 1 && set.poller.mode == REMOTE_ONLINE) { + db_connect(REMOTE, mysqlr); + mode = REMOTE; + + /* setup remote connection pool for hosts */ + db_pool_remote = (pool_t *) calloc(set.poller.threads, sizeof(pool_t)); + if (db_pool_remote == NULL) die("ERROR: Fatal calloc error: spine.c db_pool_remote!"); + db_create_connection_pool(REMOTE); + } else { + mode = LOCAL; + } + + + /* check for device 0 items */ + result = db_query(mysql, LOCAL, "SELECT * FROM (SELECT COUNT(*) AS items FROM poller_item WHERE host_id = 0 AND poller_id = 1) AS rs WHERE rs.items > 0"); + if (mysql_num_rows(result)) { + set.hosts.has_device_0 = TRUE; + } + db_free_result(result); + + /* check if poller_item has the output_regex column (added in Cacti 1.3.1) */ + if (db_column_exists(mysql, LOCAL, "poller_item", "output_regex")) { + set.hosts.has_output_regex = TRUE; + SPINE_LOG_DEBUG(("DEBUG: poller_item.output_regex column detected")); + } + + /* Since MySQL 5.7 the sql_mode defaults are too strict for cacti */ + db_insert(mysql, LOCAL, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'NO_ZERO_DATE', ''))"); + db_insert(mysql, LOCAL, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY', ''))"); + + return mode; + +} + +static void initialize_main_php(void) { + /* initialize the script server */ + if (set.php.php_required && !set.availability.ping_only) { + /* A server that started but did not answer stays BUSY for the restart + * path; polling can proceed while any server answers. */ + int ready = 0; + bool started = php_init(PHP_INIT); + for (int i = 0; started && i < set.php.php_servers; i++) { + if (php_processes[i].php_state == PHP_READY) ready++; + } + if (!started || ready == 0) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: PHP Script Server initialization failed"); + } + if (ready < set.php.php_servers) { + SPINE_LOG(("WARNING: %i of %i PHP Script Servers did not answer at startup", set.php.php_servers - ready, set.php.php_servers)); + } + set.php.php_initialized = TRUE; + set.php.php_current_server = 0; + } + +} + +static void persist_poll_completion(MYSQL *mysql, MYSQL *mysqlr, int mode) { + char querybuf[MEGA_BUFSIZE]; + /* push data back to the main server */ + if (set.poller.poller_id > 1 && set.poller.mode == REMOTE_ONLINE && !set.poller.SQL_readonly) { + poller_push_data_to_main(); + } + + /* update the db for |data_time| on graphs */ + if (!set.availability.ping_only) { + if (set.poller.poller_id == 1) { + db_insert(mysql, LOCAL, "REPLACE INTO settings (name,value) VALUES ('date',NOW())"); + } + + snprintf(querybuf, BIG_BUFSIZE, "UPDATE poller_time SET end_time=NOW() WHERE poller_id=%i AND pid=%i", set.poller.poller_id, getpid()); + + if (mode == REMOTE) { + db_insert(mysqlr, REMOTE, querybuf); + } else { + db_insert(mysql, LOCAL, querybuf); + } + } + + if (db_pool_local) { + db_close_connection_pool(LOCAL); + } + + if (db_pool_remote) { + db_close_connection_pool(REMOTE); + } + +} + +static void close_main_php(void) { + /* close the php script server */ + if (set.php.php_required && !set.availability.ping_only) php_close(PHP_INIT); + + SPINE_LOG_DEBUG(("DEBUG: PHP Script Server Pipes Closed")); + +} + +static void free_worker_storage(int num_rows, pthread_t *threads, int *ids, + char *conf_file, char *host_time) { + /* free malloc'd variables */ + for (int i = 0; i < num_rows; i++) { + if (details[i] != NULL) { + SPINE_FREE(details[i]); + } + } + + SPINE_FREE(details); + SPINE_FREE(threads); + SPINE_FREE(ids); + SPINE_FREE(conf_file); + SPINE_FREE(debug_devices); + SPINE_FREE(host_time); + SPINE_FREE(php_processes); + + SPINE_LOG_DEBUG(("DEBUG: Allocated Variable Memory Freed")); + +} + +static void report_poll_statistics(double begin_time, int num_rows) { + double end_time; + /* finally add some statistics to the log and exit */ + end_time = get_time_as_double(); + + if (set.logging.log_level >= POLLER_VERBOSITY_MEDIUM) { + SPINE_LOG(("Time: %.4f s, Threads: %i, Devices: %i", (end_time - begin_time), set.poller.threads, num_rows)); + } else { + /* provide output if running from command line */ + if (!set.console.stdout_notty) { + fprintf(stdout, "Time: %.4f s, Threads: %i, Devices: %i\n", (end_time - begin_time), set.poller.threads, num_rows); + } + } + +} + int main(int argc, char *argv[]) { char *conf_file = NULL; - double begin_time, end_time, cur_time; + double begin_time; int num_rows = 0; - int device_counter = 0; - int valid_conf_file = FALSE; - char querybuf[MEGA_BUFSIZE], *qp = querybuf; + char querybuf[MEGA_BUFSIZE]; char *host_time = NULL; - double host_time_double = 0; - int items_per_thread = 0; - int device_threads; - spine_sem_t thread_init_sem; + spine_permits_t thread_init_sem; int a_threads_value; - //struct timespec until_spec; - - pthread_t* threads = NULL; - poller_thread_t* poller_details = NULL; - pthread_attr_t attr; - int* ids = NULL; - int mode = REMOTE; - MYSQL mysql; - MYSQL mysqlr; - MYSQL_RES *result = NULL; - MYSQL_RES *tresult = NULL; - MYSQL_ROW mysql_row; - int canexit = FALSE; - int host_id = 0; - int i; - int thread_status = 0; - int total_items = 0; - int change_host = TRUE; - int current_thread; - int threads_final = 0; - int threads_missing = -1; - int threads_count; - struct snmp_session session; start_time = get_time_as_double(); total_time = 0; @@ -237,65 +1077,23 @@ int main(int argc, char *argv[]) { } #endif /* HAVE_LCAP */ - /* we must initialize snmp in the main thread */ + pthread_t* threads = NULL; + pthread_attr_t attr; + + int* ids = NULL; + int mode = REMOTE; + MYSQL mysql; + MYSQL mysqlr; + MYSQL_RES *result = NULL; + int threads_final = 0; + UNUSED_PARAMETER(argc); /* we operate strictly with argv */ /* install the spine signal handler */ install_spine_signal_handler(); - /* establish php processes and initialize space */ - php_processes = (php_t*) calloc(MAX_PHP_SERVERS, sizeof(php_t)); - if (php_processes == NULL) { - die("ERROR: Fatal calloc error: spine.c php_processes!"); - } - php_processes_initialize(php_processes, MAX_PHP_SERVERS); - - /* create the array of debug devices */ - debug_devices = calloc(MAX_DEBUG_DEVICES, sizeof(int)); - if (debug_devices == NULL) { - die("ERROR: Fatal calloc error: spine.c debug_devices!"); - } - - /* initialize icmp_avail */ - set.icmp_avail = TRUE; - set.icmp_uses_caps = FALSE; - - /* initialize number of threads */ - set.threads = 1; - set.threads_set = FALSE; - - /* detect and compensate for stdin/stderr ttys */ - if (!isatty(fileno(stdout))) { - set.stdout_notty = TRUE; - } else { - set.stdout_notty = FALSE; - } - - if (!isatty(fileno(stderr))) { - set.stderr_notty = TRUE; - } else { - set.stderr_notty = FALSE; - } - - /* set start time for cacti */ - begin_time = get_time_as_double(); - - /* set default verbosity */ - set.log_level = POLLER_VERBOSITY_LOW; - - /* set default log separator */ - set.log_datetime_separator = GDC_DEFAULT; - - /* set default log format */ - set.log_datetime_format = GD_DEFAULT; - - /* set the default exit code */ - set.exit_code = 0; - set.exit_size = 0; - - /* get static defaults for system */ - config_defaults(); + begin_time = initialize_process_defaults(); /*! ---------------------------------------------------------------- * PROCESS COMMAND LINE @@ -323,151 +1121,21 @@ int main(int argc, char *argv[]) { */ /* initialize some global variables */ - set.poller_id = 1; - set.start_host_id = -1; - set.end_host_id = -1; - set.host_id_list[0] = '\0'; - set.php_initialized = FALSE; - set.logfile_processed = FALSE; - set.parent_fork = SPINE_PARENT; - set.mode = REMOTE_ONLINE; - set.has_device_0 = FALSE; - set.has_output_regex = FALSE; + set.poller.poller_id = 1; + set.hosts.start_host_id = -1; + set.hosts.end_host_id = -1; + set.hosts.host_id_list[0] = '\0'; + set.php.php_initialized = FALSE; + set.logging.logfile_processed = FALSE; + set.poller.parent_fork = SPINE_PARENT; + set.poller.mode = REMOTE_ONLINE; + set.hosts.has_device_0 = FALSE; + set.hosts.has_output_regex = FALSE; - for (argv++; *argv; argv++) { - char *arg = *argv; - char *opt = strchr(arg, '='); /* pick off the =VALUE part */ + parse_command_line(argv, &conf_file); - if (opt) *opt++ = '\0'; - - if (STRIMATCH(arg, "-f") || STRMATCH(arg, "--first")) { - if (HOSTID_DEFINED(set.start_host_id)) { - die("ERROR: %s can only be used once", arg); - } - - set.start_host_id = atoi(opt = getarg(opt, &argv)); - - if (!HOSTID_DEFINED(set.start_host_id)) { - die("ERROR: '%s=%s' is invalid first-host ID", arg, opt); - } - } - - else if (STRIMATCH(arg, "-l") || STRIMATCH(arg, "--last")) { - if (HOSTID_DEFINED(set.end_host_id)) { - die("ERROR: %s can only be used once", arg); - } - - set.end_host_id = atoi(opt = getarg(opt, &argv)); - - if (!HOSTID_DEFINED(set.end_host_id)) { - die("ERROR: '%s=%s' is invalid last-host ID", arg, opt); - } - } - - else if (STRIMATCH(arg, "-p") || STRIMATCH(arg, "--poller")) { - set.poller_id = atoi(getarg(opt, &argv)); - } - - else if (STRMATCH(arg, "-t") || STRIMATCH(arg, "--threads")) { - set.threads = atoi(getarg(opt, &argv)); - set.threads_set = TRUE; - } - - else if (STRMATCH(arg, "-P") || STRIMATCH(arg, "--pingonly")) { - set.ping_only = TRUE; - } - - else if (STRMATCH(arg, "-N") || STRIMATCH(arg, "--mode")) { - if (STRIMATCH(getarg(opt, &argv), "online")) { - set.mode = REMOTE_ONLINE; - } else if (STRIMATCH(getarg(opt, &argv), "offline")) { - set.mode = REMOTE_OFFLINE; - } else if (STRIMATCH(getarg(opt, &argv), "recovery")) { - set.mode = REMOTE_RECOVERY; - } else { - die("ERROR: invalid polling mode '%s' specified", opt); - } - } - - else if (STRIMATCH(arg, "-H") || STRIMATCH(arg, "--hostlist")) { - snprintf(set.host_id_list, BIG_BUFSIZE, "%s", getarg(opt, &argv)); - - /* Validate host_id_list contains only digits and commas */ - { - const char *p = set.host_id_list; - while (*p) { - if (!isdigit((unsigned char)*p) && *p != ',' && *p != ' ') { - die("ERROR: --hostlist contains invalid characters. Only digits and commas are allowed."); - } - p++; - } - } - } - - else if (STRIMATCH(arg, "-M") || STRMATCH(arg, "--mibs")) { - set.mibs = 1; - } - - else if (STRIMATCH(arg, "-h") || STRMATCH(arg, "--help")) { - display_help(FALSE); - - exit(EXIT_SUCCESS); - } - - else if (STRMATCH(arg, "-v") || STRMATCH(arg, "--version")) { - display_help(TRUE); - - exit(EXIT_SUCCESS); - } - - else if (STRIMATCH(arg, "-O") || STRIMATCH(arg, "--option")) { - char *setting = getarg(opt, &argv); - char *value = strchr(setting, ':'); - - if (value == NULL) { - die("ERROR: -O requires setting:value"); - } else { - *value++ = '\0'; - } - - set_option(setting, value); - } - - else if (STRIMATCH(arg, "-R") || STRMATCH(arg, "--readonly") || STRMATCH(arg, "--read-only")) { - set.SQL_readonly = TRUE; - } - - else if (STRIMATCH(arg, "-C") || STRMATCH(arg, "--conf")) { - conf_file = strdup(getarg(opt, &argv)); - } - - else if (STRIMATCH(arg, "-S") || STRMATCH(arg, "--stdout")) { - set_option("log_destination", "STDOUT"); - } - - else if (STRIMATCH(arg, "-D") || STRMATCH(arg, "--log")) { - set_option("log_destination", getarg(opt, &argv)); - } - - else if (STRMATCH(arg, "-V") || STRMATCH(arg, "--verbosity")) { - set_option("log_verbosity", getarg(opt, &argv)); - } - - else if (!HOSTID_DEFINED(set.start_host_id) && all_digits(arg)) { - set.start_host_id = atoi(arg); - } - - else if (!HOSTID_DEFINED(set.end_host_id) && all_digits(arg)) { - set.end_host_id = atoi(arg); - } - - else { - die("ERROR: %s is an unknown command-line parameter", arg); - } - } - - if (set.ping_only) { - set.mibs = 0; + if (set.availability.ping_only) { + set.snmp.mibs = 0; } /* we attempt to support scripts better in cygwin */ @@ -475,164 +1143,45 @@ int main(int argc, char *argv[]) { setenv("CYGWIN", "nodosfilewarning", 1); if (file_exists("./sh.exe")) { set.cygwinshloc = 0; - if (set.log_level == POLLER_VERBOSITY_DEBUG) { + if (set.logging.log_level == POLLER_VERBOSITY_DEBUG) { printf("The Shell Command Exists in the current directory\n"); } } else { set.cygwinshloc = 1; - if (set.log_level == POLLER_VERBOSITY_DEBUG) { + if (set.logging.log_level == POLLER_VERBOSITY_DEBUG) { printf("The Shell Command Exists in the /bin directory\n"); } } #endif /* we require either both the first and last hosts, or neither host */ - if ((HOSTID_DEFINED(set.start_host_id) != HOSTID_DEFINED(set.end_host_id)) && - (!strlen(set.host_id_list))) { + if ((HOSTID_DEFINED(set.hosts.start_host_id) != HOSTID_DEFINED(set.hosts.end_host_id)) && + (!strlen(set.hosts.host_id_list))) { die("ERROR: must provide both -f/-l, a hostlist (-H/--hostlist), or neither"); } - if (set.start_host_id > set.end_host_id) { + if (set.hosts.start_host_id > set.hosts.end_host_id) { die("ERROR: Invalid row spec; first host_id must be less than the second"); } - /* read configuration file to establish local environment */ - if (conf_file) { - if ((read_spine_config(conf_file)) < 0) { - die("ERROR: Could not read config file: %s", conf_file); - } else { - valid_conf_file = TRUE; - } - } else { - if (!(conf_file = calloc(CONFIG_PATHS, DBL_BUFSIZE))) { - die("ERROR: Fatal malloc error: spine.c conf_file!"); - } - - for (i=0; i= 0) { - valid_conf_file = TRUE; - break; - } - - if (i == CONFIG_PATHS-1) { - snprintf(conf_file, DBL_BUFSIZE, "%.*s%s", (int) sizeof(config_paths[0]) - 1, config_paths[0], DEFAULT_CONF_FILE); - } - } - } - - if (valid_conf_file) { - /* read settings table from the database to further establish environment */ - read_config_options(); - } else { - die("FATAL: Unable to read configuration file!"); - } + conf_file = load_startup_configuration(conf_file); /* set the poller interval for those who use less than 5 minute intervals */ - if (set.poller_interval == 0) { - set.poller_interval = 300; + if (set.poller.poller_interval == 0) { + set.poller.poller_interval = 300; } /* tokenize the debug devices */ - if (strlen(set.selective_device_debug)) { - int debug_idx = 0; - char *token; - SPINE_LOG_DEBUG(("DEBUG: Selective Debug Devices %s", set.selective_device_debug)); - token = strtok(set.selective_device_debug, ","); - while(token && debug_idx < MAX_DEBUG_DEVICES - 1) { - debug_devices[debug_idx] = atoi(token); - debug_devices[debug_idx+1] = '\0'; - token = strtok(NULL, ","); - debug_idx++; - } + if (strlen(set.logging.selective_device_debug)) { + SPINE_LOG_DEBUG(("DEBUG: Selective Debug Devices %s", set.logging.selective_device_debug)); + parse_debug_devices(set.logging.selective_device_debug, debug_devices, MAX_DEBUG_DEVICES); } else { debug_devices[0] = '\0'; } - /* initialize mysql objects for threads */ - mysql_library_init(0, NULL, NULL); + mode = initialize_main_database(&mysql, &mysqlr); - /* connect for main loop */ - db_connect(LOCAL, &mysql); - - /* setup local connection pool for hosts */ - if (!(db_pool_local = (pool_t *) calloc(set.threads, sizeof(pool_t)))) { - die("ERROR: Fatal calloc error: spine.c db_pool_local!"); - } - - db_create_connection_pool(LOCAL); - - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { - db_connect(REMOTE, &mysqlr); - mode = REMOTE; - - /* setup remote connection pool for hosts */ - if (!(db_pool_remote = (pool_t *) calloc(set.threads, sizeof(pool_t)))) { - die("ERROR: Fatal calloc error: spine.c db_pool_remote!"); - } - - db_create_connection_pool(REMOTE); - } else { - mode = LOCAL; - } - - - /* check for device 0 items */ - result = db_query(&mysql, LOCAL, "SELECT * FROM (SELECT COUNT(*) AS items FROM poller_item WHERE host_id = 0 AND poller_id = 1) AS rs WHERE rs.items > 0"); - if (mysql_num_rows(result)) { - set.has_device_0 = TRUE; - } - db_free_result(result); - - /* check if poller_item has the output_regex column (added in Cacti 1.3.1) */ - if (db_column_exists(&mysql, LOCAL, "poller_item", "output_regex")) { - set.has_output_regex = TRUE; - SPINE_LOG_DEBUG(("DEBUG: poller_item.output_regex column detected")); - } - - /* Since MySQL 5.7 the sql_mode defaults are too strict for cacti */ - db_insert(&mysql, LOCAL, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'NO_ZERO_DATE', ''))"); - db_insert(&mysql, LOCAL, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY', ''))"); - - if (set.log_level == POLLER_VERBOSITY_DEBUG) { - SPINE_LOG_DEBUG(("DEBUG: Version %s starting", VERSION)); - - if (set.poller_id > 1) { - if (mode == REMOTE) { - SPINE_LOG_DEBUG(("DEBUG: Sending entries to remote database in 'online' mode")); - } else { - SPINE_LOG_DEBUG(("DEBUG: Sending entries to local database in 'offline', or 'recovery' mode")); - } - } - } else { - if (!set.stdout_notty) { - printf("Version %s starting\n", VERSION); - - if (set.poller_id > 1) { - if (mode == REMOTE) { - printf("Sending entries to remote database in 'online' mode\n"); - } else { - printf("Sending entries to local database in 'offline', or 'recovery' mode\n"); - } - } - } - } - - if (set.has_device_0) { - SPINE_LOG_MEDIUM(("Device 0 Poller Items found. Ensure that these entries are accurate")); - } else { - SPINE_LOG_MEDIUM(("No Device 0 Poller Items found.")); - } - - /* see if mysql is thread safe */ - if (mysql_thread_safe()) { - if (set.log_level == POLLER_VERBOSITY_DEBUG) { - SPINE_LOG(("DEBUG: MySQL is Thread Safe!")); - } - } else { - SPINE_LOG(("WARNING: MySQL is NOT Thread Safe!")); - } + report_startup(mode); /* test for asroot permissions for ICMP */ checkAsRoot(); @@ -645,84 +1194,20 @@ int main(int argc, char *argv[]) { SPINE_LOG_DEBUG(("DEBUG: Initializing PHP Script Server(s)")); /* tell spine that it is parent, and set the poller id */ - set.parent_fork = SPINE_PARENT; + set.poller.parent_fork = SPINE_PARENT; - /* initialize the script server */ - if (set.php_required && !set.ping_only) { - php_init(PHP_INIT); - set.php_initialized = TRUE; - set.php_current_server = 0; - } + initialize_main_php(); - /* obtain the list of hosts to poll */ - { - size_t remaining = MEGA_BUFSIZE - (qp - querybuf); - int query_ok = spine_appendf(&qp, &remaining, "SELECT SQL_NO_CACHE id, device_threads, picount, picount/device_threads AS tppi FROM host AS h LEFT JOIN (SELECT host_id, COUNT(*) AS picount FROM poller_item GROUP BY host_id) AS pi ON h.id = pi.host_id"); - remaining = MEGA_BUFSIZE - (qp - querybuf); - query_ok &= spine_appendf(&qp, &remaining, " WHERE disabled = ''"); + result = select_poll_hosts(&mysql); - remaining = MEGA_BUFSIZE - (qp - querybuf); - query_ok &= spine_appendf(&qp, &remaining, " AND availability_method != %d", AVAIL_STREAM); - - if (!strlen(set.host_id_list)) { - qp += append_hostrange(qp, "h.id"); /* AND id BETWEEN a AND b */ - } else { - remaining = MEGA_BUFSIZE - (qp - querybuf); - query_ok &= spine_appendf(&qp, &remaining, " AND h.id IN(%s)", set.host_id_list); - } - - remaining = MEGA_BUFSIZE - (qp - querybuf); - query_ok &= spine_appendf(&qp, &remaining, " AND h.poller_id = %i", set.poller_id); - remaining = MEGA_BUFSIZE - (qp - querybuf); - query_ok &= spine_appendf(&qp, &remaining, " ORDER BY picount DESC"); - - if (!query_ok) { - die("ERROR: Host selection query exceeded its buffer"); - } - } - - SPINE_LOG_DEVDBG(("DEVDBG: Host SQL:%s", querybuf)); - result = db_query(&mysql, LOCAL, querybuf); - - if (set.poller_id == 1) { - if (set.has_device_0) { - num_rows = mysql_num_rows(result) + 1; /* pollerid 1 takes care of non host based data sources */ - } else { - num_rows = mysql_num_rows(result); /* pollerid 1 takes care of non host based data sources */ - } - } else { - num_rows = mysql_num_rows(result); - } - - if (num_rows > 0) { - if (!(threads = (pthread_t *)malloc(num_rows * sizeof(pthread_t)))) { - die("ERROR: Fatal malloc error: spine.c threads!"); - } - - /* calloc, not malloc: the device loop can exit early on a poller - overrun, and both the NULL test below and the free loop at the end - walk every slot up to num_rows. */ - if (!(details = (poller_thread_t **)calloc(num_rows, sizeof(poller_thread_t*)))) { - die("ERROR: Fatal calloc error: spine.c details!"); - } - - if (!(ids = (int *)malloc(num_rows * sizeof(int)))) { - die("ERROR: Fatal malloc error: spine.c host id's!"); - } - - if (!(host_time = (char *) malloc(SMALL_BUFSIZE))) { - die("ERROR: Fatal malloc error: util.c host_time"); - } - - memset(host_time, 0, SMALL_BUFSIZE); - } + prepare_worker_storage(result, &num_rows, &threads, &ids, &host_time); /* initialize winsock library on Windows */ SOCK_STARTUP; /* mark the spine process as started */ - if (!set.ping_only) { - snprintf(querybuf, BIG_BUFSIZE, "INSERT INTO poller_time (poller_id, pid, start_time, end_time) VALUES (%i, %i, NOW(), '0000-00-00 00:00:00')", set.poller_id, getpid()); + if (!set.availability.ping_only) { + snprintf(querybuf, BIG_BUFSIZE, "INSERT INTO poller_time (poller_id, pid, start_time, end_time) VALUES (%i, %i, NOW(), '0000-00-00 00:00:00')", set.poller.poller_id, getpid()); if (mode == REMOTE) { db_insert(&mysqlr, REMOTE, querybuf); } else { @@ -736,416 +1221,51 @@ int main(int argc, char *argv[]) { init_mutexes(); - /* initialize available_threads semaphore */ - spine_sem_init(&available_threads, set.threads); - - /* initialize available_scripts semaphore */ - spine_sem_init(&available_scripts, MAX_SIMULTANEOUS_SCRIPTS); - - /* initialize thread initialization semaphore */ - spine_sem_init(&thread_init_sem, 1); + /* Initialize process-local concurrency permits before worker startup. */ + if (spine_permits_init(&available_threads, set.poller.threads) != 0 || + spine_permits_init(&available_scripts, MAX_SIMULTANEOUS_SCRIPTS) != 0 || + spine_permits_init(&thread_init_sem, 1) != 0) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Unable to initialize process permits"); + } /* specify the point of timeout for timedwait semaphores */ - //until_spec.tv_sec = (time_t)(set.poller_interval + begin_time - 0.2); - //until_spec.tv_nsec = 0; - spine_sem_getvalue(&available_threads, &a_threads_value); - SPINE_LOG_HIGH(("DEBUG: Initial Value of Available Threads is %i (%i outstanding)", a_threads_value, set.threads - a_threads_value)); + a_threads_value = spine_permits_available(&available_threads); + SPINE_LOG_HIGH(("DEBUG: Initial Value of Available Threads is %i (%i outstanding)", a_threads_value, set.poller.threads - a_threads_value)); /* tell fork processes that they are now active */ - set.parent_fork = SPINE_FORK; + set.poller.parent_fork = SPINE_FORK; - /* initialize the threading code */ - device_threads = 1; - current_thread = 0; + launch_poll_workers(&mysql, result, num_rows, threads, &thread_init_sem, &attr, host_time); - /* poller 1 always polls host 0 but only if it exists */ - if (set.poller_id == 1 && set.has_device_0 == TRUE) { - host_id = 0; - change_host = FALSE; - } else { - change_host = TRUE; - } + a_threads_value = wait_for_workers(begin_time); - /** - * We must initialize the first snmp session - * in the main thread to initialize the mib files - * and other structures. After which it's snmp - * is thread safe in threads - */ - snmp_sess_init(&session); - - /* loop through devices until done */ - while (canexit == FALSE && device_counter < num_rows) { - int loop_count = 0; - double progress_time = 0; - int sem_err = 0; - int spine_timeout = FALSE; - - if (change_host) { - mysql_row = mysql_fetch_row(result); - - if (mysql_row == NULL) { - /* fewer device rows than expected; stop processing */ - break; - } - - host_id = atoi(mysql_row[0]); - device_threads = atoi(mysql_row[1]); - current_thread = 1; - - if (device_threads < 1) { - device_threads = 1; - } - } else { - current_thread++; - } - - /* adjust device threads in cases where the host does not have sufficient data sources */ - if (!set.ping_only) { - if (set.active_profiles != 1) { - snprintf(querybuf, BIG_BUFSIZE, "SELECT SQL_NO_CACHE COUNT(local_data_id) FROM poller_item WHERE host_id=%i AND rrd_next_step <=0", host_id); - } else { - snprintf(querybuf, BIG_BUFSIZE, "SELECT SQL_NO_CACHE COUNT(local_data_id) FROM poller_item WHERE host_id=%i", host_id); - } - - tresult = db_query(&mysql, LOCAL, querybuf); - mysql_row = (tresult != NULL) ? mysql_fetch_row(tresult) : NULL; - - if (mysql_row == NULL) { - total_items = 0; - } else { - total_items = atoi(mysql_row[0]); - } - - db_free_result(tresult); - - if (total_items && total_items < device_threads) { - device_threads = total_items; - } - } else { - device_threads = 1; - } - - change_host = (current_thread >= device_threads) ? TRUE : FALSE; - - /* determine how many items will be polled per thread */ - if (device_threads > 1) { - if (current_thread == 1) { - if (set.active_profiles != 1) { - snprintf(querybuf, BIG_BUFSIZE, "SELECT SQL_NO_CACHE CEIL(COUNT(local_data_id)/%i) FROM poller_item WHERE host_id=%i AND rrd_next_step <=0", device_threads, host_id); - } else { - snprintf(querybuf, BIG_BUFSIZE, "SELECT SQL_NO_CACHE CEIL(COUNT(local_data_id)/%i) FROM poller_item WHERE host_id=%i", device_threads, host_id); - } - - tresult = db_query(&mysql, LOCAL, querybuf); - mysql_row = (tresult != NULL) ? mysql_fetch_row(tresult) : NULL; - - if (mysql_row == NULL) { - items_per_thread = 0; - } else { - items_per_thread = atoi(mysql_row[0]); - } - - db_free_result(tresult); - - snprintf(host_time, SMALL_BUFSIZE, "%lu", (unsigned long) time(NULL)); - host_time_double = get_time_as_double(); - } else if (host_time_double == 0) { - snprintf(host_time, SMALL_BUFSIZE, "%lu", (unsigned long) time(NULL)); - host_time_double = get_time_as_double(); - } - } else { - snprintf(querybuf, BIG_BUFSIZE, "SELECT SQL_NO_CACHE COUNT(local_data_id) FROM poller_item WHERE host_id=%i AND rrd_next_step <=0", host_id); - tresult = db_query(&mysql, LOCAL, querybuf); - mysql_row = (tresult != NULL) ? mysql_fetch_row(tresult) : NULL; - - if (mysql_row == NULL) { - items_per_thread = 0; - } else { - items_per_thread = atoi(mysql_row[0]); - } - - db_free_result(tresult); - - snprintf(host_time, SMALL_BUFSIZE, "%lu", (unsigned long) time(NULL)); - host_time_double = get_time_as_double(); - } - - /* populate the thread structure */ - if (!(poller_details = (poller_thread_t *)malloc(sizeof(poller_thread_t)))) { - die("ERROR: Fatal malloc error: spine.c poller_details!"); - } - - poller_details->device_counter = device_counter; - poller_details->host_id = host_id; - poller_details->host_thread = current_thread; - poller_details->host_threads = device_threads; - poller_details->host_data_ids = items_per_thread; - - snprintf(poller_details->host_time, 40, "%s", host_time); - - poller_details->host_time_double = host_time_double; - poller_details->thread_init_sem = &thread_init_sem; - poller_details->complete = FALSE; - poller_details->threads_complete = 0; - - thread_mutex_lock(LOCK_THDET); - details[device_counter] = poller_details; - thread_mutex_unlock(LOCK_THDET); - - /* dev note - errno was never primed at this point in previous version of code */ - loop_count = 0; - progress_time = 0; - sem_err = 0; - spine_timeout = FALSE; - - while (TRUE) { - sem_err = spine_sem_trywait(&available_threads); - - if (sem_err == 0) { - /* acquired a thread */ - break; - } else if (errno == EINTR) { - /* interrupted by signal handler */ - } else if (errno == EDEADLK) { - SPINE_LOG_DEVDBG(("WARNING: Device[%i] HT[%i] would have deadlocked acquiring Available Thread Lock", host_id, current_thread)); - } else if (errno == EAGAIN) { - /* keep trying */ - } - - loop_count++; - - if (loop_count == 10) { - progress_time = get_time_as_double() - start_time; - - if (progress_time + 1 > set.poller_interval) { - SPINE_LOG(("ERROR: Device[%i] HT[%i] polling timed out while acquiring Available Thread Lock", host_id, current_thread)); - spine_timeout = TRUE; - break; - } - - loop_count = 0; - } - - usleep(10000); - - total_time = get_time_as_double(); - - if (total_time - start_time > set.poller_interval) { - SPINE_LOG(("ERROR: Device[%i] HT[%i] Spine Timed Out While Processing Devices External", host_id, current_thread)); - spine_timeout = TRUE; - canexit = TRUE; - break; - } - } - - loop_count = 0; - - while (!spine_timeout) { - sem_err = spine_sem_trywait(&thread_init_sem); - - if (sem_err == 0) { - /* acquired the thread init lock */ - break; - } else { - int sem_errno = errno; - - if (sem_errno != EAGAIN) { - SPINE_LOG_DEVDBG(("WARNING: Device[%i] HT[%i] errored with %d while acquiring Thread Initialization Lock", host_id, current_thread, sem_errno)); - } - } - - if (loop_count == 10) { - progress_time = get_time_as_double() - start_time; - - if (progress_time + 1 > set.poller_interval) { - SPINE_LOG(("ERROR: Device[%i] HT[%i] polling timed out while acquiring Thread Init Lock", host_id, current_thread)); - spine_timeout = TRUE; - break; - } - - loop_count = 0; - } - - usleep(10000); - - total_time = get_time_as_double(); - - if (total_time - start_time > set.poller_interval) { - SPINE_LOG(("ERROR: Device[%i] HT[%i] Spine Timed Out While Processing Devices Internal", host_id, current_thread)); - spine_timeout = TRUE; - canexit = TRUE; - break; - } - } - - if (!spine_timeout) { - /* create child process */ - thread_retry: - - thread_mutex_lock(LOCK_HOST_TIME); - - thread_status = pthread_create(&threads[device_counter], &attr, child, poller_details); - - if (thread_status == 0) { - SPINE_LOG_DEBUG(("Device[%i] DEBUG: Valid Thread to be Created (%ld)", poller_details->host_id, (unsigned long int)threads[device_counter])); - - if (change_host) { - device_counter++; - } - - spine_sem_getvalue(&available_threads, &a_threads_value); - SPINE_LOG_HIGH(("Device[%i] DEBUG: Available Threads is %i (%i outstanding)", poller_details->host_id, a_threads_value, set.threads - a_threads_value)); - - /* the child releases thread_init_sem once it has copied poller_details - * and dropped LOCK_HOST_TIME; posting here too double-counts the semaphore */ - - SPINE_LOG_DEVDBG(("DEBUG: DTS: device = %d, host_id = %d, host_thread = %d," - " host_threads = %d, host_data_ids = %d, complete = %d", - device_counter-1, - poller_details->host_id, - poller_details->host_thread, - poller_details->host_threads, - poller_details->host_data_ids, - poller_details->complete)); - } else if (thread_status == EAGAIN) { - thread_mutex_unlock(LOCK_HOST_TIME); - usleep(10000); - goto thread_retry; - } else if (thread_status == EINVAL) { - SPINE_LOG(("ERROR: The Thread Attribute is Not Initialized")); - } - - /* Restore thread initialization semaphore if thread creation failed */ - if (thread_status) { - thread_mutex_unlock(LOCK_HOST_TIME); - spine_sem_post(&thread_init_sem); - } - } - } - - spine_sem_getvalue(&available_threads, &a_threads_value); - - /* wait for all threads to 'complete' - * using the mutex here as the semaphore will - * show zero before the children are done */ - while (a_threads_value < set.threads) { - cur_time = get_time_as_double(); - - if (cur_time - begin_time > set.poller_interval) { - SPINE_LOG(("ERROR: Polling timed out while waiting for %d Threads to End", set.threads - a_threads_value)); - break; - } - - SPINE_LOG_HIGH(("NOTE: Polling sleeping while waiting for %d Threads to End", set.threads - a_threads_value)); - usleep(500000); - spine_sem_getvalue(&available_threads, &a_threads_value); - } - - threads_final = set.threads - a_threads_value; + threads_final = set.poller.threads - a_threads_value; SPINE_LOG_HIGH(("The final count of Threads is %i", threads_final)); - if (!set.ping_only) { - thread_mutex_lock(LOCK_THDET); - - for (threads_count = 0; threads_count < num_rows; threads_count++) { - poller_thread_t* det = details[threads_count]; - - if (threads_missing == -1 && det == NULL) { - threads_missing = threads_count; - } - - if (det != NULL) { // && !det->complete) { - SPINE_LOG_HIGH(("INFO: Device[%i] Thread %scomplete and %d to %d sources", - det->host_id, - det->complete ? "":"in", - det->host_data_ids * (det->host_thread - 1), - det->host_data_ids * (det->host_thread))); - - SPINE_LOG_DEVDBG(("DEBUG: DTF: device = %d, host_id = %d, host_thread = %d," - " host_threads = %d, host_data_ids = %d, complete = %d", - threads_count, - det->host_id, - det->host_thread, - det->host_threads, - det->host_data_ids, - det->complete)); - } - } - - thread_mutex_unlock(LOCK_THDET); - } - - if (threads_missing > -1) { - SPINE_LOG(("WARNING: There were %d threads which did not run", num_rows - threads_missing)); - } + report_worker_completion(num_rows); /* tell Spine that it is now parent */ - set.parent_fork = SPINE_PARENT; + set.poller.parent_fork = SPINE_PARENT; - /* push data back to the main server */ - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE && !set.SQL_readonly) { - poller_push_data_to_main(); - } - - /* update the db for |data_time| on graphs */ - if (!set.ping_only) { - if (set.poller_id == 1) { - db_insert(&mysql, LOCAL, "REPLACE INTO settings (name,value) VALUES ('date',NOW())"); - } - - snprintf(querybuf, BIG_BUFSIZE, "UPDATE poller_time SET end_time=NOW() WHERE poller_id=%i AND pid=%i", set.poller_id, getpid()); - - if (mode == REMOTE) { - db_insert(&mysqlr, REMOTE, querybuf); - } else { - db_insert(&mysql, LOCAL, querybuf); - } - } - - if (db_pool_local) { - db_close_connection_pool(LOCAL); - } - - if (db_pool_remote) { - db_close_connection_pool(REMOTE); - } + persist_poll_completion(&mysql, &mysqlr, mode); /* cleanup and exit program */ pthread_attr_destroy(&attr); SPINE_LOG_DEBUG(("DEBUG: Thread Cleanup Complete")); - /* close the php script server */ - if (set.php_required && !set.ping_only) { - php_close(PHP_INIT); - } + close_main_php(); - SPINE_LOG_DEBUG(("DEBUG: PHP Script Server Pipes Closed")); - - /* free malloc'd variables */ - for (i = 0; i < num_rows; i++) { - if (details[i] != NULL) { - SPINE_FREE(details[i]); - } - } - - SPINE_FREE(details); - SPINE_FREE(threads); - SPINE_FREE(ids); - SPINE_FREE(conf_file); - SPINE_FREE(host_time); - SPINE_FREE(php_processes); - - SPINE_LOG_DEBUG(("DEBUG: Allocated Variable Memory Freed")); + free_worker_storage(num_rows, threads, ids, conf_file, host_time); /* close mysql */ db_free_result(result); db_disconnect(&mysql); - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { + if (set.poller.poller_id > 1 && set.poller.mode == REMOTE_ONLINE) { db_disconnect(&mysqlr); } @@ -1156,26 +1276,11 @@ int main(int argc, char *argv[]) { SPINE_LOG_DEBUG(("DEBUG: Net-SNMP Close Completed")); - /* finally add some statistics to the log and exit */ - end_time = get_time_as_double(); - - if (set.log_level >= POLLER_VERBOSITY_MEDIUM) { - SPINE_LOG(("Time: %.4f s, Threads: %i, Devices: %i", (end_time - begin_time), set.threads, num_rows)); - } else { - /* provide output if running from command line */ - if (!set.stdout_notty) { - fprintf(stdout, "Time: %.4f s, Threads: %i, Devices: %i\n", (end_time - begin_time), set.threads, num_rows); - } - } + report_poll_statistics(begin_time, num_rows); /* zero sensitive credentials before exit */ - { - volatile char *vp; - vp = (volatile char *)set.db_pass; - memset((char *)vp, 0, sizeof(set.db_pass)); - vp = (volatile char *)set.rdb_pass; - memset((char *)vp, 0, sizeof(set.rdb_pass)); - } + spine_clear_sensitive(set.database.password, sizeof(set.database.password)); + spine_clear_sensitive(set.remote_database.password, sizeof(set.remote_database.password)); /* uninstall the spine signal handler */ uninstall_spine_signal_handler(); @@ -1183,7 +1288,7 @@ int main(int argc, char *argv[]) { /* clueanup winsock library on Windows */ SOCK_CLEANUP; - exit(EXIT_SUCCESS); + exit(set.exit.exit_code); } /*! \fn static void display_help() @@ -1275,3 +1380,7 @@ static char *getarg(char *opt, char ***pargv) { die("ERROR: option %s requires a parameter", optname); } + +#ifdef SPINE_TEST_PROGRAM_ENTRY +#undef main +#endif diff --git a/spine.h b/spine.h index 47d904e..2d65559 100644 --- a/spine.h +++ b/spine.h @@ -34,585 +34,7 @@ #ifndef _SPINE_H_ #define _SPINE_H_ -/* Defines */ -#ifndef FALSE -#define FALSE 0 -#endif -#ifndef TRUE -#define TRUE 1 -#endif - -#define LOCAL 0 -#define REMOTE 1 - -#define SPINE_NONE 0 -#define SPINE_IPV4 1 -#define SPINE_IPV6 2 - -#ifndef __GNUC__ -# define __attribute__(x) /* NOTHING */ -#endif - -/* Windows does not support stderr. Therefore, don't use it. */ -#ifdef __CYGWIN__ -#define DISABLE_STDERR -#endif - -#ifdef HAS_EXECINFO_H -#include -#endif - -/* if a host is legal, return TRUE */ -#define HOSTID_DEFINED(x) ((x) >= 0) - -/* warning-suppression macros - * - * There are times when we cannot avoid using a parameter or variable which - * is not used, and these correctly generate compiler warnings. But when we - * *know* that the variable is actually intended to be unused, we can use one - * of these macros inside the function to suppress it. This has the effect - * of suppressing the warning (a good thing), plus documenting to the reader - * that this is intentional. - * - * Both do the same thing - they're just for different semantics. - */ - -#define UNUSED_VARIABLE(p) (void)(p) -#define UNUSED_PARAMETER(p) (void)(p) - -/* logging macros - * - * These all perform conditional logging based on the current runtime logging - * level, and it relies on a bit of tricky (but entirely portable) preprocessor - * techniques. - * - * Standard C does not support variadic macros (macros with a variable number - * of parameters), and though GNU C does, it's not at all portable. So we instead - * rely on the fact that putting parens around something turn multiple params - * into one: - * - * SPINE_LOG_DEBUG(("n=%d string=%s foo=%f", n, string, foo)); - * - * This macros has *one* parameter: - * - * ("n=%d string=%s foo=%f", n, string, foo) - * - * and the parentheses are part of it. When we call this macro, we pass the - * "single" parameter unadorned, so that - * - * spine_log args - * - * expands to - * - * spine_log ("n=%d string=%s foo=%f", n, string, foo) - * - * Voila: it's a normal printf-like call. - * - * The second part of this is the conditional test, and the obvious approach - * of using an "if" statement is exceptionally bad form: there are all kinds - * of pitfalls which arise in this case. Instead, we should try to use an - * *expression*, which has none of these problems. - * - * The conditional tests are modelled after the assert() mechanism, which - * checks the first parameter, and if it's true, it evaluates the second - * parameter. If the test is not true, then the second part is *guaranteed* - * not to be evaluated. - * - * The (void) prefix is to forestall compiler warnings about expressions - * not being used. - */ -#define SPINE_LOG(format_and_args) (spine_log format_and_args) -#define SPINE_LOG_LOW(format_and_args) (void)(set.log_level >= POLLER_VERBOSITY_LOW && spine_log format_and_args) -#define SPINE_LOG_MEDIUM(format_and_args) (void)(set.log_level >= POLLER_VERBOSITY_MEDIUM && spine_log format_and_args) -#define SPINE_LOG_HIGH(format_and_args) (void)(set.log_level >= POLLER_VERBOSITY_HIGH && spine_log format_and_args) -#define SPINE_LOG_DEBUG(format_and_args) (void)(set.log_level >= POLLER_VERBOSITY_DEBUG && spine_log format_and_args) -#define SPINE_LOG_DEVDBG(format_and_args) (void)(set.log_level >= POLLER_VERBOSITY_DEVDBG && spine_log format_and_args) - -/* general constants */ -#define MAX_THREADS 100 -#define MAX_DEBUG_DEVICES 100 -#define TINY_BUFSIZE 16 -#define SMALL_BUFSIZE 256 -#define MEDIUM_BUFSIZE 512 -#define BUFSIZE 1024 -#define DBL_BUFSIZE 2048 -#define LRG_BUFSIZE 8096 -#define BIG_BUFSIZE 65535 -#define MEGA_BUFSIZE 1024000 -#define HUGE_BUFSIZE 2048000 -#define LOGSIZE 65535 -#define LRG_LOGSIZE 1024000 -#define BITSINBYTE 8 -#define THIRTYTWO 4294967295ul -#define SIXTYFOUR 18446744073709551615ul -#define STAT_DESCRIP_ERROR 99 -#define SPINE_PARENT 1 -#define SPINE_FORK 0 - -/* locations to search for the config file */ -#define CONFIG_PATHS 4 -#define CONFIG_PATH_1 "" -#define CONFIG_PATH_2 "/etc/" -#define CONFIG_PATH_3 "/etc/cacti/" -#define CONFIG_PATH_4 "../etc/" - -/* config file defaults */ -#define DEFAULT_CONF_FILE "spine.conf" -#define DEFAULT_THREADS 5 -#define DEFAULT_DB_HOST "localhost" -#define DEFAULT_DB_DB "cacti" -#define DEFAULT_DB_USER "cactiuser" -#define DEFAULT_DB_PASS "cactiuser" -#define DEFAULT_DB_PORT 3306 -#define DEFAULT_DB_PREG 0 -#define DEFAULT_LOGFILE "/var/www/html/cacti/log/cacti.log" -#define DEFAULT_TIMEOUT 294000000 - -/* threads constants */ -#define LOCK_SNMP 0 -#define LOCK_SETEUID 2 -#define LOCK_GHBN 3 -#define LOCK_POOL 4 -#define LOCK_PHP 6 -#define LOCK_PHP_PROC_0 7 -#define LOCK_PHP_PROC_1 8 -#define LOCK_PHP_PROC_2 9 -#define LOCK_PHP_PROC_3 10 -#define LOCK_PHP_PROC_4 11 -#define LOCK_PHP_PROC_5 12 -#define LOCK_PHP_PROC_6 13 -#define LOCK_PHP_PROC_7 14 -#define LOCK_PHP_PROC_8 15 -#define LOCK_PHP_PROC_9 16 -#define LOCK_PHP_PROC_10 17 -#define LOCK_PHP_PROC_11 18 -#define LOCK_PHP_PROC_12 19 -#define LOCK_PHP_PROC_13 20 -#define LOCK_PHP_PROC_14 21 -#define LOCK_THDET 40 -#define LOCK_HOST_TIME 41 - -#define LOCK_SNMP_O 0 -#define LOCK_SETEUID_O 2 -#define LOCK_GHBN_O 3 -#define LOCK_POOL_O 4 -#define LOCK_PHP_O 6 -#define LOCK_PHP_PROC_0_O 7 -#define LOCK_PHP_PROC_1_O 8 -#define LOCK_PHP_PROC_2_O 9 -#define LOCK_PHP_PROC_3_O 10 -#define LOCK_PHP_PROC_4_O 11 -#define LOCK_PHP_PROC_5_O 12 -#define LOCK_PHP_PROC_6_O 13 -#define LOCK_PHP_PROC_7_O 14 -#define LOCK_PHP_PROC_8_O 15 -#define LOCK_PHP_PROC_9_O 16 -#define LOCK_PHP_PROC_10_O 17 -#define LOCK_PHP_PROC_11_O 18 -#define LOCK_PHP_PROC_12_O 19 -#define LOCK_PHP_PROC_13_O 20 -#define LOCK_PHP_PROC_14_O 21 -#define LOCK_THDET_O 40 -#define LOCK_HOST_TIME_O 41 - -/* poller actions */ -#define POLLER_ACTION_SNMP 0 -#define POLLER_ACTION_SCRIPT 1 -#define POLLER_ACTION_PHP_SCRIPT_SERVER 2 -#define POLLER_ACTION_SNMP_COUNT 10 -#define POLLER_ACTION_SCRIPT_COUNT 11 -#define POLLER_ACTION_PHP_SCRIPT_SERVER_COUNT 12 - -/* reindex constants */ -#define POLLER_COMMAND_REINDEX 1 - -/* log destinations */ -#define LOGDEST_FILE 1 -#define LOGDEST_BOTH 2 -#define LOGDEST_SYSLOG 3 -#define LOGDEST_STDOUT 4 - -#define IS_LOGGING_TO_FILE() ((set.log_destination) == LOGDEST_FILE || (set.log_destination) == LOGDEST_BOTH) -#define IS_LOGGING_TO_SYSLOG() ((set.log_destination) == LOGDEST_SYSLOG || (set.log_destination) == LOGDEST_BOTH) -#define IS_LOGGING_TO_STDOUT() ((set.log_destination) == LOGDEST_STDOUT ) - -#define SPINE_FREE(s) do { if (s) { free((void *)s); s = NULL; } } while(0) - -/* logging levels */ -#define POLLER_VERBOSITY_NONE 1 -#define POLLER_VERBOSITY_LOW 2 -#define POLLER_VERBOSITY_MEDIUM 3 -#define POLLER_VERBOSITY_HIGH 4 -#define POLLER_VERBOSITY_DEBUG 5 -#define POLLER_VERBOSITY_DEVDBG 6 - -/* logging separator constants */ -#define GDC_MIN 0 -#define GDC_HYPHEN 0 -#define GDC_SLASH 1 -#define GDC_DOT 2 -#define GDC_MAX 2 -#define GDC_DEFAULT 1 - -/* logging format constants */ -#define GD_FMT_SIZE 21 -#define GD_MIN 0 -#define GD_MO_D_Y 0 -#define GD_MN_D_Y 1 -#define GD_D_MO_Y 2 -#define GD_D_MN_Y 3 -#define GD_Y_MO_D 4 -#define GD_Y_MN_D 5 -#define GD_MAX 5 -#define GD_DEFAULT 5 - -/* host availability statistics */ -#define AVAIL_NONE 0 -#define AVAIL_SNMP_AND_PING 1 -#define AVAIL_SNMP 2 -#define AVAIL_PING 3 -#define AVAIL_SNMP_OR_PING 4 -#define AVAIL_SNMP_GET_SYSDESC 5 -#define AVAIL_SNMP_GET_NEXT 6 -#define AVAIL_STREAM 7 - -#define PING_ICMP 1 -#define PING_UDP 2 -#define PING_TCP 3 -#define PING_SNMP 4 -#define PING_TCP_CLOSED 5 - -#define HOST_UNKNOWN 0 -#define HOST_DOWN 1 -#define HOST_RECOVERING 2 -#define HOST_UP 3 - -/* required for ICMP and UDP ping */ -#define ICMP_ECHO 8 -#define ICMP_HDR_SIZE 8 - -/* required for PHP Script Server */ -#define MAX_PHP_SERVERS 15 -#define PHP_READY 0 -#define PHP_BUSY 1 -#define PHP_INIT 999 -#define PHP_ERROR 99 - -/* required for validation of script results */ -#define RESULT_INIT 0 -#define RESULT_ARGX 1 -#define RESULT_VALX 2 -#define RESULT_SEPARATOR 3 -#define RESULT_SPACE 4 -#define RESULT_ALPHA 5 -#define RESULT_DIGIT 6 - -/* snmp session status */ -#define SNMP_1 0 -#define SNMP_2c 1 -#define SNMP_3 3 -#define SNMP_NONE 4 - -/* Constants for remote polling */ -#define REMOTE_ONLINE 0 -#define REMOTE_OFFLINE 1 -#define REMOTE_RECOVERY 2 - -/* These are used to perform string matches, returning TRUE/VALUE values. - * For strcmp() this is not really that useful, but the case-insensitive - * one has slight portability issues. Better to abstract them here. - */ -#define STRMATCH(a,b) (strcmp((a),(b)) == 0) -#define STRIMATCH(a,b) (strcasecmp((a),(b)) == 0) - -/* When any kind of poller wants to set an undefined value; this particular - * value used ('U') springs from the requirements of rrdupdate. We also - * include the corresponding test macro which looks for the literal string - * "U". This *could* use strcmp(), but this is more efficient. - */ -#define SET_UNDEFINED(buf) ( (buf)[0] = 'U', (buf)[1] = '\0' ) -#define IS_UNDEFINED(buf) ( (buf)[0] == 'U' && (buf)[1] == '\0' ) - -/*! Config Structure - * - * This structure holds Spine database configuration information and/or override values - * obtained via either accessing the database or reading the runtime options. In addition, - * it contains runtime status information. - * - */ -typedef struct config_struct { - /* stdout, stderr TTY protection */ - int stdout_notty; - int stderr_notty; - /* general configuration/runtime settings */ - int poller_id; - int poller_interval; - int parent_fork; - int num_parent_processes; - int script_timeout; - int active_profiles; - int total_snmp_ports; - int threads; - int threads_set; - int logfile_processed; - int boost_enabled; - int boost_redirect; - int cygwinshloc; - /* debugging options */ - int snmponly; - int SQL_readonly; - /* host range to be poller with this spine process */ - int start_host_id; - int end_host_id; - char host_id_list[BIG_BUFSIZE]; - int has_device_0; - int has_output_regex; - /* database connection information */ - char db_host[BUFSIZE]; - char db_db[BUFSIZE]; - char db_user[BUFSIZE]; - char db_pass[BUFSIZE]; - int db_ssl; - char db_ssl_key[BUFSIZE]; - char db_ssl_cert[BUFSIZE]; - char db_ssl_ca[BUFSIZE]; - int d_b; - unsigned int db_port; - char dbversion[BUFSIZE]; - int dbonupdate; - int cacti_version; - /* path information */ - char path_logfile[DBL_BUFSIZE]; - char path_php[BUFSIZE]; - char path_php_server[BUFSIZE]; - /* logging options */ - int log_level; - int log_destination; - int log_perror; - int log_pwarn; - int log_pstats; - char selective_device_debug[LRG_BUFSIZE]; - int spine_log_level; - int log_datetime_separator; - int log_datetime_format; - /* ping settings */ - int icmp_avail; - int icmp_uses_caps; - int availability_method; - int ping_method; - int ping_retries; - int ping_timeout; - int ping_failure_count; - int ping_recovery_count; - int ping_only; - /* snmp options */ - int snmp_max_get_size; - int snmp_retries; - char snmp_clientaddr[BUFSIZE]; - int mibs; - /* PHP Script Server Options */ - int php_required; - int php_initialized; - int php_servers; - int php_current_server; - /* Exit code if we need it */ - int exit_code; - size_t exit_size; - void* exit_stack[10]; - - /* Remote polling mode */ - int mode; - /* remote database connection information */ - char rdb_host[BUFSIZE]; - char rdb_db[BUFSIZE]; - char rdb_user[BUFSIZE]; - char rdb_pass[BUFSIZE]; - int rdb_ssl; - char rdb_ssl_key[BUFSIZE]; - char rdb_ssl_cert[BUFSIZE]; - char rdb_ssl_ca[BUFSIZE]; - unsigned int rdb_port; - char rdbversion[BUFSIZE]; - int rdbonupdate; -} config_t; - -/*! Target Structure - * - * This structure holds the contents of the Poller Items table and the results - * of each polling action. - * - */ -typedef struct target_struct { - int target_id; - char result[RESULTS_BUFFER]; - int local_data_id; - int action; - char command[256]; - char hostname[250]; - char snmp_community[100]; - int snmp_version; - char snmp_username[50]; - char snmp_password[50]; - char snmp_auth_protocol[7]; - char snmp_priv_passphrase[200]; - char snmp_priv_protocol[8]; - char snmp_context[65]; - char snmp_engine_id[30]; - int snmp_port; - int snmp_timeout; - int availability_method; - int ping_method; - int ping_port; - int ping_timeout; - int ping_retries; - char rrd_name[30]; - char rrd_path[255]; - int rrd_num; - char arg1[1024]; - char arg2[255]; - char arg3[255]; - char output_regex[255]; -} target_t; - -/*! SNMP OID's Structure - * - * This structure holds SNMP get results temporarily while polling is taking place. - * - */ -typedef struct snmp_oids { - int array_position; - char oid[1024]; - char result[RESULTS_BUFFER]; -} snmp_oids_t; - -/*! Poller Structure - * - * This structure holds thread polling instructions. - * - */ -typedef struct poller_thread { - int device_counter; - int host_id; - int host_thread; - int host_threads; - int host_data_ids; - int threads_complete; - int complete; - char host_time[40]; - double host_time_double; - spine_sem_t *thread_init_sem; -} poller_thread_t; - -/*! PHP Script Server Structure - * - * This structure holds status and PID information for all the running - * PHP Script Server processes. - * - */ -typedef struct php_processes { - int php_state; - pid_t php_pid; - int php_write_fd; - int php_read_fd; -} php_t; - -/*! Host Structure - * - * This structure holds host information from the host table and is used throughout - * the application. - * - */ -typedef struct host_struct { - int id; - char hostname[250]; - char snmp_community[100]; - int snmp_version; - char snmp_username[50]; - char snmp_password[50]; - char snmp_auth_protocol[7]; - char snmp_priv_passphrase[200]; - char snmp_priv_protocol[8]; - char snmp_context[65]; - char snmp_engine_id[30]; - int snmp_port; - int snmp_timeout; - int snmp_retries; - char snmp_sysDescr[600]; - char snmp_sysObjectID[160]; - unsigned long long snmp_sysUpTimeInstance; - char snmp_sysContact[300]; - char snmp_sysName[300]; - char snmp_sysLocation[600]; - int max_oids; - int availability_method; - int ping_method; - int ping_port; - int ping_timeout; - int ping_retries; - int status; - int status_event_count; - char status_fail_date[40]; - char status_rec_date[40]; - char status_last_error[BUFSIZE * 2 + 2]; - double min_time; - double max_time; - double cur_time; - double avg_time; - int total_polls; - int failed_polls; - double availability; - int ignore_host; - void *snmp_session; - int snmp_status; -} host_t; - -/*! Host Reindex Structure - * - * This structure holds the results of the host re-index checks and values. - * - */ -typedef struct host_reindex_struct { - char op[4]; - char assert_value[100]; - char arg1[1024]; - int data_query_id; - int action; -} reindex_t; - -/*! Ping Result Structure - * - * This structure holds the results of a host ping. - * - */ -typedef struct ping_results { - char hostname[BUFSIZE]; - char ping_status[50]; - char ping_response[BUFSIZE]; - char snmp_status[50]; - char snmp_response[BUFSIZE]; -} ping_t; - -/*! Name Result Structure - * - * This structure holds the results of a name/port split - * - */ -typedef struct name_port { - // Method = 0 - default, 1 - tcp, 2 - udp - char hostname[BUFSIZE]; - int method; - int port; -} name_t; - -/*! MySQL Connection Pool Structure - * - * This structure holds the mysql connection pool object. - */ -typedef struct db_connection { - int id; - int free; - MYSQL mysql; -} pool_t; +#include "spine_types.h" /* Include all Standard Spine Headers */ #include "poller.h" @@ -636,8 +58,8 @@ extern config_t set; extern php_t *php_processes; extern char start_datetime[20]; extern char config_paths[CONFIG_PATHS][BUFSIZE]; -extern spine_sem_t available_threads; -extern spine_sem_t available_scripts; +extern spine_permits_t available_threads; +extern spine_permits_t available_scripts; extern pool_t *db_pool_remote; extern pool_t *db_pool_local; diff --git a/spine_types.h b/spine_types.h new file mode 100644 index 0000000..3eadc35 --- /dev/null +++ b/spine_types.h @@ -0,0 +1,668 @@ +/* + ex: set tabstop=4 shiftwidth=4 autoindent: + +-------------------------------------------------------------------------+ + | Copyright (C) 2004-2026 The Cacti Group | + | | + | This program is free software; you can redistribute it and/or | + | modify it under the terms of the GNU Lesser General Public | + | License as published by the Free Software Foundation; either | + | version 2.1 of the License, or (at your option) any later version. | + | | + | This program is distributed in the hope that it will be useful, | + | but WITHOUT ANY WARRANTY; without even the implied warranty of | + | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | + | GNU Lesser General Public License for more details. | + | | + | You should have received a copy of the GNU Lesser General Public | + | License along with this library; if not, write to the Free Software | + | Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA | + | 02110-1301, USA | + | | + +-------------------------------------------------------------------------+ + | spine: a backend data gatherer for cacti | + +-------------------------------------------------------------------------+ + | This poller would not have been possible without: | + | - Larry Adams (current development and enhancements) | + | - Rivo Nurges (rrd support, mysql poller cache, misc functions) | + | - RTG (core poller code, pthreads, snmp, autoconf examples) | + | - Brady Alleman/Doug Warner (threading ideas, implementation details) | + +-------------------------------------------------------------------------+ + | - Cacti - http://www.cacti.net/ | + +-------------------------------------------------------------------------+ +*/ + +#ifndef SPINE_TYPES_H +#define SPINE_TYPES_H + +#include "common.h" + +/* Defines */ +#ifndef FALSE +#define FALSE 0 +#endif +#ifndef TRUE +#define TRUE 1 +#endif + +#define LOCAL 0 +#define REMOTE 1 + +#define SPINE_NONE 0 +#define SPINE_IPV4 1 +#define SPINE_IPV6 2 + +#ifndef __GNUC__ +# define __attribute__(x) /* NOTHING */ +#endif + +/* Windows does not support stderr. Therefore, don't use it. */ +#ifdef __CYGWIN__ +#define DISABLE_STDERR +#endif + +#ifdef HAS_EXECINFO_H +#include +#endif + +/* if a host is legal, return TRUE */ +#define HOSTID_DEFINED(x) ((x) >= 0) + +/* warning-suppression macros + * + * There are times when we cannot avoid using a parameter or variable which + * is not used, and these correctly generate compiler warnings. But when we + * *know* that the variable is actually intended to be unused, we can use one + * of these macros inside the function to suppress it. This has the effect + * of suppressing the warning (a good thing), plus documenting to the reader + * that this is intentional. + * + * Both do the same thing - they're just for different semantics. + */ + +#define UNUSED_VARIABLE(p) (void)(p) +#define UNUSED_PARAMETER(p) (void)(p) + +/* logging macros + * + * These all perform conditional logging based on the current runtime logging + * level, and it relies on a bit of tricky (but entirely portable) preprocessor + * techniques. + * + * Standard C does not support variadic macros (macros with a variable number + * of parameters), and though GNU C does, it's not at all portable. So we instead + * rely on the fact that putting parens around something turn multiple params + * into one: + * + * SPINE_LOG_DEBUG(("n=%d string=%s foo=%f", n, string, foo)); + * + * This macros has *one* parameter: + * + * ("n=%d string=%s foo=%f", n, string, foo) + * + * and the parentheses are part of it. When we call this macro, we pass the + * "single" parameter unadorned, so that + * + * spine_log args + * + * expands to + * + * spine_log ("n=%d string=%s foo=%f", n, string, foo) + * + * Voila: it's a normal printf-like call. + * + * The second part of this is the conditional test, and the obvious approach + * of using an "if" statement is exceptionally bad form: there are all kinds + * of pitfalls which arise in this case. Instead, we should try to use an + * *expression*, which has none of these problems. + * + * The conditional tests are modelled after the assert() mechanism, which + * checks the first parameter, and if it's true, it evaluates the second + * parameter. If the test is not true, then the second part is *guaranteed* + * not to be evaluated. + * + * The (void) prefix is to forestall compiler warnings about expressions + * not being used. + */ +#define SPINE_LOG(format_and_args) (spine_log format_and_args) +#define SPINE_LOG_LOW(format_and_args) (void)(set.logging.log_level >= POLLER_VERBOSITY_LOW && spine_log format_and_args) +#define SPINE_LOG_MEDIUM(format_and_args) (void)(set.logging.log_level >= POLLER_VERBOSITY_MEDIUM && spine_log format_and_args) +#define SPINE_LOG_HIGH(format_and_args) (void)(set.logging.log_level >= POLLER_VERBOSITY_HIGH && spine_log format_and_args) +#define SPINE_LOG_DEBUG(format_and_args) (void)(set.logging.log_level >= POLLER_VERBOSITY_DEBUG && spine_log format_and_args) +#define SPINE_LOG_DEVDBG(format_and_args) (void)(set.logging.log_level >= POLLER_VERBOSITY_DEVDBG && spine_log format_and_args) +#define SPINE_LOG_DEVICE(host_id, verbosity, format_and_args) \ + (void)(spine_should_log_device(host_id, verbosity) && spine_log format_and_args) + +/* general constants */ +#define MAX_THREADS 100 +#define MAX_DEBUG_DEVICES 100 +#define TINY_BUFSIZE 16 +#define SMALL_BUFSIZE 256 +#define MEDIUM_BUFSIZE 512 +#define BUFSIZE 1024 +#define DBL_BUFSIZE 2048 +#define LRG_BUFSIZE 8096 +#define BIG_BUFSIZE 65535 +#define MEGA_BUFSIZE 1024000 +#define HUGE_BUFSIZE 2048000 +#define LOGSIZE 65535 +#define LRG_LOGSIZE 1024000 +#define BITSINBYTE 8 +#define THIRTYTWO 4294967295ul +#define SIXTYFOUR 18446744073709551615ul +#define STAT_DESCRIP_ERROR 99 +#define SPINE_PARENT 1 +#define SPINE_FORK 0 + +/* locations to search for the config file */ +#define CONFIG_PATHS 4 +#define CONFIG_PATH_1 "" +#define CONFIG_PATH_2 "/etc/" +#define CONFIG_PATH_3 "/etc/cacti/" +#define CONFIG_PATH_4 "../etc/" + +/* config file defaults */ +#define DEFAULT_CONF_FILE "spine.conf" +#define DEFAULT_THREADS 5 +#define DEFAULT_DB_HOST "localhost" +#define DEFAULT_DB_DB "cacti" +#define DEFAULT_DB_USER "cactiuser" +#define DEFAULT_DB_PASS "cactiuser" +#define DEFAULT_DB_PORT 3306 +#define DEFAULT_DB_PREG 0 +#define DEFAULT_LOGFILE "/var/www/html/cacti/log/cacti.log" +#define DEFAULT_TIMEOUT 294000000 + +/* threads constants */ +#define LOCK_SNMP 0 +#define LOCK_SETEUID 2 +#define LOCK_GHBN 3 +#define LOCK_POOL 4 +#define LOCK_PHP 6 +#define LOCK_PHP_PROC_0 7 +#define LOCK_PHP_PROC_1 8 +#define LOCK_PHP_PROC_2 9 +#define LOCK_PHP_PROC_3 10 +#define LOCK_PHP_PROC_4 11 +#define LOCK_PHP_PROC_5 12 +#define LOCK_PHP_PROC_6 13 +#define LOCK_PHP_PROC_7 14 +#define LOCK_PHP_PROC_8 15 +#define LOCK_PHP_PROC_9 16 +#define LOCK_PHP_PROC_10 17 +#define LOCK_PHP_PROC_11 18 +#define LOCK_PHP_PROC_12 19 +#define LOCK_PHP_PROC_13 20 +#define LOCK_PHP_PROC_14 21 +#define LOCK_THDET 40 +#define LOCK_HOST_TIME 41 + +#define LOCK_SNMP_O 0 +#define LOCK_SETEUID_O 2 +#define LOCK_GHBN_O 3 +#define LOCK_POOL_O 4 +#define LOCK_PHP_O 6 +#define LOCK_PHP_PROC_0_O 7 +#define LOCK_PHP_PROC_1_O 8 +#define LOCK_PHP_PROC_2_O 9 +#define LOCK_PHP_PROC_3_O 10 +#define LOCK_PHP_PROC_4_O 11 +#define LOCK_PHP_PROC_5_O 12 +#define LOCK_PHP_PROC_6_O 13 +#define LOCK_PHP_PROC_7_O 14 +#define LOCK_PHP_PROC_8_O 15 +#define LOCK_PHP_PROC_9_O 16 +#define LOCK_PHP_PROC_10_O 17 +#define LOCK_PHP_PROC_11_O 18 +#define LOCK_PHP_PROC_12_O 19 +#define LOCK_PHP_PROC_13_O 20 +#define LOCK_PHP_PROC_14_O 21 +#define LOCK_THDET_O 40 +#define LOCK_HOST_TIME_O 41 + +/* poller actions */ +#define POLLER_ACTION_SNMP 0 +#define POLLER_ACTION_SCRIPT 1 +#define POLLER_ACTION_PHP_SCRIPT_SERVER 2 +#define POLLER_ACTION_SNMP_COUNT 10 +#define POLLER_ACTION_SCRIPT_COUNT 11 +#define POLLER_ACTION_PHP_SCRIPT_SERVER_COUNT 12 + +/* reindex constants */ +#define POLLER_COMMAND_REINDEX 1 + +/* log destinations */ +#define LOGDEST_FILE 1 +#define LOGDEST_BOTH 2 +#define LOGDEST_SYSLOG 3 +#define LOGDEST_STDOUT 4 + +#define IS_LOGGING_TO_FILE() ((set.logging.log_destination) == LOGDEST_FILE || (set.logging.log_destination) == LOGDEST_BOTH) +#define IS_LOGGING_TO_SYSLOG() ((set.logging.log_destination) == LOGDEST_SYSLOG || (set.logging.log_destination) == LOGDEST_BOTH) +#define IS_LOGGING_TO_STDOUT() ((set.logging.log_destination) == LOGDEST_STDOUT ) + +#define SPINE_FREE(s) do { if (s) { free((void *)s); s = NULL; } } while(0) + +/* logging levels */ +#define POLLER_VERBOSITY_NONE 1 +#define POLLER_VERBOSITY_LOW 2 +#define POLLER_VERBOSITY_MEDIUM 3 +#define POLLER_VERBOSITY_HIGH 4 +#define POLLER_VERBOSITY_DEBUG 5 +#define POLLER_VERBOSITY_DEVDBG 6 + +/* logging separator constants */ +#define GDC_MIN 0 +#define GDC_HYPHEN 0 +#define GDC_SLASH 1 +#define GDC_DOT 2 +#define GDC_MAX 2 +#define GDC_DEFAULT 1 + +/* logging format constants */ +#define GD_FMT_SIZE 21 +#define GD_MIN 0 +#define GD_MO_D_Y 0 +#define GD_MN_D_Y 1 +#define GD_D_MO_Y 2 +#define GD_D_MN_Y 3 +#define GD_Y_MO_D 4 +#define GD_Y_MN_D 5 +#define GD_MAX 5 +#define GD_DEFAULT 5 + +/* host availability statistics */ +#define AVAIL_NONE 0 +#define AVAIL_SNMP_AND_PING 1 +#define AVAIL_SNMP 2 +#define AVAIL_PING 3 +#define AVAIL_SNMP_OR_PING 4 +#define AVAIL_SNMP_GET_SYSDESC 5 +#define AVAIL_SNMP_GET_NEXT 6 +#define AVAIL_STREAM 7 + +#define PING_ICMP 1 +#define PING_UDP 2 +#define PING_TCP 3 +#define PING_SNMP 4 +#define PING_TCP_CLOSED 5 + +#define HOST_UNKNOWN 0 +#define HOST_DOWN 1 +#define HOST_RECOVERING 2 +#define HOST_UP 3 + +/* required for ICMP and UDP ping */ +#define ICMP_ECHO 8 +#define ICMP_HDR_SIZE 8 + +/* required for PHP Script Server */ +#define MAX_PHP_SERVERS 15 +#define PHP_READY 0 +#define PHP_BUSY 1 +#define PHP_INIT 999 +#define PHP_ERROR 99 + +/* required for validation of script results */ +#define RESULT_INIT 0 +#define RESULT_ARGX 1 +#define RESULT_VALX 2 +#define RESULT_SEPARATOR 3 +#define RESULT_SPACE 4 +#define RESULT_ALPHA 5 +#define RESULT_DIGIT 6 + +/* snmp session status */ +#define SNMP_1 0 +#define SNMP_2c 1 +#define SNMP_3 3 +#define SNMP_NONE 4 + +/* Constants for remote polling */ +#define REMOTE_ONLINE 0 +#define REMOTE_OFFLINE 1 +#define REMOTE_RECOVERY 2 + +/* These are used to perform string matches, returning TRUE/VALUE values. + * For strcmp() this is not really that useful, but the case-insensitive + * one has slight portability issues. Better to abstract them here. + */ +#define STRMATCH(a,b) (strcmp((a),(b)) == 0) +#define STRIMATCH(a,b) (strcasecmp((a),(b)) == 0) + +/* When any kind of poller wants to set an undefined value; this particular + * value used ('U') springs from the requirements of rrdupdate. We also + * include the corresponding test macro which looks for the literal string + * "U". This *could* use strcmp(), but this is more efficient. + */ +#define SET_UNDEFINED(buf) ( (buf)[0] = 'U', (buf)[1] = '\0' ) +#define IS_UNDEFINED(buf) ( (buf)[0] == 'U' && (buf)[1] == '\0' ) + +/*! Config Structure + * + * This structure holds Spine database configuration information and/or override values + * obtained via either accessing the database or reading the runtime options. In addition, + * it contains runtime status information. + * + */ +/* Console configuration and runtime state. */ +typedef struct { + int stdout_notty; + int stderr_notty; +} spine_console_config_t; + +/* Poller configuration and runtime state. */ +typedef struct { + int poller_id; + int poller_interval; + int parent_fork; + int num_parent_processes; + int active_profiles; + int threads; + int threads_set; + int snmponly; + int SQL_readonly; + int mode; +} spine_poller_config_t; + +/* Hosts configuration and runtime state. */ +typedef struct { + int start_host_id; + int end_host_id; + char host_id_list[BIG_BUFSIZE]; + int has_device_0; + int has_output_regex; +} spine_hosts_config_t; + +/* Database configuration and runtime state. */ +typedef struct { + char host[BUFSIZE]; + char database[BUFSIZE]; + char user[BUFSIZE]; + char password[BUFSIZE]; + int ssl; + char ssl_key[BUFSIZE]; + char ssl_cert[BUFSIZE]; + char ssl_ca[BUFSIZE]; + unsigned int port; + char version[BUFSIZE]; + int onupdate; +} spine_database_config_t; + +/* Php configuration and runtime state. */ +typedef struct { + char path_php[BUFSIZE]; + char path_php_server[BUFSIZE]; + int script_timeout; + int php_required; + int php_initialized; + int php_servers; + int php_current_server; +} spine_php_config_t; + +/* Logging configuration and runtime state. */ +typedef struct { + char path_logfile[DBL_BUFSIZE]; + int logfile_processed; + int log_level; + int log_destination; + int log_perror; + int log_pwarn; + int log_pstats; + char selective_device_debug[LRG_BUFSIZE]; + int spine_log_level; + int log_datetime_separator; + int log_datetime_format; +} spine_logging_config_t; + +/* Availability configuration and runtime state. */ +typedef struct { + int icmp_avail; + int icmp_uses_caps; + int availability_method; + int ping_method; + int ping_retries; + int ping_timeout; + int ping_failure_count; + int ping_recovery_count; + int ping_only; +} spine_availability_config_t; + +/* Snmp configuration and runtime state. */ +typedef struct { + int total_snmp_ports; + int snmp_max_get_size; + int snmp_retries; + char snmp_clientaddr[BUFSIZE]; + int mibs; +} spine_snmp_config_t; + +/* Boost configuration and runtime state. */ +typedef struct { + int boost_enabled; + int boost_redirect; +} spine_boost_config_t; + +/* Exit configuration and runtime state. */ +typedef struct { + int exit_code; + size_t exit_size; + void* exit_stack[10]; +} spine_exit_config_t; + +/* Process-local aggregate. A full rebuild is required when its layout changes. */ +typedef struct config_struct { + spine_console_config_t console; + spine_poller_config_t poller; + spine_hosts_config_t hosts; + spine_database_config_t database; + spine_database_config_t remote_database; + spine_php_config_t php; + spine_logging_config_t logging; + spine_availability_config_t availability; + spine_snmp_config_t snmp; + spine_boost_config_t boost; + spine_exit_config_t exit; + int d_b; + int cacti_version; + int cygwinshloc; +} config_t; + +/*! Target Structure + * + * This structure holds the contents of the Poller Items table and the results + * of each polling action. + * + */ +/* Owned SNMP identity and transport options. The borrowed snmp_connection_t + * adapter remains separate; these arrays are never external pointers. */ +typedef struct { + char community[100]; + int version; + char username[50]; + char password[50]; + char auth_protocol[7]; + char priv_passphrase[200]; + char priv_protocol[8]; + char context[65]; + char engine_id[30]; + int port; + int timeout; +} snmp_profile_t; + +typedef struct { + int method; + int ping_method; + int port; + int timeout; + int retries; +} device_availability_t; + +typedef struct target_struct { + int target_id; + char result[RESULTS_BUFFER]; + int local_data_id; + int action; + char command[256]; + char hostname[250]; + char rrd_name[30]; + char rrd_path[255]; + int rrd_num; + char arg1[1024]; + char arg2[255]; + char arg3[255]; + char output_regex[255]; + snmp_profile_t snmp; + device_availability_t availability; +} target_t; + +/*! SNMP OID's Structure + * + * This structure holds SNMP get results temporarily while polling is taking place. + * + */ +typedef struct snmp_oids { + int array_position; + char oid[1024]; + char result[RESULTS_BUFFER]; +} snmp_oids_t; + +/*! Poller Structure + * + * This structure holds thread polling instructions. + * + */ +/* Process-local permits use a mutex because Darwin does not implement + * unnamed POSIX semaphores. No caller needs a blocking semaphore wait. */ +typedef struct spine_permits { + pthread_mutex_t mutex; + int available; +} spine_permits_t; + +typedef struct poller_thread { + int device_counter; + int host_id; + int host_thread; + int host_threads; + int host_data_ids; + int threads_complete; + int complete; + int output_failed; + char host_time[40]; + double host_time_double; + spine_permits_t *thread_init_sem; +} poller_thread_t; + +/*! PHP Script Server Structure + * + * This structure holds status and PID information for all the running + * PHP Script Server processes. + * + */ +typedef struct php_processes { + int php_state; + pid_t php_pid; + int php_write_fd; + int php_read_fd; +} php_t; + +/*! Host Structure + * + * This structure holds host information from the host table and is used throughout + * the application. + * + */ +typedef struct { + snmp_profile_t profile; + int retries; + int max_oids; + void* session; + int status; +} host_snmp_t; + +typedef struct { + char snmp_sysDescr[600]; + char snmp_sysObjectID[160]; + unsigned long long snmp_sysUpTimeInstance; + char snmp_sysContact[300]; + char snmp_sysName[300]; + char snmp_sysLocation[600]; +} host_system_t; + +typedef struct { + int status; + int status_event_count; + char status_fail_date[40]; + char status_rec_date[40]; + char status_last_error[BUFSIZE * 2 + 2]; +} host_state_t; + +typedef struct { + double min_time; + double max_time; + double cur_time; + double avg_time; + int total_polls; + int failed_polls; + double availability; +} host_statistics_t; + +typedef struct host_struct { + int id; + char hostname[250]; + int ignore_host; + host_snmp_t snmp; + host_system_t system; + device_availability_t availability; + host_state_t state; + host_statistics_t statistics; +} host_t; + +/*! Host Reindex Structure + * + * This structure holds the results of the host re-index checks and values. + * + */ +typedef struct host_reindex_struct { + char op[4]; + char assert_value[100]; + char arg1[1024]; + int data_query_id; + int action; +} reindex_t; + +/*! Ping Result Structure + * + * This structure holds the results of a host ping. + * + */ +typedef struct ping_results { + char hostname[BUFSIZE]; + char ping_status[50]; + char ping_response[BUFSIZE]; + char snmp_status[50]; + char snmp_response[BUFSIZE]; +} ping_t; + +/*! Name Result Structure + * + * This structure holds the results of a name/port split + * + */ +typedef struct name_port { + // Method = 0 - default, 1 - tcp, 2 - udp + char hostname[BUFSIZE]; + int method; + int port; +} name_t; + +/*! MySQL Connection Pool Structure + * + * This structure holds the mysql connection pool object. + */ +typedef struct db_connection { + int id; + int free; + MYSQL mysql; +} pool_t; + +#endif /* SPINE_TYPES_H */ diff --git a/sql.c b/sql.c index 682f396..a1037ce 100644 --- a/sql.c +++ b/sql.c @@ -46,69 +46,48 @@ * \return TRUE if successful, or FALSE if not. * */ -int db_insert(MYSQL *mysql, int type, const char *query) { - int error; - int error_count = 0; - char query_frag[LRG_BUFSIZE]; - - /* save a fragment just in case */ - memset(query_frag, 0, LRG_BUFSIZE); - snprintf(query_frag, LRG_BUFSIZE, "%s", query); - - /* show the sql query */ - SPINE_LOG_DEVDBG(("DEVDBG: SQL:%s", query_frag)); - - while(1) { - if (set.SQL_readonly == FALSE) { - if (mysql_query(mysql, query)) { - error = mysql_errno(mysql); - - if (error == 2013 || error == 2006) { - if (errno != EINTR) { - db_reconnect(mysql, type, error, "db_insert"); - - error_count++; - - if (error_count > 30) { - die("FATAL: Too many Reconnect Attempts!"); - } - - continue; - } else { - usleep(50000); - continue; - } - } - - if ((error == 1213) || (error == 1205)) { - usleep(50000); - error_count++; - - if (error_count > 30) { - SPINE_LOG(("ERROR: Too many Lock/Deadlock errors occurred!, SQL Fragment:'%s'", query_frag)); - return FALSE; - } - - continue; - } else { - SPINE_LOG(("ERROR: SQL Failed! Error:'%i', Message:'%s', SQL Fragment:'%s'", error, mysql_error(mysql), query_frag)); - return FALSE; - } - } else { - return TRUE; - } - } else { - return TRUE; - } +static void retry_disconnected_query(MYSQL *mysql, int type, int error, const char *function, int *error_count) { + if (errno == EINTR) { + spine_sleep_usec(50000); + return; } + db_reconnect(mysql, type, error, function); + if (++*error_count > 30) die("FATAL: Too many Reconnect Attempts!"); +} + +int db_insert(MYSQL *mysql, int type, const char *query) { + int error_count = 0; + char query_frag[LRG_BUFSIZE]; + snprintf(query_frag, sizeof(query_frag), "%s", query); + SPINE_LOG_DEVDBG(("DEVDBG: SQL:%s", query_frag)); + if (set.poller.SQL_readonly != FALSE) return TRUE; + while (mysql_query(mysql, query) != 0) { + int error = mysql_errno(mysql); + if (error == 2013 || error == 2006) { + retry_disconnected_query(mysql, type, error, "db_insert", &error_count); + continue; + } + if (error == 1213 || error == 1205) { + spine_sleep_usec(50000); + if (++error_count > 30) { + SPINE_LOG(("ERROR: Too many Lock/Deadlock errors occurred!, SQL Fragment:'%s'", query_frag)); + return FALSE; + } + continue; + } + SPINE_LOG(("ERROR: SQL Failed! Error:'%i', Message:'%s', SQL Fragment:'%s'", error, mysql_error(mysql), query_frag)); + return FALSE; + } + return TRUE; } int db_reconnect(MYSQL *mysql, int type, int error, const char *function) { unsigned long mysql_thread = 0; char query[100]; + int ping_status; mysql_thread = mysql_thread_id(mysql); - mysql_ping(mysql); + ping_status = mysql_ping(mysql); if (mysql_thread_id(mysql) != mysql_thread) { SPINE_LOG(("WARNING: Connection Broken in Function %s with Error %i. Reconnect via mysql_ping() successful.", function, error)); @@ -126,6 +105,11 @@ int db_reconnect(MYSQL *mysql, int type, int error, const char *function) { return TRUE; } + /* A live session on the same thread has nothing to reconnect. */ + if (ping_status == 0) { + return FALSE; + } + /* mysql_ping() did not reconnect; do it explicitly */ SPINE_LOG(("WARNING: Connection Broken in Function %s with Error %i. Attempting explicit reconnect.", function, error)); @@ -152,64 +136,29 @@ int db_reconnect(MYSQL *mysql, int type, int error, const char *function) { * */ MYSQL_RES *db_query(MYSQL *mysql, int type, const char *query) { - MYSQL_RES *mysql_res = 0; - - int error = 0; - int error_count = 0; - - char query_frag[LRG_BUFSIZE]; - - /* save a fragment just in case */ - memset(query_frag, 0, LRG_BUFSIZE); - snprintf(query_frag, LRG_BUFSIZE, "%s", query); - - /* show the sql query */ + int error_count = 0; + char query_frag[LRG_BUFSIZE]; + snprintf(query_frag, sizeof(query_frag), "%s", query); SPINE_LOG_DEVDBG(("DEVDBG: SQL:%s", query_frag)); - - while (1) { - if (mysql_query(mysql, query)) { - error = mysql_errno(mysql); - - if (error == 2013 || error == 2006) { - if (errno != EINTR) { - db_reconnect(mysql, type, error, "db_query"); - - error_count++; - - if (error_count > 30) { - die("FATAL: Too many Reconnect Attempts!"); - } - - continue; - } else { - usleep(50000); - continue; - } - } - - if (error == 1213 || error == 1205) { - usleep(50000); - error_count++; - - if (error_count > 30) { - SPINE_LOG(("FATAL: Too many Lock/Deadlock errors occurred!, SQL Fragment:'%s'", query_frag)); - exit(1); - } - - continue; - } else { - SPINE_LOG(("FATAL: Database Error:'%i', Message:'%s'", error, mysql_error(mysql))); - SPINE_LOG(("ERROR: The Query Was:'%s'", query)); + while (mysql_query(mysql, query) != 0) { + int error = mysql_errno(mysql); + if (error == 2013 || error == 2006) { + retry_disconnected_query(mysql, type, error, "db_query", &error_count); + continue; + } + if (error == 1213 || error == 1205) { + spine_sleep_usec(50000); + if (++error_count > 30) { + SPINE_LOG(("FATAL: Too many Lock/Deadlock errors occurred!, SQL Fragment:'%s'", query_frag)); exit(1); } - } else { - mysql_res = mysql_store_result(mysql); - - break; + continue; } + SPINE_LOG(("FATAL: Database Error:'%i', Message:'%s'", error, mysql_error(mysql))); + SPINE_LOG(("ERROR: The Query Was:'%s'", query)); + exit(1); } - - return mysql_res; + return mysql_store_result(mysql); } /*! \fn void db_connect(char *database, MYSQL *mysql) @@ -222,58 +171,70 @@ MYSQL_RES *db_query(MYSQL *mysql, int type, const char *query) { * fails more than 20 times, the function will fail and Spine will terminate. * */ +void db_address_init(db_address_t *address, const char *value, bool parse_socket) { + address->storage = strdup(value); + if (address->storage == NULL) die("ERROR: Fatal malloc error: database address!"); + address->hostname = address->storage; + address->socket = NULL; + if (!parse_socket) return; + struct stat socket_stat; + if (stat(address->storage, &socket_stat) == 0) { + if (S_ISSOCK(socket_stat.st_mode)) { + address->socket = address->storage; + address->hostname = NULL; + } + return; + } + address->socket = strchr(address->storage, ':'); + if (address->socket != NULL) *address->socket++ = '\0'; +} + +void db_address_release(db_address_t *address) { + free(address->storage); + address->storage = NULL; + address->hostname = NULL; + address->socket = NULL; +} + +void db_set_option(MYSQL *mysql, enum mysql_option option, const void *value, const char *description) { + if (mysql_options(mysql, option, value) != 0) { + set.exit.exit_code = EXIT_FAILURE; + die("FATAL: MySQL options unable to set %s option", description); + } +} + +#ifdef HAS_MYSQL_OPT_SSL_KEY +static void db_set_ssl_options(MYSQL *mysql, int type) { + #ifdef HAS_MYSQL_OPT_SSL_VERIFY_SERVER_CERT + int ssl_enabled = type == LOCAL ? set.database.ssl : set.remote_database.ssl; + if (ssl_enabled == 0) { + bool ssl_enforce = false; + db_set_option(mysql, MYSQL_OPT_SSL_VERIFY_SERVER_CERT, &ssl_enforce, "ssl disable"); + } + #endif + const char *key = type == REMOTE ? set.remote_database.ssl_key : set.database.ssl_key; + const char *ca = type == REMOTE ? set.remote_database.ssl_ca : set.database.ssl_ca; + const char *cert = type == REMOTE ? set.remote_database.ssl_cert : set.database.ssl_cert; + if (key[0] != '\0') db_set_option(mysql, MYSQL_OPT_SSL_KEY, key, "ssl key"); + if (ca[0] != '\0') db_set_option(mysql, MYSQL_OPT_SSL_CA, ca, "ssl ca"); + if (cert[0] != '\0') db_set_option(mysql, MYSQL_OPT_SSL_CERT, cert, "ssl cert"); +} +#endif + void db_connect(int type, MYSQL *mysql) { int tries; int attempts; int timeout; int rtimeout; int wtimeout; - int options_error; int success; int error = 0; - MYSQL *connect_error; - char *hostname = NULL; - char *socket = NULL; - struct stat socket_stat; + const MYSQL *connect_error; + db_address_t address; static int connections = 0; - #ifdef HAS_MYSQL_OPT_SSL_KEY - char *ssl_key = NULL; - char *ssl_ca = NULL; - char *ssl_cert = NULL; - #endif - /* see if the hostname variable is a file reference. If so, - * and if it is a socket file, setup mysql to use it. - */ - if (set.poller_id > 1) { - if (type == LOCAL) { - STRDUP_OR_DIE(hostname, set.db_host, "db_host") - - if (stat(hostname, &socket_stat) == 0) { - if (socket_stat.st_mode & S_IFSOCK) { - socket = strdup (set.db_host); - free(hostname); - hostname = NULL; - } - } else if ((socket = strstr(hostname,":"))) { - *socket++ = 0x0; - } - } else { - STRDUP_OR_DIE(hostname, set.rdb_host, "rdb_host") - } - } else { - STRDUP_OR_DIE(hostname, set.db_host, "db_host") - - if (stat(hostname, &socket_stat) == 0) { - if (socket_stat.st_mode & S_IFSOCK) { - socket = strdup (set.db_host); - free(hostname); - hostname = NULL; - } - } else if ((socket = strstr(hostname,":"))) { - *socket++ = 0x0; - } - } + bool local_address = set.poller.poller_id <= 1 || type == LOCAL; + db_address_init(&address, local_address ? set.database.host : set.remote_database.host, local_address); /* initialalize variables */ tries = 2; @@ -284,69 +245,41 @@ void db_connect(int type, MYSQL *mysql) { attempts = 1; if (mysql_init(mysql) == NULL) { + db_address_release(&address); printf("FATAL: Database unable to allocate memory and therefore can not connect\n"); exit(1); } - MYSQL_SET_OPTION(MYSQL_OPT_READ_TIMEOUT, (int *)&rtimeout, "read timeout"); - MYSQL_SET_OPTION(MYSQL_OPT_WRITE_TIMEOUT, (int *)&wtimeout, "write timeout"); - MYSQL_SET_OPTION(MYSQL_OPT_CONNECT_TIMEOUT, (int *)&timeout, "general timeout"); + db_set_option(mysql, MYSQL_OPT_READ_TIMEOUT, &rtimeout, "read timeout"); + db_set_option(mysql, MYSQL_OPT_WRITE_TIMEOUT, &wtimeout, "write timeout"); + db_set_option(mysql, MYSQL_OPT_CONNECT_TIMEOUT, &timeout, "general timeout"); #ifdef HAS_MYSQL_OPT_RETRY_COUNT - MYSQL_SET_OPTION(MYSQL_OPT_RETRY_COUNT, &tries, "retry count"); + db_set_option(mysql, MYSQL_OPT_RETRY_COUNT, &tries, "retry count"); #endif - /* set SSL options if available */ #ifdef HAS_MYSQL_OPT_SSL_KEY - /* if the users has explicitly said to disable SSL, do that now */ - #ifdef HAS_MYSQL_OPT_SSL_VERIFY_SERVER_CERT - if (type == LOCAL) { - if (set.db_ssl == 0) { - bool ssl_enforce = 0; - MYSQL_SET_OPTION(MYSQL_OPT_SSL_VERIFY_SERVER_CERT, &ssl_enforce, "ssl disable"); - } - } else { - if (set.rdb_ssl == 0) { - bool ssl_enforce = 0; - MYSQL_SET_OPTION(MYSQL_OPT_SSL_VERIFY_SERVER_CERT, &ssl_enforce, "ssl disable"); - } - } - #endif - - if (type == REMOTE) { - STRDUP_OR_DIE(ssl_key, set.rdb_ssl_key, "rdb_ssl_key"); - STRDUP_OR_DIE(ssl_ca, set.rdb_ssl_ca, "rdb_ssl_ca"); - STRDUP_OR_DIE(ssl_cert, set.rdb_ssl_cert, "rdb_ssl_cert"); - } else { - STRDUP_OR_DIE(ssl_key, set.db_ssl_key, "db_ssl_key"); - STRDUP_OR_DIE(ssl_ca, set.db_ssl_ca, "db_ssl_ca"); - STRDUP_OR_DIE(ssl_cert, set.db_ssl_cert, "db_ssl_cert"); - } - - if (strlen(ssl_key)) MYSQL_SET_OPTION(MYSQL_OPT_SSL_KEY, ssl_key, "ssl key"); - if (strlen(ssl_ca)) MYSQL_SET_OPTION(MYSQL_OPT_SSL_CA, ssl_ca, "ssl ca"); - if (strlen(ssl_cert)) MYSQL_SET_OPTION(MYSQL_OPT_SSL_CERT, ssl_cert, "ssl cert"); - + db_set_ssl_options(mysql, type); #endif while (tries > 0) { tries--; - if (set.poller_id > 1) { + if (set.poller.poller_id > 1) { if (type == LOCAL) { - connect_error = mysql_real_connect(mysql, hostname, set.db_user, set.db_pass, set.db_db, set.db_port, socket, 0); + connect_error = mysql_real_connect(mysql, address.hostname, set.database.user, set.database.password, set.database.database, set.database.port, address.socket, 0); } else { - connect_error = mysql_real_connect(mysql, hostname, set.rdb_user, set.rdb_pass, set.rdb_db, set.rdb_port, socket, 0); + connect_error = mysql_real_connect(mysql, address.hostname, set.remote_database.user, set.remote_database.password, set.remote_database.database, set.remote_database.port, address.socket, 0); } } else { - connect_error = mysql_real_connect(mysql, hostname, set.db_user, set.db_pass, set.db_db, set.db_port, socket, 0); + connect_error = mysql_real_connect(mysql, address.hostname, set.database.user, set.database.password, set.database.database, set.database.port, address.socket, 0); } if (!connect_error) { error = mysql_errno(mysql); if ((error == 2002 || error == 2003 || error == 2006 || error == 2013) && errno == EINTR) { - usleep(5000); + spine_sleep_usec(5000); tries++; success = FALSE; } else if (error == 2002) { @@ -356,13 +289,12 @@ void db_connect(int type, MYSQL *mysql) { } else if (error != 1049 && error != 2005 && error != 1045) { printf("Database: Connection Failed: Error:'%d', Message:'%s'\n", error, mysql_error(mysql)); success = FALSE; - usleep(50000); + spine_sleep_usec(50000); } else { tries = 0; success = FALSE; } } else { - tries = 0; success = TRUE; break; } @@ -370,23 +302,9 @@ void db_connect(int type, MYSQL *mysql) { attempts++; } - if (hostname != NULL) { - free(hostname); - } + db_address_release(&address); - #ifdef HAS_MYSQL_OPT_SSL_KEY - if (ssl_key != NULL) { - free(ssl_key); - } - if (ssl_ca != NULL) { - free(ssl_ca); - } - - if (ssl_cert != NULL) { - free(ssl_cert); - } - #endif if (!success){ printf("FATAL: Connection Failed, Error:'%i', Message:'%s'\n", error, mysql_error(mysql)); @@ -419,9 +337,9 @@ void db_create_connection_pool(int type) { int id; if (type == LOCAL) { - SPINE_LOG_DEBUG(("DEBUG: Creating Local Connection Pool of %i threads.", set.threads)); + SPINE_LOG_DEBUG(("DEBUG: Creating Local Connection Pool of %i threads.", set.poller.threads)); - for(id = 0; id < set.threads; id++) { + for(id = 0; id < set.poller.threads; id++) { SPINE_LOG_DEBUG(("DEBUG: Creating Local Connection %i.", id)); db_connect(type, &db_pool_local[id].mysql); @@ -438,9 +356,9 @@ void db_create_connection_pool(int type) { db_pool_local[id].id = id; } } else { - SPINE_LOG_DEBUG(("DEBUG: Creating Remote Connection Pool of %i threads.", set.threads)); + SPINE_LOG_DEBUG(("DEBUG: Creating Remote Connection Pool of %i threads.", set.poller.threads)); - for(id = 0; id < set.threads; id++) { + for(id = 0; id < set.poller.threads; id++) { SPINE_LOG_DEBUG(("DEBUG: Creating Remote Connection %i.", id)); db_connect(type, &db_pool_remote[id].mysql); @@ -468,14 +386,14 @@ void db_close_connection_pool(int type) { int id; if (type == LOCAL) { - for(id = 0; id < set.threads; id++) { + for(id = 0; id < set.poller.threads; id++) { SPINE_LOG_DEBUG(("DEBUG: Closing Local Connection Pool ID %i", id)); db_disconnect(&db_pool_local[id].mysql); } free(db_pool_local); } else { - for(id = 0; id < set.threads; id++) { + for(id = 0; id < set.poller.threads; id++) { SPINE_LOG_DEBUG(("DEBUG: Closing Remote Connection Pool ID %i", id)); db_disconnect(&db_pool_remote[id].mysql); } @@ -496,7 +414,7 @@ pool_t *db_get_connection(int type) { if (type == LOCAL) { SPINE_LOG_DEBUG(("DEBUG: Traversing Local Connection Pool for free connection.")); - for (id = 0; id < set.threads; id++) { + for (id = 0; id < set.poller.threads; id++) { SPINE_LOG_DEBUG(("DEBUG: Checking Local Pool ID %i.", id)); if (db_pool_local[id].free == TRUE) { SPINE_LOG_DEBUG(("DEBUG: Allocating Local Pool ID %i.", id)); @@ -507,7 +425,7 @@ pool_t *db_get_connection(int type) { } } else { SPINE_LOG_DEBUG(("DEBUG: Traversing Remote Connection Pool for free connection.")); - for (id = 0; id < set.threads; id++) { + for (id = 0; id < set.poller.threads; id++) { SPINE_LOG_DEBUG(("DEBUG: Checking Remote Pool ID %i.", id)); if (db_pool_remote[id].free == TRUE) { SPINE_LOG_DEBUG(("DEBUG: Allocating Remote Pool ID %i.", id)); @@ -560,12 +478,12 @@ void db_release_connection(int type, int id) { * \return the number of characters added to the end of the character buffer * */ -int append_hostrange(char *obuf, const char *colname) { - if (HOSTID_DEFINED(set.start_host_id) && HOSTID_DEFINED(set.end_host_id)) { - return snprintf(obuf, BUFSIZE, " AND %s BETWEEN %d AND %d", +int append_hostrange(char *obuf, size_t capacity, const char *colname) { + if (HOSTID_DEFINED(set.hosts.start_host_id) && HOSTID_DEFINED(set.hosts.end_host_id)) { + return spine_snprintf(obuf, capacity, " AND %s BETWEEN %d AND %d", colname, - set.start_host_id, - set.end_host_id); + set.hosts.start_host_id, + set.hosts.end_host_id); } else { return 0; } @@ -584,40 +502,28 @@ int append_hostrange(char *obuf, const char *colname) { * */ void db_escape(MYSQL *mysql, char *output, int max_size, const char *input) { - char *input_trimmed; - size_t max_escaped_input_size; - size_t trim_len; + size_t input_len; + size_t max_input; - if (input == NULL) return; - if (max_size <= 0) return; + if (input == NULL || output == NULL) return; - /* input_trimmed only ever needs to hold what can fit escaped into - * output, so size it to the caller's max_size instead of a fixed - * DBL_BUFSIZE. The previous fixed cap meant a larger max_size never - * actually admitted a longer input, silently truncating it anyway. */ - if (!(input_trimmed = (char *) malloc((size_t) max_size))) { - die("ERROR: Fatal malloc error: sql.c db_escape!"); + if (max_size <= 1) { + if (max_size == 1) { + *output = '\0'; + } + return; } - max_escaped_input_size = (strlen(input) * 2) + 1; - - /* Escaping can double every byte, so input_trimmed may hold at most - * (max_size - 1) / 2 characters and the escaped result still fits within - * max_size, including the terminator. This is always >= 1 for any - * max_size >= 1, so the terminator is always reserved: max_size of 1-3 - * previously drove this to 0 (an unterminated output) or -1 (converted - * to a huge size_t that let snprintf overflow the allocation). */ - trim_len = ((size_t) max_size - 1) / 2 + 1; - - if (max_escaped_input_size > (size_t) max_size) { - snprintf(input_trimmed, trim_len, "%s", input); - } else { - snprintf(input_trimmed, (size_t) max_size, "%s", input); + /* Escaping can double every byte and adds a NUL terminator. Derive the + * input limit from the caller's destination rather than a fixed staging + * buffer, so full RESULTS_BUFFER values survive in a 2N+1 destination. */ + max_input = ((size_t) max_size - 1) / 2; + input_len = strlen(input); + if (input_len > max_input) { + input_len = max_input; } - mysql_real_escape_string(mysql, output, input_trimmed, strlen(input_trimmed)); - - free(input_trimmed); + mysql_real_escape_string(mysql, output, input, (unsigned long) input_len); } void db_free_result(MYSQL_RES *result) { diff --git a/sql.h b/sql.h index de735b3..787a219 100644 --- a/sql.h +++ b/sql.h @@ -33,6 +33,14 @@ #ifndef SPINE_SQL_H #define SPINE_SQL_H +typedef struct db_address { + char *storage; + char *hostname; + char *socket; +} db_address_t; +extern void db_address_init(db_address_t *address, const char *value, bool parse_socket); +extern void db_address_release(db_address_t *address); + extern int db_insert(MYSQL *mysql, int type, const char *query); extern MYSQL_RES *db_query(MYSQL *mysql, int type, const char *query); extern void db_connect(int type, MYSQL *mysql); @@ -46,14 +54,8 @@ extern void db_release_connection(int type, int id); extern int db_reconnect(MYSQL *mysql, int type, int error, const char *location); extern int db_column_exists(MYSQL *mysql, int type, const char *table, const char *column); -extern int append_hostrange(char *obuf, const char *colname); +extern int append_hostrange(char *obuf, size_t capacity, const char *colname); -#define MYSQL_SET_OPTION(opt, value, desc) \ -{\ - options_error = mysql_options(mysql, opt, value); \ - if (options_error < 0) {\ - die("FATAL: MySQL options unable to set %s option", desc);\ - }\ -} +extern void db_set_option(MYSQL *mysql, enum mysql_option option, const void *value, const char *description); #endif /* SPINE_SQL_H */ diff --git a/src/core/locks.c b/src/core/locks.c index 1b4846b..da83317 100644 --- a/src/core/locks.c +++ b/src/core/locks.c @@ -72,32 +72,32 @@ DEFINE_SPINE_LOCK(php_proc_14) DEFINE_SPINE_LOCK(thdet) DEFINE_SPINE_LOCK(host_time) -void init_mutexes(void) { - pthread_once((pthread_once_t*) get_attr(LOCK_SNMP_O), init_snmp_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_SETEUID_O), init_seteuid_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_GHBN_O), init_ghbn_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_POOL_O), init_pool_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_O), init_php_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_0_O), init_php_proc_0_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_1_O), init_php_proc_1_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_2_O), init_php_proc_2_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_3_O), init_php_proc_3_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_4_O), init_php_proc_4_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_5_O), init_php_proc_5_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_6_O), init_php_proc_6_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_7_O), init_php_proc_7_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_8_O), init_php_proc_8_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_9_O), init_php_proc_9_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_10_O), init_php_proc_10_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_11_O), init_php_proc_11_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_12_O), init_php_proc_12_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_13_O), init_php_proc_13_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_PHP_PROC_14_O), init_php_proc_14_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_THDET_O), init_thdet_lock); - pthread_once((pthread_once_t*) get_attr(LOCK_HOST_TIME_O), init_host_time_lock); +void init_mutexes() { + pthread_once(get_attr(LOCK_SNMP_O), init_snmp_lock); + pthread_once(get_attr(LOCK_SETEUID_O), init_seteuid_lock); + pthread_once(get_attr(LOCK_GHBN_O), init_ghbn_lock); + pthread_once(get_attr(LOCK_POOL_O), init_pool_lock); + pthread_once(get_attr(LOCK_PHP_O), init_php_lock); + pthread_once(get_attr(LOCK_PHP_PROC_0_O), init_php_proc_0_lock); + pthread_once(get_attr(LOCK_PHP_PROC_1_O), init_php_proc_1_lock); + pthread_once(get_attr(LOCK_PHP_PROC_2_O), init_php_proc_2_lock); + pthread_once(get_attr(LOCK_PHP_PROC_3_O), init_php_proc_3_lock); + pthread_once(get_attr(LOCK_PHP_PROC_4_O), init_php_proc_4_lock); + pthread_once(get_attr(LOCK_PHP_PROC_5_O), init_php_proc_5_lock); + pthread_once(get_attr(LOCK_PHP_PROC_6_O), init_php_proc_6_lock); + pthread_once(get_attr(LOCK_PHP_PROC_7_O), init_php_proc_7_lock); + pthread_once(get_attr(LOCK_PHP_PROC_8_O), init_php_proc_8_lock); + pthread_once(get_attr(LOCK_PHP_PROC_9_O), init_php_proc_9_lock); + pthread_once(get_attr(LOCK_PHP_PROC_10_O), init_php_proc_10_lock); + pthread_once(get_attr(LOCK_PHP_PROC_11_O), init_php_proc_11_lock); + pthread_once(get_attr(LOCK_PHP_PROC_12_O), init_php_proc_12_lock); + pthread_once(get_attr(LOCK_PHP_PROC_13_O), init_php_proc_13_lock); + pthread_once(get_attr(LOCK_PHP_PROC_14_O), init_php_proc_14_lock); + pthread_once(get_attr(LOCK_THDET_O), init_thdet_lock); + pthread_once(get_attr(LOCK_HOST_TIME_O), init_host_time_lock); } -static const char *get_name(int lock) { +const char *get_name(int lock) { switch (lock) { case LOCK_SNMP: return "snmp"; case LOCK_SETEUID: return "seteuid"; @@ -121,9 +121,9 @@ static const char *get_name(int lock) { case LOCK_PHP_PROC_14: return "php_proc_14"; case LOCK_THDET: return "thdet"; case LOCK_HOST_TIME: return "host_time"; + default: return "Unknown lock"; } - return "Unknown lock"; } pthread_cond_t* get_cond(int lock) { @@ -152,6 +152,7 @@ pthread_cond_t* get_cond(int lock) { case LOCK_PHP_PROC_14: ret_val = &php_proc_14_cond; break; case LOCK_THDET: ret_val = &thdet_cond; break; case LOCK_HOST_TIME: ret_val = &host_time_cond; break; + default: break; } SPINE_LOG_DEVDBG(("LOCKS: [RET] Returning cond for %s", get_name(lock))); @@ -185,6 +186,7 @@ pthread_mutex_t* get_lock(int lock) { case LOCK_PHP_PROC_14: ret_val = &php_proc_14_lock; break; case LOCK_THDET: ret_val = &thdet_lock; break; case LOCK_HOST_TIME: ret_val = &host_time_lock; break; + default: break; } SPINE_LOG_DEVDBG(("LOCKS: [RET] Returning lock for %s", get_name(lock))); @@ -218,6 +220,7 @@ pthread_once_t* get_attr(int locko) { case LOCK_PHP_PROC_14_O: ret_val = &php_proc_14_lock_o; break; case LOCK_THDET_O: ret_val = &thdet_lock_o; break; case LOCK_HOST_TIME_O: ret_val = &host_time_lock_o; break; + default: break; } SPINE_LOG_DEVDBG(("LOCKS: [RET] Returning attr for %s", get_name(locko))); diff --git a/src/core/locks.h b/src/core/locks.h index a5a8e82..c4e4fa5 100644 --- a/src/core/locks.h +++ b/src/core/locks.h @@ -33,6 +33,7 @@ #ifndef SPINE_LOCKS_H #define SPINE_LOCKS_H +extern const char* get_name(int lock); extern void init_mutexes(void); extern void thread_mutex_lock(int mutex); extern void thread_mutex_unlock(int mutex); diff --git a/tests/fixtures/configuration.sql b/tests/fixtures/configuration.sql new file mode 100644 index 0000000..06e8520 --- /dev/null +++ b/tests/fixtures/configuration.sql @@ -0,0 +1,111 @@ +/* + +-------------------------------------------------------------------------+ + | Copyright (C) 2004-2026 The Cacti Group | + | | + | This program is free software; you can redistribute it and/or | + | modify it under the terms of the GNU General Public License | + | as published by the Free Software Foundation; either version 2 | + | of the License, or (at your option) any later version. | + | | + | This program is distributed in the hope that it will be useful, | + | but WITHOUT ANY WARRANTY; without even the implied warranty of | + | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | + | GNU General Public License for more details. | + +-------------------------------------------------------------------------+ + | Cacti: The Complete RRDTool-based Graphing Solution | + +-------------------------------------------------------------------------+ + | This code is designed, written, and maintained by the Cacti Group. See | + | about.php and/or the AUTHORS file for specific developer information. | + +-------------------------------------------------------------------------+ + | http://www.cacti.net/ | + +-------------------------------------------------------------------------+ +*/ + +-- Schema excerpt from Cacti cacti.sql; isolated regression fixtures only. +CREATE TABLE settings ( + name varchar(255) NOT NULL default '', + value varchar(4096) NOT NULL default '', + PRIMARY KEY (name) +) ENGINE=InnoDB ROW_FORMAT=Dynamic; + +CREATE TABLE `poller` ( + `id` smallint(5) unsigned NOT NULL AUTO_INCREMENT, + `disabled` char(2) DEFAULT '', + `name` varchar(30) DEFAULT NULL, + `notes` varchar(1024) DEFAULT '', + `status` int(10) unsigned NOT NULL DEFAULT '0', + `timezone` varchar(40) DEFAULT '', + `hostname` varchar(100) NOT NULL DEFAULT '', + `dbdefault` varchar(20) NOT NULL DEFAULT '', + `dbhost` varchar(64) NOT NULL DEFAULT '', + `dbuser` varchar(20) NOT NULL DEFAULT '', + `dbpass` varchar(64) NOT NULL DEFAULT '', + `dbport` int(10) unsigned DEFAULT '3306', + `dbretries` int(10) unsigned DEFAULT '2', + `dbssl` char(3) DEFAULT '', + `dbsslkey` varchar(255) DEFAULT NULL, + `dbsslcert` varchar(255) DEFAULT NULL, + `dbsslca` varchar(255) DEFAULT NULL, + `total_time` double DEFAULT '0', + `max_time` double DEFAULT NULL, + `min_time` double DEFAULT NULL, + `avg_time` double DEFAULT NULL, + `total_polls` int(10) unsigned DEFAULT '0', + `processes` int(10) unsigned DEFAULT '1', + `threads` int(10) unsigned DEFAULT '1', + `sync_interval` int(10) unsigned DEFAULT '7200', + `snmp` mediumint(8) unsigned DEFAULT '0', + `script` mediumint(8) unsigned DEFAULT '0', + `server` mediumint(8) unsigned DEFAULT '0', + `last_update` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00', + `last_status` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00', + `last_sync` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00', + `requires_sync` char(2) DEFAULT '', + PRIMARY KEY (`id`), + KEY `name` (`name`), + KEY `disabled` (`disabled`) +) ENGINE=InnoDB ROW_FORMAT=Dynamic COMMENT='Pollers for Cacti'; + +CREATE TABLE poller_item ( + `local_data_id` int(10) unsigned NOT NULL default '0', + `poller_id` int(10) unsigned NOT NULL default '1', + `host_id` mediumint(8) unsigned NOT NULL default '0', + `action` tinyint(3) unsigned NOT NULL default '1', + `present` tinyint(3) unsigned NOT NULL DEFAULT '1', + `last_updated` timestamp DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + `hostname` varchar(100) NOT NULL default '', + `snmp_community` varchar(100) NOT NULL default '', + `snmp_version` tinyint(3) unsigned NOT NULL default '0', + `snmp_username` varchar(50) NOT NULL default '', + `snmp_password` varchar(50) NOT NULL default '', + `snmp_auth_protocol` char(6) NOT NULL default '', + `snmp_priv_passphrase` varchar(200) NOT NULL default '', + `snmp_priv_protocol` char(7) NOT NULL default '', + `snmp_context` varchar(64) default '', + `snmp_engine_id` varchar(64) default '', + `snmp_port` mediumint(8) unsigned NOT NULL default '161', + `snmp_timeout` mediumint(8) unsigned NOT NULL default '0', + `rrd_name` varchar(19) NOT NULL default '', + `rrd_path` varchar(255) NOT NULL default '', + `rrd_num` tinyint(3) unsigned NOT NULL default '0', + `rrd_step` mediumint(8) NOT NULL default '300', + `rrd_next_step` mediumint(8) NOT NULL default '0', + `arg1` TEXT, + `arg2` varchar(255) default NULL, + `arg3` varchar(255) default NULL, + PRIMARY KEY (`local_data_id`,`rrd_name`), + KEY `host_id` (`host_id`), + KEY `action` (`action`), + KEY `present` (`present`), + KEY `poller_id_host_id` (`poller_id`,`host_id`), + KEY `poller_id_rrd_next_step` (`poller_id`,`rrd_next_step`), + KEY `poller_id_action` (`poller_id`,`action`), + KEY `poller_id_last_updated` (`poller_id`, `last_updated`) +) ENGINE=InnoDB ROW_FORMAT=Dynamic; + +CREATE TABLE version ( + cacti char(20) default '', + PRIMARY KEY (cacti) +) ENGINE=InnoDB ROW_FORMAT=Dynamic; +INSERT INTO version VALUES ('1.2.32'); +INSERT INTO poller (id,threads) VALUES (1,7); diff --git a/tests/fixtures/poller.sql b/tests/fixtures/poller.sql new file mode 100644 index 0000000..c165bc7 --- /dev/null +++ b/tests/fixtures/poller.sql @@ -0,0 +1,153 @@ +/* + +-------------------------------------------------------------------------+ + | Copyright (C) 2004-2026 The Cacti Group | + | | + | This program is free software; you can redistribute it and/or | + | modify it under the terms of the GNU General Public License | + | as published by the Free Software Foundation; either version 2 | + | of the License, or (at your option) any later version. | + | | + | This program is distributed in the hope that it will be useful, | + | but WITHOUT ANY WARRANTY; without even the implied warranty of | + | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the | + | GNU General Public License for more details. | + +-------------------------------------------------------------------------+ + | Cacti: The Complete RRDTool-based Graphing Solution | + +-------------------------------------------------------------------------+ + | This code is designed, written, and maintained by the Cacti Group. See | + | about.php and/or the AUTHORS file for specific developer information. | + +-------------------------------------------------------------------------+ + | http://www.cacti.net/ | + +-------------------------------------------------------------------------+ +*/ + +-- Schema excerpt from Cacti develop cacti.sql; isolated poller regressions. +CREATE TABLE `host` ( + id mediumint(8) unsigned NOT NULL auto_increment, + poller_id int(10) unsigned NOT NULL default '1', + site_id int(10) unsigned NOT NULL default '1', + host_template_id mediumint(8) unsigned NOT NULL default '0', + description varchar(150) NOT NULL default '', + hostname varchar(100) default NULL, + location varchar(40) default NULL, + notes text, + external_id varchar(40) default NULL, + snmp_options tinyint(3) unsigned NOT NULL default '0', + snmp_community varchar(100) default NULL, + snmp_version tinyint(3) unsigned NOT NULL default '1', + snmp_username varchar(50) default NULL, + snmp_password varchar(50) default NULL, + snmp_auth_protocol char(6) default '', + snmp_priv_passphrase varchar(200) default '', + snmp_priv_protocol char(7) default '', + snmp_context varchar(64) default '', + snmp_engine_id varchar(64) default '', + snmp_port mediumint(8) unsigned NOT NULL default '161', + snmp_timeout mediumint(8) unsigned NOT NULL default '500', + snmp_retries tinyint(3) unsigned NOT NULL default '3', + snmp_sysDescr varchar(300) NOT NULL default '', + snmp_sysObjectID varchar(128) NOT NULL default '', + snmp_sysUpTimeInstance bigint(20) unsigned NOT NULL default '0', + snmp_sysContact varchar(300) NOT NULL default '', + snmp_sysName varchar(300) NOT NULL default '', + snmp_sysLocation varchar(300) NOT NULL default '', + availability_method smallint(5) unsigned NOT NULL default '1', + ping_method smallint(5) unsigned default '0', + ping_port int(10) unsigned default '0', + ping_timeout int(10) unsigned default '500', + ping_retries int(10) unsigned default '2', + max_oids int(10) unsigned default '10', + bulk_walk_size int(11) default '-1', + device_threads tinyint(3) unsigned NOT NULL default '1', + deleted char(2) NOT NULL default '', + disabled char(2) NOT NULL default '', + graphs int(10) unsigned NOT NULL default '0', + data_sources int(10) unsigned NOT NULL default '0', + status tinyint(3) unsigned NOT NULL default '0', + status_event_count mediumint(8) unsigned NOT NULL default '0', + status_fail_date timestamp NOT NULL default '0000-00-00 00:00:00', + status_rec_date timestamp NOT NULL default '0000-00-00 00:00:00', + status_options_date timestamp NOT NULL default '0000-00-00 00:00:00', + status_last_error varchar(255) default '', + min_time decimal(10,5) default '9.99999', + max_time decimal(10,5) default '0.00000', + cur_time decimal(10,5) default '0.00000', + avg_time decimal(10,5) default '0.00000', + polling_time DOUBLE default '0', + current_errors int(10) unsigned NOT NULL default '0', + total_polls int(10) unsigned default '0', + failed_polls int(10) unsigned default '0', + availability decimal(8,5) NOT NULL default '100.00000', + last_updated timestamp default CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + created timestamp default CURRENT_TIMESTAMP, + PRIMARY KEY (id), + KEY poller_id_disabled (poller_id, disabled), + KEY host_template_id (host_template_id), + KEY external_id (external_id), + KEY disabled (disabled), + KEY status (status), + KEY current_errors (current_errors), + KEY site_id_location (site_id, location), + KEY hostname (hostname), + KEY poller_id_last_updated (poller_id, last_updated) +) ENGINE=InnoDB ROW_FORMAT=Dynamic; + +CREATE TABLE `poller_reindex` ( + host_id mediumint(8) unsigned NOT NULL default '0', + data_query_id mediumint(8) unsigned NOT NULL default '0', + action tinyint(3) unsigned NOT NULL default '0', + present tinyint(3) unsigned NOT NULL default '1', + op char(1) NOT NULL default '', + assert_value varchar(100) NOT NULL default '', + arg1 varchar(255) NOT NULL default '', + PRIMARY KEY (host_id, data_query_id, arg1(187)), + KEY present (present) +) ENGINE=InnoDB ROW_FORMAT=Dynamic; + +CREATE TABLE `poller_output` ( + local_data_id int(10) unsigned NOT NULL default '0', + rrd_name varchar(19) NOT NULL default '', + time timestamp NOT NULL default '0000-00-00 00:00:00', + output varchar(512) NOT NULL default '', + PRIMARY KEY (local_data_id, rrd_name, time) /*!50060 USING BTREE */ +) ENGINE=MEMORY; + +CREATE TABLE `poller_output_boost` ( + `local_data_id` int(10) unsigned NOT NULL default '0', + `rrd_name` varchar(19) NOT NULL default '', + `time` timestamp NOT NULL default '0000-00-00 00:00:00', + `output` varchar(512) NOT NULL, + `last_updated` timestamp NOT NULL default current_timestamp, + PRIMARY KEY USING BTREE (`local_data_id`, `time`, `rrd_name`), + KEY `last_updated` (`last_updated`), + KEY `time` (`time`) +) ENGINE=InnoDB ROW_FORMAT=Dynamic; + +CREATE TABLE `host_errors` ( + `host_id` mediumint(8) unsigned NOT NULL default 0, + `poller_id` int(10) unsigned NOT NULL default 1, + `errors` mediumint(8) unsigned NOT NULL default 0, + `local_data_ids` text default NULL, + PRIMARY KEY (`host_id`), + KEY `poller_id` (`poller_id`) +) ENGINE=InnoDB ROW_FORMAT=DYNAMIC COMMENT='Holds Device Error buffer for Spine'; + +CREATE TABLE `poller_command` ( + poller_id smallint(5) unsigned NOT NULL default '1', + time timestamp NOT NULL default '0000-00-00 00:00:00', + action tinyint(3) unsigned NOT NULL default '0', + command varchar(191) NOT NULL default '', + last_updated timestamp default CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + PRIMARY KEY (poller_id, action, command), + KEY poller_id_last_updated (poller_id, last_updated) +) ENGINE=InnoDB ROW_FORMAT=Dynamic; + +CREATE TABLE `poller_time` ( + id bigint(20) unsigned NOT NULL auto_increment, + pid int(10) unsigned NOT NULL default '0', + poller_id int(10) unsigned NOT NULL default '1', + start_time timestamp NOT NULL default '0000-00-00 00:00:00', + end_time timestamp NOT NULL default '0000-00-00 00:00:00', + PRIMARY KEY (id), + KEY `poller_id_end_time` (`poller_id`, `end_time`) +) ENGINE=InnoDB ROW_FORMAT=Dynamic; diff --git a/tests/fixtures/snmpd.conf b/tests/fixtures/snmpd.conf new file mode 100644 index 0000000..c308821 --- /dev/null +++ b/tests/fixtures/snmpd.conf @@ -0,0 +1,7 @@ +# Copyright (C) 2026 The Cacti Group +# Licensed under the GNU Lesser General Public License, version 2.1 or later. +# Isolated regression agent; never use this configuration in production. +agentaddress udp:127.0.0.1:1161 +rocommunity regression 127.0.0.1 +sysLocation isolated-regression-agent +sysContact regression diff --git a/tests/fixtures/snmpv3-agent.conf b/tests/fixtures/snmpv3-agent.conf new file mode 100644 index 0000000..95b6be4 --- /dev/null +++ b/tests/fixtures/snmpv3-agent.conf @@ -0,0 +1,9 @@ +# Copyright (C) 2026 The Cacti Group +# Licensed under the GNU Lesser General Public License, version 2.1 or later. +# Isolated test agent. These usernames and passphrases are public test fixtures. +agentaddress udp:127.0.0.1:1162 +rouser regression-v3-noauth noauth +rouser regression-v3-auth auth +rouser regression-v3-priv priv +sysLocation isolated-regression-v3-agent +sysContact regression diff --git a/tests/fixtures/snmpv3-users.conf b/tests/fixtures/snmpv3-users.conf new file mode 100644 index 0000000..b1d876f --- /dev/null +++ b/tests/fixtures/snmpv3-users.conf @@ -0,0 +1,7 @@ +# Copyright (C) 2026 The Cacti Group +# Licensed under the GNU Lesser General Public License, version 2.1 or later. +# Copy into an owned temporary persistent snmpd.conf; never a system directory. +# The authentication passphrase is the public RFC3414 Appendix A.3.2 vector. +createUser regression-v3-noauth +createUser regression-v3-auth SHA maplesyrup +createUser regression-v3-priv SHA maplesyrup AES regression-privacy diff --git a/tests/fuzz/stubs.c b/tests/fuzz/stubs.c index 6e62ecf..065e9d9 100644 --- a/tests/fuzz/stubs.c +++ b/tests/fuzz/stubs.c @@ -8,8 +8,8 @@ #include "spine.h" #include "php.h" -spine_sem_t available_threads; -spine_sem_t available_scripts; +spine_permits_t available_threads; +spine_permits_t available_scripts; double start_time; double total_time; config_t set; @@ -18,4 +18,4 @@ int *debug_devices; pool_t *db_pool_local; pool_t *db_pool_remote; php_t *php_processes; -poller_thread_t *details; +poller_thread_t **details; diff --git a/tests/regression/test_remote_push.sh b/tests/regression/test_remote_push.sh index e748510..8dcb413 100755 --- a/tests/regression/test_remote_push.sh +++ b/tests/regression/test_remote_push.sh @@ -10,26 +10,27 @@ fail() { exit 1 } -body=$(awk '/^void poller_push_data_to_main\(void\) \{/{f=1} f{print} f&&/^\}/{exit}' util.c) -poll_host_body=$(awk '/^void poll_host\(/{f=1} f{print} f&&/^\}/{exit}' poller.c) +queries_body=$(awk '/^static void transfer_queries\(/{f=1} f{print} f&&/^\}/{exit}' util.c) +table_body=$(awk '/^static bool transfer_table\(/{f=1} f{print} f&&/^\}/{exit}' util.c) +prepare_body=$(awk '/^void poller_prepare_queries\(/{f=1} f{print} f&&/^\}/{exit}' poller.c) -[ -n "$body" ] || fail "could not find poller_push_data_to_main() in util.c" -[ -n "$poll_host_body" ] || fail "could not find poll_host() in poller.c" +[ -n "$queries_body" ] || fail "could not find transfer_queries() in util.c" +[ -n "$table_body" ] || fail "could not find transfer_table() in util.c" +[ -n "$prepare_body" ] || fail "could not find poller_prepare_queries() in poller.c" -printf '%s\n' "$body" | grep -q 'AS rs ON DUPLICATE KEY UPDATE' && +printf '%s\n' "$queries_body" | grep -q 'AS rs' && fail "remote pushes must not use row-alias syntax selected from the local server" -printf '%s\n' "$body" | grep -q 'if (set.dbonupdate' && +printf '%s\n' "$queries_body" | grep -q 'onupdate' && fail "remote pushes must not branch on the local server version" -awk '/^static void push_flush_batch/{f=1} f{print} f&&/^\}/{exit}' util.c | - grep -q 'if (!spine_appendf' || - fail "overflowed remote push batches must not be sent" +printf '%s\n' "$table_body" | grep -q 'length + suffix_length + 3 > HUGE_BUFSIZE - used' || + fail "remote push batches must flush before a row that would overflow them" -printf '%s\n' "$poll_host_body" | grep -q 'if (set\.dbonupdate' && +printf '%s\n' "$prepare_body" | grep -q 'onupdate' && fail "poller_output must not select SQL syntax from the local server version" -printf '%s\n' "$poll_host_body" | grep -q 'output=VALUES(output)' || +printf '%s\n' "$prepare_body" | grep -q 'output=VALUES(output)' || fail "poller_output must use the cross-vendor upsert form" exit 0 diff --git a/tests/snmpv3/Dockerfile.coverage b/tests/snmpv3/Dockerfile.coverage index 87689a4..369557a 100644 --- a/tests/snmpv3/Dockerfile.coverage +++ b/tests/snmpv3/Dockerfile.coverage @@ -1,6 +1,6 @@ # Single stage: the .gcda files land beside the objects, so /src must survive # into the running container. -FROM debian:bookworm-slim +FROM mirror.gcr.io/library/debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ gcc make autoconf automake libtool pkg-config \ libmariadb-dev libsnmp-dev libssl-dev gcovr \ diff --git a/tests/snmpv3/docker-compose.mysql.yml b/tests/snmpv3/docker-compose.mysql.yml index 1c34e1a..52d3724 100644 --- a/tests/snmpv3/docker-compose.mysql.yml +++ b/tests/snmpv3/docker-compose.mysql.yml @@ -1,6 +1,6 @@ services: db: - image: mysql:8.4 + image: mirror.gcr.io/library/mysql:8.4 # The Cacti 1.x schema intentionally uses zero-date sentinel values. command: ["--sql-mode=NO_ENGINE_SUBSTITUTION"] environment: @@ -15,6 +15,6 @@ services: retries: 20 verify: - image: mysql:8.4 + image: mirror.gcr.io/library/mysql:8.4 environment: DB_CLIENT: mysql diff --git a/tests/snmpv3/docker-compose.yml b/tests/snmpv3/docker-compose.yml index 6d4f18e..8d9e737 100644 --- a/tests/snmpv3/docker-compose.yml +++ b/tests/snmpv3/docker-compose.yml @@ -1,7 +1,7 @@ services: db: - image: mariadb:11 + image: mirror.gcr.io/library/mariadb:11 environment: MARIADB_ROOT_PASSWORD: root MARIADB_DATABASE: cacti @@ -51,7 +51,7 @@ services: # Asserts on the database spine wrote to. Without this the fixture only # proves spine exited 0, which a broken poll would also do. verify: - image: mariadb:11 + image: mirror.gcr.io/library/mariadb:11 depends_on: db: condition: service_healthy diff --git a/tests/snmpv3/snmpd/Dockerfile b/tests/snmpv3/snmpd/Dockerfile index 1975725..7d8b9e8 100644 --- a/tests/snmpv3/snmpd/Dockerfile +++ b/tests/snmpv3/snmpd/Dockerfile @@ -1,4 +1,4 @@ -FROM debian:bookworm-slim +FROM mirror.gcr.io/library/debian:bookworm-slim RUN apt-get update && apt-get install -y --no-install-recommends \ iproute2 \ diff --git a/tests/test_spine_cli_contracts.c b/tests/test_spine_cli_contracts.c new file mode 100644 index 0000000..bda90f6 --- /dev/null +++ b/tests/test_spine_cli_contracts.c @@ -0,0 +1,109 @@ +/* + * Copyright (C) 2004-2026 The Cacti Group + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License + * for more details. + */ +#include "common.h" +#include "spine.h" + +/* Exercise the built command-line entry point, without reaching a DB. + * An explicit absent config keeps a misparsed --version from polling. */ +static void run_cli_contract(char *const arguments[], int expected, char *output, size_t capacity) { + int descriptors[2]; + assert(pipe(descriptors) == 0); + fflush(NULL); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + alarm(15); + assert(close(descriptors[0]) == 0); + assert(dup2(descriptors[1], STDOUT_FILENO) == STDOUT_FILENO); + assert(dup2(descriptors[1], STDERR_FILENO) == STDERR_FILENO); + assert(close(descriptors[1]) == 0); + execv("./spine", arguments); + _exit(127); + } + assert(close(descriptors[1]) == 0); + size_t used = 0; + for (;;) { + assert(used < capacity - 1); + ssize_t received = read(descriptors[0], output + used, capacity - used - 1); + if (received < 0 && errno == EINTR) continue; + assert(received >= 0); + if (received == 0) break; + used += (size_t)received; + } + output[used] = '\0'; + assert(close(descriptors[0]) == 0); + int status; + while (waitpid(child, &status, 0) < 0) assert(errno == EINTR); + assert(WIFEXITED(status) && WEXITSTATUS(status) == expected); +} + +void test_cli_alias_contracts(void) { + char output[8192]; + char version[8192]; + char help[8192]; + char *version_args[] = {"spine", "--version", NULL}; + run_cli_contract(version_args, EXIT_SUCCESS, version, sizeof(version)); + assert(strstr(version, "SPINE ") != NULL && strstr(version, "Usage:") == NULL); + char *help_args[] = {"spine", "--help", NULL}; + run_cli_contract(help_args, EXIT_SUCCESS, help, sizeof(help)); + assert(strstr(help, "-h/--help") != NULL && strstr(help, "-H/--hostlist") != NULL); + assert(strstr(help, "-p/--poller") != NULL && strstr(help, "-P/--pingonly") != NULL); + char *short_help[] = {"spine", "-h", NULL}; + run_cli_contract(short_help, EXIT_SUCCESS, output, sizeof(output)); + assert(strcmp(help, output) == 0); + char *value_forms[][6] = { + {"spine", "-p", "17", "--version", NULL}, + {"spine", "-p=17", "--version", NULL}, + {"spine", "--poller=17", "--version", NULL}, + {"spine", "-H", "17,18", "--version", NULL}, + {"spine", "-H=17,18", "--version", NULL}, + {"spine", "--hostlist=17,18", "--version", NULL}, + {"spine", "-H", " 0 , 0017, 18 ", "--version", NULL}, + {"spine", "-H", "2147483647", "--version", NULL}, + {"spine", "-H", "17,17", "--version", NULL}, + }; + for (size_t index = 0; index < sizeof(value_forms) / sizeof(value_forms[0]); index++) { + run_cli_contract(value_forms[index], EXIT_SUCCESS, output, sizeof(output)); + assert(strcmp(version, output) == 0); + } + char *ping_forms[][6] = { + {"spine", "-C", "/nonexistent/spine-cli-contract.conf", "-P", "--version", NULL}, + {"spine", "-C", "/nonexistent/spine-cli-contract.conf", "--pingonly", "--version", NULL}, + }; + for (size_t index = 0; index < sizeof(ping_forms) / sizeof(ping_forms[0]); index++) { + run_cli_contract(ping_forms[index], EXIT_SUCCESS, output, sizeof(output)); + assert(strcmp(version, output) == 0); + } + char *missing_poller[] = {"spine", "-p", NULL}; + run_cli_contract(missing_poller, EXIT_FAILURE, output, sizeof(output)); + assert(strstr(output, "ERROR: option -p requires a parameter") != NULL); + char *missing_hostlist[] = {"spine", "-H", NULL}; + run_cli_contract(missing_hostlist, EXIT_FAILURE, output, sizeof(output)); + assert(strstr(output, "ERROR: option -H requires a parameter") != NULL); + char invalid_lists[][48] = {"", " ", ",17", "17,", "17,,18", "17 18", "-1", "+1", "1.5", "1e2", "17x", "17;18", "2147483648", "999999999999999999999999999999999999"}; + for (size_t index = 0; index < sizeof(invalid_lists) / sizeof(invalid_lists[0]); index++) { + char *arguments[] = {"spine", "-H", invalid_lists[index], "--version", NULL}; + run_cli_contract(arguments, EXIT_FAILURE, output, sizeof(output)); + assert(strstr(output, "ERROR: invalid or oversized host list") != NULL); + } + char *oversized = malloc(BIG_BUFSIZE + 1); + assert(oversized != NULL); + memset(oversized, '0', BIG_BUFSIZE); + oversized[BIG_BUFSIZE] = '\0'; + char *oversized_args[] = {"spine", "-H", oversized, "--version", NULL}; + run_cli_contract(oversized_args, EXIT_FAILURE, output, sizeof(output)); + assert(strstr(output, "ERROR: invalid or oversized host list") != NULL); + free(oversized); + puts("production CLI alias regressions passed"); +} diff --git a/tests/test_spine_contracts.c b/tests/test_spine_contracts.c new file mode 100644 index 0000000..32ecd65 --- /dev/null +++ b/tests/test_spine_contracts.c @@ -0,0 +1,305 @@ +/* + * Copyright (C) 2004-2026 The Cacti Group + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License + * for more details. + */ +#include "common.h" +#include "spine.h" +#include + +static void test_keyword_roundtrips(void) { + typedef struct { const char *word; int value; } keyword_case_t; + static const keyword_case_t levels[] = { + {"NONE", POLLER_VERBOSITY_NONE}, {"LOW", POLLER_VERBOSITY_LOW}, + {"MEDIUM", POLLER_VERBOSITY_MEDIUM}, {"HIGH", POLLER_VERBOSITY_HIGH}, + {"DEBUG", POLLER_VERBOSITY_DEBUG} + }; + static const keyword_case_t destinations[] = { + {"FILE", LOGDEST_FILE}, {"SYSLOG", LOGDEST_SYSLOG}, + {"BOTH", LOGDEST_BOTH}, {"STDOUT", LOGDEST_STDOUT} + }; + static const keyword_case_t actions[] = { + {"SNMP", POLLER_ACTION_SNMP}, {"SCRIPT", POLLER_ACTION_SCRIPT}, + {"PHPSCRIPT", POLLER_ACTION_PHP_SCRIPT_SERVER}, + {"SNMP_CT", POLLER_ACTION_SNMP_COUNT}, {"SCRIPT_CT", POLLER_ACTION_SCRIPT_COUNT}, + {"PHPSCRIPT_CT", POLLER_ACTION_PHP_SCRIPT_SERVER_COUNT} + }; + for (size_t i = 0; i < sizeof(levels)/sizeof(levels[0]); i++) { + assert(parse_log_level(levels[i].word, -1) == levels[i].value); + assert(strcmp(printable_log_level(levels[i].value), levels[i].word) == 0); + } + for (size_t i = 0; i < sizeof(destinations)/sizeof(destinations[0]); i++) { + assert(parse_logdest(destinations[i].word, -1) == destinations[i].value); + assert(strcmp(printable_logdest(destinations[i].value), destinations[i].word) == 0); + } + for (size_t i = 0; i < sizeof(actions)/sizeof(actions[0]); i++) { + assert(parse_action(actions[i].word, -1) == actions[i].value); + assert(strcmp(printable_action(actions[i].value), actions[i].word) == 0); + } + assert(parse_log_level("dEbUg", -1) == POLLER_VERBOSITY_DEBUG); + assert(parse_logdest("sYsLoG", -1) == LOGDEST_SYSLOG); + assert(parse_action("pHpScRiPt_Ct", -1) == POLLER_ACTION_PHP_SCRIPT_SERVER_COUNT); + assert(parse_log_level("42", -1) == 42); + assert(parse_logdest("007", -1) == 7); + assert(parse_action("0", -1) == 0); + assert(parse_log_level("", 97) == 97); + assert(parse_log_level("unknown", 97) == 97); + assert(parse_logdest("-1", 98) == 98); + assert(parse_action(" 2", 99) == 99); + assert(parse_action("2x", 99) == 99); + assert(strcmp(printable_log_level(-1), "-unknown-") == 0); + assert(strcmp(printable_logdest(-1), "-unknown-") == 0); + assert(strcmp(printable_action(-1), "-unknown-") == 0); +} + +static void *try_busy_lock(void *argument) { + int lock = *(int *)argument; + int result = thread_mutex_trylock(lock); + if (result == 0) thread_mutex_unlock(lock); + return (void *)(intptr_t)result; +} + +static void test_lock_contracts(void) { + static const int locks[] = {LOCK_SNMP, LOCK_SETEUID, LOCK_GHBN, LOCK_POOL, LOCK_PHP, + LOCK_PHP_PROC_0, LOCK_PHP_PROC_1, LOCK_PHP_PROC_2, LOCK_PHP_PROC_3, + LOCK_PHP_PROC_4, LOCK_PHP_PROC_5, LOCK_PHP_PROC_6, LOCK_PHP_PROC_7, + LOCK_PHP_PROC_8, LOCK_PHP_PROC_9, LOCK_PHP_PROC_10, LOCK_PHP_PROC_11, + LOCK_PHP_PROC_12, LOCK_PHP_PROC_13, LOCK_PHP_PROC_14, LOCK_THDET, LOCK_HOST_TIME}; + static const char *const names[] = {"snmp", "seteuid", "ghbn", "pool", "php", + "php_proc_0", "php_proc_1", "php_proc_2", "php_proc_3", "php_proc_4", + "php_proc_5", "php_proc_6", "php_proc_7", "php_proc_8", "php_proc_9", + "php_proc_10", "php_proc_11", "php_proc_12", "php_proc_13", "php_proc_14", "thdet", "host_time"}; + init_mutexes(); + init_mutexes(); /* pthread_once must preserve the already-created locks. */ + for (size_t i = 0; i < sizeof(locks)/sizeof(locks[0]); i++) { + assert(strcmp(get_name(locks[i]), names[i]) == 0); + assert(get_lock(locks[i]) != NULL && get_cond(locks[i]) != NULL && get_attr(locks[i]) != NULL); + assert(get_lock(locks[i]) == get_lock(locks[i])); + for (size_t j = 0; j < i; j++) { + assert(get_lock(locks[i]) != get_lock(locks[j])); + assert(get_cond(locks[i]) != get_cond(locks[j])); + assert(get_attr(locks[i]) != get_attr(locks[j])); + } + assert(thread_mutex_trylock(locks[i]) == 0); + pthread_t competitor; + int lock = locks[i]; + assert(pthread_create(&competitor, NULL, try_busy_lock, &lock) == 0); + void *result; + assert(pthread_join(competitor, &result) == 0); + assert((intptr_t)result == EBUSY); + thread_mutex_unlock(locks[i]); + assert(thread_mutex_trylock(locks[i]) == 0); + thread_mutex_unlock(locks[i]); + } + assert(strcmp(get_name(-1), "Unknown lock") == 0); + assert(get_lock(-1) == NULL && get_cond(-1) == NULL && get_attr(-1) == NULL); + assert(get_lock(1) == NULL && get_cond(1) == NULL && get_attr(1) == NULL); +} + +static pthread_mutex_t distribution_mutex = PTHREAD_MUTEX_INITIALIZER; +static int distribution[MAX_PHP_SERVERS]; +static void *next_php_process(void *unused) { + (void)unused; + for (int i = 0; i < 300; i++) { + int process = php_get_process(); + assert(process >= 0 && process < 3); + assert(pthread_mutex_lock(&distribution_mutex) == 0); + distribution[process]++; + assert(pthread_mutex_unlock(&distribution_mutex) == 0); + } + return NULL; +} + +static void test_php_roundrobin(void) { + int previous_count = set.php.php_servers; + int previous_current = set.php.php_current_server; + /* php_get_process() hands out healthy slots, so every slot looks like a + * live, ready server here. */ + php_t *previous_processes = php_processes; + php_t healthy[MAX_PHP_SERVERS]; + php_processes_initialize(healthy, MAX_PHP_SERVERS); + for (int i = 0; i < MAX_PHP_SERVERS; i++) { + healthy[i].php_state = PHP_READY; + healthy[i].php_pid = 2; + healthy[i].php_read_fd = STDIN_FILENO; + healthy[i].php_write_fd = STDOUT_FILENO; + } + php_processes = healthy; + set.php.php_servers = MAX_PHP_SERVERS; + set.php.php_current_server = 0; + for (int i = 0; i < 2 * MAX_PHP_SERVERS; i++) assert(php_get_process() == i % MAX_PHP_SERVERS); + set.php.php_servers = 1; + set.php.php_current_server = MAX_PHP_SERVERS; + assert(php_get_process() == 0 && php_get_process() == 0); + set.php.php_servers = 3; + set.php.php_current_server = 0; + memset(distribution, 0, sizeof(distribution)); + pthread_t workers[4]; + for (size_t i = 0; i < 4; i++) assert(pthread_create(&workers[i], NULL, next_php_process, NULL) == 0); + for (size_t i = 0; i < 4; i++) assert(pthread_join(workers[i], NULL) == 0); + /* The cursor is fair, but a slot busy in another thread hands that caller + * the next healthy one, so only the total and coverage are exact. */ + assert(distribution[0] + distribution[1] + distribution[2] == 1200); + for (int i = 0; i < 3; i++) assert(distribution[i] > 0); + assert(set.php.php_current_server == 3); + set.php.php_servers = previous_count; + set.php.php_current_server = previous_current; + php_processes = previous_processes; +} + +static void test_legacy_ip_predicate(void) { + /* This unused legacy helper recognizes characters, not IP syntax. */ + assert(is_ipaddress("127.0.0.1")); + assert(is_ipaddress("::1")); + assert(is_ipaddress("")); + assert(is_ipaddress("999.999.999.999")); + assert(is_ipaddress(".:")); + assert(!is_ipaddress("example.org")); + assert(!is_ipaddress("2001:db8::1")); + assert(!is_ipaddress("127.0.0.1 ")); + assert(!is_ipaddress("127.0.0.1/8")); +} + +static void test_multipart_boundaries(void) { + static const struct { const char *input; int expected; } cases[] = { + {NULL, FALSE}, {"", FALSE}, {"123", FALSE}, {"hello", FALSE}, + {"a:1", TRUE}, {"a!1", TRUE}, {":", TRUE}, {"!", TRUE}, + {"a:1 b:2", TRUE}, {"a!1 b!2", TRUE}, {"a:1 b!2", TRUE}, + {"a:1 b:2", FALSE}, {" a:1", FALSE}, {"a:1 ", FALSE}, + {" a:1 b:2 ", FALSE}, {"a:1 b:2 c:3", TRUE}, + {"a::1 b:2", FALSE}, {"a::1", TRUE}, + {"a:1\tb:2", TRUE}, {"a:1\nb!2", TRUE}, + {"a:1\t b!2", TRUE}, {"a:1 b:2\n c!3", TRUE}, + {"a:1 \t b:2", FALSE}, {"a b", FALSE} + }; + for (size_t i = 0; i < sizeof(cases)/sizeof(cases[0]); i++) assert(is_multipart_output(cases[i].input) == cases[i].expected); +} + +static void test_fatal_signal_contracts(void) { + static const struct { int signal; const char *message; } cases[] = { + {SIGINT, "Console Operator"}, + {SIGSEGV, "Segmentation Fault"}, {SIGBUS, "Bus Error"}, + {SIGFPE, "Floating Point Exception"}, {SIGQUIT, "Keyboard Quit"}, + {SIGSYS, "Unhandled Exception"}, {SIGABRT, "Abort Signal"} + }; + for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) { + int diagnostic[2]; + assert(pipe(diagnostic) == 0); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + alarm(5); + close(diagnostic[0]); + assert(dup2(diagnostic[1], STDERR_FILENO) == STDERR_FILENO); + close(diagnostic[1]); + assert(signal(cases[i].signal, SIG_DFL) != SIG_ERR); + install_spine_signal_handler(); + assert(raise(cases[i].signal) == 0); + assert(set.exit.exit_code == cases[i].signal); + struct sigaction restored; + assert(sigaction(cases[i].signal, NULL, &restored) == 0); + assert(restored.sa_handler == SIG_DFL); + uninstall_spine_signal_handler(); + _exit(0); + } + close(diagnostic[1]); + char message[512] = {0}; + size_t used = 0; + ssize_t count; + while ((count = read(diagnostic[0], message + used, sizeof(message) - used - 1)) != 0) { + if (count < 0 && errno == EINTR) continue; + assert(count > 0); + used += (size_t)count; + assert(used < sizeof(message) - 1); + } + close(diagnostic[0]); + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status)); + assert(WEXITSTATUS(status) == (cases[i].signal == SIGSEGV ? 1 : 0)); + assert(strstr(message, cases[i].message) != NULL); + } + /* A broken pipe is an ordinary write failure: it must neither terminate + * Spine nor be reported as fatal, and the handler stays installed. */ + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + alarm(5); + set.exit.exit_code = 0; + install_spine_signal_handler(); + assert(raise(SIGPIPE) == 0 && raise(SIGPIPE) == 0); + assert(set.exit.exit_code == 0); + struct sigaction installed; + assert(sigaction(SIGPIPE, NULL, &installed) == 0 && installed.sa_handler != SIG_DFL); + uninstall_spine_signal_handler(); + assert(sigaction(SIGPIPE, NULL, &installed) == 0 && installed.sa_handler == SIG_DFL); + _exit(0); + } + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); +} + +void test_additional_contracts(void) { + test_keyword_roundtrips(); + test_lock_contracts(); + test_php_roundrobin(); + test_legacy_ip_predicate(); + test_multipart_boundaries(); + test_fatal_signal_contracts(); + puts("production additional contracts passed"); +} + +void test_additional_database_contracts(MYSQL *mysql) { + assert(db_column_exists(mysql, LOCAL, "host", "id") == TRUE); + assert(db_column_exists(mysql, LOCAL, "host", "regression_missing_column") == FALSE); + /* Only kill a connection created and owned by this test. */ + MYSQL victim; + db_connect(LOCAL, &victim); + unsigned long original = mysql_thread_id(&victim); + char query[100]; + snprintf(query, sizeof(query), "KILL CONNECTION %lu", original); + assert(mysql_query(mysql, query) == 0); + assert(db_reconnect(&victim, LOCAL, 2006, "regression_owned_session") == TRUE); + assert(mysql_thread_id(&victim) != original); + assert(mysql_query(&victim, "SELECT 123") == 0); + MYSQL_RES *result = mysql_store_result(&victim); + assert(result != NULL); + MYSQL_ROW row = mysql_fetch_row(result); + assert(row != NULL && strcmp(row[0], "123") == 0); + mysql_free_result(result); + /* An already-healthy session does not count as a new reconnection. */ + assert(db_reconnect(&victim, LOCAL, 2006, "regression_healthy_session") == FALSE); + /* Exercise query and mutation callers, not just explicit reconnect. */ + for (int insert = 0; insert < 2; insert++) { + original = mysql_thread_id(&victim); + snprintf(query, sizeof(query), "KILL CONNECTION %lu", original); + assert(mysql_query(mysql, query) == 0); + errno = 0; + if (insert) assert(db_insert(&victim, LOCAL, "SET @spine_regression_retry=456")); + else { + result = db_query(&victim, LOCAL, "SELECT 456"); + assert(result != NULL); + row = mysql_fetch_row(result); + assert(row != NULL && row[0] != NULL && strcmp(row[0], "456") == 0); + db_free_result(result); + } + assert(mysql_thread_id(&victim) != original); + if (insert) { + result = db_query(&victim, LOCAL, "SELECT @spine_regression_retry"); + assert(result != NULL); + row = mysql_fetch_row(result); + assert(row != NULL && row[0] != NULL && strcmp(row[0], "456") == 0); + db_free_result(result); + } + } + db_disconnect(&victim); +} diff --git a/tests/test_spine_faults.c b/tests/test_spine_faults.c new file mode 100644 index 0000000..012e60c --- /dev/null +++ b/tests/test_spine_faults.c @@ -0,0 +1,712 @@ +/* + * Copyright (C) 2004-2026 The Cacti Group + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License + * for more details. + */ +#include "common.h" +#include "spine.h" +#include +#include +#include + +/* GNU ld wraps the actual API boundary. No production test-only code is used. */ +extern size_t __real_strftime(char *, size_t, const char *, const struct tm *); +extern int __real_mysql_query(MYSQL *, const char *); +extern int __real_mysql_ping(MYSQL *); +extern unsigned int __real_mysql_errno(MYSQL *); +extern int __real_pipe(int [2]); +extern int __real_socketpair(int, int, int, int [2]); +extern int __real_posix_spawn(pid_t *, const char *, const posix_spawn_file_actions_t *, + const posix_spawnattr_t *, char *const [], char *const []); +extern void *__real_malloc(size_t); +extern char *__real_strdup(const char *); +extern void *__real_snmp_sess_open(netsnmp_session *); +extern int __real_snmp_sess_close(void *); +extern int __real_pthread_create(pthread_t *, const pthread_attr_t *, void *(*)(void *), void *); +extern int spine_program_main(int, char **); +extern poller_thread_t **details; + +static int format_failures; +static int format_fault_calls; +static MYSQL *query_fault_connection; +static const char *query_fault_statement; +static int query_fault_armed; +static int observed_query_error; +static int query_fault_calls; +static int query_ping_calls; +static MYSQL *interrupted_connection; +static int interrupted_queries; +static int interrupted_query_calls; +static int pipe_failures; +static int socketpair_failures; +static int spawn_failures; +static int spawn_failure_errno; +static int spawn_fault_calls; +static int allocation_failures; +static int copy_failures; +static const char *copy_failure_text; +static bool account_snmp_sessions; +static void *owned_snmp_session; +static int session_opens; +static int session_close_attempts; +static int session_closes; +static bool launch_case_active; +static int launch_error; +static int launch_failures; +static int launch_expected_faults; +static int launch_calls; +static int launch_fault_calls; +static int launch_checkpoints; +static int launch_device_counter; +static spine_permits_t *launch_startup; + +int __wrap_pthread_create(pthread_t *thread, const pthread_attr_t *attributes, + void *(*start)(void *), void *arg) { + if (!launch_case_active || start != &child) return __real_pthread_create(thread, attributes, start, arg); + const poller_thread_t *work = arg; + assert(work != NULL && work->host_id == 902); + launch_calls++; + launch_device_counter = work->device_counter; + launch_startup = work->thread_init_sem; + if (launch_failures > 0) { + launch_failures--; + launch_fault_calls++; + return launch_error; + } + return __real_pthread_create(thread, attributes, start, arg); +} + +static void inspect_worker_launch_completion(void) { + assert(launch_startup != NULL && launch_calls > 0); + assert(spine_permits_available(&available_threads) == 1); + assert(spine_permits_available(launch_startup) == 1); + assert(spine_permits_available(&available_scripts) == MAX_SIMULTANEOUS_SCRIPTS); + assert(db_pool_local != NULL && db_pool_local[0].free); + thread_mutex_lock(LOCK_THDET); + const poller_thread_t *device = details[launch_device_counter]; + assert(device != NULL && device->host_id == 902); + if (launch_error == EPERM) { + assert(set.exit.exit_code == EXIT_FAILURE && !device->complete); + assert(device->threads_complete == 0 && !device->output_failed); + } else { + assert(set.exit.exit_code == EXIT_SUCCESS && device->complete); + assert(device->threads_complete == 1 && !device->output_failed); + } + thread_mutex_unlock(LOCK_THDET); + launch_checkpoints++; +} + +static void verify_worker_launch_exit(void) { + fflush(NULL); + fprintf(stderr, "worker launch checkpoint: checkpoints=%i calls=%i faults=%i pending=%i exit=%i\n", launch_checkpoints, launch_calls, launch_fault_calls, launch_failures, set.exit.exit_code); + assert(launch_checkpoints == 1 && launch_failures == 0); + assert(launch_fault_calls == launch_expected_faults); + assert(launch_calls == (launch_error == EAGAIN ? 2 : 1)); + puts("production worker launch completion checkpoint passed"); +} + +void *__wrap_snmp_sess_open(netsnmp_session *session) { + void *handle = __real_snmp_sess_open(session); + if (account_snmp_sessions && handle != NULL) { + assert(owned_snmp_session == NULL); + owned_snmp_session = handle; + session_opens++; + } + return handle; +} + +int __wrap_snmp_sess_close(void *handle) { + bool owned = account_snmp_sessions && handle == owned_snmp_session && handle != NULL; + if (owned) session_close_attempts++; + int result = __real_snmp_sess_close(handle); + if (owned && result == 1) { + session_closes++; + owned_snmp_session = NULL; + } + return result; +} + +size_t __wrap_strftime(char *output, size_t capacity, const char *format, const struct tm *time) { + if (format_failures > 0) { + format_failures--; + format_fault_calls++; + return 0; /* strftime documents zero when its destination is insufficient. */ + } + return __real_strftime(output, capacity, format, time); +} + +int __wrap_mysql_query(MYSQL *mysql, const char *query) { + static const char completion[] = "UPDATE poller_time SET end_time=NOW() WHERE poller_id="; + /* The interrupted handle is never connected; it must not reach the client library. */ + if (mysql == interrupted_connection) { + interrupted_query_calls++; + if (interrupted_queries > 0) { + interrupted_queries--; + errno = EINTR; + return 1; + } + return 0; + } + if (launch_case_active && strncmp(query, completion, sizeof(completion) - 1) == 0) { + inspect_worker_launch_completion(); + } + if (query_fault_armed && mysql == query_fault_connection && strcmp(query, query_fault_statement) == 0) { + query_fault_armed = 0; + bool reconnect = FALSE; + assert(mysql_options(mysql, MYSQL_OPT_RECONNECT, &reconnect) == 0); + int result = __real_mysql_query(mysql, query); + observed_query_error = (int)mysql_errno(mysql); + assert(result != 0 && (observed_query_error == 2006 || observed_query_error == 2013)); + reconnect = TRUE; + assert(mysql_options(mysql, MYSQL_OPT_RECONNECT, &reconnect) == 0); + assert((int)mysql_errno(mysql) == observed_query_error); + query_fault_calls++; + return result; + } + return __real_mysql_query(mysql, query); +} + +int __wrap_mysql_ping(MYSQL *mysql) { + if (mysql == query_fault_connection) query_ping_calls++; + return __real_mysql_ping(mysql); +} + +unsigned int __wrap_mysql_errno(MYSQL *mysql) { + if (mysql == interrupted_connection) return 2013; + return __real_mysql_errno(mysql); +} + +int __wrap_pipe(int descriptors[2]) { + if (pipe_failures > 0) { pipe_failures--; errno = EMFILE; return -1; } + return __real_pipe(descriptors); +} + +int __wrap_socketpair(int domain, int type, int protocol, int descriptors[2]) { + if (socketpair_failures > 0) { socketpair_failures--; errno = EMFILE; return -1; } + return __real_socketpair(domain, type, protocol, descriptors); +} + +/* Script children are created with posix_spawn(), which reports failure + * through its return value rather than errno. */ +int __wrap_posix_spawn(pid_t *pid, const char *path, const posix_spawn_file_actions_t *actions, + const posix_spawnattr_t *attributes, char *const arguments[], char *const environment[]) { + if (spawn_failures > 0) { spawn_failures--; spawn_fault_calls++; return spawn_failure_errno; } + return __real_posix_spawn(pid, path, actions, attributes, arguments, environment); +} + +void *__wrap_malloc(size_t size) { + if (allocation_failures > 0) { allocation_failures--; errno = ENOMEM; return NULL; } + return __real_malloc(size); +} + +char *__wrap_strdup(const char *text) { + if (copy_failures > 0) { copy_failures--; errno = ENOMEM; return NULL; } + if (copy_failure_text != NULL && strcmp(text, copy_failure_text) == 0) { errno = ENOMEM; return NULL; } + return __real_strdup(text); +} + +static void test_logger_format_failure(void) { + config_t previous = set; + char filename[] = "spine-log-fault-XXXXXX"; + int log = mkstemp(filename); + assert(log >= 0 && close(log) == 0); + set.logging.log_destination = LOGDEST_FILE; + set.logging.log_level = POLLER_VERBOSITY_LOW; + set.logging.logfile_processed = TRUE; + set.console.stdout_notty = TRUE; + set.console.stderr_notty = FALSE; + STRNCOPY(set.logging.path_logfile, filename); + #ifdef DISABLE_STDERR + const int diagnostic = STDOUT_FILENO; + set.console.stdout_notty = FALSE; + #else + const int diagnostic = STDERR_FILENO; + #endif + int saved = dup(diagnostic); + int capture[2]; + assert(saved >= 0 && pipe(capture) == 0); + fflush(NULL); + assert(dup2(capture[1], diagnostic) == diagnostic && close(capture[1]) == 0); + format_failures = 1; + format_fault_calls = 0; + /* Date formatting failure reports a warning and preserves sink delivery. */ + assert(spine_log("regression date fallback") == TRUE); + assert(format_failures == 0 && format_fault_calls == 1); + fflush(NULL); + assert(dup2(saved, diagnostic) == diagnostic && close(saved) == 0); + char response[LOGSIZE * 2]; + ssize_t received = read(capture[0], response, sizeof(response) - 1); + assert(received > 0 && (size_t)received < sizeof(response) && close(capture[0]) == 0); + response[received] = '\0'; + assert(strstr(response, "ERROR: Could not get string from strftime()") != NULL); + assert(strstr(response, "regression date fallback") != NULL); + + set.console.stdout_notty = TRUE; + set.console.stderr_notty = TRUE; + saved = dup(diagnostic); + assert(saved >= 0 && pipe(capture) == 0); + assert(dup2(capture[1], diagnostic) == diagnostic && close(capture[1]) == 0); + format_failures = 1; + assert(spine_log("regression quiet date fallback") == TRUE); + fflush(NULL); + assert(dup2(saved, diagnostic) == diagnostic && close(saved) == 0); + assert(read(capture[0], response, sizeof(response)) == 0 && close(capture[0]) == 0); + assert(format_failures == 0 && format_fault_calls == 2); + FILE *file = fopen(filename, "r"); + assert(file != NULL); + size_t bytes = fread(response, 1, sizeof(response) - 1, file); + assert(bytes > 0 && bytes < sizeof(response) && !ferror(file) && fclose(file) == 0); + response[bytes] = '\0'; + assert(strstr(response, "regression date fallback") != NULL); + assert(strstr(response, "regression quiet date fallback") != NULL); + assert(strncmp(response, "SPINE: Poller[", strlen("SPINE: Poller[")) == 0); + assert(unlink(filename) == 0); + set = previous; + puts("production logger date-format failure regressions passed"); +} + +static unsigned int open_descriptor_count(void) { + DIR *directory = opendir("/proc/self/fd"); + assert(directory != NULL); + unsigned int count = 0; + const struct dirent *entry; + while ((entry = readdir(directory)) != NULL) { + if (strcmp(entry->d_name, ".") != 0 && strcmp(entry->d_name, "..") != 0) count++; + } + assert(closedir(directory) == 0); + return count; +} + +static void assert_cancel_state(int expected) { + int previous; + assert(pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &previous) == 0); + assert(previous == expected); + assert(pthread_setcancelstate(previous, NULL) == 0); +} + +static void test_child_setup_failures(unsigned int expected_descriptors) { + /* Redirection and exec now happen inside posix_spawn(), out of reach of + * the dup2()/execve() wrappers, so drive the failure through the shell: + * a command that cannot execute must still be reaped with 127 and must not + * leak a descriptor in either pipe mode. */ + for (int index = 0; index < 2; index++) { + int descriptor = nft_popen("/spine-regression/nonexistent-command 2>/dev/null", index == 1 ? "r+" : "r"); + assert(descriptor >= 0); + char bytes[3]; + assert(read(descriptor, bytes, sizeof(bytes)) == 0); + int status = nft_pclose(descriptor); + assert(status == -1 || (WIFEXITED(status) && WEXITSTATUS(status) == 127)); + assert(open_descriptor_count() == expected_descriptors); + } +} + +static void test_process_creation_failures(void) { + unsigned int descriptors = open_descriptor_count(); + int unrelated[2]; + assert(pipe(unrelated) == 0); + unsigned int with_unrelated = open_descriptor_count(); + assert(with_unrelated == descriptors + 2); + pipe_failures = 1; + assert(nft_popen("printf unused", "r") == -1 && errno == EMFILE); + assert(pipe_failures == 0 && open_descriptor_count() == with_unrelated); + socketpair_failures = 1; + assert(nft_popen("printf unused", "r+") == -1 && errno == EMFILE); + assert(socketpair_failures == 0 && open_descriptor_count() == with_unrelated); + allocation_failures = 1; + assert(nft_popen("printf unused", "r") == -1 && errno == ENOMEM); + assert(allocation_failures == 0 && open_descriptor_count() == with_unrelated); + assert_cancel_state(PTHREAD_CANCEL_ENABLE); + copy_failures = 1; + assert(nft_popen("printf unused", "r") == -1 && errno == ENOMEM); + assert(copy_failures == 0 && open_descriptor_count() == with_unrelated); + assert_cancel_state(PTHREAD_CANCEL_ENABLE); + spawn_failures = 4; + spawn_failure_errno = EAGAIN; + spawn_fault_calls = 0; + assert(nft_popen("printf unused", "r") == -1 && errno == EAGAIN); + assert(spawn_failures == 0 && spawn_fault_calls == 4 && open_descriptor_count() == with_unrelated); + assert_cancel_state(PTHREAD_CANCEL_ENABLE); + spawn_failures = 1; + spawn_failure_errno = EPERM; + spawn_fault_calls = 0; + assert(nft_popen("printf unused", "r") == -1 && errno == EPERM); + assert(spawn_failures == 0 && spawn_fault_calls == 1 && open_descriptor_count() == with_unrelated); + spawn_failures = 2; + spawn_failure_errno = EAGAIN; + spawn_fault_calls = 0; + int descriptor = nft_popen("printf ok", "r"); + assert(descriptor >= 0 && spawn_failures == 0 && spawn_fault_calls == 2); + char bytes[3] = {0}; + assert(read(descriptor, bytes, 2) == 2 && strcmp(bytes, "ok") == 0); + int status = nft_pclose(descriptor); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); + assert(open_descriptor_count() == with_unrelated); + test_child_setup_failures(with_unrelated); + assert(write(unrelated[1], "x", 1) == 1 && read(unrelated[0], bytes, 1) == 1 && bytes[0] == 'x'); + assert(close(unrelated[0]) == 0 && close(unrelated[1]) == 0); + assert(open_descriptor_count() == descriptors); + puts("production process creation failure regressions passed"); +} + +static void kill_owned_connection(MYSQL *administrator, MYSQL *victim) { + char query[100]; + spine_snprintf(query, sizeof(query), "KILL CONNECTION %lu", mysql_thread_id(victim)); + assert(mysql_query(administrator, query) == 0); +} + +static void arm_lost_connection(MYSQL *mysql, const char *statement) { + query_fault_connection = mysql; + query_fault_statement = statement; + query_fault_armed = 1; + query_fault_calls = 0; + query_ping_calls = 0; + observed_query_error = 0; +} + +/* A lost-connection error raised while a signal interrupted the call is + * retried in place: db_insert() must neither give up nor ping for a reconnect. */ +static void test_interrupted_insert_retry(void) { + MYSQL offline; + assert(mysql_init(&offline) == &offline); + set.poller.SQL_readonly = FALSE; + interrupted_connection = &offline; + interrupted_queries = 1; + interrupted_query_calls = 0; + query_fault_connection = &offline; + query_ping_calls = 0; + assert(db_insert(&offline, LOCAL, "INSERT INTO interrupted(value) VALUES(1)")); + assert(interrupted_queries == 0 && interrupted_query_calls == 2 && query_ping_calls == 0); + interrupted_connection = NULL; + query_fault_connection = NULL; + mysql_close(&offline); + puts("production interrupted insert retry regressions passed"); +} + +/* die() ends the process, so the copy failure runs in a child. The + * authentication copy has already succeeded and must be released first. */ +static void test_snmpv3_privacy_copy_failure(void) { + int output[2]; + assert(pipe(output) == 0); + fflush(stdout); + fflush(stderr); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + if (dup2(output[1], STDERR_FILENO) != STDERR_FILENO) _exit(3); + close(output[0]); + close(output[1]); + snmp_spine_init(); + copy_failure_text = "fault-privacy-passphrase"; + snmp_connection_t options = { + .host_id = 47, .hostname = "127.0.0.1", .snmp_version = 3, + .snmp_community = "", .snmp_username = "fault-v3-user", + .snmp_password = "fault-auth-passphrase", .snmp_auth_protocol = "SHA", + .snmp_priv_passphrase = "fault-privacy-passphrase", .snmp_priv_protocol = "AES", + .snmp_context = "", .snmp_engine_id = "", .snmp_port = 1162, + .snmp_timeout = 500 + }; + snmp_host_init(&options); + _exit(4); + } + assert(close(output[1]) == 0); + char message[BUFSIZE] = {0}; + size_t used = 0; + ssize_t count; + while (used < sizeof(message) - 1 && (count = read(output[0], message + used, sizeof(message) - 1 - used)) > 0) { + used += (size_t)count; + } + assert(close(output[0]) == 0); + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == EXIT_FAILURE); + assert(strstr(message, "ERROR: Fatal malloc error: SNMP privacy passphrase") != NULL); + assert(strstr(message, "fault-auth-passphrase") == NULL); + puts("production SNMPv3 privacy copy failure regressions passed"); +} + +static void test_real_database_retry(void) { + const char *hostname = getenv("SPINE_TEST_DB_HOST"); + assert(hostname != NULL && hostname[0] != '\0'); + config_defaults(); + strncopy(set.database.host, hostname, sizeof(set.database.host)); + STRNCOPY(set.database.user, "root"); + STRNCOPY(set.database.database, "spine_regressions"); + set.database.password[0] = '\0'; + set.database.port = 3306; + set.poller.SQL_readonly = FALSE; + set.logging.log_destination = 0; + set.console.stdout_notty = TRUE; + set.console.stderr_notty = TRUE; + MYSQL administrator; + MYSQL victim; + db_connect(LOCAL, &administrator); + db_connect(LOCAL, &victim); + unsigned long original = mysql_thread_id(&victim); + kill_owned_connection(&administrator, &victim); + const char *read_statement = "SELECT 123 AS recovered_value"; + arm_lost_connection(&victim, read_statement); + MYSQL_RES *result = db_query(&victim, LOCAL, read_statement); + assert(result != NULL && mysql_num_rows(result) == 1); + MYSQL_ROW row = mysql_fetch_row(result); + assert(row != NULL && strcmp(row[0], "123") == 0); + db_free_result(result); + assert(query_fault_calls == 1 && query_ping_calls == 1 && !query_fault_armed && observed_query_error != 0); + assert(mysql_thread_id(&victim) != original); + char table[100]; + spine_snprintf(table, sizeof(table), "spine_fault_%ld", (long)getpid()); + char query[BUFSIZE]; + spine_snprintf(query, sizeof(query), "CREATE TABLE %s (value INT NOT NULL) ENGINE=InnoDB", table); + assert(db_insert(&administrator, LOCAL, query)); + original = mysql_thread_id(&victim); + kill_owned_connection(&administrator, &victim); + spine_snprintf(query, sizeof(query), "INSERT INTO %s(value) VALUES(456)", table); + arm_lost_connection(&victim, query); + assert(db_insert(&victim, LOCAL, query)); + assert(query_fault_calls == 1 && query_ping_calls == 1 && !query_fault_armed && observed_query_error != 0); + assert(mysql_thread_id(&victim) != original); + spine_snprintf(query, sizeof(query), "SELECT value FROM %s", table); + result = db_query(&administrator, LOCAL, query); + assert(result != NULL && mysql_num_rows(result) == 1); + row = mysql_fetch_row(result); + assert(row != NULL && strcmp(row[0], "456") == 0); + db_free_result(result); + spine_snprintf(query, sizeof(query), "DROP TABLE %s", table); + assert(db_insert(&administrator, LOCAL, query)); + db_disconnect(&victim); + db_disconnect(&administrator); + query_fault_connection = NULL; + query_fault_statement = NULL; + puts("production real lost-connection retry regressions passed"); +} + +static unsigned long long fault_database_count(MYSQL *mysql, const char *query) { + MYSQL_RES *result = db_query(mysql, LOCAL, query); + assert(result != NULL && mysql_num_rows(result) == 1); + MYSQL_ROW row = mysql_fetch_row(result); + assert(row != NULL && row[0] != NULL); + unsigned long long count = strtoull(row[0], NULL, 10); + db_free_result(result); + return count; +} + +static void *run_ping_only_worker(void *argument) { + assert(mysql_thread_init() == 0); + int errors = -1; + poll_host(argument, &errors); + assert(errors == -1); /* Early ping-only return preserves this output. */ + return NULL; +} + +static void test_ping_only_session_lifetime(void) { + extern int *debug_devices; + const char *agent = getenv("SPINE_TEST_SNMP_HOST"); + assert(agent != NULL && agent[0] != '\0'); + config_t previous = set; + int *previous_debug_devices = debug_devices; + int debug_fixture[100] = {0}; + debug_devices = debug_fixture; + pool_t *previous_pool = db_pool_local; + set.poller.threads = 1; + set.poller.poller_id = 1; + set.poller.mode = REMOTE_OFFLINE; + set.availability.ping_only = TRUE; + set.snmp.snmp_retries = 0; + set.snmp.mibs = 0; + MYSQL administrator; + db_connect(LOCAL, &administrator); + /* Refuse to replace pre-existing records, even in the isolated fixture. */ + assert(fault_database_count(&administrator, "SELECT COUNT(*) FROM host WHERE id=900") == 0); + assert(fault_database_count(&administrator, "SELECT COUNT(*) FROM poller_item WHERE host_id=900") == 0); + unsigned long long outputs = fault_database_count(&administrator, "SELECT COUNT(*) FROM poller_output"); + unsigned long long boosted = fault_database_count(&administrator, "SELECT COUNT(*) FROM poller_output_boost"); + db_pool_local = calloc(1, sizeof(*db_pool_local)); + assert(db_pool_local != NULL); + db_pool_local[0].free = TRUE; + db_connect(LOCAL, &db_pool_local[0].mysql); + assert(spine_permits_init(&available_threads, 2) == 0); + assert(spine_permits_init(&available_scripts, 3) == 0); + char escaped_agent[BUFSIZE]; + db_escape(&administrator, escaped_agent, sizeof(escaped_agent), agent); + snmp_spine_init(); + for (int with_session = 1; with_session >= 0; with_session--) { + char query[LRG_BUFSIZE]; + spine_snprintf(query, sizeof(query), "INSERT INTO host(id,hostname,snmp_version,snmp_community,snmp_port,snmp_timeout,availability_method,status,total_polls,failed_polls,status_fail_date,status_rec_date,snmp_sysLocation) VALUES(900,'%s',2,'%s',1161,500,%i,%i,0,0,'2026-10-06 00:00:00','2026-10-06 00:00:00','ping-only-metadata')", escaped_agent, with_session ? "regression" : "", AVAIL_SNMP, HOST_UP); + assert(db_insert(&administrator, LOCAL, query)); + poller_thread_t work = {0}; + work.host_id = 900; + work.host_thread = 1; + work.host_threads = 1; + work.host_time_double = get_time_as_double(); + STRNCOPY(work.host_time, "1791244800"); + owned_snmp_session = NULL; + session_opens = 0; + session_close_attempts = 0; + session_closes = 0; + account_snmp_sessions = TRUE; + pthread_t worker; + assert(pthread_create(&worker, NULL, run_ping_only_worker, &work) == 0); + assert(pthread_join(worker, NULL) == 0); + account_snmp_sessions = FALSE; + int observed_opens = session_opens; + int observed_attempts = session_close_attempts; + int observed_closes = session_closes; + /* Preserve an old-code failure without leaking the task-owned fixture + * handle: this real cleanup is excluded from production close counts. */ + if (owned_snmp_session != NULL) { + assert(__real_snmp_sess_close(owned_snmp_session) == 1); + owned_snmp_session = NULL; + } + assert(db_pool_local[0].free); + assert(spine_permits_available(&available_threads) == 2); + assert(spine_permits_available(&available_scripts) == 3); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM host WHERE id=900 AND status=%i AND total_polls=1 AND failed_polls=0 AND snmp_sysLocation='ping-only-metadata'", HOST_UP); + assert(fault_database_count(&administrator, query) == 1); + assert(fault_database_count(&administrator, "SELECT COUNT(*) FROM poller_output") == outputs); + assert(fault_database_count(&administrator, "SELECT COUNT(*) FROM poller_output_boost") == boosted); + assert(db_insert(&administrator, LOCAL, "DELETE FROM host WHERE id=900")); + fprintf(stderr, "ping-only owned session accounting: session=%d opens=%d attempts=%d closes=%d\n", with_session, observed_opens, observed_attempts, observed_closes); + assert(observed_opens == with_session); + assert(observed_attempts == with_session && observed_closes == with_session); + } + snmp_spine_close(); + assert(spine_permits_destroy(&available_scripts) == 0); + assert(spine_permits_destroy(&available_threads) == 0); + db_close_connection_pool(LOCAL); + db_pool_local = previous_pool; + db_disconnect(&administrator); + debug_devices = previous_debug_devices; + set = previous; + puts("production ping-only SNMP session ownership regressions passed"); +} + +static int execute_worker_launch_case(const char *scenario, char *config) { + assert(strcmp(scenario, "admitted") == 0 || strcmp(scenario, "rejected") == 0 || strcmp(scenario, "retry") == 0); + launch_case_active = TRUE; + if (strcmp(scenario, "rejected") == 0) launch_error = EPERM; + else if (strcmp(scenario, "retry") == 0) launch_error = EAGAIN; + launch_expected_faults = launch_error == 0 ? 0 : 1; + launch_failures = launch_expected_faults; + assert(atexit(verify_worker_launch_exit) == 0); + alarm(15); + char interval[] = "poller_interval:5"; + char profiles[] = "active_profiles:2"; + char boost[] = "boost_rrd_update_enable:0"; + char redirect[] = "boost_redirect:0"; + char *arguments[] = {"spine", "-C", "/nonexistent/spine-fault.conf", "--conf", config, + "-p", "1", "-t", "1", "-H", "902", "-O", interval, "-O", profiles, + "-O", boost, "-O", redirect, "-S", "-V", "2", NULL}; + return spine_program_main((int)(sizeof(arguments) / sizeof(arguments[0]) - 1), arguments); +} + +static pid_t run_worker_launch_case(const char *scenario, const char *config, int expected) { + fflush(NULL); + pid_t process = fork(); + assert(process >= 0); + if (process == 0) { + execl("./test_spine_faults", "test_spine_faults", "--worker-launch-case", scenario, config, NULL); + _exit(127); + } + int status; + assert(waitpid(process, &status, 0) == process); + fprintf(stderr, "worker launch case=%s expected=%i status=%i\n", scenario, expected, status); + assert(WIFEXITED(status) && WEXITSTATUS(status) == expected); + return process; +} + +static void test_worker_launch_failures(void) { + MYSQL mysql; + db_connect(LOCAL, &mysql); + /* Refuse any pre-existing spelling of this fixture identity. Grants are + * limited to DML on the isolated fixture; no global administrative rights. */ + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM mysql.user WHERE User='spine_owned_launch'") == 0); + assert(db_insert(&mysql, LOCAL, "CREATE USER 'spine_owned_launch'@'%' IDENTIFIED BY 'regression-only'")); + assert(db_insert(&mysql, LOCAL, "GRANT SELECT,INSERT,UPDATE,DELETE ON spine_regressions.* TO 'spine_owned_launch'@'%'")); + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM information_schema.SCHEMA_PRIVILEGES WHERE GRANTEE=CONCAT(CHAR(39),'spine_owned_launch',CHAR(39),'@',CHAR(39),'%',CHAR(39))") == 4); + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM information_schema.SCHEMA_PRIVILEGES WHERE GRANTEE=CONCAT(CHAR(39),'spine_owned_launch',CHAR(39),'@',CHAR(39),'%',CHAR(39)) AND TABLE_SCHEMA='spine_regressions' AND PRIVILEGE_TYPE IN('SELECT','INSERT','UPDATE','DELETE') AND IS_GRANTABLE='NO'") == 4); + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM information_schema.USER_PRIVILEGES WHERE GRANTEE=CONCAT(CHAR(39),'spine_owned_launch',CHAR(39),'@',CHAR(39),'%',CHAR(39)) AND PRIVILEGE_TYPE<>'USAGE'") == 0); + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM host WHERE id=902") == 0); + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=902 OR local_data_id=942001") == 0); + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=942001") == 0); + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id=942001") == 0); + /* Poller1 always includes device0 if present, regardless of hostlist. */ + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=0 AND poller_id=1") == 0); + assert(db_insert(&mysql, LOCAL, "INSERT INTO host(id,hostname,poller_id,disabled,device_threads,availability_method,snmp_version,status_fail_date,status_rec_date) VALUES(902,'127.0.0.1',1,'',1,0,0,'2026-10-06 00:00:00','2026-10-06 00:00:00')")); + char query[BUFSIZE]; + spine_snprintf(query, sizeof(query), "INSERT INTO poller_item(local_data_id,host_id,poller_id,action,arg1,rrd_name,snmp_port,rrd_step,rrd_next_step) VALUES(942001,902,1,%i,'/usr/bin/printf 123','owned',161,300,0)", POLLER_ACTION_SCRIPT); + assert(db_insert(&mysql, LOCAL, query)); + char directory[] = "/tmp/spine-worker-launch-XXXXXX"; + assert(mkdtemp(directory) != NULL); + struct stat owned; + assert(lstat(directory, &owned) == 0 && S_ISDIR(owned.st_mode)); + assert(owned.st_uid == geteuid() && (owned.st_mode & 0777) == 0700); + char config[SMALL_BUFSIZE]; + char log_path[SMALL_BUFSIZE]; + spine_snprintf(config, sizeof(config), "%s/spine.conf", directory); + spine_snprintf(log_path, sizeof(log_path), "%s/spine.log", directory); + int descriptor = open(config, O_WRONLY | O_CREAT | O_EXCL, 0600); + assert(descriptor >= 0); + assert(fstat(descriptor, &owned) == 0 && S_ISREG(owned.st_mode)); + assert(owned.st_uid == geteuid() && (owned.st_mode & 0777) == 0600); + FILE *file = fdopen(descriptor, "w"); + assert(file != NULL); + /* Public fixture-only credentials follow the parser's two-token grammar. + * A value-less DB_Pass line retains its configured default. */ + assert(fprintf(file, "DB_Host %s\nDB_Database spine_regressions\nDB_User spine_owned_launch\nDB_Pass regression-only\nDB_Port 3306\nCacti_Log %s\n", set.database.host, log_path) > 0); + assert(fclose(file) == 0); + const char *const scenarios[] = {"admitted", "rejected", "retry"}; + for (size_t index = 0; index < sizeof(scenarios) / sizeof(scenarios[0]); index++) { + assert(db_insert(&mysql, LOCAL, "DELETE FROM poller_output WHERE local_data_id=942001")); + assert(db_insert(&mysql, LOCAL, "UPDATE poller_item SET rrd_next_step=0 WHERE host_id=902 AND local_data_id=942001")); + unsigned long long first_id = fault_database_count(&mysql, "SELECT COALESCE(MAX(id),0) FROM poller_time"); + bool rejected = index == 1; + pid_t process = run_worker_launch_case(scenarios[index], config, rejected ? EXIT_FAILURE : EXIT_SUCCESS); + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=942001 AND output='123'") == (rejected ? 0 : 1)); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_item WHERE host_id=902 AND local_data_id=942001 AND rrd_next_step=%i", rejected ? 0 : 295); + assert(fault_database_count(&mysql, query) == 1); + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id=942001") == 0); + spine_snprintf(query, sizeof(query), "DELETE FROM poller_time WHERE poller_id=1 AND pid=%ld AND id>%llu", (long)process, first_id); + assert(db_insert(&mysql, LOCAL, query)); + } + assert(db_insert(&mysql, LOCAL, "DELETE FROM poller_output WHERE local_data_id=942001")); + assert(db_insert(&mysql, LOCAL, "DELETE FROM poller_item WHERE host_id=902 AND local_data_id=942001")); + assert(db_insert(&mysql, LOCAL, "DELETE FROM host WHERE id=902")); + assert(db_insert(&mysql, LOCAL, "DROP USER 'spine_owned_launch'@'%'")); + assert(fault_database_count(&mysql, "SELECT COUNT(*) FROM mysql.user WHERE User='spine_owned_launch'") == 0); + assert(unlink(config) == 0); + if (unlink(log_path) != 0) assert(errno == ENOENT); + assert(rmdir(directory) == 0); + db_disconnect(&mysql); + puts("production actual main worker launch failure and retry regressions passed"); +} + +int main(int argc, char **argv) { + /* Fresh exec enters production initialization exactly once, without + * reinitializing the ordinary fault harness's inherited mutexes. */ + if (argc == 4 && strcmp(argv[1], "--worker-launch-case") == 0) { + return execute_worker_launch_case(argv[2], argv[3]); + } + config_defaults(); + init_mutexes(); + alarm(20); + test_logger_format_failure(); + test_process_creation_failures(); + test_interrupted_insert_retry(); + test_snmpv3_privacy_copy_failure(); + if (argc == 2 && strcmp(argv[1], "--database") == 0) { + test_real_database_retry(); + test_ping_only_session_lifetime(); + test_worker_launch_failures(); + } + else assert(argc == 1); + alarm(0); + puts("production linker fault regressions passed"); + return 0; +} diff --git a/tests/test_spine_output_contracts.c b/tests/test_spine_output_contracts.c new file mode 100644 index 0000000..66ff84d --- /dev/null +++ b/tests/test_spine_output_contracts.c @@ -0,0 +1,609 @@ +/* + * Copyright (C) 2004-2026 The Cacti Group + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License + * for more details. + */ +#include "common.h" +#include "spine.h" +#include + +extern poller_thread_t **details; +extern int *debug_devices; + +static unsigned long long output_count(MYSQL *mysql, const char *query) { + MYSQL_RES *result = db_query(mysql, LOCAL, query); + assert(result != NULL && mysql_num_rows(result) == 1); + MYSQL_ROW row = mysql_fetch_row(result); + assert(row != NULL && row[0] != NULL); + unsigned long long value = strtoull(row[0], NULL, 10); + db_free_result(result); + return value; +} + +static void assert_output_engine(MYSQL *mysql, const char *table, const char *engine) { + char query[BUFSIZE]; + spine_snprintf(query, sizeof(query), "SELECT ENGINE FROM information_schema.TABLES WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME='%s'", table); + MYSQL_RES *result = db_query(mysql, LOCAL, query); + assert(result != NULL && mysql_num_rows(result) == 1); + MYSQL_ROW row = mysql_fetch_row(result); + assert(row != NULL && row[0] != NULL && STRIMATCH(row[0], engine)); + db_free_result(result); +} + +static void run_output_partition(const poller_thread_t *aggregate, int partition) { + poller_thread_t *work = malloc(sizeof(*work)); + assert(work != NULL); + *work = *aggregate; + work->host_thread = partition; + spine_permits_t startup; + assert(spine_permits_init(&startup, 1) == 0); + assert(spine_permits_try_acquire(&startup) == 0); + work->thread_init_sem = &startup; + assert(spine_permits_try_acquire(&available_threads) == 0); + pthread_t worker; + /* Production child owns/frees instructions and releases thread/startup + * permits after poll_host, including output failures. */ + assert(pthread_create(&worker, NULL, child, work) == 0); + assert(pthread_join(worker, NULL) == 0); + assert(spine_permits_available(&startup) == 1); + assert(spine_permits_destroy(&startup) == 0); + assert(spine_permits_available(&available_threads) == 1); + assert(db_pool_local[0].free && spine_permits_available(&available_scripts) == 2); +} + +/* A real bounded PHP-server producer: consume command lines and send numeric + * samples. The production PHP transport and poll_host do all interpretation. */ +static void send_output_sample(int fd, const char *result, size_t length) { + size_t sent = 0; + while (sent < length) { + ssize_t count = write(fd, result + sent, length - sent); + if (count < 0 && errno == EINTR) continue; + assert(count > 0); + sent += (size_t)count; + } +} + +static pid_t start_output_server(php_t *server, int payload) { + int requests[2]; + int responses[2]; + assert(pipe(requests) == 0 && pipe(responses) == 0); + fflush(NULL); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + alarm(180); + assert(close(requests[1]) == 0 && close(responses[0]) == 0); + FILE *input = fdopen(requests[0], "r"); + assert(input != NULL); + char command[BUFSIZE]; + char result[512]; + assert(payload >= 3 && payload < (int)sizeof(result)); + memset(result, '0', payload - 3); + memcpy(result + payload - 3, "123\n", 4); + while (fgets(command, sizeof(command), input) != NULL) { + send_output_sample(responses[1], result, (size_t)payload + 1); + } + assert(!ferror(input) && fclose(input) == 0 && close(responses[1]) == 0); + _exit(0); + } + assert(close(requests[0]) == 0 && close(responses[1]) == 0); + server->php_state = PHP_READY; + server->php_pid = child; + server->php_write_fd = requests[1]; + server->php_read_fd = responses[0]; + return child; +} + +static void reset_output_work(poller_thread_t *work, int rows, int partitions) { + memset(work, 0, sizeof(*work)); + work->host_id = 901; + work->host_threads = partitions; + work->host_thread = partitions; + work->host_data_ids = rows / partitions; + work->host_time_double = get_time_as_double(); + STRNCOPY(work->host_time, "1791244800"); + set.exit.exit_code = EXIT_SUCCESS; +} + +static void assert_sample(MYSQL *mysql, const char *table, int id, const char *expected) { + char query[BUFSIZE]; + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM %s WHERE local_data_id=%i AND output=%s", table, id, expected); + assert(output_count(mysql, query) == 1); +} + +static void assert_sample_at(MYSQL *mysql, const char *table, int id, + const char *timestamp, const char *expected) { + char query[BUFSIZE]; + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM %s WHERE local_data_id=%i AND time=FROM_UNIXTIME(%s) AND output='%s'", table, id, timestamp, expected); + assert(output_count(mysql, query) == 1); +} + +static void test_output_recollection(MYSQL *mysql) { + poller_thread_t **previous_details = details; + char query[BUFSIZE]; + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_item WHERE host_id=901")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output WHERE local_data_id BETWEEN 930001 AND 939999")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost WHERE local_data_id BETWEEN 930001 AND 939999")); + spine_snprintf(query, sizeof(query), "INSERT INTO poller_item(local_data_id,host_id,poller_id,action,arg1,rrd_name,snmp_port,rrd_step,rrd_next_step) VALUES(930001,901,1,%i,'/usr/bin/printf 123','owned',161,300,0),(930002,901,1,%i,'/usr/bin/printf 123','owned',162,300,0)", POLLER_ACTION_SCRIPT, POLLER_ACTION_SCRIPT); + assert(db_insert(mysql, LOCAL, query)); + assert(db_insert(mysql, LOCAL, "INSERT INTO poller_output(local_data_id,rrd_name,time,output) VALUES(930001,'owned',FROM_UNIXTIME(1791244800),'old'),(930002,'owned',FROM_UNIXTIME(1791244800),'old')")); + assert(db_insert(mysql, LOCAL, "INSERT INTO poller_output_boost(local_data_id,rrd_name,time,output) VALUES(930001,'owned',FROM_UNIXTIME(1791244800),'old'),(930002,'owned',FROM_UNIXTIME(1791244800),'old')")); + assert(db_insert(mysql, LOCAL, "CREATE TRIGGER spine_owned_output_reject BEFORE INSERT ON poller_output FOR EACH ROW BEGIN IF NEW.local_data_id=930002 THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='owned output rejection'; END IF; END")); + poller_thread_t *work = calloc(1, sizeof(*work)); + assert(work != NULL); + reset_output_work(work, 2, 1); + poller_thread_t **owned_details = calloc(1, sizeof(*owned_details)); + assert(owned_details != NULL); + owned_details[0] = work; + details = owned_details; + run_output_partition(work, 1); + assert(set.exit.exit_code == EXIT_FAILURE && work->output_failed && !work->complete); + assert(work->threads_complete == 1); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=901 AND rrd_next_step=0") == 2); + assert_sample_at(mysql, "poller_output", 930001, "1791244800", "123"); + assert_sample_at(mysql, "poller_output", 930002, "1791244800", "old"); + assert_sample_at(mysql, "poller_output_boost", 930001, "1791244800", "123"); + assert_sample_at(mysql, "poller_output_boost", 930002, "1791244800", "123"); + assert(db_insert(mysql, LOCAL, "DROP TRIGGER spine_owned_output_reject")); + assert(db_insert(mysql, LOCAL, "UPDATE poller_item SET arg1='/usr/bin/printf 456' WHERE host_id=901")); + /* A later poll recollects current values, rather than replaying failed + * samples. Supply distinct work timestamps without depending on the clock. */ + reset_output_work(work, 2, 1); + STRNCOPY(work->host_time, "1791244805"); + run_output_partition(work, 1); + assert(set.exit.exit_code == EXIT_SUCCESS && !work->output_failed && work->complete); + assert(work->threads_complete == 1); + for (int id = 930001; id <= 930002; id++) { + assert_sample_at(mysql, "poller_output", id, "1791244805", "456"); + assert_sample_at(mysql, "poller_output_boost", id, "1791244805", "456"); + assert_sample_at(mysql, "poller_output_boost", id, "1791244800", "123"); + } + assert_sample_at(mysql, "poller_output", 930001, "1791244800", "123"); + assert_sample_at(mysql, "poller_output", 930002, "1791244800", "old"); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=901 AND rrd_next_step=295") == 2); + assert(output_count(mysql, "SELECT COUNT(*) FROM host_errors WHERE host_id=901") == 0); + assert_sample(mysql, "poller_output", 930000, "'unchanged'"); + assert_sample(mysql, "poller_output_boost", 930000, "'unchanged'"); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id BETWEEN 930000 AND 939999") == 5); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id BETWEEN 930000 AND 939999") == 5); + details = previous_details; + free(owned_details); + free(work); + puts("production output recollection retains incomplete historical samples passed"); +} + +typedef struct { + poller_thread_t *work; + int finished_fd; +} controlled_output_worker_t; + +static void output_worker_finished(void *arg) { + const controlled_output_worker_t *worker = arg; + /* This outer handler runs after production child cleanup frees instructions + * and releases its permit, even though child exits through pthread_exit. */ + send_output_sample(worker->finished_fd, "F", 1); +} + +static void *controlled_output_child(void *arg) { + const controlled_output_worker_t *worker = arg; + pthread_cleanup_push(output_worker_finished, arg); + child(worker->work); + pthread_cleanup_pop(1); + return NULL; +} + +static void output_descriptor_flags(int fd) { + int flags = fcntl(fd, F_GETFD); + assert(flags >= 0 && fcntl(fd, F_SETFD, flags | FD_CLOEXEC) == 0); + flags = fcntl(fd, F_GETFL); + assert(flags >= 0 && fcntl(fd, F_SETFL, flags | O_NONBLOCK) == 0); +} + +static void await_output_byte(int fd, char expected, double deadline) { + assert(spine_wait_readable(fd, deadline) == 1); + char value; + ssize_t count; + do { count = read(fd, &value, 1); } while (count < 0 && errno == EINTR); + assert(count == 1 && value == expected); +} + +static void close_output_signal(int ready, int release, const int finished[2], const char *ready_path, const char *release_path) { + assert(close(ready) == 0 && close(release) == 0); + assert(close(finished[0]) == 0 && close(finished[1]) == 0); + assert(unlink(ready_path) == 0 && unlink(release_path) == 0); +} + +static void test_concurrent_output_failure(MYSQL *mysql) { + pool_t *previous_pool = db_pool_local; + poller_thread_t **previous_details = details; + int previous_threads = set.poller.threads; + int previous_timeout = set.php.script_timeout; + assert(previous_threads == 1 && db_pool_local[0].free); + assert(spine_permits_available(&available_threads) == 1); + assert(spine_permits_available(&available_scripts) == 2); + assert(spine_permits_destroy(&available_threads) == 0); + assert(spine_permits_init(&available_threads, 2) == 0); + set.poller.threads = 2; + set.php.script_timeout = 30; + db_pool_local = calloc(2, sizeof(*db_pool_local)); + assert(db_pool_local != NULL); + db_create_connection_pool(LOCAL); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_item WHERE host_id=901")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output WHERE local_data_id BETWEEN 930001 AND 939999")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost WHERE local_data_id BETWEEN 930001 AND 939999")); + char directory[] = "/tmp/spine-output-overlap-XXXXXX"; + assert(mkdtemp(directory) != NULL); + struct stat directory_status; + assert(lstat(directory, &directory_status) == 0); + assert(S_ISDIR(directory_status.st_mode)); + assert((directory_status.st_mode & 0777) == 0700 && directory_status.st_uid == geteuid()); + char ready_paths[2][SMALL_BUFSIZE]; + char release_paths[2][SMALL_BUFSIZE]; + int ready[2]; + int release[2]; + int finished[2][2]; + char query[LRG_BUFSIZE]; + for (int index = 0; index < 2; index++) { + spine_snprintf(ready_paths[index], sizeof(ready_paths[index]), "%s/ready%i", directory, index); + spine_snprintf(release_paths[index], sizeof(release_paths[index]), "%s/release%i", directory, index); + assert(mkfifo(ready_paths[index], 0600) == 0 && mkfifo(release_paths[index], 0600) == 0); + ready[index] = open(ready_paths[index], O_RDWR | O_NONBLOCK); + release[index] = open(release_paths[index], O_RDWR | O_NONBLOCK); + assert(ready[index] >= 0 && release[index] >= 0 && pipe(finished[index]) == 0); + output_descriptor_flags(ready[index]); + output_descriptor_flags(release[index]); + output_descriptor_flags(finished[index][0]); + output_descriptor_flags(finished[index][1]); + /* Quote-free SQL string; the real shell producer signals selection then + * blocks on its own gate before publishing a sample to Spine. */ + spine_snprintf(query, sizeof(query), "INSERT INTO poller_item(local_data_id,host_id,poller_id,action,arg1,rrd_name,snmp_port,rrd_step,rrd_next_step) VALUES(%i,901,1,%i,'/usr/bin/printf R > %s; read token < %s; /usr/bin/printf 123','owned',%i,300,0)", 930001 + index, POLLER_ACTION_SCRIPT, ready_paths[index], release_paths[index], 161 + index); + assert(db_insert(mysql, LOCAL, query)); + } + assert(db_insert(mysql, LOCAL, "CREATE TRIGGER spine_owned_output_reject BEFORE INSERT ON poller_output FOR EACH ROW BEGIN IF NEW.local_data_id=930002 THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='owned output rejection'; END IF; END")); + poller_thread_t *aggregate = calloc(1, sizeof(*aggregate)); + assert(aggregate != NULL); + reset_output_work(aggregate, 2, 2); + poller_thread_t **owned_details = calloc(1, sizeof(*owned_details)); + assert(owned_details != NULL); + owned_details[0] = aggregate; + details = owned_details; + spine_permits_t startup; + assert(spine_permits_init(&startup, 2) == 0); + controlled_output_worker_t workers[2]; + pthread_t threads[2]; + for (int index = 0; index < 2; index++) { + workers[index].work = malloc(sizeof(*workers[index].work)); + assert(workers[index].work != NULL); + thread_mutex_lock(LOCK_THDET); + *workers[index].work = *aggregate; + thread_mutex_unlock(LOCK_THDET); + workers[index].work->host_thread = index + 1; + workers[index].work->thread_init_sem = &startup; + workers[index].finished_fd = finished[index][1]; + assert(spine_permits_try_acquire(&startup) == 0); + assert(spine_permits_try_acquire(&available_threads) == 0); + assert(pthread_create(&threads[index], NULL, controlled_output_child, &workers[index]) == 0); + } + double deadline = spine_monotonic_time() + 10; + await_output_byte(ready[0], 'R', deadline); + await_output_byte(ready[1], 'R', deadline); + assert(spine_permits_available(&startup) == 2); + assert(spine_permits_available(&available_threads) == 0); + assert(spine_permits_available(&available_scripts) == 0); + thread_mutex_lock(LOCK_POOL); + assert(!db_pool_local[0].free && !db_pool_local[1].free); + thread_mutex_unlock(LOCK_POOL); + /* Both real workers have selected their own partition and hold resources. + * Allow only the rejected partition to finish first. */ + send_output_sample(release[1], "resume\n", 7); + await_output_byte(finished[1][0], 'F', spine_monotonic_time() + 10); + assert(pthread_join(threads[1], NULL) == 0); + thread_mutex_lock(LOCK_THDET); + assert(set.exit.exit_code == EXIT_FAILURE && aggregate->output_failed); + assert(aggregate->threads_complete == 1 && !aggregate->complete); + thread_mutex_unlock(LOCK_THDET); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=901 AND rrd_next_step=0") == 2); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id IN(930001,930002)") == 0); + assert_sample_at(mysql, "poller_output_boost", 930002, "1791244800", "123"); + assert(spine_permits_available(&available_threads) == 1); + assert(spine_permits_available(&available_scripts) == 1); + thread_mutex_lock(LOCK_POOL); + assert(db_pool_local[0].free + db_pool_local[1].free == 1); + thread_mutex_unlock(LOCK_POOL); + send_output_sample(release[0], "resume\n", 7); + await_output_byte(finished[0][0], 'F', spine_monotonic_time() + 10); + assert(pthread_join(threads[0], NULL) == 0); + assert(set.exit.exit_code == EXIT_FAILURE && aggregate->output_failed && !aggregate->complete); + assert(aggregate->threads_complete == 2); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=901 AND rrd_next_step=0") == 2); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=930002") == 0); + assert_sample_at(mysql, "poller_output", 930001, "1791244800", "123"); + assert_sample_at(mysql, "poller_output_boost", 930001, "1791244800", "123"); + assert_sample_at(mysql, "poller_output_boost", 930002, "1791244800", "123"); + assert_sample(mysql, "poller_output", 930000, "'unchanged'"); + assert_sample(mysql, "poller_output_boost", 930000, "'unchanged'"); + assert(output_count(mysql, "SELECT COUNT(*) FROM host_errors WHERE host_id=901") == 0); + assert(spine_permits_available(&available_threads) == 2); + assert(spine_permits_available(&available_scripts) == 2); + assert(db_pool_local[0].free && db_pool_local[1].free); + assert(db_insert(mysql, LOCAL, "DROP TRIGGER spine_owned_output_reject")); + for (int index = 0; index < 2; index++) { + close_output_signal(ready[index], release[index], finished[index], ready_paths[index], release_paths[index]); + } + assert(rmdir(directory) == 0); + assert(spine_permits_destroy(&startup) == 0); + assert(spine_permits_destroy(&available_threads) == 0); + assert(spine_permits_init(&available_threads, 1) == 0); + db_close_connection_pool(LOCAL); + db_pool_local = previous_pool; + details = previous_details; + free(owned_details); + free(aggregate); + set.poller.threads = previous_threads; + set.php.script_timeout = previous_timeout; + puts("production simultaneous output failure ordering regressions passed"); +} + +static void assert_remote_resources_and_local_output(MYSQL *mysql) { + assert(db_pool_local[0].free && db_pool_remote[0].free); + assert(spine_permits_available(&available_threads) == 1); + assert(spine_permits_available(&available_scripts) == 2); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id BETWEEN 930001 AND 939999") == 0); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id BETWEEN 930001 AND 939999") == 0); + assert_sample(mysql, "poller_output", 930000, "'unchanged'"); + assert_sample(mysql, "poller_output_boost", 930000, "'unchanged'"); + assert(output_count(mysql, "SELECT COUNT(*) FROM host_errors WHERE host_id=901") == 0); +} + +static void assert_remote_samples(MYSQL *mysql, const char *timestamp, const char *sample) { + for (int id = 930001; id <= 930002; id++) { + assert_sample_at(mysql, "spine_owned_output_remote.poller_output", id, timestamp, sample); + assert_sample_at(mysql, "spine_owned_output_remote.poller_output_boost", id, timestamp, sample); + } +} + +static void test_remote_output_destination(MYSQL *mysql) { + pool_t *previous_remote_pool = db_pool_remote; + poller_thread_t **previous_details = details; + spine_database_config_t previous_remote_database = set.remote_database; + int previous_id = set.poller.poller_id; + int previous_mode = set.poller.mode; + int previous_exit = set.exit.exit_code; + assert(set.poller.threads == 1 && db_pool_local[0].free); + assert(output_count(mysql, "SELECT COUNT(*) FROM information_schema.SCHEMATA WHERE SCHEMA_NAME='spine_owned_output_remote'") == 0); + assert(db_insert(mysql, LOCAL, "CREATE DATABASE spine_owned_output_remote")); + /* Clone the actual fixture tables, preserving their keys, column defaults, + * and different engines rather than inventing a remote schema. */ + assert(db_insert(mysql, LOCAL, "CREATE TABLE spine_owned_output_remote.poller_output LIKE poller_output")); + assert(db_insert(mysql, LOCAL, "CREATE TABLE spine_owned_output_remote.poller_output_boost LIKE poller_output_boost")); + assert(output_count(mysql, "SELECT COUNT(*) FROM information_schema.TABLES WHERE TABLE_SCHEMA='spine_owned_output_remote' AND TABLE_NAME='poller_output' AND ENGINE='MEMORY'") == 1); + assert(output_count(mysql, "SELECT COUNT(*) FROM information_schema.TABLES WHERE TABLE_SCHEMA='spine_owned_output_remote' AND TABLE_NAME='poller_output_boost' AND ENGINE='InnoDB'") == 1); + set.remote_database = set.database; + STRNCOPY(set.remote_database.database, "spine_owned_output_remote"); + set.poller.poller_id = 2; + set.poller.mode = REMOTE_ONLINE; + db_pool_remote = calloc(1, sizeof(*db_pool_remote)); + assert(db_pool_remote != NULL); + db_create_connection_pool(REMOTE); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_item WHERE host_id=901")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output WHERE local_data_id BETWEEN 930001 AND 939999")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost WHERE local_data_id BETWEEN 930001 AND 939999")); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_reindex WHERE host_id=901") == 0); + char query[BUFSIZE]; + spine_snprintf(query, sizeof(query), "INSERT INTO poller_item(local_data_id,host_id,poller_id,action,arg1,rrd_name,snmp_port,rrd_step,rrd_next_step) VALUES(930001,901,2,%i,'/usr/bin/printf 123','owned',161,300,0),(930002,901,2,%i,'/usr/bin/printf 123','owned',162,300,0)", POLLER_ACTION_SCRIPT, POLLER_ACTION_SCRIPT); + assert(db_insert(mysql, LOCAL, query)); + poller_thread_t *work = calloc(1, sizeof(*work)); + assert(work != NULL); + reset_output_work(work, 2, 1); + poller_thread_t **owned_details = calloc(1, sizeof(*owned_details)); + assert(owned_details != NULL); + owned_details[0] = work; + details = owned_details; + run_output_partition(work, 1); + assert(set.exit.exit_code == EXIT_SUCCESS && work->complete && !work->output_failed); + assert(work->threads_complete == 1); + assert_remote_samples(mysql, "1791244800", "123"); + assert_remote_resources_and_local_output(mysql); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=901 AND poller_id=2 AND rrd_next_step=295") == 2); + assert(db_insert(mysql, LOCAL, "UPDATE poller_item SET rrd_next_step=0,arg1='/usr/bin/printf 456' WHERE host_id=901 AND poller_id=2")); + assert(db_insert(mysql, LOCAL, "CREATE TRIGGER spine_owned_output_remote.spine_owned_output_reject BEFORE INSERT ON spine_owned_output_remote.poller_output FOR EACH ROW BEGIN IF NEW.local_data_id=930002 THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='owned remote output rejection'; END IF; END")); + reset_output_work(work, 2, 1); + STRNCOPY(work->host_time, "1791244805"); + run_output_partition(work, 1); + assert(set.exit.exit_code == EXIT_FAILURE && work->output_failed && !work->complete); + assert(work->threads_complete == 1); + assert_remote_resources_and_local_output(mysql); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=901 AND poller_id=2 AND rrd_next_step=0") == 2); + assert_remote_samples(mysql, "1791244800", "123"); + assert_sample_at(mysql, "spine_owned_output_remote.poller_output", 930001, "1791244805", "456"); + assert(output_count(mysql, "SELECT COUNT(*) FROM spine_owned_output_remote.poller_output WHERE local_data_id=930002 AND time=FROM_UNIXTIME(1791244805)") == 0); + assert_sample_at(mysql, "spine_owned_output_remote.poller_output_boost", 930001, "1791244805", "456"); + assert_sample_at(mysql, "spine_owned_output_remote.poller_output_boost", 930002, "1791244805", "456"); + assert(db_insert(mysql, LOCAL, "DROP TRIGGER spine_owned_output_remote.spine_owned_output_reject")); + assert(db_insert(mysql, LOCAL, "UPDATE poller_item SET arg1='/usr/bin/printf 789' WHERE host_id=901 AND poller_id=2")); + reset_output_work(work, 2, 1); + STRNCOPY(work->host_time, "1791244810"); + run_output_partition(work, 1); + assert(set.exit.exit_code == EXIT_SUCCESS && !work->output_failed && work->complete); + assert(work->threads_complete == 1); + assert_remote_resources_and_local_output(mysql); + assert_remote_samples(mysql, "1791244810", "789"); + assert_remote_samples(mysql, "1791244800", "123"); + assert_sample_at(mysql, "spine_owned_output_remote.poller_output", 930001, "1791244805", "456"); + assert(output_count(mysql, "SELECT COUNT(*) FROM spine_owned_output_remote.poller_output WHERE local_data_id=930002 AND time=FROM_UNIXTIME(1791244805)") == 0); + assert_sample_at(mysql, "spine_owned_output_remote.poller_output_boost", 930001, "1791244805", "456"); + assert_sample_at(mysql, "spine_owned_output_remote.poller_output_boost", 930002, "1791244805", "456"); + assert(output_count(mysql, "SELECT COUNT(*) FROM spine_owned_output_remote.poller_output") == 5); + assert(output_count(mysql, "SELECT COUNT(*) FROM spine_owned_output_remote.poller_output_boost") == 6); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=901 AND poller_id=2 AND rrd_next_step=295") == 2); + db_close_connection_pool(REMOTE); + db_pool_remote = previous_remote_pool; + assert(db_insert(mysql, LOCAL, "DROP DATABASE spine_owned_output_remote")); + assert(output_count(mysql, "SELECT COUNT(*) FROM information_schema.SCHEMATA WHERE SCHEMA_NAME='spine_owned_output_remote'") == 0); + details = previous_details; + free(owned_details); + free(work); + set.remote_database = previous_remote_database; + set.poller.poller_id = previous_id; + set.poller.mode = previous_mode; + set.exit.exit_code = previous_exit; + puts("production remote output destination failure and recollection regressions passed"); +} + +static void run_output_failure_scenario(MYSQL *mysql, int scenario, int payload) { + poller_thread_t **previous_details = details; + bool boundary = scenario >= 2; + bool boost_failure = (scenario & 1) != 0; + int rows = boundary ? MAX_MYSQL_BUF_SIZE / payload + 4 : 2; + assert(rows < 9999); + int partitions = scenario == 0 ? 2 : 1; + char query[LRG_BUFSIZE]; + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_item WHERE host_id=901")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output WHERE local_data_id BETWEEN 930001 AND 939999")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost WHERE local_data_id BETWEEN 930001 AND 939999")); + for (int index = 1; index <= rows; index++) { + spine_snprintf(query, sizeof(query), "INSERT INTO poller_item(local_data_id,host_id,poller_id,action,arg1,rrd_name,snmp_port,rrd_step,rrd_next_step) VALUES(%i,901,1,%i,'%s','owned',%i,300,0)", 930000 + index, boundary ? POLLER_ACTION_PHP_SCRIPT_SERVER : POLLER_ACTION_SCRIPT, boundary ? "owned numeric request" : "/usr/bin/printf 123", 160 + index); + assert(db_insert(mysql, LOCAL, query)); + spine_snprintf(query, sizeof(query), "INSERT INTO poller_output(local_data_id,rrd_name,time,output) VALUES(%i,'owned',FROM_UNIXTIME(1791244800),'old')", 930000 + index); + assert(db_insert(mysql, LOCAL, query)); + spine_snprintf(query, sizeof(query), "INSERT INTO poller_output_boost(local_data_id,rrd_name,time,output) VALUES(%i,'owned',FROM_UNIXTIME(1791244800),'old')", 930000 + index); + assert(db_insert(mysql, LOCAL, query)); + } + spine_snprintf(query, sizeof(query), "CREATE TRIGGER spine_owned_output_reject BEFORE INSERT ON %s FOR EACH ROW BEGIN IF NEW.local_data_id=930002 THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='owned output rejection'; END IF; END", boost_failure ? "poller_output_boost" : "poller_output"); + assert(db_insert(mysql, LOCAL, query)); + poller_thread_t *work = calloc(1, sizeof(*work)); + assert(work != NULL); + reset_output_work(work, rows, partitions); + poller_thread_t **owned_details = calloc(1, sizeof(*owned_details)); + assert(owned_details != NULL); + owned_details[0] = work; + details = owned_details; + if (partitions == 2) { + /* Failure finishes before the other partition even selects its rows. */ + run_output_partition(work, 2); + fprintf(stderr, "output failure partition: exit=%i threads=%i complete=%i failed=%i\n", set.exit.exit_code, work->threads_complete, work->complete, work->output_failed); + assert(work->threads_complete == 1 && !work->complete); + assert(set.exit.exit_code == EXIT_FAILURE); + } + run_output_partition(work, 1); + fprintf(stderr, "output persistence scenario=%i boundary=%i rows=%i exit=%i completed=%i\n", scenario, boundary, rows, set.exit.exit_code, work->complete); + assert(set.exit.exit_code == EXIT_FAILURE); + assert(work->output_failed && !work->complete && work->threads_complete == partitions); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_item WHERE host_id=901 AND rrd_next_step=0"); + assert(output_count(mysql, query) == (unsigned long long)rows); + assert(output_count(mysql, "SELECT COUNT(*) FROM host_errors WHERE host_id=901") == 0); + assert_sample(mysql, "poller_output", 930000, "'unchanged'"); + assert_sample(mysql, "poller_output_boost", 930000, "'unchanged'"); + const char *failed_table = boost_failure ? "poller_output_boost" : "poller_output"; + const char *other_table = boost_failure ? "poller_output" : "poller_output_boost"; + assert_sample(mysql, failed_table, 930002, "'old'"); + char expected[100]; + spine_snprintf(expected, sizeof(expected), "CONCAT(REPEAT('0',%i),'123')", boundary ? payload - 3 : 0); + assert_sample(mysql, other_table, 930002, expected); + if (boundary) { + /* Sample bytes alone exceed the configured batch limit, so the + * failure is in an early flush and later writes still execute. */ + assert((size_t)rows * (size_t)payload > MAX_MYSQL_BUF_SIZE); + assert_sample(mysql, failed_table, 930000 + rows, expected); + assert_sample(mysql, "poller_output", 930001, expected); + } + assert(db_insert(mysql, LOCAL, "DROP TRIGGER spine_owned_output_reject")); + /* This controlled retry reuses its timestamp and resets completion/exit + * state. It proves same-key upsert repair, including partial MEMORY rows; + * a later CLI invocation recollects data with the current timestamp. */ + reset_output_work(work, rows, partitions); + if (partitions == 2) run_output_partition(work, 2); + run_output_partition(work, 1); + assert(set.exit.exit_code == EXIT_SUCCESS && !work->output_failed && work->complete); + assert(work->threads_complete == partitions); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_output WHERE local_data_id BETWEEN 930001 AND %i AND output=%s", 930000 + rows, expected); + assert(output_count(mysql, query) == (unsigned long long)rows); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id BETWEEN 930001 AND %i AND output=%s", 930000 + rows, expected); + assert(output_count(mysql, query) == (unsigned long long)rows); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=901 AND rrd_next_step=295") == (unsigned long long)rows); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id BETWEEN 930000 AND 939999") == (unsigned long long)rows + 1); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id BETWEEN 930000 AND 939999") == (unsigned long long)rows + 1); + details = previous_details; + free(owned_details); + free(work); +} + +void test_output_write_contracts(MYSQL *mysql) { + config_t previous = set; + pool_t *previous_pool = db_pool_local; + poller_thread_t **previous_details = details; + php_t *previous_servers = php_processes; + int *previous_debug = debug_devices; + static int owned_debug[100]; + debug_devices = owned_debug; + assert_output_engine(mysql, "poller_output", "MEMORY"); + assert_output_engine(mysql, "poller_output_boost", "InnoDB"); + assert(output_count(mysql, "SELECT COUNT(*) FROM host WHERE id=901") == 0); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=901 OR local_data_id BETWEEN 930000 AND 939999") == 0); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id BETWEEN 930000 AND 939999") == 0); + assert(output_count(mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id BETWEEN 930000 AND 939999") == 0); + set.poller.threads = 1; + set.poller.poller_id = 1; + set.poller.mode = REMOTE_OFFLINE; + set.poller.active_profiles = 2; + set.poller.poller_interval = 5; + set.poller.SQL_readonly = FALSE; + set.availability.ping_only = FALSE; + set.snmp.total_snmp_ports = 0; + set.snmp.mibs = FALSE; + set.boost.boost_enabled = TRUE; + set.boost.boost_redirect = TRUE; + set.logging.spine_log_level = 0; + set.logging.log_destination = 0; + set.php.script_timeout = 5; + set.php.php_servers = 1; + set.php.php_current_server = 0; + set.php.php_initialized = TRUE; + set.php.php_required = TRUE; + db_pool_local = calloc(1, sizeof(*db_pool_local)); + assert(db_pool_local != NULL); + db_create_connection_pool(LOCAL); + assert(spine_permits_init(&available_threads, 1) == 0); + assert(spine_permits_init(&available_scripts, 2) == 0); + php_t server = {0}; + int payload = RESULTS_BUFFER - 2 < 400 ? RESULTS_BUFFER - 2 : 400; + assert(payload >= 3); + pid_t producer = start_output_server(&server, payload); + php_processes = &server; + assert(db_insert(mysql, LOCAL, "INSERT INTO host(id,hostname,availability_method,snmp_version,status_fail_date,status_rec_date) VALUES(901,'127.0.0.1',0,0,'2026-10-06 00:00:00','2026-10-06 00:00:00')")); + assert(db_insert(mysql, LOCAL, "INSERT INTO poller_output(local_data_id,rrd_name,time,output) VALUES(930000,'sentinel',FROM_UNIXTIME(1791244800),'unchanged')")); + assert(db_insert(mysql, LOCAL, "INSERT INTO poller_output_boost(local_data_id,rrd_name,time,output) VALUES(930000,'sentinel',FROM_UNIXTIME(1791244800),'unchanged')")); + for (int scenario = 0; scenario < 4; scenario++) { + run_output_failure_scenario(mysql, scenario, payload); + assert(details == previous_details); + } + test_output_recollection(mysql); + test_concurrent_output_failure(mysql); + test_remote_output_destination(mysql); + assert(close(server.php_write_fd) == 0); + int status; + assert(waitpid(producer, &status, 0) == producer); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); + assert(close(server.php_read_fd) == 0); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_item WHERE host_id=901")); + assert(db_insert(mysql, LOCAL, "DELETE FROM host WHERE id=901")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output WHERE local_data_id BETWEEN 930000 AND 939999")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost WHERE local_data_id BETWEEN 930000 AND 939999")); + assert(spine_permits_destroy(&available_scripts) == 0); + assert(spine_permits_destroy(&available_threads) == 0); + db_close_connection_pool(LOCAL); + db_pool_local = previous_pool; + details = previous_details; + php_processes = previous_servers; + debug_devices = previous_debug; + set = previous; + puts("production output write failure and retry regressions passed"); +} diff --git a/tests/test_spine_regressions.c b/tests/test_spine_regressions.c new file mode 100644 index 0000000..2f59b71 --- /dev/null +++ b/tests/test_spine_regressions.c @@ -0,0 +1,2523 @@ +/* + * Copyright (C) 2004-2026 The Cacti Group + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License + * for more details. + */ +#include "common.h" +#include "spine.h" +#include +#include +#include + +extern int spine_program_main(int argc, char **argv); +extern void test_output_write_contracts(MYSQL *mysql); +extern void test_additional_contracts(void); +extern void test_cli_alias_contracts(void); +extern int run_script_stream_fixture(const char *scenario); +extern void test_script_stream_contracts(void); +extern void test_additional_database_contracts(MYSQL *mysql); +extern void test_settings_write_contracts(MYSQL *mysql); +extern void test_additional_reindex_contracts(MYSQL *mysql); +extern void test_additional_snmp_session_boundaries(void); +extern void test_snmpv3_agent_contracts(void); + +static void test_copy_bounds(void) { + struct { char text[8]; unsigned char guard; } output; + memset(&output, 0xa5, sizeof output); + strncopy(output.text, "12345678", sizeof(output.text)); + assert(strcmp(output.text, "1234567") == 0); + assert(output.guard == 0xa5); + strncopy(output.text, "abcdefghijk", sizeof(output.text)); + assert(strcmp(output.text, "abcdefg") == 0); + assert(output.guard == 0xa5); + strncopy(output.text, "unchanged", 0); + assert(strcmp(output.text, "abcdefg") == 0); + strncopy(output.text, "x", 1); + assert(output.text[0] == '\0'); + strncopy(output.text, "", sizeof(output.text)); + assert(output.text[0] == '\0'); +} + +static void test_string_conversions(void) { + char empty[] = ""; + char one[] = "x"; + char word[] = "abcd"; + assert(strcmp(reverse(empty), "") == 0); + assert(strcmp(reverse(one), "x") == 0); + assert(strcmp(reverse(word), "dcba") == 0); + char hex[] = "\"FF ff\t\""; + char maximum[] = "FFFFFFFFFFFFFFFF"; + char overflow[] = "10000000000000000"; + char invalid[] = "not hex"; + unsigned long long value = 0; + assert(hex2dec(hex, &value) && value == 65535); + assert(hex2dec(maximum, &value) && value == ULLONG_MAX); + assert(!hex2dec(overflow, &value)); + assert(!hex2dec(invalid, &value) && !hex2dec(NULL, &value)); + char input[BUFSIZE * 2]; + memset(input, '\\', sizeof(input) - 1); + input[sizeof(input) - 1] = '\0'; + char *escaped = add_slashes(input); + assert(strlen(escaped) == 2 * (sizeof(input) - 1)); + for (size_t i = 0; escaped[i] != '\0'; i++) assert(escaped[i] == '\\'); + free(escaped); + escaped = add_slashes("a\\b"); + assert(strcmp(escaped, "a\\\\b") == 0); + free(escaped); + escaped = add_slashes(""); + assert(escaped[0] == '\0'); + free(escaped); + char decorated[] = "text +12.5 Bytes"; + assert(strcmp(strip_alpha(decorated), "12.5") == 0); + char negative[] = "text -12.5 Bytes"; + assert(strcmp(strip_alpha(negative), "-12.5") == 0); + char nonnumeric[] = "no sample"; + assert(strcmp(strip_alpha(nonnumeric), "") == 0); +} + +static void test_result_count_range(void) { + assert(spine_count_to_int(0) == 0); + assert(spine_count_to_int(INT_MAX) == INT_MAX); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + spine_count_to_int((unsigned long long)INT_MAX + 1); + _exit(99); + } + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == EXIT_FAILURE); +} + +static void test_cli_case(const char *flag, const char *input, int expected, const char *error) { + int errors[2]; + assert(pipe(errors) == 0); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + assert(close(errors[0]) == 0); + assert(dup2(errors[1], STDERR_FILENO) == STDERR_FILENO); + assert(close(errors[1]) == 0); + char program[] = "spine"; + char option[32]; + strncopy(option, flag, sizeof(option)); + char version[] = "--version"; + char value[64]; + strncopy(value, input, sizeof(value)); + char *args[] = {program, option, value, version, NULL}; + spine_program_main(4, args); + _exit(99); + } + assert(close(errors[1]) == 0); + char message[512]; + ssize_t received = read(errors[0], message, sizeof(message) - 1); + if (received < 0 || received >= (ssize_t)sizeof(message)) abort(); + message[received] = '\0'; + assert(close(errors[0]) == 0); + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == expected); + if (expected == EXIT_FAILURE) assert(strstr(message, error) != NULL); + else assert(message[0] == '\0'); +} + +static void test_cli_option_shape(void) { + test_cli_case("--option", "missing-colon", EXIT_FAILURE, "ERROR: -O requires setting:value"); + test_cli_case("--option", ":value", EXIT_FAILURE, "ERROR: -O requires setting:value"); + test_cli_case("--option", "regression:", EXIT_SUCCESS, NULL); + test_cli_case("--option", "regression:value:colon", EXIT_SUCCESS, NULL); + test_cli_case("--mode", "online", EXIT_SUCCESS, NULL); + test_cli_case("--mode", "offline", EXIT_SUCCESS, NULL); + test_cli_case("--mode", "recovery", EXIT_SUCCESS, NULL); + test_cli_case("--mode", "invalid", EXIT_FAILURE, "ERROR: invalid polling mode 'invalid' specified"); + int errors[2]; + assert(pipe(errors) == 0); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + assert(close(errors[0]) == 0); + assert(dup2(errors[1], STDERR_FILENO) == STDERR_FILENO); + assert(close(errors[1]) == 0); + set.exit.exit_code = EXIT_SUCCESS; + for (int i = 0; i < 257; i++) set_option("regression", "value"); + _exit(99); + } + assert(close(errors[1]) == 0); + char message[512]; + ssize_t received = read(errors[0], message, sizeof(message) - 1); + if (received < 0 || received >= (ssize_t)sizeof(message)) abort(); + message[received] = '\0'; + assert(close(errors[0]) == 0); + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == EXIT_FAILURE); + assert(strstr(message, "Invalid or excessive command-line setting overrides") != NULL); +} + +static spine_permits_t test_permits; +static pthread_mutex_t permit_test_mutex = PTHREAD_MUTEX_INITIALIZER; +static int active_permit_workers; + +static void *exercise_permit(void *unused) { + (void)unused; + for (int iteration = 0; iteration < 100; iteration++) { + int status; + while ((status = spine_permits_try_acquire(&test_permits)) == EAGAIN) spine_sleep_usec(100); + assert(status == 0); + assert(pthread_mutex_lock(&permit_test_mutex) == 0); + assert(active_permit_workers == 0); + active_permit_workers++; + assert(pthread_mutex_unlock(&permit_test_mutex) == 0); + spine_sleep_usec(100); + assert(pthread_mutex_lock(&permit_test_mutex) == 0); + assert(active_permit_workers == 1); + active_permit_workers--; + assert(pthread_mutex_unlock(&permit_test_mutex) == 0); + assert(spine_permits_release(&test_permits) == 0); + } + return NULL; +} + +static void test_concurrent_permits(void) { + assert(spine_permits_init(&test_permits, -1) == EINVAL); + assert(spine_permits_init(&test_permits, INT_MAX) == 0); + assert(spine_permits_release(&test_permits) == EOVERFLOW); + assert(spine_permits_available(&test_permits) == INT_MAX); + assert(spine_permits_destroy(&test_permits) == 0); + assert(spine_permits_init(&test_permits, 1) == 0); + assert(spine_permits_try_acquire(&test_permits) == 0); + assert(spine_permits_try_acquire(&test_permits) == EAGAIN); + assert(spine_permits_available(&test_permits) == 0); + assert(spine_permits_release(&test_permits) == 0); + pthread_t workers[4]; + for (size_t i = 0; i < sizeof(workers) / sizeof(workers[0]); i++) { + assert(pthread_create(&workers[i], NULL, exercise_permit, NULL) == 0); + } + for (size_t i = 0; i < sizeof(workers) / sizeof(workers[0]); i++) assert(pthread_join(workers[i], NULL) == 0); + assert(spine_permits_available(&test_permits) == 1 && active_permit_workers == 0); + assert(spine_permits_destroy(&test_permits) == 0); +} + +static void test_database_escape(void) { + MYSQL *mysql = mysql_init(NULL); + assert(mysql != NULL); + char input[RESULTS_BUFFER]; + char output[RESULTS_BUFFER * 2 + 1]; + memset(input, 'x', sizeof(input) - 1); + input[sizeof(input) - 1] = '\0'; + db_escape(mysql, output, sizeof(output), input); + assert(strcmp(output, input) == 0); + memset(input, '\'', sizeof(input) - 1); + db_escape(mysql, output, sizeof(output), input); + assert(strlen(output) == 2 * (sizeof(input) - 1)); + for (size_t i = 0; i < sizeof(input) - 1; i++) { + assert(output[2 * i] == '\\' && output[2 * i + 1] == '\''); + } + unsigned char guarded[13]; + memset(guarded, 0xa5, sizeof(guarded)); + db_escape(mysql, (char *)guarded + 1, 11, input); + assert(guarded[0] == 0xa5 && guarded[12] == 0xa5); + assert(strlen((char *)guarded + 1) == 10); + char tiny[] = "abc"; + db_escape(mysql, tiny, 0, input); + assert(strcmp(tiny, "abc") == 0); + db_escape(mysql, tiny, -1, input); + assert(strcmp(tiny, "abc") == 0); + db_escape(mysql, tiny, 1, input); + assert(tiny[0] == '\0' && tiny[1] == 'b'); + db_escape(mysql, tiny, sizeof(tiny), NULL); + assert(tiny[1] == 'b'); + db_escape(mysql, NULL, 16, input); + mysql_close(mysql); +} + +static void test_database_addresses(void) { + db_address_t address; + db_address_init(&address, "localhost", true); + assert(strcmp(address.hostname, "localhost") == 0 && address.socket == NULL); + db_address_release(&address); + assert(address.storage == NULL && address.hostname == NULL && address.socket == NULL); + db_address_init(&address, "localhost:service", true); + assert(strcmp(address.hostname, "localhost") == 0 && strcmp(address.socket, "service") == 0); + db_address_release(&address); + db_address_init(&address, "remote:service", false); + assert(strcmp(address.hostname, "remote:service") == 0 && address.socket == NULL); + db_address_release(&address); + char directory[] = "spine-socket-test-XXXXXX"; + assert(mkdtemp(directory) != NULL); + db_address_init(&address, directory, true); + assert(strcmp(address.hostname, directory) == 0 && address.socket == NULL); + db_address_release(&address); + struct sockaddr_un socket_address = {0}; + socket_address.sun_family = AF_UNIX; + #ifdef __APPLE__ + socket_address.sun_len = sizeof(socket_address); + #endif + spine_snprintf(socket_address.sun_path, sizeof(socket_address.sun_path), "%s/db.sock", directory); + int fd = socket(AF_UNIX, SOCK_STREAM, 0); + assert(fd >= 0); + int bound = bind(fd, (struct sockaddr *)&socket_address, sizeof(socket_address)); + if (bound != 0) perror("bind regression socket"); + assert(bound == 0); + db_address_init(&address, socket_address.sun_path, true); + assert(address.hostname == NULL && strcmp(address.socket, socket_address.sun_path) == 0); + db_address_release(&address); + assert(close(fd) == 0); + assert(unlink(socket_address.sun_path) == 0); + assert(rmdir(directory) == 0); +} + +static void test_database_option_failure(void) { + MYSQL *mysql = mysql_init(NULL); + assert(mysql != NULL); + unsigned int timeout = 5; + db_set_option(mysql, MYSQL_OPT_CONNECT_TIMEOUT, &timeout, "test timeout"); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + db_set_option(mysql, (enum mysql_option)999999, NULL, "unsupported test option"); + _exit(0); + } + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == EXIT_FAILURE); + mysql_close(mysql); +} + +static void test_regex_storage(void) { + char first[] = "abc123xyz"; + char second[] = "x4y"; + char absent[] = "no digits"; + const char *match = regex_replace("[0-9][0-9]*", first); + assert(strcmp(match, "123") == 0); + /* The result remains valid across another function's stack use. */ + char stack[4096]; + memset(stack, 'z', sizeof stack); + assert(strcmp(match, "123") == 0); + assert(strcmp(regex_replace("[0-9][0-9]*", second), "4") == 0); + assert(regex_replace("[0-9][0-9]*", absent) == absent); + assert(regex_replace("[", first) == first); + char large[RESULTS_BUFFER + 1]; + memset(large, '9', sizeof large - 1); + large[sizeof large - 1] = '\0'; + assert(regex_replace("[0-9][0-9]*", large) == large); +} + +static pthread_mutex_t regex_test_mutex = PTHREAD_MUTEX_INITIALIZER; +static pthread_cond_t regex_test_ready = PTHREAD_COND_INITIALIZER; +static int regex_threads_ready; + +static void *test_regex_thread(void *argument) { + char *input = argument; + const char *match = regex_replace("[0-9][0-9]*", input); + assert(pthread_mutex_lock(®ex_test_mutex) == 0); + regex_threads_ready++; + assert(pthread_cond_broadcast(®ex_test_ready) == 0); + while (regex_threads_ready < 2) { + assert(pthread_cond_wait(®ex_test_ready, ®ex_test_mutex) == 0); + } + /* Both threads have produced a result; neither may overwrite the other. */ + assert(strcmp(match, input) == 0); + assert(pthread_mutex_unlock(®ex_test_mutex) == 0); + return NULL; +} + +static void test_regex_concurrency(void) { + char first[] = "123"; + char second[] = "4567"; + pthread_t threads[2]; + assert(pthread_create(&threads[0], NULL, test_regex_thread, first) == 0); + assert(pthread_create(&threads[1], NULL, test_regex_thread, second) == 0); + assert(pthread_join(threads[0], NULL) == 0); + assert(pthread_join(threads[1], NULL) == 0); +} + +static void test_formatting(void) { + char output[8]; + assert(spine_snprintf(output, sizeof output, "%s", "1234567") == 7); + assert(strcmp(output, "1234567") == 0); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + spine_snprintf(output, sizeof output, "%s", "12345678"); + _exit(0); + } + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == EXIT_FAILURE); +} + +static void test_secret_clear(void) { + unsigned char bytes[8]; + memset(bytes, 0xa5, sizeof bytes); + spine_clear_sensitive(bytes + 1, 6); + assert(bytes[0] == 0xa5 && bytes[7] == 0xa5); + for (size_t i = 1; i < 7; i++) assert(bytes[i] == 0); + spine_clear_sensitive(NULL, 0); +} + +static void test_log_boundary(void) { + char *message = malloc(LOGSIZE + 1); + assert(message != NULL); + memset(message, 'x', LOGSIZE); + message[LOGSIZE] = '\0'; + set.logging.log_level = POLLER_VERBOSITY_NONE; + set.logging.log_destination = 0; + set.console.stdout_notty = TRUE; + spine_log("%s", message); + free(message); +} + +static void test_log_sanitization(void) { + char message[] = "a\r\nb\t\033\177c\302\205d\342\200\250e\342\200\251f"; + spine_sanitize_log_message(message); + assert(strcmp(message, "a b c d e f") == 0); + char unicode[] = "caf\303\251"; + spine_sanitize_log_message(unicode); + assert(strcmp(unicode, "caf\303\251") == 0); + char truncated[] = "\342\200"; + spine_sanitize_log_message(truncated); + assert(strcmp(truncated, "\342\200") == 0); + char path[] = "spine-log-test-XXXXXX"; + int fd = mkstemp(path); + assert(fd >= 0 && close(fd) == 0); + set.logging.log_destination = LOGDEST_FILE; + set.logging.log_level = POLLER_VERBOSITY_LOW; + set.logging.logfile_processed = TRUE; + strncopy(set.logging.path_logfile, path, sizeof(set.logging.path_logfile)); + spine_log("device: %s", "first\r\nsecond"); + FILE *file = fopen(path, "r"); + assert(file != NULL); + char line[BUFSIZE]; + assert(fgets(line, sizeof(line), file) != NULL); + assert(strstr(line, "device: first second") != NULL); + assert(fgets(line, sizeof(line), file) == NULL && !ferror(file)); + assert(fclose(file) == 0 && unlink(path) == 0); + set.logging.log_destination = 0; +} + +static void *test_log_writer(void *argument) { + const char *message = argument; + for (int count = 0; count < 5; count++) assert(spine_log("%s", message)); + return NULL; +} + +static void test_log_append_and_failures(void) { + config_t previous = set; + char path[] = "spine-log-append-XXXXXX"; + int fd = mkstemp(path); + assert(fd >= 0); + static const char retained[] = "retained record\n"; + assert(write(fd, retained, sizeof(retained) - 1) == (ssize_t)(sizeof(retained) - 1)); + assert(close(fd) == 0); + set.logging.log_destination = LOGDEST_FILE; + set.logging.log_level = POLLER_VERBOSITY_LOW; + set.logging.logfile_processed = TRUE; + set.console.stdout_notty = TRUE; + set.console.stderr_notty = TRUE; + strncopy(set.logging.path_logfile, path, sizeof(set.logging.path_logfile)); + char first[20001]; + char second[20001]; + memset(first, 'a', sizeof(first) - 1); + memset(second, 'b', sizeof(second) - 1); + first[sizeof(first) - 1] = '\0'; + second[sizeof(second) - 1] = '\0'; + pthread_t threads[2]; + assert(pthread_create(&threads[0], NULL, test_log_writer, first) == 0); + assert(pthread_create(&threads[1], NULL, test_log_writer, second) == 0); + assert(pthread_join(threads[0], NULL) == 0); + assert(pthread_join(threads[1], NULL) == 0); + FILE *file = fopen(path, "r"); + assert(file != NULL); + char *line = NULL; + size_t capacity = 0; + assert(getline(&line, &capacity, file) == (ssize_t)(sizeof(retained) - 1)); + assert(strcmp(line, retained) == 0); + int first_count = 0; + int second_count = 0; + while (getline(&line, &capacity, file) >= 0) { + const char *body = strstr(line, first); + if (body != NULL) first_count++; + else { body = strstr(line, second); second_count++; } + assert(body != NULL && strcmp(body + 20000, "\n") == 0); + } + assert(!ferror(file) && first_count == 5 && second_count == 5); + free(line); + assert(fclose(file) == 0 && unlink(path) == 0); + /* Append mode must also create a missing file. */ + assert(spine_log("created record")); + assert(file_exists(path) && unlink(path) == 0); + char directory[] = "spine-log-directory-XXXXXX"; + assert(mkdtemp(directory) != NULL); + strncopy(set.logging.path_logfile, directory, sizeof(set.logging.path_logfile)); + assert(!spine_log("must fail to open directory")); + assert(rmdir(directory) == 0); + #ifdef __linux__ + STRNCOPY(set.logging.path_logfile, "/dev/full"); + assert(!spine_log("must detect buffered write/close failure")); + #endif + set = previous; +} + +static void test_config_directives(void) { + char path[] = "spine-config-test-XXXXXX"; + int fd = mkstemp(path); + assert(fd >= 0); + FILE *file = fdopen(fd, "w"); + assert(file != NULL); + fputs("# comment\nDB_Host localhost\nDB_Port 3307\nRDB_UseSSL 1\n" + "Poller 2\nCacti_Log /tmp/spine-test.log\n" + "SNMP_Clientaddr 127.0.0.1\nDB_Pass\nDB_Database final_line", file); + assert(fclose(file) == 0); + memset(&set, 0, sizeof set); + set.console.stdout_notty = TRUE; + set.console.stderr_notty = TRUE; + config_defaults(); + assert(read_spine_config(path) == 0); + assert(strcmp(set.database.host, "localhost") == 0); + assert(set.database.port == 3307 && set.remote_database.ssl == 1 && set.poller.poller_id == 2); + assert(strcmp(set.logging.path_logfile, "/tmp/spine-test.log") == 0); + assert(strcmp(set.snmp.snmp_clientaddr, "127.0.0.1") == 0); + assert(strcmp(set.database.database, "final_line") == 0); + assert(set.logging.logfile_processed == 1 && set.logging.log_destination == LOGDEST_BOTH); + assert(unlink(path) == 0); + assert(read_spine_config(path) == -1); +} + +static void test_config_structure_bindings(void) { + config_t previous = set; + memset(&set, 0, sizeof set); + set.console.stdout_notty = TRUE; + set.console.stderr_notty = TRUE; + set.d_b = 37; + set.cacti_version = 1232; + set.cygwinshloc = 1; + config_defaults(); + assert(strcmp(set.database.host, DEFAULT_DB_HOST) == 0); + assert(strcmp(set.remote_database.host, DEFAULT_DB_HOST) == 0); + assert(set.database.port == DEFAULT_DB_PORT && set.remote_database.port == DEFAULT_DB_PORT); + assert(set.poller.threads == DEFAULT_THREADS && set.logging.log_destination == LOGDEST_FILE); + char path[] = "spine-config-bindings-XXXXXX"; + int fd = mkstemp(path); + assert(fd >= 0); + FILE *file = fdopen(fd, "w"); + assert(file != NULL); + assert(fputs("DB_Host local-fixture\nDB_Database local-schema\nDB_User local-user\n" + "DB_Pass fixture-local-password\nDB_SSL_Key local-key\nDB_SSL_Cert local-cert\nDB_SSL_CA local-ca\n" + "RDB_Host remote-fixture\nRDB_Database remote-schema\nRDB_User remote-user\n" + "RDB_Pass fixture-remote-password\nRDB_SSL_Key remote-key\nRDB_SSL_Cert remote-cert\nRDB_SSL_CA remote-ca\n" + "DB_Port 3307\nRDB_Port 3308\nDB_UseSSL 1\nRDB_UseSSL 2\nPoller 3\n" + "SNMP_Clientaddr 127.0.0.2\nCacti_Log /tmp/spine-bindings.log\n", file) != EOF); + assert(fclose(file) == 0 && read_spine_config(path) == 0); + assert(strcmp(set.database.host, "local-fixture") == 0); + assert(strcmp(set.database.database, "local-schema") == 0); + assert(strcmp(set.database.user, "local-user") == 0); + assert(strcmp(set.database.password, "fixture-local-password") == 0); + assert(strcmp(set.database.ssl_key, "local-key") == 0); + assert(strcmp(set.database.ssl_cert, "local-cert") == 0); + assert(strcmp(set.database.ssl_ca, "local-ca") == 0); + assert(strcmp(set.remote_database.host, "remote-fixture") == 0); + assert(strcmp(set.remote_database.database, "remote-schema") == 0); + assert(strcmp(set.remote_database.user, "remote-user") == 0); + assert(strcmp(set.remote_database.password, "fixture-remote-password") == 0); + assert(strcmp(set.remote_database.ssl_key, "remote-key") == 0); + assert(strcmp(set.remote_database.ssl_cert, "remote-cert") == 0); + assert(strcmp(set.remote_database.ssl_ca, "remote-ca") == 0); + assert(set.database.port == 3307 && set.remote_database.port == 3308); + assert(set.database.ssl == 1 && set.remote_database.ssl == 2); + assert(set.poller.poller_id == 3 && set.poller.threads == DEFAULT_THREADS); + assert(strcmp(set.snmp.snmp_clientaddr, "127.0.0.2") == 0); + assert(strcmp(set.logging.path_logfile, "/tmp/spine-bindings.log") == 0); + assert(set.logging.logfile_processed == 1 && set.logging.log_destination == LOGDEST_BOTH); + assert(set.d_b == 37 && set.cacti_version == 1232 && set.cygwinshloc == 1); + assert(set.console.stdout_notty && set.console.stderr_notty); + assert(unlink(path) == 0); + set = previous; + puts("production config structure bindings passed"); +} + +static void test_date_formats(void) { + const char *const expected[] = { + "%m/%d/%Y %H:%M:%S - ", "%b/%d/%Y %H:%M:%S - ", + "%d/%m/%Y %H:%M:%S - ", "%d/%b/%Y %H:%M:%S - ", + "%Y/%m/%d %H:%M:%S - ", "%Y/%b/%d %H:%M:%S - " + }; + set.logging.log_datetime_separator = GDC_SLASH; + for (int i = GD_MIN; i <= GD_MAX; i++) { + set.logging.log_datetime_format = i; + set_date_format(); + assert(strcmp(get_date_format(), expected[i]) == 0); + } + set.logging.log_datetime_format = -1; + set.logging.log_datetime_separator = -1; + set_date_format(); + assert(strcmp(get_date_format(), expected[GD_DEFAULT]) == 0); +} + +static void test_device_logging(void) { + extern int *debug_devices; + int devices[100] = {7, 0}; + int *previous = debug_devices; + debug_devices = devices; + for (int level = POLLER_VERBOSITY_NONE; level <= POLLER_VERBOSITY_DEVDBG; level++) { + set.logging.log_level = level; + for (int minimum = POLLER_VERBOSITY_LOW; minimum <= POLLER_VERBOSITY_DEVDBG; minimum++) { + assert(spine_should_log_device(7, minimum)); + assert(spine_should_log_device(8, minimum) == (level >= minimum)); + } + } + set.logging.log_level = POLLER_VERBOSITY_NONE; + int evaluated = 0; + SPINE_LOG_DEVICE(8, POLLER_VERBOSITY_DEBUG, ("hidden %i", ++evaluated)); + assert(evaluated == 0); + SPINE_LOG_DEVICE(7, POLLER_VERBOSITY_DEBUG, ("visible %i", ++evaluated)); + assert(evaluated == 1); + debug_devices = previous; +} + +static void test_debug_device_bounds(void) { + struct { int before; int devices[100]; int after; } output; + output.before = 123; + output.after = 456; + char list[2048]; + size_t used = 0; + for (int i = 1; i <= 110; i++) { + used += (size_t)spine_snprintf(list + used, sizeof(list) - used, "%i,", i); + } + parse_debug_devices(list, output.devices, 100); + assert(output.before == 123 && output.after == 456); + for (int i = 0; i < 99; i++) assert(output.devices[i] == i + 1); + assert(output.devices[99] == 0); + char empty[] = ""; + parse_debug_devices(empty, output.devices, 100); + assert(output.devices[0] == 0); + char single[] = "7"; + parse_debug_devices(single, output.devices, 1); + assert(output.devices[0] == 0); + output.devices[0] = 7; + parse_debug_devices(single, output.devices, 0); + assert(output.devices[0] == 7); +} + +static void test_poll_result_formats(void) { + const struct { + const char *input; + const char *output; + enum poll_result_status status; + } cases[] = { + {"123", "123", POLL_RESULT_VALID}, + {"-12.5", "-12.5", POLL_RESULT_VALID}, + {"4096 Bytes", "4096", POLL_RESULT_VALID}, + {"a:1 b!2", "a:1 b!2", POLL_RESULT_VALID}, + {"a!1", "a!1", POLL_RESULT_VALID}, + {"U", "U", POLL_RESULT_UNDEFINED}, + {"nonsense", "", POLL_RESULT_INVALID} + }; + for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) { + for (int snmp = 0; snmp <= 1; snmp++) { + char result[RESULTS_BUFFER]; + strncopy(result, cases[i].input, sizeof(result)); + assert(normalize_poll_result(result, snmp) == cases[i].status); + assert(strcmp(result, cases[i].output) == 0); + } + } + char undefined[RESULTS_BUFFER] = "u"; + assert(normalize_poll_result(undefined, true) == POLL_RESULT_UNDEFINED); + assert(strcmp(undefined, "u") == 0); + assert(normalize_poll_result(undefined, false) == POLL_RESULT_INVALID); +} + +static void test_hostnames(void) { + const char *const inputs[] = {"router.example", "router.example:161", "TCP:router.example:443", + "udp:router.example:53", "[2001:db8::1]"}; + const char *const outputs[] = {"router.example", "router.example", "router.example", "router.example", "[2001:db8::1]"}; + const int ports[] = {0, 161, 443, 53, 0}; + const int methods[] = {0, 0, 1, 2, 0}; + for (size_t i = 0; i < sizeof(inputs) / sizeof(inputs[0]); i++) { + char input[128]; + strncopy(input, inputs[i], sizeof(input)); + name_t *name = get_namebyhost(input, NULL); + assert(strcmp(name->hostname, outputs[i]) == 0); + assert(name->port == ports[i] && name->method == methods[i]); + free(name); + } +} + +static void test_snmp_initialization_failure(void) { + /* These inputs fail before a network session opens. Temporary connection + * parameters and passphrase copies must be released on every error path. */ + assert(snmp_host_init(&(snmp_connection_t){ + .host_id = 1, + .hostname = "127.0.0.1", + .snmp_version = 99, + .snmp_community = "public", + .snmp_username = "", + .snmp_password = "", + .snmp_auth_protocol = "SHA", + .snmp_priv_passphrase = "", + .snmp_priv_protocol = "[None]", + .snmp_context = "", + .snmp_engine_id = "", + .snmp_port = 161, + .snmp_timeout = 1000, + }) == NULL); + assert(snmp_host_init(&(snmp_connection_t){ + .host_id = 1, + .hostname = "127.0.0.1", + .snmp_version = 3, + .snmp_community = "public", + .snmp_username = "user", + .snmp_password = "short", + .snmp_auth_protocol = "INVALID", + .snmp_priv_passphrase = "", + .snmp_priv_protocol = "[None]", + .snmp_context = "", + .snmp_engine_id = "", + .snmp_port = 161, + .snmp_timeout = 1000, + }) == NULL); + assert(snmp_host_init(&(snmp_connection_t){ + .host_id = 1, + .hostname = "127.0.0.1", + .snmp_version = 3, + .snmp_community = "public", + .snmp_username = "user", + .snmp_password = "short", + .snmp_auth_protocol = "SHA", + .snmp_priv_passphrase = "short", + .snmp_priv_protocol = "AES", + .snmp_context = "", + .snmp_engine_id = "", + .snmp_port = 161, + .snmp_timeout = 1000, + }) == NULL); + assert(snmp_host_init(&(snmp_connection_t){ + .host_id = 1, + .hostname = "127.0.0.1", + .snmp_version = 3, + .snmp_community = "public", + .snmp_username = "user", + .snmp_password = "regression-password", + .snmp_auth_protocol = "SHA", + .snmp_priv_passphrase = "short", + .snmp_priv_protocol = "AES", + .snmp_context = "", + .snmp_engine_id = "", + .snmp_port = 161, + .snmp_timeout = 1000, + }) == NULL); +} + +static void test_snmp_security_protocols(void) { + snmp_spine_init(); + snmp_connection_t options = { + .host_id = 1, .hostname = "127.0.0.1", .snmp_version = 3, + .snmp_community = "", .snmp_username = "regression-user", + .snmp_password = "", .snmp_auth_protocol = "SHA", + .snmp_priv_passphrase = "", .snmp_priv_protocol = "[None]", + .snmp_context = "regression-context", .snmp_engine_id = "", + .snmp_port = 1161, .snmp_timeout = 500, + }; + const int levels[] = {SNMP_SEC_LEVEL_NOAUTH, SNMP_SEC_LEVEL_AUTHNOPRIV, SNMP_SEC_LEVEL_AUTHPRIV}; + for (size_t index = 0; index < sizeof(levels) / sizeof(levels[0]); index++) { + options.snmp_password = index == 0 ? "" : "regression-password"; + options.snmp_priv_protocol = index == 2 ? "AES" : "[None]"; + options.snmp_priv_passphrase = index == 2 ? "regression-privacy" : ""; + void *handle = snmp_host_init(&options); + assert(handle != NULL); + const struct snmp_session *session = snmp_sess_session(handle); + assert(session != NULL && session->version == SNMP_VERSION_3); + assert(session->securityLevel == levels[index]); + assert(strcmp(session->securityName, options.snmp_username) == 0); + assert(strcmp(session->contextName, options.snmp_context) == 0); + assert(session->securityAuthProto != NULL && session->securityPrivProto != NULL); + if (index == 2) { + assert(session->securityAuthKeyLen == 20); /* SHA-1 Ku length. */ + assert(session->securityPrivKeyLen == 20); + } + snmp_host_cleanup(handle); + } + test_additional_snmp_session_boundaries(); + options.snmp_priv_protocol = "INVALID"; + assert(snmp_host_init(&options) == NULL); + snmp_spine_close(); +} + +static void test_child_process(void) { + int fd = nft_popen("printf spine-test", "r"); + assert(fd >= 0); + char output[32] = {0}; + size_t used = 0; + ssize_t received; + while ((received = read(fd, output + used, sizeof(output) - 1 - used)) != 0) { + if (received < 0 && errno == EINTR) continue; + assert(received > 0); + used += (size_t)received; + assert(used < sizeof(output) - 1); + } + assert(used == 10); + assert(strcmp(output, "spine-test") == 0); + int status = nft_pclose(fd); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); + assert(nft_popen("printf ignored", "invalid") == -1); + assert(nft_popen("printf ignored", "invalid+") == -1 && errno == EINVAL); + assert(nft_popen(NULL, "r") == -1 && errno == EINVAL); + assert(nft_popen("printf ignored", NULL) == -1 && errno == EINVAL); +} + +static void test_duplex_process(void) { + int fd = nft_popen("IFS= read -r value; printf '%s' \"$value\"", "r+"); + assert(fd >= 0); + assert(write(fd, "duplex\n", 7) == 7); + char output[7] = {0}; + size_t used = 0; + while (used < 6) { + ssize_t received = read(fd, output + used, 6 - used); + if (received < 0 && errno == EINTR) continue; + assert(received > 0); + used += (size_t)received; + } + assert(strcmp(output, "duplex") == 0); + int status = nft_pclose(fd); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); +} + +static void test_write_process(void) { + char path[] = "spine-write-process-XXXXXX"; + int temporary = mkstemp(path); + assert(temporary >= 0 && close(temporary) == 0); + char command[BUFSIZE]; + spine_snprintf(command, sizeof(command), "cat > %s", path); + int fd = nft_popen(command, "w"); + assert(fd >= 0 && write(fd, "written", 7) == 7); + /* cat copies each read straight through, so the data reaches the file + * before EOF. Check it while the writer is still open: nft_pclose() never + * blocks the poller on a slow child and may signal it once closed. */ + char output[8] = {0}; + FILE *file = NULL; + for (int attempt = 0; attempt < 200; attempt++) { + file = fopen(path, "r"); + assert(file != NULL); + memset(output, 0, sizeof(output)); + if (fread(output, 1, 7, file) == 7) break; + assert(fclose(file) == 0); + file = NULL; + spine_sleep_usec(10000); + } + assert(file != NULL && strcmp(output, "written") == 0); + int status = nft_pclose(fd); + assert(status == -1 || (WIFEXITED(status) && WEXITSTATUS(status) == 0) || WIFSIGNALED(status)); + assert(fgetc(file) == EOF && !ferror(file)); + assert(fclose(file) == 0 && unlink(path) == 0); +} + +static void test_process_descriptor_aliases(void) { + for (int mask = 1; mask <= 3; mask++) { + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + alarm(5); + if (mask & 1) assert(close(STDIN_FILENO) == 0); + if (mask & 2) assert(close(STDOUT_FILENO) == 0); + /* Keep an earlier parent pipe in the list, possibly at 0 or 1. */ + int retained = nft_popen("printf retained", "r"); + assert(retained >= 0); + test_duplex_process(); + char output[16] = {0}; + assert(read(retained, output, sizeof(output) - 1) == 8); + assert(strcmp(output, "retained") == 0); + int status = nft_pclose(retained); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); + _exit(0); + } + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); + } +} + +/* Drive the shipped php_readpipe() against a pipe that stands in for a + * script server. A failed read restarts the slot, so point the restart at a + * missing executable to keep it local. */ +static char *read_php_pipe(int fd, int timeout) { + config_t previous_config = set; + php_t *previous_processes = php_processes; + php_t process; + php_processes_initialize(&process, 1); + process.php_read_fd = fd; + process.php_state = PHP_BUSY; + php_processes = &process; + set.php.php_servers = 1; + set.php.script_timeout = timeout; + STRNCOPY(set.php.path_php, "/nonexistent-spine-regression-executable"); + char *result = php_readpipe(0, "regression"); + if (process.php_read_fd >= 0) close(process.php_read_fd); + if (process.php_write_fd >= 0) close(process.php_write_fd); + php_processes = previous_processes; + set = previous_config; + return result; +} + +static void test_php_response(size_t length, bool newline) { + int pipes[2]; + assert(pipe(pipes) == 0); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + close(pipes[0]); + char *data = malloc(length); + assert(data != NULL); + memset(data, '7', length); + if (newline) data[length - 1] = '\n'; + size_t sent = 0; + while (sent < length) { + ssize_t count = write(pipes[1], data + sent, length - sent); + if (count < 0 && errno == EINTR) continue; + if (count < 0 && errno == EPIPE) break; + assert(count > 0); + sent += (size_t)count; + } + free(data); + close(pipes[1]); + _exit(0); + } + close(pipes[1]); + char *result = read_php_pipe(pipes[0], 5); + if (newline) { + assert(strlen(result) == length && result[length - 1] == '\n'); + } else { + assert(strcmp(result, "U") == 0); + } + free(result); + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); +} + +static void test_php_partial_response_timeout(void) { + /* A server that writes part of a line and stalls must not hold the + * poller thread past the response deadline. */ + int pipes[2]; + assert(pipe(pipes) == 0); + assert(write(pipes[1], "7", 1) == 1); + double begin = spine_monotonic_time(); + char *result = read_php_pipe(pipes[0], 1); + assert(spine_monotonic_time() - begin < 5); + assert(strcmp(result, "U") == 0); + free(result); + close(pipes[1]); +} + +static void test_script_execution(void) { + assert(spine_permits_init(&available_scripts, 1) == 0); + host_t host = {0}; + STRNCOPY(host.hostname, "regression-device"); + int previous_timeout = set.php.script_timeout; + set.php.script_timeout = 1; + char command[] = "/usr/bin/printf 7"; + char *result = exec_poll(&host, command, 1, "DS"); + assert(strcmp(result, "7") == 0); + free(result); + char delayed[] = "/bin/sleep 2"; + double begin = spine_monotonic_time(); + result = exec_poll(&host, delayed, 1, "DS"); + assert(strcmp(result, "U") == 0); + assert(spine_monotonic_time() - begin < 5); + free(result); + assert(spine_permits_available(&available_scripts) == 1); + for (int timeout = -1; timeout <= 0; timeout++) { + set.php.script_timeout = timeout; + result = exec_poll(&host, command, 1, "DS"); + assert(strcmp(result, "U") == 0); + free(result); + assert(spine_permits_available(&available_scripts) == 1); + } + set.php.script_timeout = 1; + assert(spine_permits_try_acquire(&available_scripts) == 0); + begin = spine_monotonic_time(); + result = exec_poll(&host, command, 1, "DS"); + assert(strcmp(result, "U") == 0); + assert(spine_monotonic_time() - begin < 1); + free(result); + assert(spine_permits_available(&available_scripts) == 0); + assert(spine_permits_release(&available_scripts) == 0); + char output_command[128]; + for (int excess = 0; excess <= 1; excess++) { + spine_snprintf(output_command, sizeof(output_command), "/usr/bin/printf '%%%ds' x", RESULTS_BUFFER - 1 + excess); + result = exec_poll(&host, output_command, 1, "DS"); + assert(strlen(result) == RESULTS_BUFFER - 1); + assert(result[RESULTS_BUFFER - 1] == '\0'); + free(result); + assert(spine_permits_available(&available_scripts) == 1); + } + char empty[] = "/usr/bin/printf ''"; + result = exec_poll(&host, empty, 1, "DQ"); + assert(strcmp(result, "U") == 0); + free(result); + char missing[] = "/spine-regression/nonexistent-executable"; + result = exec_poll(&host, missing, 1, "DS"); + assert(strcmp(result, "U") == 0); + free(result); + assert(spine_permits_available(&available_scripts) == 1); + assert(spine_permits_destroy(&available_scripts) == 0); + set.php.script_timeout = previous_timeout; +} + +static void test_php_command(size_t length) { + char command[BUFSIZE]; + assert(length <= sizeof(command) - 3); + memset(command, 'c', length); + command[length] = '\0'; + int requests[2]; + int responses[2]; + assert(pipe(requests) == 0 && pipe(responses) == 0); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + close(requests[1]); + close(responses[0]); + char received[BUFSIZE] = {0}; + size_t used = 0; + while (used < length + 2) { + ssize_t bytes = read(requests[0], received + used, length + 2 - used); + if (bytes < 0 && errno == EINTR) continue; + assert(bytes > 0); + used += (size_t)bytes; + } + assert(memcmp(received, command, length) == 0); + assert(received[length] == '\r' && received[length + 1] == '\n'); + assert(write(responses[1], "7\n", 2) == 2); + close(requests[0]); + close(responses[1]); + _exit(0); + } + close(requests[0]); + close(responses[1]); + php_t process = {0}; + /* php_cmd() only talks to a READY slot that owns a live child. */ + process.php_state = PHP_READY; + process.php_pid = child; + process.php_write_fd = requests[1]; + process.php_read_fd = responses[0]; + php_t *previous = php_processes; + php_processes = &process; + set.php.php_servers = 1; + set.php.script_timeout = 5; + char *result = php_cmd(command, 0); + assert(strcmp(result, "7\n") == 0); + free(result); + php_processes = previous; + close(requests[1]); + close(responses[0]); + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); +} + +static void test_invalid_php_commands(void) { + /* Invalid requests must return before touching server state or writing. */ + char *result = php_cmd("invalid\r\nline", 0); + assert(strcmp(result, "U") == 0); + free(result); + char oversized[BUFSIZE]; + memset(oversized, 'c', sizeof(oversized) - 1); + oversized[sizeof(oversized) - 1] = '\0'; + result = php_cmd(oversized, 0); + assert(strcmp(result, "U") == 0); + free(result); + result = php_cmd("valid", -1); + assert(strcmp(result, "U") == 0); + free(result); +} + +static int run_test_script_server(int argc, char **argv) { + assert(argc >= 5); + if (strncmp(argv[2], "regression-server:", 18) == 0) { + char *end; + errno = 0; + long descriptor = strtol(argv[2] + 18, &end, 10); + assert(errno == 0 && *end == 0 && descriptor >= 3 && descriptor <= INT_MAX); + char token; + ssize_t received = read((int)descriptor, &token, 1); + assert(close((int)descriptor) == 0); + if (received == 0) return 127; + assert(received == 1 && token == 82); + } else assert(strcmp(argv[2], "regression-server") == 0); + if (strcmp(argv[3], "spine") == 0) { + assert(argc == 5 && strcmp(argv[4], "1") == 0); + } else { + assert(strcmp(argv[3], "--environ=spine") == 0); + assert(strcmp(argv[4], "--poller=1") == 0 || strcmp(argv[4], "--poller=2") == 0); + if (strcmp(argv[4], "--poller=2") == 0) { + assert(argc == 6 && (strcmp(argv[5], "--mode=online") == 0 || strcmp(argv[5], "--mode=offline") == 0)); + } else assert(argc == 5); + } + puts("Started"); + fflush(stdout); + char command[BUFSIZE]; + while (fgets(command, sizeof(command), stdin) != NULL) { + if (strcmp(command, "quit\r\n") == 0) return 0; + puts("7"); + fflush(stdout); + } + return 17; /* EOF without quit must not count as graceful protocol shutdown. */ +} + +static void assert_php_responses(const php_t *processes, int count) { + for (int index = 0; index < count; index++) { + assert(processes[index].php_state == PHP_READY); + char *result = php_cmd("regression request", index); + assert(strcmp(result, "7\n") == 0); + free(result); + } +} + +static void assert_php_children_reaped(const php_t *processes, const pid_t *children, size_t count) { + for (size_t index = 0; index < count; index++) { + int status; + assert(waitpid(children[index], &status, WNOHANG) == -1 && errno == ECHILD); + assert(processes[index].php_pid == -1); + } +} + +static void test_php_startup(const char *executable) { + char *absolute = realpath(executable, NULL); + assert(absolute != NULL); + config_t previous_config = set; + php_t *previous_processes = php_processes; + php_t processes[2]; + memset(processes, 0, sizeof(processes)); + php_processes = processes; + STRNCOPY(set.php.path_php, absolute); + free(absolute); + STRNCOPY(set.php.path_php_server, "regression-server"); + set.php.script_timeout = 5; + set.php.php_servers = 2; + const int versions[] = {1222, 1223, 1223, 1223}; + const int pollers[] = {1, 1, 2, 2}; + const int modes[] = {REMOTE_ONLINE, REMOTE_ONLINE, REMOTE_ONLINE, REMOTE_OFFLINE}; + for (size_t i = 0; i < sizeof(versions) / sizeof(versions[0]); i++) { + set.cacti_version = versions[i]; + set.poller.poller_id = pollers[i]; + set.poller.mode = modes[i]; + assert(php_init(PHP_INIT)); + assert_php_responses(processes, set.php.php_servers); + pid_t children[] = {processes[0].php_pid, processes[1].php_pid}; + php_close(PHP_INIT); + assert_php_children_reaped(processes, children, sizeof(children) / sizeof(children[0])); + } + /* An inherited one-byte pipe admits the first child and rejects the second. */ + int admissions[2]; + assert(pipe(admissions) == 0); + assert(write(admissions[1], "R", 1) == 1 && close(admissions[1]) == 0); + spine_snprintf(set.php.path_php_server, sizeof(set.php.path_php_server), "regression-server:%d", admissions[0]); + assert(php_init(PHP_INIT)); + /* A server that never answered stays BUSY for the restart path; shutdown + * still reaps both children. */ + assert(processes[0].php_state == PHP_READY && processes[1].php_state == PHP_BUSY); + php_close(PHP_INIT); + assert(processes[0].php_pid == -1 && processes[1].php_pid == -1); + assert(close(admissions[0]) == 0); + STRNCOPY(set.php.path_php_server, "regression-server"); + assert(!php_init(-2) && !php_init(set.php.php_servers)); + STRNCOPY(set.php.path_php, "/nonexistent-spine-regression-executable"); + assert(!php_init(0)); + assert(processes[0].php_state == PHP_BUSY); + assert(processes[0].php_read_fd == -1 && processes[0].php_write_fd == -1); + /* posix_spawn reports a missing executable to the parent (or the child + * exits 127 where the libc defers exec errors), so no server is left. */ + assert(processes[0].php_pid == -1); + char *failed_command = php_cmd("request on stopped server", 0); + assert(strcmp(failed_command, "U") == 0); + free(failed_command); + assert(processes[0].php_pid == -1 && processes[0].php_state == PHP_BUSY); + php_processes = previous_processes; + set = previous_config; +} + +static void test_php_owned_shutdown(void) { + int requests[2]; + int responses[2]; + assert(pipe(requests) == 0 && pipe(responses) == 0); + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + close(requests[1]); + close(responses[0]); + assert(signal(SIGTERM, SIG_IGN) != SIG_ERR); + assert(write(responses[1], "R", 1) == 1); + for (;;) pause(); + } + close(requests[0]); + close(responses[1]); + char ready; + assert(read(responses[0], &ready, 1) == 1 && ready == 'R'); + int flags = fcntl(requests[1], F_GETFL); + assert(flags >= 0 && fcntl(requests[1], F_SETFL, flags | O_NONBLOCK) == 0); + char padding[1024] = {0}; + while (write(requests[1], padding, sizeof(padding)) > 0) { /* Fill the pipe until the nonblocking write refuses more data. */ } + assert(errno == EAGAIN || errno == EWOULDBLOCK); + assert(fcntl(requests[1], F_SETFL, flags) == 0); + php_t server = {0}; + server.php_pid = child; + server.php_write_fd = requests[1]; + server.php_read_fd = responses[0]; + php_t *previous_processes = php_processes; + int previous_count = set.php.php_servers; + php_processes = &server; + set.php.php_servers = 1; + double begin = spine_monotonic_time(); + alarm(5); + php_close(0); + alarm(0); + assert(spine_monotonic_time() - begin < 4.0); + assert(server.php_pid == -1 && server.php_read_fd == -1 && server.php_write_fd == -1); + assert(fcntl(requests[1], F_GETFD) == -1 && errno == EBADF); + assert(fcntl(responses[0], F_GETFD) == -1 && errno == EBADF); + /* php_close() sends SIGKILL and parks a child that has not exited yet + * instead of blocking the poller, so it is reaped here or by the sweep. */ + int status; + pid_t reaped = waitpid(child, &status, 0); + assert((reaped == child && WIFSIGNALED(status) && WTERMSIG(status) == SIGKILL) || + (reaped == -1 && errno == ECHILD)); + php_close(0); + /* A stale PID for a non-child must be retired without sending a signal. */ + server.php_pid = getpid(); + php_close(0); + assert(server.php_pid == -1); + php_processes = previous_processes; + set.php.php_servers = previous_count; +} + +static void test_udp_deadline(void) { + struct sockaddr_in address = {0}; + assert(init_sockaddr(&address, "127.0.0.1", 0)); + assert(address.sin_family == AF_INET && address.sin_addr.s_addr == htonl(INADDR_LOOPBACK)); + assert(!init_sockaddr(&address, "127.0.0.1", -1)); + assert(!init_sockaddr(&address, "127.0.0.1", 65536)); + int server = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP); + assert(server >= 0); + assert(bind(server, (struct sockaddr *)&address, sizeof(address)) == 0); + socklen_t length = sizeof(address); + assert(getsockname(server, (struct sockaddr *)&address, &length) == 0); + host_t host = {0}; + STRNCOPY(host.hostname, "127.0.0.1"); + host.availability.port = ntohs(address.sin_port); + host.availability.timeout = 650; + host.availability.retries = 0; + ping_t ping = {0}; + double begin = spine_monotonic_time(); + assert(ping_udp(&host, &ping) == HOST_DOWN); + double elapsed = spine_monotonic_time() - begin; + assert(elapsed >= 0.60 && elapsed < 1.25); + assert(strcmp(ping.ping_response, "UDP: Ping timed out") == 0); + char request[64]; + ssize_t received = recv(server, request, sizeof(request), 0); + static const char expected[] = "cacti-monitoring-system"; + assert(received == (ssize_t)(sizeof(expected) - 1)); + assert(memcmp(request, expected, sizeof(expected) - 1) == 0); + assert(close(server) == 0); + host.availability.timeout = 0; + assert(ping_udp(&host, &ping) == HOST_DOWN); +} + +static bool icmp_reply_matches(const unsigned char *reply, size_t length, uint16_t id, uint16_t sequence) { + return spine_icmp_classify_reply(reply, (ssize_t) length, id, sequence, NULL) == SPINE_ICMP_REPLY_OK; +} + +static void test_icmp_reply_bounds(void) { + unsigned char storage[29] = {0}; + unsigned char *reply = storage + 1; + reply[0] = 0x45; + reply[9] = IPPROTO_ICMP; + uint16_t id = htons(123); + uint16_t sequence = htons(456); + memcpy(reply + 24, &id, sizeof(id)); + memcpy(reply + 26, &sequence, sizeof(sequence)); + assert(icmp_reply_matches(reply, 28, id, sequence)); + assert(!icmp_reply_matches(NULL, 28, id, sequence)); + for (size_t length = 0; length < 28; length++) assert(!icmp_reply_matches(reply, length, id, sequence)); + assert(!icmp_reply_matches(reply, 28, htons(124), sequence)); + assert(!icmp_reply_matches(reply, 28, id, htons(457))); + reply[0] = 0x44; + assert(!icmp_reply_matches(reply, 28, id, sequence)); + reply[0] = 0x4f; + assert(!icmp_reply_matches(reply, 28, id, sequence)); + reply[0] = 0x65; + assert(!icmp_reply_matches(reply, 28, id, sequence)); + reply[0] = 0x45; + reply[20] = ICMP_ECHO; + assert(!icmp_reply_matches(reply, 28, id, sequence)); + reply[20] = ICMP_ECHOREPLY; + reply[21] = 1; + assert(!icmp_reply_matches(reply, 28, id, sequence)); + reply[21] = 0; + reply[9] = IPPROTO_TCP; + assert(!icmp_reply_matches(reply, 28, id, sequence)); +} + +static void test_icmp_loopback(void) { + host_t host = {0}; + STRNCOPY(host.hostname, "127.0.0.1"); + host.availability.timeout = 500; + host.availability.retries = 0; + ping_t ping = {0}; + assert(ping_icmp(&host, &ping) == HOST_UP); + assert(strcmp(ping.ping_response, "ICMP: Device is Alive") == 0); + assert(geteuid() == getuid()); +} + +static void test_icmp_socket_failure(void) { + assert(geteuid() != 0); + host_t host = {0}; + STRNCOPY(host.hostname, "127.0.0.1"); + host.availability.timeout = 100; + ping_t ping = {0}; + /* Where net.ipv4.ping_group_range admits this user, ICMP works through an + * unprivileged datagram socket and needs no capability at all. */ + int datagram = socket(AF_INET, SOCK_DGRAM, IPPROTO_ICMP); + if (datagram >= 0) { + close(datagram); + host.availability.timeout = 500; + assert(ping_icmp(&host, &ping) == HOST_UP); + assert(strcmp(ping.ping_response, "ICMP: Device is Alive") == 0); + assert(geteuid() == getuid()); + return; + } + /* Failed socket retries must release the privilege mutex each time. */ + alarm(6); + assert(ping_icmp(&host, &ping) == HOST_DOWN); + alarm(0); + assert(strcmp(ping.ping_response, "ICMP: Ping unable to create ICMP Socket") == 0); + assert(geteuid() == getuid()); +} + +static unsigned int query_sample_count(MYSQL *mysql, const char *query) { + MYSQL_RES *result = db_query(mysql, LOCAL, query); + assert(result != NULL && mysql_num_fields(result) == 2); + unsigned int samples = 0; + MYSQL_ROW row; + while ((row = mysql_fetch_row(result))) samples += (unsigned int)atoi(row[1]); + db_free_result(result); + return samples; +} + +static void test_poller_query_case(MYSQL *mysql, int poller, int active, int ports) { + static const unsigned int all_rows[] = {3, 2, 1}; + static const unsigned int due_rows[] = {2, 1, 1}; + set.poller.poller_id = poller; + set.poller.active_profiles = active; + set.snmp.total_snmp_ports = ports; + poller_queries_t queries; + poller_prepare_queries(&queries, 42, 1, 0); + MYSQL_RES *result = db_query(mysql, LOCAL, queries.items); + assert(result != NULL && mysql_num_fields(result) == 20); + assert(mysql_num_rows(result) == all_rows[poller]); + db_free_result(result); + result = db_query(mysql, LOCAL, queries.due_items); + unsigned int expected_due = active ? all_rows[poller] : due_rows[poller]; + assert(result != NULL && mysql_num_rows(result) == expected_due); + db_free_result(result); + assert(query_sample_count(mysql, queries.agents) == all_rows[poller]); + assert(query_sample_count(mysql, queries.due_agents) == expected_due); +} + +static void test_poller_schedule(MYSQL *mysql) { + poller_queries_t queries; + set.poller.poller_interval = 60; + set.poller.poller_id = 1; + assert(db_insert(mysql, LOCAL, "UPDATE poller_item SET rrd_step=300,rrd_next_step=180 WHERE host_id=42") == TRUE); + poller_prepare_queries(&queries, 42, 1, 0); + assert(db_insert(mysql, LOCAL, queries.schedule) == TRUE); + MYSQL_RES *result = db_query(mysql, LOCAL, "SELECT local_data_id,rrd_next_step FROM poller_item WHERE host_id=42 ORDER BY local_data_id"); + assert(result != NULL && mysql_num_rows(result) == 3); + MYSQL_ROW row = mysql_fetch_row(result); + assert(row != NULL && strcmp(row[0], "101") == 0 && strcmp(row[1], "120") == 0); + row = mysql_fetch_row(result); + assert(row != NULL && strcmp(row[0], "102") == 0 && strcmp(row[1], "120") == 0); + row = mysql_fetch_row(result); + assert(row != NULL && strcmp(row[0], "103") == 0 && strcmp(row[1], "180") == 0); + db_free_result(result); + set.poller.poller_id = 0; + poller_prepare_queries(&queries, 42, 1, 0); + assert(db_insert(mysql, LOCAL, queries.schedule) == TRUE); + result = db_query(mysql, LOCAL, "SELECT rrd_next_step FROM poller_item WHERE local_data_id=103"); + assert(result != NULL && mysql_num_rows(result) == 1); + row = mysql_fetch_row(result); + assert(row != NULL && strcmp(row[0], "120") == 0); + db_free_result(result); +} + +static void test_poller_queries(MYSQL *mysql) { + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_item") == TRUE); + assert(db_insert(mysql, LOCAL, "DELETE FROM host") == TRUE); + assert(db_insert(mysql, LOCAL, "INSERT INTO host (id,hostname) VALUES (42,'127.0.0.1')") == TRUE); + assert(db_insert(mysql, LOCAL, "INSERT INTO poller_item (local_data_id,host_id,poller_id,rrd_next_step,snmp_port) VALUES (101,42,1,0,161),(102,42,1,1,162),(103,42,2,0,161),(104,99,1,0,161),(105,0,1,0,161)") == TRUE); + poller_queries_t queries; + for (int poller = 0; poller <= 2; poller++) { + for (int active = 0; active <= 1; active++) { + for (int ports = 1; ports <= 2; ports++) { + test_poller_query_case(mysql, poller, active, ports); + } + } + } + set.poller.poller_id = 1; + poller_prepare_queries(&queries, 42, 1, 1); + MYSQL_RES *result = db_query(mysql, LOCAL, queries.items); + assert(result != NULL && mysql_num_rows(result) == 1); + db_free_result(result); + poller_prepare_queries(&queries, 42, 3, 1); + result = db_query(mysql, LOCAL, queries.items); + assert(result != NULL && mysql_num_rows(result) == 0); + db_free_result(result); + poller_prepare_queries(&queries, 42, INT_MAX, INT_MAX); + result = db_query(mysql, LOCAL, queries.items); + assert(result != NULL && mysql_num_rows(result) == 0); + db_free_result(result); + poller_prepare_queries(&queries, 42, 1, 0); + result = db_query(mysql, LOCAL, queries.host); + assert(result != NULL && mysql_num_fields(result) == 37 && mysql_num_rows(result) == 1); + db_free_result(result); + assert(db_insert(mysql, LOCAL, "UPDATE host SET deleted='on' WHERE id=42") == TRUE); + result = db_query(mysql, LOCAL, queries.host); + assert(result != NULL && mysql_num_rows(result) == 0); + db_free_result(result); + result = db_query(mysql, LOCAL, queries.reindex); + assert(result != NULL && mysql_num_fields(result) == 5 && mysql_num_rows(result) == 0); + db_free_result(result); + poller_prepare_queries(&queries, 0, 1, 0); + result = db_query(mysql, LOCAL, queries.items); + assert(result != NULL && mysql_num_rows(result) == 1); + db_free_result(result); + test_poller_schedule(mysql); +} + +static unsigned long database_count(MYSQL *mysql, const char *query) { + MYSQL_RES *result = db_query(mysql, LOCAL, query); + assert(result != NULL && mysql_num_rows(result) == 1 && mysql_num_fields(result) == 1); + MYSQL_ROW row = mysql_fetch_row(result); + assert(row != NULL && row[0] != NULL); + unsigned long value = strtoul(row[0], NULL, 10); + db_free_result(result); + return value; +} + +static void seed_transfer_rows(MYSQL *source) { + assert(db_insert(source, LOCAL, "DELETE FROM host")); + assert(db_insert(source, LOCAL, "DELETE FROM poller_item")); + for (int index = 1; index <= 502; index++) { + char query[BUFSIZE]; + spine_snprintf(query, sizeof(query), "INSERT INTO host (id,poller_id,snmp_sysDescr,status_last_error,last_updated,min_time) VALUES (%d,%d,'quote\\\' and slash\\\\',NULL,NULL,NULL)", index, index == 502 ? 3 : 2); + assert(db_insert(source, LOCAL, query)); + } + assert(db_insert(source, LOCAL, "UPDATE host SET snmp_sysDescr=REPEAT(CONVERT(0xF09F8CB5 USING utf8mb4),300) WHERE id=501")); + for (int begin = 1; begin <= 10002; begin += 100) { + char query[BUFSIZE * 8]; + size_t used = (size_t)spine_snprintf(query, sizeof(query), "INSERT INTO poller_item (local_data_id,host_id,poller_id,rrd_name,rrd_step,rrd_next_step) VALUES "); + int end = begin + 100; + if (end > 10003) end = 10003; + for (int index = begin; index < end; index++) { + used += (size_t)spine_snprintf(query + used, sizeof(query) - used, "%s(%d,%d,%d,'r\\\'r',300,120)", index == begin ? "" : ",", index, index == 10002 ? 502 : 501, index == 10002 ? 3 : 2); + } + assert(db_insert(source, LOCAL, query)); + } +} + +static void test_transfer_boundary(MYSQL *source, MYSQL *destination) { + assert(poller_transfer_status(source, destination)); + assert(database_count(destination, "SELECT COUNT(*) FROM host") == 501); + assert(database_count(destination, "SELECT COUNT(*) FROM poller_item") == 10001); + assert(database_count(destination, "SELECT COUNT(*) FROM host WHERE status_last_error IS NULL AND min_time IS NULL") == 501); + assert(database_count(destination, "SELECT COUNT(*) FROM host d JOIN spine_regressions.host s ON s.id=d.id WHERE NOT (d.last_updated <=> s.last_updated)") == 0); + assert(database_count(destination, "SELECT CHAR_LENGTH(snmp_sysDescr) FROM host WHERE id=501") == 300); + assert(database_count(destination, "SELECT OCTET_LENGTH(snmp_sysDescr) FROM host WHERE id=501") == 1200); + assert(database_count(destination, "SELECT COUNT(*) FROM host WHERE snmp_sysDescr=CONCAT('quote',CHAR(39),' and slash',CHAR(92))") == 500); + assert(database_count(destination, "SELECT COUNT(*) FROM poller_item WHERE rrd_name='r\\\'r' AND rrd_next_step=120") == 10001); + /* Upsert retries must update only the existing documented columns. */ + assert(db_insert(destination, REMOTE, "UPDATE poller_item SET rrd_step=900,rrd_next_step=999")); + assert(poller_transfer_status(source, destination)); + assert(database_count(destination, "SELECT COUNT(*) FROM poller_item WHERE rrd_step=900 AND rrd_next_step=120") == 10001); +} + +static void test_transfer_failure_and_filter(MYSQL *source, MYSQL *destination) { + assert(db_insert(destination, REMOTE, "DELETE FROM host")); + assert(db_insert(destination, REMOTE, "DELETE FROM poller_item")); + assert(db_insert(destination, REMOTE, "CREATE TRIGGER reject_transfer BEFORE INSERT ON host FOR EACH ROW BEGIN IF NEW.id=501 THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='regression late-batch failure'; END IF; END")); + assert(!poller_transfer_status(source, destination)); + set.exit.exit_code = EXIT_SUCCESS; + poller_push_data_to_main(); + assert(set.exit.exit_code == EXIT_FAILURE); + assert(database_count(destination, "SELECT COUNT(*) FROM host") == 500); + assert(database_count(destination, "SELECT COUNT(*) FROM poller_item") == 0); + assert(database_count(source, "SELECT COUNT(*) FROM host") == 502); + assert(database_count(source, "SELECT COUNT(*) FROM poller_item") == 10002); + assert(db_insert(destination, REMOTE, "DROP TRIGGER reject_transfer")); + set.exit.exit_code = EXIT_SUCCESS; + poller_push_data_to_main(); + assert(set.exit.exit_code == EXIT_SUCCESS); + assert(database_count(destination, "SELECT COUNT(*) FROM host") == 501); + assert(database_count(destination, "SELECT COUNT(*) FROM poller_item") == 10001); + assert(db_insert(destination, REMOTE, "DELETE FROM poller_item")); + assert(db_insert(destination, REMOTE, "CREATE TRIGGER reject_item_transfer BEFORE INSERT ON poller_item FOR EACH ROW BEGIN IF NEW.local_data_id=10001 THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='regression final-item failure'; END IF; END")); + assert(!poller_transfer_status(source, destination)); + assert(database_count(destination, "SELECT COUNT(*) FROM poller_item") == 10000); + assert(database_count(source, "SELECT COUNT(*) FROM poller_item") == 10002); + assert(db_insert(destination, REMOTE, "DROP TRIGGER reject_item_transfer")); + assert(poller_transfer_status(source, destination)); + assert(database_count(destination, "SELECT COUNT(*) FROM poller_item") == 10001); + /* Full-width Unicode across all text fields must flush by bytes before 500 rows. */ + assert(db_insert(source, LOCAL, "UPDATE host SET snmp_sysDescr=REPEAT(CONVERT(0xF09F8CB5 USING utf8mb4),300),snmp_sysContact=snmp_sysDescr,snmp_sysName=snmp_sysDescr,snmp_sysLocation=snmp_sysDescr WHERE poller_id=2")); + assert(db_insert(destination, REMOTE, "DELETE FROM host")); + assert(poller_transfer_status(source, destination)); + assert(database_count(destination, "SELECT COUNT(*) FROM host WHERE OCTET_LENGTH(snmp_sysDescr)=1200 AND OCTET_LENGTH(snmp_sysLocation)=1200") == 501); + assert(db_insert(destination, REMOTE, "DELETE FROM host")); + assert(db_insert(destination, REMOTE, "DELETE FROM poller_item")); + STRNCOPY(set.hosts.host_id_list, "1"); + assert(poller_transfer_status(source, destination)); + assert(database_count(destination, "SELECT COUNT(*) FROM host") == 1); + assert(database_count(destination, "SELECT COUNT(*) FROM poller_item") == 0); + set.hosts.host_id_list[0] = '\0'; + assert(db_insert(source, LOCAL, "DELETE FROM host")); + assert(db_insert(source, LOCAL, "DELETE FROM poller_item")); + assert(poller_transfer_status(source, destination)); + assert(database_count(destination, "SELECT COUNT(*) FROM host") == 1); +} + +static void test_collector_transfer(MYSQL *source) { + config_t previous = set; + set.poller.poller_id = 2; + set.database.onupdate = 0; + set.hosts.host_id_list[0] = '\0'; + char database[80]; + char query[BUFSIZE]; + spine_snprintf(database, sizeof(database), "spine_transfer_%ld", (long)getpid()); + STRNCOPY(set.remote_database.host, set.database.host); + STRNCOPY(set.remote_database.user, set.database.user); + STRNCOPY(set.remote_database.password, set.database.password); + STRNCOPY(set.remote_database.database, database); + set.remote_database.port = set.database.port; + set.remote_database.ssl = FALSE; + spine_snprintf(query, sizeof(query), "CREATE DATABASE %s CHARACTER SET utf8mb4", database); + assert(db_insert(source, LOCAL, query)); + MYSQL destination; + db_connect(LOCAL, &destination); + assert(mysql_select_db(&destination, database) == 0); + assert(mysql_set_character_set(source, "utf8mb4") == 0 && mysql_set_character_set(&destination, "utf8mb4") == 0); + assert(db_insert(&destination, REMOTE, "CREATE TABLE host LIKE spine_regressions.host")); + assert(db_insert(&destination, REMOTE, "CREATE TABLE poller_item LIKE spine_regressions.poller_item")); + seed_transfer_rows(source); + test_transfer_boundary(source, &destination); + test_transfer_failure_and_filter(source, &destination); + db_disconnect(&destination); + spine_snprintf(query, sizeof(query), "DROP DATABASE %s", database); + assert(db_insert(source, LOCAL, query)); + set = previous; +} + +static void test_database_version(MYSQL *mysql) { + const struct { + const char *text; + int expected; + } cases[] = { + {"1.2.32", 1232}, {"1.3.0_develop", 1300}, {"1.2.33-beta1", 1233}, + {"2147483.6.47", INT_MAX}, {"2147483.6.48", 0}, {"2147483647.0.0", 0}, + {"99999999999999999999", 0}, {"new_install", 0}, {"", 0}, + {"1.2", 0}, {"1.x.3", 0}, {"1.2.-1", 0}, {"-1.2.3", 0} + }; + for (size_t index = 0; index < sizeof(cases) / sizeof(cases[0]); index++) { + assert(db_insert(mysql, LOCAL, "DELETE FROM version")); + char query[BUFSIZE]; + spine_snprintf(query, sizeof(query), "INSERT INTO version(cacti) VALUES ('%s')", cases[index].text); + assert(db_insert(mysql, LOCAL, query)); + assert(get_cacti_version(mysql, LOCAL) == cases[index].expected); + } + assert(db_insert(mysql, LOCAL, "DELETE FROM version")); + assert(get_cacti_version(mysql, LOCAL) == 0); + assert(db_insert(mysql, LOCAL, "INSERT INTO version(cacti) VALUES ('1.2.32')")); +} + +typedef struct { + poller_thread_t thread; + int errors; +} test_poll_work_t; + +static void *test_poll_worker(void *argument) { + test_poll_work_t *work = argument; + assert(mysql_thread_init() == 0); + poll_host(&work->thread, &work->errors); + return NULL; +} + +static void test_poll_missing_connection(const poller_thread_t *work) { + for (int remote = 0; remote <= 1; remote++) { + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + pool_t unavailable = {0}; + if (remote) { + set.poller.poller_id = 2; + set.poller.mode = REMOTE_ONLINE; + db_pool_remote = &unavailable; + } else db_pool_local[0].free = FALSE; + int errors = 0; + poll_host(work, &errors); + _exit(0); + } + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == EXIT_FAILURE); + } +} + +static void test_reindex_pipeline(MYSQL *mysql, test_poll_work_t *work) { + static const struct { + const char *op; + const char *expected; + const char *command; + int action; + int queued; + const char *stored; + const char *output; + } cases[] = { + {"=", "123", "/usr/bin/printf 123", POLLER_ACTION_SCRIPT, 0, "123", "123"}, + {"=", "122", "/usr/bin/printf 123", POLLER_ACTION_SCRIPT, 1, "123", "123"}, + {">", "122", "/usr/bin/printf 123", POLLER_ACTION_SCRIPT, 1, "123", "123"}, + {">", "124", "/usr/bin/printf 123", POLLER_ACTION_SCRIPT, 0, "123", "123"}, + {"<", "124", "/usr/bin/printf 123", POLLER_ACTION_SCRIPT, 1, "123", "U"}, + {"<", "122", "/usr/bin/printf 123", POLLER_ACTION_SCRIPT, 0, "123", "123"}, + {"<", "0", "/usr/bin/printf 123", POLLER_ACTION_SCRIPT, 0, "123", "123"}, + {"=", "123", "/usr/bin/printf 'a\\nb\\n'", POLLER_ACTION_SCRIPT_COUNT, 1, "2", "123"}, + {"=", "123", "/usr/bin/printf U", POLLER_ACTION_SCRIPT, 0, "123", "123"}, + {"<", "124", "/usr/bin/printf U", POLLER_ACTION_SCRIPT, 0, "U", "123"}, + {"<", "124", "unknown", 99, 0, "124", "123"} + }; + for (int level = 0; level <= 2; level++) { + set.logging.spine_log_level = level; + for (size_t index = 0; index < sizeof(cases) / sizeof(cases[0]); index++) { + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_reindex")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_command")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost")); + char escaped_command[BUFSIZE]; + db_escape(mysql, escaped_command, sizeof(escaped_command), cases[index].command); + char query[LRG_BUFSIZE]; + spine_snprintf(query, sizeof(query), "INSERT INTO poller_reindex(host_id,data_query_id,action,op,assert_value,arg1) VALUES (%d,7,%d,'%s','%s','%s')", work->thread.host_id, cases[index].action, cases[index].op, cases[index].expected, escaped_command); + assert(db_insert(mysql, LOCAL, query)); + work->thread.complete = FALSE; + work->thread.threads_complete = 0; + work->errors = 0; + pthread_t worker; + assert(pthread_create(&worker, NULL, test_poll_worker, work) == 0); + assert(pthread_join(worker, NULL) == 0); + assert(work->thread.complete && work->thread.threads_complete == 1); + int expected_errors = 1; + if (level == 1) expected_errors += cases[index].queued + (cases[index].queued && STRMATCH(cases[index].output, "U")); + assert(work->errors == expected_errors); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_command WHERE poller_id=1 AND action=%d AND command='%d:7'", POLLER_COMMAND_REINDEX, work->thread.host_id); + assert(database_count(mysql, query) == (unsigned long)cases[index].queued); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_reindex WHERE assert_value='%s'", cases[index].stored); + assert(database_count(mysql, query) == 1); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_output WHERE local_data_id=601 AND output='%s'", cases[index].output); + assert(database_count(mysql, query) == 1); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id=601 AND output='%s'", cases[index].output); + assert(database_count(mysql, query) == 1); + assert(db_pool_local[0].free && spine_permits_available(&available_scripts) == 2); + } + } + set.logging.spine_log_level = 0; + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_reindex")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_command")); +} + +static void test_reindex_query_shortcut(MYSQL *mysql, test_poll_work_t *work) { + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_reindex")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_command")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost")); + assert(db_insert(mysql, LOCAL, "INSERT INTO poller_reindex(host_id,data_query_id,action,op,assert_value,arg1) VALUES (43,7,1,'=','122','/usr/bin/printf 123'),(43,7,1,'=','777','/usr/bin/printf 888'),(43,8,1,'=','455','/usr/bin/printf 456')")); + work->thread.complete = FALSE; + work->thread.threads_complete = 0; + work->errors = 0; + pthread_t worker; + assert(pthread_create(&worker, NULL, test_poll_worker, work) == 0); + assert(pthread_join(worker, NULL) == 0); + assert(work->thread.complete && work->thread.threads_complete == 1 && work->errors == 1); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_command") == 2); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_reindex WHERE (data_query_id=7 AND arg1='/usr/bin/printf 123' AND assert_value='123') OR (data_query_id=7 AND arg1='/usr/bin/printf 888' AND assert_value='777') OR (data_query_id=8 AND arg1='/usr/bin/printf 456' AND assert_value='456')") == 3); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE (local_data_id=601 AND output='123') OR (local_data_id=602 AND output='U')") == 2); + assert(db_pool_local[0].free && spine_permits_available(&available_scripts) == 2); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_reindex")); +} + +static void test_profile_schedule_completion(MYSQL *mysql, test_poll_work_t *aggregate) { + config_t previous = set; + test_poll_work_t original = *aggregate; + set.poller.active_profiles = 2; + set.poller.poller_interval = 5; + set.snmp.total_snmp_ports = 1; + assert(db_insert(mysql, LOCAL, "UPDATE poller_item SET rrd_step=300,rrd_next_step=0 WHERE host_id=43")); + assert(db_insert(mysql, LOCAL, "INSERT INTO poller_item (local_data_id,host_id,poller_id,action,arg1,rrd_name,rrd_step,rrd_next_step) VALUES (603,43,1,1,'/usr/bin/printf 789','not-due',300,100)")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost")); + aggregate->thread.host_thread = 2; + aggregate->thread.host_threads = 2; + aggregate->thread.host_data_ids = 1; + aggregate->thread.complete = FALSE; + aggregate->thread.threads_complete = 0; + /* Deliberately finish partition 2 before partition 1 has selected its rows. */ + test_poll_work_t second = *aggregate; + second.thread.host_thread = 2; + pthread_t worker; + assert(pthread_create(&worker, NULL, test_poll_worker, &second) == 0); + assert(pthread_join(worker, NULL) == 0); + assert(aggregate->thread.threads_complete == 1 && !aggregate->thread.complete); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=43 AND rrd_next_step=0") == 2); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=602 AND output='U'") == 1); + test_poll_work_t first = *aggregate; + first.thread.host_thread = 1; + assert(pthread_create(&worker, NULL, test_poll_worker, &first) == 0); + assert(pthread_join(worker, NULL) == 0); + assert(aggregate->thread.threads_complete == 2 && aggregate->thread.complete); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE (local_data_id=601 AND output='123') OR (local_data_id=602 AND output='U')") == 2); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE (local_data_id=601 AND output='123') OR (local_data_id=602 AND output='U')") == 2); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=603") == 0); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=43 AND ((local_data_id IN (601,602) AND rrd_next_step=295) OR (local_data_id=603 AND rrd_next_step=95))") == 3); + assert(db_pool_local[0].free && spine_permits_available(&available_scripts) == 2); + *aggregate = original; + set = previous; +} + +static void prepare_nullable_snmp_profile(MYSQL *mysql) { + /* These fields are nullable in the shipped schema; missing values + * must survive the real row-loader/session handoff. */ + assert(db_insert(mysql, LOCAL, "UPDATE host SET snmp_context=NULL,snmp_engine_id=NULL WHERE id=44")); + assert(db_insert(mysql, LOCAL, "UPDATE poller_item SET snmp_context=NULL,snmp_engine_id=NULL,arg2=NULL,arg3=NULL WHERE host_id=44")); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_item WHERE host_id=44 AND snmp_context IS NULL AND snmp_engine_id IS NULL AND arg2 IS NULL AND arg3 IS NULL") == 2); +} + +static void assert_snmp_pipeline_samples(MYSQL *mysql, bool spike, bool change_version) { + char query[LRG_BUFSIZE]; + bool first_discarded = spike && !change_version; + const char *first_check = first_discarded ? "output='U'" : "output REGEXP '^[0-9]+$' AND CAST(output AS UNSIGNED)>0"; + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_output WHERE local_data_id=601 AND %s", first_check); + assert(database_count(mysql, query) == 1); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id=601 AND %s", first_check); + assert(database_count(mysql, query) == 1); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=602 AND output='U'") == 1); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id=602 AND output='U'") == 1); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output") == 2); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output_boost") == 2); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_command") == (spike ? 1 : 0)); +} + +static void test_snmp_item_pipeline(MYSQL *mysql, test_poll_work_t *work, const char *agent) { + char escaped_agent[BUFSIZE]; + db_escape(mysql, escaped_agent, sizeof(escaped_agent), agent); + for (int scenario = 0; scenario < 9; scenario++) { + bool change_version = (scenario & 2) != 0; + bool spike = (scenario & 4) != 0; + char query[LRG_BUFSIZE]; + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_reindex")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_command")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost")); + spine_snprintf(query, sizeof(query), "UPDATE host SET max_oids=%d WHERE id=44", 1 + (scenario & 1)); + assert(db_insert(mysql, LOCAL, query)); + spine_snprintf(query, sizeof(query), "UPDATE poller_item SET action=0,hostname='%s',snmp_community='regression',snmp_version=2,snmp_port=1161,snmp_timeout=500,arg1='.1.3.6.1.2.1.1.3.0' WHERE host_id=44", escaped_agent); + assert(db_insert(mysql, LOCAL, query)); + spine_snprintf(query, sizeof(query), "UPDATE poller_item SET snmp_version=%d,arg1='%s' WHERE local_data_id=602", change_version ? 1 : 2, spike ? ".1.3.6.1.2.1.1.3.0" : ".1.3.6.1.2.1.1.999.0"); + assert(db_insert(mysql, LOCAL, query)); + if (spike) assert(db_insert(mysql, LOCAL, "INSERT INTO poller_reindex(host_id,data_query_id,action,op,assert_value,arg1) VALUES (44,7,1,'<','124','/usr/bin/printf 123')")); + if (scenario == 8) { + prepare_nullable_snmp_profile(mysql); + } + work->thread.complete = FALSE; + work->thread.threads_complete = 0; + work->errors = 0; + pthread_t worker; + assert(pthread_create(&worker, NULL, test_poll_worker, work) == 0); + assert(pthread_join(worker, NULL) == 0); + assert(work->thread.complete && work->thread.threads_complete == 1); + assert(work->errors == (spike ? 0 : 1)); + assert(db_pool_local[0].free && spine_permits_available(&available_scripts) == 2); + assert_snmp_pipeline_samples(mysql, spike, change_version); + if (scenario == 8) puts("production nullable SNMP profile handoff passed"); + } + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_reindex")); + assert(db_insert(mysql, LOCAL, "UPDATE host SET snmp_context='',snmp_engine_id='' WHERE id=44")); + assert(db_insert(mysql, LOCAL, "UPDATE poller_item SET snmp_context='',snmp_engine_id='' WHERE host_id=44")); +} + +static void assert_polled_host_statistics(MYSQL *mysql, int host_id) { + if (host_id == 44) { + assert(database_count(mysql, "SELECT COUNT(*) FROM host WHERE id=44 AND total_polls=1 AND failed_polls=0 AND status_last_error='' AND snmp_sysUpTimeInstance>0 AND snmp_sysDescr!='' AND snmp_sysObjectID!='' AND snmp_sysContact='regression' AND snmp_sysName!='' AND snmp_sysLocation='isolated-regression-agent'") == 1); + } else if (host_id != 0) { + char query[256]; + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM host WHERE id=%d AND total_polls=1 AND failed_polls=0 AND min_time=0 AND status_last_error=''", host_id); + assert(database_count(mysql, query) == 1); + } +} + +static void test_poll_pipeline(MYSQL *mysql) { + extern poller_thread_t **details; + config_t previous = set; + pool_t *previous_pool = db_pool_local; + poller_thread_t **previous_details = details; + set.poller.threads = 1; + set.poller.poller_id = 1; + set.poller.poller_interval = 0; + set.poller.active_profiles = 1; + set.boost.boost_enabled = TRUE; + set.boost.boost_redirect = TRUE; + set.availability.ping_only = FALSE; + set.php.script_timeout = 2; + set.logging.spine_log_level = 0; + set.logging.log_destination = 0; + db_pool_local = calloc(1, sizeof(*db_pool_local)); + assert(db_pool_local != NULL); + db_create_connection_pool(LOCAL); + assert(spine_permits_init(&available_scripts, 2) == 0); + set.snmp.mibs = FALSE; + set.availability.ping_recovery_count = 1; + set.availability.ping_failure_count = 1; + const char *agent = getenv("SPINE_TEST_SNMP_HOST"); + const int hosts[] = {0, 42, 43, 44}; + size_t host_count = sizeof(hosts) / sizeof(hosts[0]); + if (agent == NULL || agent[0] == '\0') host_count--; + else snmp_spine_init(); + for (size_t index = 0; index < host_count; index++) { + int host_id = hosts[index]; + set.snmp.mibs = host_id == 44; + assert(db_insert(mysql, LOCAL, "DELETE FROM host")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_item")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost")); + assert(db_insert(mysql, LOCAL, "DELETE FROM host_errors")); + assert(db_insert(mysql, LOCAL, "INSERT INTO poller_item (local_data_id,host_id,poller_id,action,arg1,rrd_name) VALUES (601,0,1,1,'/usr/bin/printf 123','valid'),(602,0,1,1,'/usr/bin/printf invalid','invalid')")); + char host_query[BUFSIZE]; + spine_snprintf(host_query, sizeof(host_query), "UPDATE poller_item SET host_id=%d", host_id); + assert(db_insert(mysql, LOCAL, host_query)); + if (host_id != 0) { + spine_snprintf(host_query, sizeof(host_query), "INSERT INTO host (id,hostname,availability_method,status_fail_date,status_rec_date,max_oids,ping_method,ping_port,status_last_error,min_time,total_polls) VALUES (%d,'127.0.0.1',%d,'2026-10-05 00:00:00','2026-10-05 00:00:00',%d,NULL,NULL,NULL,NULL,NULL)", host_id, host_id == 42 ? AVAIL_STREAM : AVAIL_NONE, host_id == 42 ? 0 : 101); + assert(db_insert(mysql, LOCAL, host_query)); + } + if (host_id == 44) { + char escaped_agent[BUFSIZE]; + db_escape(mysql, escaped_agent, sizeof(escaped_agent), agent); + spine_snprintf(host_query, sizeof(host_query), "UPDATE host SET hostname='%s',availability_method=%d,snmp_version=2,snmp_community='regression',snmp_port=1161,snmp_timeout=500 WHERE id=44", escaped_agent, AVAIL_SNMP); + assert(db_insert(mysql, LOCAL, host_query)); + } + test_poll_work_t *work = calloc(1, sizeof(*work)); + assert(work != NULL); + work->thread.host_id = host_id; + work->thread.host_thread = 1; + work->thread.host_threads = 1; + work->thread.host_data_ids = 2; + work->thread.host_time_double = get_time_as_double(); + STRNCOPY(work->thread.host_time, "1791158400"); + poller_thread_t **owned_details = calloc(1, sizeof(*owned_details)); + assert(owned_details != NULL); + owned_details[0] = &work->thread; + details = owned_details; + pthread_t worker; + assert(pthread_create(&worker, NULL, test_poll_worker, work) == 0); + assert(pthread_join(worker, NULL) == 0); + assert(work->thread.complete && work->thread.threads_complete == 1 && work->errors == 1); + assert(db_pool_local[0].free && spine_permits_available(&available_scripts) == 2); + if (host_id == 0) test_poll_missing_connection(&work->thread); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE (local_data_id=601 AND output='123') OR (local_data_id=602 AND output='U')") == 2); + assert(database_count(mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE (local_data_id=601 AND output='123') OR (local_data_id=602 AND output='U')") == 2); + char query[256]; + spine_snprintf(query, sizeof(query), "SELECT errors FROM host_errors WHERE host_id=%d", host_id); + assert(database_count(mysql, query) == 1); + assert_polled_host_statistics(mysql, host_id); + if (host_id == 44) test_snmp_item_pipeline(mysql, work, agent); + if (host_id == 43) { + test_reindex_pipeline(mysql, work); + test_reindex_query_shortcut(mysql, work); + test_profile_schedule_completion(mysql, work); + } + /* End this fixture binding before the next host or caller runs. */ + details = previous_details; + free(owned_details); + free(work); + } + assert(spine_permits_destroy(&available_scripts) == 0); + db_close_connection_pool(LOCAL); + db_pool_local = previous_pool; + details = previous_details; + set = previous; +} + +static void run_cli_poll_profile(const char *config, const char *poller, const char *threads, const char *interval, const char *profiles, const char *hostlist, int expected) { + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + alarm(15); + if (hostlist != NULL) { + execl("./spine", "spine", "--conf", config, "-p", poller, "-t", threads, "-H", hostlist, "--mode=online", "-O", interval, "-O", profiles, "-S", "-V", "2", NULL); + _exit(127); + } + execl("./spine", "spine", "-C", "/nonexistent/spine-regression.conf", "--conf", config, "-p", poller, "-t", threads, "--mode=online", "-O", interval, "-O", profiles, "-S", "-V", "2", NULL); + _exit(127); + } + int status; + assert(waitpid(child, &status, 0) == child); + printf("Spine executable exit: expected=%d raw_status=%d\n", expected, status); + fflush(stdout); + assert(WIFEXITED(status) && WEXITSTATUS(status) == expected); +} + +static void run_cli_poll(const char *config, const char *poller, const char *threads, const char *interval, int expected) { + run_cli_poll_profile(config, poller, threads, interval, "active_profiles:1", NULL, expected); +} + +static void test_cli_workers(MYSQL *source, const char *config) { + assert(db_insert(source, LOCAL, "REPLACE INTO poller(id,threads) VALUES (1,2)")); + assert(db_insert(source, LOCAL, "DELETE FROM poller_time WHERE poller_id=1")); + assert(db_insert(source, LOCAL, "DELETE FROM poller_output")); + assert(db_insert(source, LOCAL, "DELETE FROM poller_item")); + char query[BUFSIZE]; + spine_snprintf(query, sizeof(query), "UPDATE host SET hostname='127.0.0.1',poller_id=1,disabled='',device_threads=2,availability_method=%i,status_fail_date='2026-10-05 00:00:00',status_rec_date='2026-10-05 00:00:00' WHERE id=42", AVAIL_NONE); + assert(db_insert(source, LOCAL, query)); + assert(db_insert(source, LOCAL, "INSERT INTO poller_item (local_data_id,host_id,poller_id,action,arg1,rrd_name) VALUES (701,42,1,1,'/usr/bin/printf 123','first'),(702,42,1,1,'/usr/bin/printf 456','second')")); + run_cli_poll(config, "1", "2", "poller_interval:5", EXIT_SUCCESS); + assert(database_count(source, "SELECT COUNT(*) FROM poller_output WHERE (local_data_id=701 AND output='123') OR (local_data_id=702 AND output='456')") == 2); + assert(db_insert(source, LOCAL, "DELETE FROM poller_output")); + assert(db_insert(source, LOCAL, "INSERT INTO host(id,hostname,poller_id,disabled,device_threads,availability_method,status_fail_date,status_rec_date) VALUES(43,'127.0.0.1',1,'',1,0,'2026-10-05 00:00:00','2026-10-05 00:00:00')")); + assert(db_insert(source, LOCAL, "INSERT INTO poller_item(local_data_id,host_id,poller_id,action,arg1,rrd_name) VALUES(704,43,1,1,'/usr/bin/printf 789','excluded')")); + run_cli_poll_profile(config, "1", "2", "poller_interval:5", "active_profiles:1", " 0042 , 42 ", EXIT_SUCCESS); + assert(database_count(source, "SELECT COUNT(*) FROM poller_output WHERE (local_data_id=701 AND output='123') OR (local_data_id=702 AND output='456')") == 2); + assert(database_count(source, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=704") == 0); + assert(db_insert(source, LOCAL, "DELETE FROM poller_item WHERE local_data_id=704")); + assert(db_insert(source, LOCAL, "DELETE FROM host WHERE id=43")); + assert(db_insert(source, LOCAL, "DELETE FROM poller_output")); + assert(db_insert(source, LOCAL, "UPDATE poller_item SET rrd_next_step=0 WHERE local_data_id IN (701,702)")); + assert(db_insert(source, LOCAL, "INSERT INTO poller_item (local_data_id,host_id,poller_id,action,arg1,rrd_name,rrd_next_step) VALUES (703,42,1,1,'/usr/bin/printf 789','not-due',100)")); + run_cli_poll_profile(config, "1", "2", "poller_interval:5", "active_profiles:2", NULL, EXIT_SUCCESS); + assert(database_count(source, "SELECT COUNT(*) FROM poller_output WHERE (local_data_id=701 AND output='123') OR (local_data_id=702 AND output='456')") == 2); + assert(database_count(source, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=703") == 0); + assert(db_insert(source, LOCAL, "DELETE FROM poller_item WHERE local_data_id=703")); + assert(db_insert(source, LOCAL, "DELETE FROM poller_output")); + assert(db_insert(source, LOCAL, "UPDATE poller_item SET arg1='/bin/sleep 3' WHERE local_data_id=702")); + double start = spine_monotonic_time(); + run_cli_poll(config, "1", "2", "poller_interval:1", EXIT_FAILURE); + assert(spine_monotonic_time() - start < 3); + assert(database_count(source, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=701 AND output='123'") == 1); + assert(database_count(source, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=702") == 0); + assert(database_count(source, "SELECT COUNT(*) FROM poller_time WHERE poller_id=1 AND end_time='0000-00-00 00:00:00'") == 1); +} + +static void test_cli_transfer_exit(MYSQL *source) { + char database[80]; + char username[80]; + char query[BUFSIZE]; + spine_snprintf(database, sizeof(database), "spine_exit_%ld", (long)getpid()); + spine_snprintf(username, sizeof(username), "spine_cli_%ld", (long)getpid()); + spine_snprintf(query, sizeof(query), "CREATE DATABASE %s CHARACTER SET utf8mb4", database); + assert(db_insert(source, LOCAL, query)); + /* Public fixture credentials, restricted to the two isolated regression databases. */ + spine_snprintf(query, sizeof(query), "CREATE USER '%s'@'%%' IDENTIFIED BY 'regression-only'", username); + assert(db_insert(source, LOCAL, query)); + spine_snprintf(query, sizeof(query), "GRANT SELECT,INSERT,UPDATE,DELETE ON spine_regressions.* TO '%s'@'%%'", username); + assert(db_insert(source, LOCAL, query)); + spine_snprintf(query, sizeof(query), "GRANT SELECT,INSERT,UPDATE,DELETE ON %s.* TO '%s'@'%%'", database, username); + assert(db_insert(source, LOCAL, query)); + MYSQL destination; + db_connect(LOCAL, &destination); + assert(mysql_select_db(&destination, database) == 0); + const char *const tables[] = {"settings", "poller", "poller_item", "version", "host", "poller_time"}; + for (size_t index = 0; index < sizeof(tables) / sizeof(tables[0]); index++) { + spine_snprintf(query, sizeof(query), "CREATE TABLE %s LIKE spine_regressions.%s", tables[index], tables[index]); + assert(db_insert(&destination, REMOTE, query)); + } + assert(db_insert(&destination, REMOTE, "INSERT INTO version VALUES ('1.2.32')")); + assert(db_insert(source, LOCAL, "REPLACE INTO poller(id,threads) VALUES (2,1)")); + assert(db_insert(&destination, REMOTE, "INSERT INTO poller(id,threads) VALUES (2,1)")); + assert(db_insert(source, LOCAL, "DELETE FROM host")); + assert(db_insert(source, LOCAL, "DELETE FROM poller_item")); + /* Disabled devices are excluded from CLI polling, but their status still syncs. */ + assert(db_insert(source, LOCAL, "INSERT INTO host (id,poller_id,disabled) VALUES (42,2,'on')")); + assert(db_insert(&destination, REMOTE, "CREATE TRIGGER reject_cli_transfer BEFORE INSERT ON host FOR EACH ROW SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='regression transfer failure'")); + char config[] = "spine-cli-exit-XXXXXX"; + int fd = mkstemp(config); + assert(fd >= 0); + FILE *file = fdopen(fd, "w"); + assert(file != NULL); + assert(fprintf(file, "DB_Host %s\nDB_Database spine_regressions\nDB_User %s\nDB_Pass regression-only\nDB_Port 3306\nRDB_Host %s\nRDB_Database %s\nRDB_User %s\nRDB_Pass regression-only\nRDB_Port 3306\nCacti_Log %s.log\n", set.database.host, username, set.database.host, database, username, config) > 0); + assert(fclose(file) == 0); + run_cli_poll(config, "2", "1", "poller_interval:5", EXIT_FAILURE); + assert(database_count(&destination, "SELECT COUNT(*) FROM host") == 0); + assert(database_count(source, "SELECT COUNT(*) FROM host") == 1); + assert(db_insert(&destination, REMOTE, "DROP TRIGGER reject_cli_transfer")); + run_cli_poll(config, "2", "1", "poller_interval:5", EXIT_SUCCESS); + assert(database_count(&destination, "SELECT COUNT(*) FROM host WHERE id=42") == 1); + test_cli_workers(source, config); + assert(unlink(config) == 0); + spine_snprintf(query, sizeof(query), "%s.log", config); + if (file_exists(query)) assert(unlink(query) == 0); + db_disconnect(&destination); + spine_snprintf(query, sizeof(query), "DROP USER '%s'@'%%'", username); + assert(db_insert(source, LOCAL, query)); + spine_snprintf(query, sizeof(query), "DROP DATABASE %s", database); + assert(db_insert(source, LOCAL, query)); +} + +static void test_database_configuration(void) { + const char *hostname = getenv("SPINE_TEST_DB_HOST"); + assert(hostname != NULL && hostname[0] != '\0'); + config_defaults(); + strncopy(set.database.host, hostname, sizeof(set.database.host)); + STRNCOPY(set.database.user, "root"); + STRNCOPY(set.database.database, "spine_regressions"); + set.database.password[0] = '\0'; + set.database.port = 3306; + set.poller.poller_id = 1; + set.hosts.start_host_id = -1; + set.hosts.end_host_id = -1; + set.poller.parent_fork = SPINE_PARENT; + MYSQL mysql; + db_connect(LOCAL, &mysql); + /* Restore the fixture input before testing configuration, including after a failed CLI run. */ + assert(db_insert(&mysql, LOCAL, "REPLACE INTO poller (id,threads) VALUES (1,7)")); + test_database_version(&mysql); + assert(db_insert(&mysql, LOCAL, "DELETE FROM settings") == TRUE); + assert(db_insert(&mysql, LOCAL, "INSERT INTO settings (name,value) VALUES ('path_webroot','/srv/cacti'),('path_cactilog',''),('ping_timeout','650'),('script_timeout','2'),('php_servers','100'),('max_get_size','200'),('default_datechar','99')") == TRUE); + assert(db_insert(&mysql, LOCAL, "DELETE FROM poller_item") == TRUE); + assert(db_insert(&mysql, LOCAL, "INSERT INTO poller_item (local_data_id,host_id,action) VALUES (1,42,2)") == TRUE); + db_disconnect(&mysql); + read_config_options(); + assert(set.cacti_version == 1232); + assert(strcmp(set.php.path_php_server, "/srv/cacti/script_server.php") == 0); + assert(strcmp(set.logging.path_logfile, "/srv/cacti/log/cacti.log") == 0); + assert(set.availability.ping_timeout == 650 && set.php.script_timeout == 5); + assert(set.php.php_servers == MAX_PHP_SERVERS && set.snmp.snmp_max_get_size == 128); + assert(set.logging.log_datetime_separator == GDC_DEFAULT); + assert(set.poller.threads == 7 && set.php.php_required); + db_connect(LOCAL, &mysql); + MYSQL_RES *result = db_query(&mysql, LOCAL, "SELECT value FROM settings WHERE name='spine_capabilities'"); + assert(result != NULL && mysql_num_rows(result) == 1); + MYSQL_ROW row = mysql_fetch_row(result); + assert(row != NULL && row[0] != NULL && strstr(row[0], "authProtocols") != NULL); + db_free_result(result); + assert(db_insert(&mysql, LOCAL, "DELETE FROM poller_item") == TRUE); + assert(db_insert(&mysql, LOCAL, "UPDATE settings SET value='/tmp/configured.log' WHERE name='path_cactilog'") == TRUE); + db_disconnect(&mysql); + set.poller.threads_set = TRUE; + set.poller.threads = 3; + STRNCOPY(set.hosts.host_id_list, "42"); + read_config_options(); + assert(set.poller.threads == 3 && !set.php.php_required); + assert(strcmp(set.logging.path_logfile, "/tmp/configured.log") == 0); + set_option("ping_timeout", "777"); + read_config_options(); + assert(set.availability.ping_timeout == 777); + read_config_options(); + assert(set.availability.ping_timeout == 777); + db_connect(LOCAL, &mysql); + test_settings_write_contracts(&mysql); + test_additional_database_contracts(&mysql); + test_poller_queries(&mysql); + test_collector_transfer(&mysql); + test_poll_pipeline(&mysql); + test_output_write_contracts(&mysql); + const char *reindex_agent = getenv("SPINE_TEST_SNMP_HOST"); + if (reindex_agent != NULL && reindex_agent[0] != '\0') test_additional_reindex_contracts(&mysql); + test_cli_transfer_exit(&mysql); + db_disconnect(&mysql); +} + +static void test_snmp_multi_responses(host_t *host) { + target_t items[4] = {0}; + snmp_oids_t oids[4] = {0}; + static const char *const names[] = {".1.3.6.1.2.1.1.6.0", "invalid-regression-oid", ".1.3.6.1.2.1.1.1.999", ".1.3.6.1.2.1.1.4.0"}; + for (int index = 0; index < 4; index++) { + strncopy(oids[index].oid, names[index], sizeof(oids[index].oid)); + oids[index].array_position = index; + items[index].local_data_id = 100 + index; + } + snmp_get_multi(host, items, oids, 4); + assert(!host->ignore_host); + assert(strstr(oids[0].result, "isolated-regression-agent") != NULL); + assert(strcmp(oids[1].result, "U") == 0); + /* v1 reports the missing instance as a PDU error and v2c as an inline + * exception varbind; neither is data, so both store U. */ + assert(strcmp(oids[2].result, "U") == 0); + assert(strstr(oids[3].result, "regression") != NULL); + snmp_get_multi(NULL, NULL, NULL, 0); host_t missing_session = {0}; + snmp_oids_t undefined[2] = {0}; + STRNCOPY(undefined[0].result, "123"); + STRNCOPY(undefined[1].result, "456"); + snmp_get_multi(&missing_session, items, undefined, 2); + assert(missing_session.snmp.status == STAT_DESCRIP_ERROR); + assert(IS_UNDEFINED(undefined[0].result) && IS_UNDEFINED(undefined[1].result)); + struct variable_list value = {0}; + u_char text[] = "regression"; + value.type = ASN_OCTET_STR; + value.val.string = text; + value.val_len = sizeof(text) - 1; + char formatted[128]; + snmp_snprint_value(formatted, sizeof(formatted), NULL, 0, &value); + assert(strstr(formatted, "regression") != NULL); + struct { char before; char value; char after; } bounded = {'a', 'x', 'z'}; + snmp_snprint_value(&bounded.value, 0, NULL, 0, &value); + assert(bounded.before == 'a' && bounded.value == 'x' && bounded.after == 'z'); + snmp_snprint_value(&bounded.value, 1, NULL, 0, &value); + assert(bounded.before == 'a' && bounded.value == '\0' && bounded.after == 'z'); + snmp_snprint_value(NULL, 0, NULL, 0, &value); + snmp_snprint_value(NULL, 1, NULL, 0, &value); + char insufficient[3] = "xx"; + snmp_snprint_value(insufficient, 2, NULL, 0, &value); + assert(strcmp(insufficient, "U") == 0 && insufficient[2] == '\0'); + +} + +static void test_snmp_scalar_responses(host_t *host) { + char *result = snmp_get(host, ".1.3.6.1.2.1.1.6.0"); + assert(strstr(result, "isolated-regression-agent") != NULL && !host->ignore_host); + free(result); + /* A malformed configured OID must not poison later requests or leak a PDU. */ + for (int count = 0; count < 50; count++) { + result = snmp_get(host, "invalid-regression-oid"); + assert(strcmp(result, "U") == 0 && host->snmp.status == STAT_ERROR && !host->ignore_host); + free(result); + result = snmp_getnext(host, "invalid-regression-oid"); + assert(strcmp(result, "U") == 0 && host->snmp.status == STAT_ERROR && !host->ignore_host); + free(result); + } + result = snmp_get_base(host, ".1.3.6.1.2.1.1.1.999", FALSE); + assert(!host->ignore_host); + if (host->snmp.profile.version == 2) assert(strcmp(result, "U") == 0); + else assert(strcmp(result, "") == 0); + free(result); + result = snmp_get(host, ".1.3.6.1.2.1.1.6.0"); + assert(strstr(result, "isolated-regression-agent") != NULL && !host->ignore_host); + free(result); + if (host->snmp.profile.version == 2) { + result = snmp_get(host, ".1.3.6.1.2.1.1.1.999"); + assert(strcmp(result, "U") == 0 && host->ignore_host); + free(result); + result = snmp_get(host, ".1.3.6.1.2.1.1.6.0"); + assert(strcmp(result, "U") == 0); + free(result); + host->ignore_host = FALSE; + } +} + +static void test_system_information(host_t *host) { + MYSQL mysql; + assert(mysql_init(&mysql) != NULL); + int previous_mibs = set.snmp.mibs; + set.snmp.mibs = FALSE; + STRNCOPY(host->system.snmp_sysLocation, "untouched"); + get_system_information(host, &mysql, FALSE); + assert(strcmp(host->system.snmp_sysLocation, "untouched") == 0); + assert(host->system.snmp_sysUpTimeInstance > 0 && !host->ignore_host); + for (int explicit_update = 0; explicit_update <= 1; explicit_update++) { + set.snmp.mibs = !explicit_update; + get_system_information(host, &mysql, explicit_update); + assert(strcmp(host->system.snmp_sysLocation, "isolated-regression-agent") == 0); + assert(strcmp(host->system.snmp_sysContact, "regression") == 0); + assert(host->system.snmp_sysDescr[0] != '\0' && host->system.snmp_sysObjectID[0] != '\0'); + assert(host->system.snmp_sysName[0] != '\0' && !host->ignore_host); + } + /* Missing sessions return allocated U responses; repeated short polls must free them. */ + void *session = host->snmp.session; + int previous_ignore = host->ignore_host; + int previous_status = host->snmp.status; + host->snmp.session = NULL; + unsigned long long previous_uptime = host->system.snmp_sysUpTimeInstance; + set.snmp.mibs = FALSE; + for (int count = 0; count < 100; count++) get_system_information(host, &mysql, FALSE); + assert(host->system.snmp_sysUpTimeInstance == previous_uptime); + host->snmp.session = session; + host->ignore_host = previous_ignore; + host->snmp.status = previous_status; + set.snmp.mibs = previous_mibs; + mysql_close(&mysql); +} + +static void test_snmp_multi_timeout(void) { + int previous_retries = set.snmp.snmp_retries; + set.snmp.snmp_retries = 1; + int silent = socket(AF_INET, SOCK_DGRAM, 0); + assert(silent >= 0); + struct sockaddr_in address = {0}; + address.sin_family = AF_INET; + address.sin_addr.s_addr = htonl(INADDR_LOOPBACK); + address.sin_port = 0; + assert(bind(silent, (struct sockaddr *)&address, sizeof(address)) == 0); + socklen_t address_length = sizeof(address); + assert(getsockname(silent, (struct sockaddr *)&address, &address_length) == 0); + assert(address.sin_family == AF_INET && ntohs(address.sin_port) > 0); + /* Keep the UDP endpoint bound, without receiving or answering requests. + * This produces a real transport timeout instead of ICMP port rejection. */ + for (int version = 1; version <= 2; version++) { + host_t host = {0}; + STRNCOPY(host.hostname, "127.0.0.1"); + STRNCOPY(host.snmp.profile.community, "regression"); + host.snmp.profile.version = version; + host.snmp.profile.port = ntohs(address.sin_port); + host.snmp.profile.timeout = 100; + host.snmp.session = snmp_host_init(&(snmp_connection_t){ + .host_id = 1, + .hostname = host.hostname, + .snmp_version = host.snmp.profile.version, + .snmp_community = host.snmp.profile.community, + .snmp_username = "", + .snmp_password = "", + .snmp_auth_protocol = "SHA", + .snmp_priv_passphrase = "", + .snmp_priv_protocol = "[None]", + .snmp_context = "", + .snmp_engine_id = "", + .snmp_port = host.snmp.profile.port, + .snmp_timeout = host.snmp.profile.timeout, + }); + assert(host.snmp.session != NULL); + const struct snmp_session *session = snmp_sess_session(host.snmp.session); + assert(session != NULL && session->timeout == 100000L && session->retries == 1); + const struct netsnmp_transport_s *transport = snmp_sess_transport(host.snmp.session); + assert(transport != NULL && transport->sock >= 0 && transport->sock != silent); + int session_fd = transport->sock; + target_t items[2] = {0}; + snmp_oids_t oids[2] = {0}; + STRNCOPY(oids[0].oid, ".1.3.6.1.2.1.1.3.0"); + STRNCOPY(oids[1].oid, ".1.3.6.1.2.1.1.1.0"); + for (int index = 0; index < 2; index++) { + oids[index].array_position = index; + items[index].local_data_id = 100 + index; + STRNCOPY(oids[index].result, "123"); + } + snmp_get_multi(&host, items, oids, 2); + assert(host.snmp.status == STAT_TIMEOUT && host.ignore_host); + assert(IS_UNDEFINED(oids[0].result) && IS_UNDEFINED(oids[1].result)); + assert(fcntl(session_fd, F_GETFD) >= 0); + snmp_host_cleanup(host.snmp.session); + host.snmp.session = NULL; + errno = 0; + assert(fcntl(session_fd, F_GETFD) == -1 && errno == EBADF); + } + /* Readiness proves requests reached our local fixture; do not drain them. */ + assert(spine_wait_readable(silent, spine_monotonic_time() + 1) == 1); + assert(close(silent) == 0); + errno = 0; + assert(fcntl(silent, F_GETFD) == -1 && errno == EBADF); + set.snmp.snmp_retries = previous_retries; + puts("production local silent UDP SNMP multi timeout regressions passed"); +} + +static void test_snmp_agent(void) { + const char *address = getenv("SPINE_TEST_SNMP_HOST"); + assert(address != NULL && address[0] != '\0'); + snmp_spine_init(); + set.snmp.snmp_retries = 0; + const int methods[] = {AVAIL_SNMP, AVAIL_SNMP_GET_SYSDESC, AVAIL_SNMP_GET_NEXT}; + for (int version = 1; version <= 2; version++) { + host_t host = {0}; + ping_t ping = {0}; + strncopy(host.hostname, address, sizeof(host.hostname)); + STRNCOPY(host.snmp.profile.community, "regression"); + host.snmp.profile.version = version; + host.snmp.session = snmp_host_init(&(snmp_connection_t){ + .host_id = 1, + .hostname = host.hostname, + .snmp_version = version, + .snmp_community = host.snmp.profile.community, + .snmp_username = "", + .snmp_password = "", + .snmp_auth_protocol = "SHA", + .snmp_priv_passphrase = "", + .snmp_priv_protocol = "[None]", + .snmp_context = "", + .snmp_engine_id = "", + .snmp_port = 1161, + .snmp_timeout = 500, + }); + assert(host.snmp.session != NULL); + test_snmp_scalar_responses(&host); + test_snmp_multi_responses(&host); + test_system_information(&host); + assert(snmp_count(&host, ".1.3.6.1.2.1.1.6") == 1 && !host.ignore_host); + assert(snmp_count(&host, ".1.3.6.1.2.1.1.9999") == 0 && !host.ignore_host); + /* At the end of the entire MIB, v1 sends an agent error. The old + * walker repeated that same request forever. */ + alarm(5); + double begin = spine_monotonic_time(); + int terminal_count = snmp_count(&host, ".2.999"); + alarm(0); + assert(terminal_count >= 0 && terminal_count <= 1); + assert(spine_monotonic_time() - begin < 2.0); + host.ignore_host = FALSE; + for (size_t index = 0; index < sizeof(methods) / sizeof(methods[0]); index++) { + host.availability.method = methods[index]; + assert(ping_host(&host, &ping) == HOST_UP); + assert(strcmp(ping.snmp_response, "Device responded to SNMP") == 0); + assert(atof(ping.snmp_status) >= 0.0); + } + host.availability.ping_method = PING_TCP; + host.availability.port = -1; + host.availability.timeout = 100; + host.availability.method = AVAIL_SNMP_OR_PING; + assert(ping_host(&host, &ping) == HOST_UP); + assert(strcmp(ping.snmp_response, "Device responded to SNMP") == 0); + STRNCOPY(host.hostname, "localhost"); + host.availability.method = AVAIL_SNMP_AND_PING; + assert(ping_host(&host, &ping) == HOST_UP); + snmp_host_cleanup(host.snmp.session); + } + test_snmp_multi_timeout(); + snmp_spine_close(); +} + +static void test_host_status_transitions(void) { + config_t previous = set; + set.logging.log_level = POLLER_VERBOSITY_NONE; + set.availability.ping_failure_count = 2; + set.availability.ping_recovery_count = 2; + host_t host = {0}; + ping_t ping = {0}; + STRNCOPY(host.snmp.profile.community, "regression"); + host.snmp.profile.version = 2; + host.state.status = HOST_UP; + host.statistics.min_time = 1000; + STRNCOPY(ping.ping_status, "4"); + STRNCOPY(ping.snmp_status, "8"); + STRNCOPY(ping.ping_response, "network unavailable"); + STRNCOPY(ping.snmp_response, "SNMP unavailable"); + update_host_status(HOST_DOWN, &host, &ping, AVAIL_SNMP_AND_PING); + assert(host.state.status == HOST_UP && host.state.status_event_count == 1 && host.state.status_fail_date[0] != '\0'); + assert(strcmp(host.state.status_last_error, "SNMP unavailable, network unavailable") == 0); + update_host_status(HOST_DOWN, &host, &ping, AVAIL_SNMP_AND_PING); + assert(host.state.status == HOST_DOWN && host.state.status_event_count == 2); + update_host_status(HOST_UP, &host, &ping, AVAIL_SNMP_AND_PING); + assert(host.state.status == HOST_RECOVERING && host.state.status_event_count == 1 && host.state.status_rec_date[0] != '\0'); + assert(host.statistics.cur_time == 6 && host.statistics.min_time == 6 && host.statistics.max_time == 6 && host.statistics.avg_time == 6); + update_host_status(HOST_DOWN, &host, &ping, AVAIL_SNMP); + assert(host.state.status == HOST_DOWN && host.state.status_event_count == 1); + update_host_status(HOST_UP, &host, &ping, AVAIL_SNMP); + update_host_status(HOST_UP, &host, &ping, AVAIL_PING); + assert(host.state.status == HOST_UP && host.state.status_event_count == 0); + assert(host.statistics.total_polls == 6 && host.statistics.failed_polls == 3 && host.statistics.availability == 50); + assert(host.statistics.cur_time == 4 && host.statistics.avg_time == 6 && host.statistics.min_time == 4 && host.statistics.max_time == 8); + const int methods[] = {AVAIL_NONE, AVAIL_SNMP, AVAIL_PING, AVAIL_SNMP_AND_PING, AVAIL_SNMP_OR_PING}; + const double expected[] = {0, 8, 4, 6, 4}; + for (size_t index = 0; index < sizeof(methods) / sizeof(methods[0]); index++) { + host_t sample = {0}; + sample.statistics.min_time = 1000; + sample.snmp.profile.version = 3; /* v3 needs no community. */ + update_host_status(HOST_UP, &sample, &ping, methods[index]); + assert(sample.state.status == HOST_UP && sample.statistics.cur_time == expected[index] && sample.statistics.avg_time == expected[index]); + } + host_t no_snmp = {0}; + no_snmp.snmp.profile.version = 2; + update_host_status(HOST_UP, &no_snmp, &ping, AVAIL_SNMP); + assert(no_snmp.statistics.cur_time == 0); + update_host_status(HOST_DOWN, &no_snmp, &ping, AVAIL_SNMP); + assert(strcmp(no_snmp.state.status_last_error, "Device does not require SNMP") == 0); + set = previous; +} + +static void test_availability_modes(void) { + host_t host = {0}; + ping_t ping = {0}; + STRNCOPY(host.hostname, "127.0.0.1"); + STRNCOPY(host.snmp.profile.community, "regression"); + host.snmp.profile.version = 2; + host.availability.ping_method = PING_TCP; + host.availability.port = -1; + host.availability.timeout = 100; + /* A failed network check must fall back to SNMP in OR mode. */ + host.availability.method = AVAIL_SNMP_OR_PING; + assert(ping_host(&host, &ping) == HOST_DOWN); + assert(strcmp(ping.snmp_response, "Invalid SNMP Session") == 0); + STRNCOPY(ping.snmp_response, "untouched"); + host.availability.method = AVAIL_SNMP_AND_PING; + assert(ping_host(&host, &ping) == HOST_DOWN); + assert(strcmp(ping.snmp_response, "untouched") == 0); + /* Preserve the existing localhost exemption, and OR short circuit. */ + STRNCOPY(host.hostname, "localhost"); + host.availability.method = AVAIL_SNMP_OR_PING; + assert(ping_host(&host, &ping) == HOST_UP); + assert(strcmp(ping.snmp_response, "untouched") == 0); + host.availability.method = AVAIL_SNMP_AND_PING; + assert(ping_host(&host, &ping) == HOST_DOWN); + assert(strcmp(ping.snmp_response, "Invalid SNMP Session") == 0); + host.availability.method = AVAIL_NONE; + assert(ping_host(&host, &ping) == HOST_UP); + host.availability.method = AVAIL_PING; + assert(ping_host(&host, &ping) == HOST_UP); + host.availability.method = AVAIL_STREAM; + assert(ping_host(&host, &ping) == HOST_DOWN); + host.availability.method = AVAIL_SNMP; + host.snmp.profile.community[0] = '\0'; + assert(ping_host(&host, &ping) == HOST_UP); + host.snmp.profile.version = 3; + assert(ping_host(&host, &ping) == HOST_DOWN); +} + +static void test_tcp_loopback(void) { + int server = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); + assert(server >= 0); + struct sockaddr_in address = {0}; + assert(init_sockaddr(&address, "127.0.0.1", 0)); + assert(bind(server, (struct sockaddr *)&address, sizeof(address)) == 0); + socklen_t length = sizeof(address); + assert(getsockname(server, (struct sockaddr *)&address, &length) == 0); + host_t host = {0}; + STRNCOPY(host.hostname, "127.0.0.1"); + host.availability.port = ntohs(address.sin_port); + host.availability.timeout = 100; + host.availability.retries = 0; + host.availability.ping_method = PING_TCP; + ping_t ping = {0}; + /* Close the reservation so both Darwin and Linux return a refusal. */ + assert(close(server) == 0); + assert(ping_tcp(&host, &ping) == HOST_DOWN); + host.availability.ping_method = PING_TCP_CLOSED; + assert(ping_tcp(&host, &ping) == HOST_UP); + assert(strcmp(ping.ping_response, "TCP: Device is Alive") == 0); + server = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); + assert(server >= 0); + address.sin_port = 0; + assert(bind(server, (struct sockaddr *)&address, sizeof(address)) == 0); + assert(getsockname(server, (struct sockaddr *)&address, &length) == 0); + host.availability.port = ntohs(address.sin_port); + assert(listen(server, 1) == 0); + host.availability.ping_method = PING_TCP; + assert(ping_tcp(&host, &ping) == HOST_UP); + int client = accept(server, NULL, NULL); + assert(client >= 0); + char byte; + assert(read(client, &byte, 1) == 0); + assert(close(client) == 0 && close(server) == 0); + host.availability.timeout = 0; + assert(ping_tcp(&host, &ping) == HOST_DOWN); +} + +static void test_error_id_buffer_boundaries(void) { + config_t previous = set; + set.logging.log_destination = 0; + set.logging.log_level = POLLER_VERBOSITY_NONE; + char storage[DBL_BUFSIZE + 1] = {0}; + storage[DBL_BUFSIZE] = 'G'; + int size = 0; + int count = 0; + buffer_output_errors(storage, &size, &count, 1, 1, INT_MIN, FALSE); + buffer_output_errors(storage, &size, &count, 1, 1, 0, FALSE); + buffer_output_errors(storage, &size, &count, 1, 1, INT_MAX, FALSE); + assert(strcmp(storage, "-2147483648, 0, 2147483647") == 0); + assert(count == 3 && size == (int)strlen(storage)); + assert(strpbrk(storage, "\r\n") == NULL && storage[DBL_BUFSIZE] == 'G'); + /* Fill the caller-owned numeric list to its real boundary, then verify + * overflow starts a fresh record rather than overrunning storage. */ + memset(storage, '7', DBL_BUFSIZE - 2); + storage[DBL_BUFSIZE - 2] = '\0'; + size = DBL_BUFSIZE - 2; + count = 1; + buffer_output_errors(storage, &size, &count, 1, 1, INT_MAX, FALSE); + assert(strcmp(storage, "2147483647") == 0 && size == 10 && count == 1); + assert(storage[DBL_BUFSIZE] == 'G'); + buffer_output_errors(storage, &size, &count, 1, 1, 0, TRUE); + assert(strcmp(storage, "2147483647") == 0 && size == 10 && count == 1); + set = previous; + puts("production numeric error-list boundary regressions passed"); +} + + +int main(int argc, char **argv) { + if (argc == 3 && strcmp(argv[1], "--script-stream") == 0) return run_script_stream_fixture(argv[2]); + if (argc > 1 && strcmp(argv[1], "-q") == 0) return run_test_script_server(argc, argv); + extern int *debug_devices; + static int devices[100]; + debug_devices = devices; + if (argc == 2 && strcmp(argv[1], "--raw-icmp") == 0) { + init_mutexes(); + test_icmp_loopback(); + puts("production ICMP loopback regression passed"); + return 0; + } + if (argc == 2 && strcmp(argv[1], "--icmp-no-capability") == 0) { + init_mutexes(); + test_icmp_socket_failure(); + puts("production ICMP socket-failure regression passed"); + return 0; + } + if (argc == 2 && strcmp(argv[1], "--snmpv3-agent") == 0) { + init_mutexes(); + test_snmpv3_agent_contracts(); + return 0; + } + if (argc == 2 && strcmp(argv[1], "--snmp-agent") == 0) { + init_mutexes(); + test_snmp_agent(); + puts("production SNMP agent regressions passed"); + return 0; + } + if (argc == 2 && strcmp(argv[1], "--database") == 0) { + init_mutexes(); + test_database_configuration(); + puts("production database configuration regressions passed"); + return 0; + } + test_string_conversions(); + test_result_count_range(); + test_cli_option_shape(); + test_concurrent_permits(); + test_copy_bounds(); + test_database_escape(); + test_database_addresses(); + test_database_option_failure(); + test_regex_concurrency(); + test_regex_storage(); + test_formatting(); + test_secret_clear(); + test_log_boundary(); + test_log_sanitization(); + test_log_append_and_failures(); + test_config_directives(); + test_config_structure_bindings(); + test_date_formats(); + test_device_logging(); + test_debug_device_bounds(); + test_poll_result_formats(); + test_hostnames(); + test_udp_deadline(); + test_tcp_loopback(); + test_availability_modes(); + test_icmp_reply_bounds(); + test_snmp_initialization_failure(); + test_snmp_security_protocols(); + test_child_process(); + alarm(10); + test_duplex_process(); + test_write_process(); + test_process_descriptor_aliases(); + alarm(0); + test_php_response(4, true); + test_php_response(RESULTS_BUFFER - 1, true); + test_php_response(RESULTS_BUFFER, false); + test_php_partial_response_timeout(); + init_mutexes(); + test_php_command(4); + test_php_command(BUFSIZE - 3); + test_invalid_php_commands(); + test_php_startup(argv[0]); + test_php_owned_shutdown(); + test_host_status_transitions(); + test_script_execution(); + test_script_stream_contracts(); + test_cli_alias_contracts(); + test_error_id_buffer_boundaries(); + test_additional_contracts(); + puts("production regression tests passed"); + return 0; +} diff --git a/tests/test_spine_reindex_contracts.c b/tests/test_spine_reindex_contracts.c new file mode 100644 index 0000000..045241e --- /dev/null +++ b/tests/test_spine_reindex_contracts.c @@ -0,0 +1,189 @@ +/* + * Copyright (C) 2004-2026 The Cacti Group + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License + * for more details. + */ +#include "common.h" +#include "spine.h" + +extern poller_thread_t **details; + +typedef struct { + poller_thread_t thread; + int errors; +} reindex_test_work_t; + +static unsigned long reindex_count(MYSQL *mysql, const char *query) { + MYSQL_RES *result = db_query(mysql, LOCAL, query); + assert(result != NULL && mysql_num_rows(result) == 1 && mysql_num_fields(result) == 1); + MYSQL_ROW row = mysql_fetch_row(result); + assert(row != NULL && row[0] != NULL); + unsigned long count = strtoul(row[0], NULL, 10); + db_free_result(result); + return count; +} + +static void *run_reindex_worker(void *argument) { + reindex_test_work_t *work = argument; + assert(mysql_thread_init() == 0); + poll_host(&work->thread, &work->errors); + return NULL; +} + +static void run_reindex_case(MYSQL *mysql, int action, const char *argument, + const char *op, const char *expected, const char *stored, int commands) { + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_reindex WHERE host_id=45")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_command WHERE command='45:7' OR command='45:8'")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output WHERE local_data_id=801")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost WHERE local_data_id=801")); + char query[LRG_BUFSIZE]; + spine_snprintf(query, sizeof(query), "INSERT INTO poller_reindex(host_id,data_query_id,action,op,assert_value,arg1) VALUES (45,7,%d,'%s','%s','%s')", action, op, expected, argument); + assert(db_insert(mysql, LOCAL, query)); + reindex_test_work_t *work = calloc(1, sizeof(*work)); + assert(work != NULL); + work->thread.host_id = 45; + work->thread.host_thread = 1; + work->thread.host_threads = 1; + work->thread.host_data_ids = 1; + work->thread.host_time_double = get_time_as_double(); + STRNCOPY(work->thread.host_time, "1791158400"); + poller_thread_t **owned_details = calloc(1, sizeof(*owned_details)); + assert(owned_details != NULL); + owned_details[0] = &work->thread; + poller_thread_t **prior_details = details; + details = owned_details; + pthread_t worker; + assert(pthread_create(&worker, NULL, run_reindex_worker, work) == 0); + assert(pthread_join(worker, NULL) == 0); + assert(work->thread.complete && work->thread.threads_complete == 1 && work->errors == 0); + assert(db_pool_local[0].free && spine_permits_available(&available_scripts) == 2); + assert(reindex_count(mysql, "SELECT COUNT(*) FROM poller_command WHERE poller_id=1 AND action=1 AND command='45:7'") == (unsigned long)commands); + spine_snprintf(query, sizeof(query), "SELECT COUNT(*) FROM poller_reindex WHERE host_id=45 AND data_query_id=7 AND assert_value='%s'", stored); + assert(reindex_count(mysql, query) == 1); + assert(reindex_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=801 AND output='123'") == 1); + assert(reindex_count(mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id=801 AND output='123'") == 1); + assert(reindex_count(mysql, "SELECT COUNT(*) FROM host_errors WHERE host_id=45") == 0); + details = prior_details; + free(owned_details); + free(work); +} + +static void run_uptime_cache_case(MYSQL *mysql) { + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_reindex WHERE host_id=45")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_command WHERE command='45:7' OR command='45:8'")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output WHERE local_data_id=801")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost WHERE local_data_id=801")); + /* Different query IDs prevent the first assertion's shortcut skipping row2. */ + assert(db_insert(mysql, LOCAL, "INSERT INTO poller_reindex(host_id,data_query_id,action,op,assert_value,arg1) VALUES (45,7,0,'>','0','.1.3.6.1.2.1.1.3.0'),(45,8,0,'>','0','.1.3.6.1.6.3.10.2.1.3.0')")); + reindex_test_work_t *work = calloc(1, sizeof(*work)); + assert(work != NULL); + work->thread.host_id = 45; + work->thread.host_thread = 1; + work->thread.host_threads = 1; + work->thread.host_data_ids = 1; + work->thread.host_time_double = get_time_as_double(); + STRNCOPY(work->thread.host_time, "1791158400"); + poller_thread_t **owned_details = calloc(1, sizeof(*owned_details)); + assert(owned_details != NULL); + owned_details[0] = &work->thread; + poller_thread_t **prior_details = details; + details = owned_details; + pthread_t worker; + assert(pthread_create(&worker, NULL, run_reindex_worker, work) == 0); + assert(pthread_join(worker, NULL) == 0); + assert(work->thread.complete && work->thread.threads_complete == 1 && work->errors == 0); + assert(db_pool_local[0].free && spine_permits_available(&available_scripts) == 2); + assert(reindex_count(mysql, "SELECT COUNT(*) FROM poller_command WHERE poller_id=1 AND action=1 AND command IN ('45:7','45:8')") == 2); + assert(reindex_count(mysql, "SELECT COUNT(*) FROM poller_reindex WHERE host_id=45 AND assert_value REGEXP '^[0-9]+$' AND CAST(assert_value AS UNSIGNED)>0") == 2); + /* A failed assertion on an uptime OID means the counters reset, so the + * cycle's samples are discarded as a spike. */ + assert(reindex_count(mysql, "SELECT COUNT(*) FROM poller_output WHERE local_data_id=801 AND output='U'") == 1); + assert(reindex_count(mysql, "SELECT COUNT(*) FROM poller_output_boost WHERE local_data_id=801 AND output='U'") == 1); + assert(reindex_count(mysql, "SELECT COUNT(*) FROM host_errors WHERE host_id=45") == 0); + details = prior_details; + free(owned_details); + free(work); +} + +void test_additional_reindex_contracts(MYSQL *mysql) { + const char *agent = getenv("SPINE_TEST_SNMP_HOST"); + assert(agent != NULL && agent[0] != '\0'); + config_t previous_config = set; + pool_t *previous_pool = db_pool_local; + poller_thread_t **previous_details = details; + php_t *previous_processes = php_processes; + set.poller.threads = 1; + set.poller.poller_id = 1; + set.poller.poller_interval = 0; + set.poller.active_profiles = 1; + set.boost.boost_enabled = TRUE; + set.boost.boost_redirect = TRUE; + set.availability.ping_only = FALSE; + set.php.script_timeout = 2; + set.logging.spine_log_level = 0; + set.logging.log_destination = 0; + set.snmp.mibs = FALSE; + set.availability.ping_recovery_count = 1; + set.availability.ping_failure_count = 1; + set.php.php_servers = 1; + set.php.php_current_server = 0; + set.cacti_version = 1232; + set.poller.mode = REMOTE_ONLINE; + db_pool_local = calloc(1, sizeof(*db_pool_local)); + assert(db_pool_local != NULL); + db_create_connection_pool(LOCAL); + assert(spine_permits_init(&available_scripts, 2) == 0); + assert(db_insert(mysql, LOCAL, "DELETE FROM host_errors WHERE host_id=45")); + assert(db_insert(mysql, LOCAL, "DELETE FROM host WHERE id=45")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_item WHERE local_data_id=801")); + char escaped_agent[BUFSIZE]; + db_escape(mysql, escaped_agent, sizeof(escaped_agent), agent); + char query[LRG_BUFSIZE]; + spine_snprintf(query, sizeof(query), "INSERT INTO host(id,hostname,availability_method,snmp_version,snmp_community,snmp_port,snmp_timeout,status_fail_date,status_rec_date,max_oids) VALUES (45,'%s',%d,2,'regression',1161,500,'2026-10-05 00:00:00','2026-10-05 00:00:00',5)", escaped_agent, AVAIL_NONE); + assert(db_insert(mysql, LOCAL, query)); + assert(db_insert(mysql, LOCAL, "INSERT INTO poller_item(local_data_id,host_id,poller_id,action,arg1,rrd_name) VALUES(801,45,1,1,'/usr/bin/printf 123','reindex_valid')")); + snmp_spine_init(); + run_reindex_case(mysql, POLLER_ACTION_SNMP, ".1.3.6.1.2.1.1.6.0", "=", "isolated-regression-agent", "isolated-regression-agent", 0); + run_reindex_case(mysql, POLLER_ACTION_SNMP, ".1.3.6.1.2.1.1.6.0", "=", "previous", "isolated-regression-agent", 1); + run_reindex_case(mysql, POLLER_ACTION_SNMP, "invalid-regression-oid", "=", "previous", "previous", 0); + run_reindex_case(mysql, POLLER_ACTION_SNMP_COUNT, ".1.3.6.1.2.1.1.6", "=", "0", "1", 1); + run_uptime_cache_case(mysql); + assert(db_insert(mysql, LOCAL, "UPDATE host SET snmp_community='' WHERE id=45")); + run_reindex_case(mysql, POLLER_ACTION_SNMP, ".1.3.6.1.2.1.1.6.0", "=", "previous", "previous", 0); + + php_t processes[1] = {0}; + php_processes = processes; + char *executable = realpath("./test_spine_regressions", NULL); + assert(executable != NULL); + strncopy(set.php.path_php, executable, sizeof(set.php.path_php)); + free(executable); + STRNCOPY(set.php.path_php_server, "regression-server"); + assert(php_init(PHP_INIT)); + run_reindex_case(mysql, POLLER_ACTION_PHP_SCRIPT_SERVER, "regression request", "=", "7", "7", 0); + run_reindex_case(mysql, POLLER_ACTION_PHP_SCRIPT_SERVER, "regression request", "=", "6", "7", 1); + run_reindex_case(mysql, POLLER_ACTION_PHP_SCRIPT_SERVER_COUNT, "regression request", "=", "2", "1", 1); + php_close(PHP_INIT); + assert(processes[0].php_pid == -1 && processes[0].php_read_fd == -1 && processes[0].php_write_fd == -1); + php_processes = previous_processes; + assert(spine_permits_destroy(&available_scripts) == 0); + db_close_connection_pool(LOCAL); + db_pool_local = previous_pool; + details = previous_details; + set = previous_config; + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_reindex WHERE host_id=45")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_command WHERE command='45:7' OR command='45:8'")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output WHERE local_data_id=801")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_output_boost WHERE local_data_id=801")); + assert(db_insert(mysql, LOCAL, "DELETE FROM poller_item WHERE local_data_id=801")); + assert(db_insert(mysql, LOCAL, "DELETE FROM host_errors WHERE host_id=45")); + assert(db_insert(mysql, LOCAL, "DELETE FROM host WHERE id=45")); + puts("production live SNMP and PHP reindex contracts passed"); +} diff --git a/tests/test_spine_settings_contracts.c b/tests/test_spine_settings_contracts.c new file mode 100644 index 0000000..485d7db --- /dev/null +++ b/tests/test_spine_settings_contracts.c @@ -0,0 +1,60 @@ +/* + * Copyright (C) 2004-2026 The Cacti Group + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License + * for more details. + */ +#include "common.h" +#include "spine.h" + +extern int putsetting(MYSQL *, int, const char *, const char *); + +static void assert_saved_setting(MYSQL *mysql, const char *name, const char *expected) { + char query[BUFSIZE]; + spine_snprintf(query, sizeof(query), "SELECT value FROM settings WHERE name='%s'", name); + MYSQL_RES *result = db_query(mysql, LOCAL, query); + assert(result != NULL && mysql_num_rows(result) == 1); + MYSQL_ROW row = mysql_fetch_row(result); + assert(row != NULL && row[0] != NULL && strcmp(row[0], expected) == 0); + db_free_result(result); +} + +void test_settings_write_contracts(MYSQL *mysql) { + config_t previous = set; + set.database.onupdate = 0; /* MariaDB's supported INSERT ... VALUES upsert dialect. */ + set.poller.SQL_readonly = FALSE; + char name[100]; + char trigger[100]; + spine_snprintf(name, sizeof(name), "spine_settings_fault_%ld", (long)getpid()); + spine_snprintf(trigger, sizeof(trigger), "spine_settings_reject_%ld", (long)getpid()); + char query[LRG_BUFSIZE]; + /* This owned key is ASCII letters/digits/underscores; reader escaping is + * a separate contract from the writer's result being verified here. */ + spine_snprintf(query, sizeof(query), "DELETE FROM settings WHERE name='%s'", name); + assert(db_insert(mysql, LOCAL, query)); + assert(putsetting(mysql, LOCAL, name, "first value") == TRUE); + assert_saved_setting(mysql, name, "first value"); + assert(putsetting(mysql, LOCAL, name, "multibyte café 🌵") == TRUE); + assert_saved_setting(mysql, name, "multibyte café 🌵"); + assert(putsetting(mysql, LOCAL, name, "multibyte café 🌵") == TRUE); + assert_saved_setting(mysql, name, "multibyte café 🌵"); + spine_snprintf(query, sizeof(query), "CREATE TRIGGER %s BEFORE INSERT ON settings FOR EACH ROW BEGIN IF NEW.name='%s' THEN SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='owned settings failure'; END IF; END", trigger, name); + assert(db_insert(mysql, LOCAL, query)); + assert(putsetting(mysql, LOCAL, name, "failed replacement") == FALSE); + assert_saved_setting(mysql, name, "multibyte café 🌵"); + spine_snprintf(query, sizeof(query), "DROP TRIGGER %s", trigger); + assert(db_insert(mysql, LOCAL, query)); + assert(putsetting(mysql, LOCAL, name, "retried replacement") == TRUE); + assert_saved_setting(mysql, name, "retried replacement"); + spine_snprintf(query, sizeof(query), "DELETE FROM settings WHERE name='%s'", name); + assert(db_insert(mysql, LOCAL, query)); + set = previous; + puts("production settings write outcome regressions passed"); +} diff --git a/tests/test_spine_snmpv3_contracts.c b/tests/test_spine_snmpv3_contracts.c new file mode 100644 index 0000000..9b78ef9 --- /dev/null +++ b/tests/test_spine_snmpv3_contracts.c @@ -0,0 +1,166 @@ +/* + * Copyright (C) 2004-2026 The Cacti Group + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License + * for more details. + */ +#include "common.h" +#include "spine.h" +#include + +/* Independent SHA-1 Ku oracle from RFC3414 Appendix A.3.2, not a duplicate + * implementation or a second call to the production key-generation API: + * https://www.rfc-editor.org/rfc/rfc3414.html#appendix-A.3.2 */ +static const unsigned char maplesyrup_sha_key[] = { + 0x9f, 0xb5, 0xcc, 0x03, 0x81, 0x49, 0x7b, 0x37, 0x93, 0x52, + 0x89, 0x39, 0xff, 0x78, 0x8d, 0x5d, 0x79, 0x14, 0x52, 0x11 +}; + +static snmp_connection_t session_options(char *hostname, char *username) { + return (snmp_connection_t){ + .host_id = 46, .hostname = hostname, .snmp_version = 3, + .snmp_community = "", .snmp_username = username, + .snmp_password = "maplesyrup", .snmp_auth_protocol = "SHA", + .snmp_priv_passphrase = "", .snmp_priv_protocol = "[None]", + .snmp_context = "", .snmp_engine_id = "", .snmp_port = 1162, + .snmp_timeout = 500 + }; +} + +void test_additional_snmp_session_boundaries(void) { + config_t previous = set; + set.snmp.snmp_retries = 0; + /* Runs inside the default suite's single Net-SNMP init/shutdown lifecycle. */ + snmp_connection_t options = session_options("127.0.0.1", "regression-v3-vector-auth"); + void *handle = snmp_host_init(&options); + assert(handle != NULL); + const struct snmp_session *session = snmp_sess_session(handle); + assert(session != NULL && session->securityLevel == SNMP_SEC_LEVEL_AUTHNOPRIV); + assert(session->securityAuthKeyLen == sizeof(maplesyrup_sha_key)); + assert(memcmp(session->securityAuthKey, maplesyrup_sha_key, sizeof(maplesyrup_sha_key)) == 0); + assert(session->securityPrivKeyLen == 0); + snmp_host_cleanup(handle); + options.snmp_username = "regression-v3-vector-priv"; + options.snmp_priv_protocol = "AES"; + options.snmp_priv_passphrase = "maplesyrup"; + handle = snmp_host_init(&options); + assert(handle != NULL); + session = snmp_sess_session(handle); + assert(session != NULL && session->securityLevel == SNMP_SEC_LEVEL_AUTHPRIV); + assert(session->securityAuthKeyLen == sizeof(maplesyrup_sha_key)); + assert(session->securityPrivKeyLen == sizeof(maplesyrup_sha_key)); + assert(memcmp(session->securityAuthKey, maplesyrup_sha_key, sizeof(maplesyrup_sha_key)) == 0); + assert(memcmp(session->securityPrivKey, maplesyrup_sha_key, sizeof(maplesyrup_sha_key)) == 0); + snmp_host_cleanup(handle); + options.snmp_priv_protocol = "[None]"; + options.snmp_priv_passphrase = ""; + options.snmp_password = "short"; + assert(snmp_host_init(&options) == NULL); + options.snmp_username = "regression-v3-vector-noauth"; + options.snmp_password = ""; + handle = snmp_host_init(&options); + assert(handle != NULL); + session = snmp_sess_session(handle); + assert(session != NULL && session->securityLevel == SNMP_SEC_LEVEL_NOAUTH); + assert(session->securityAuthKeyLen == 0 && session->securityPrivKeyLen == 0); + snmp_host_cleanup(handle); + + options.snmp_version = 2; + options.snmp_community = "regression"; + const int milliseconds[] = {1, 500, INT_MAX / 1000, INT_MAX / 1000 + 1, INT_MAX}; + for (size_t index = 0; index < sizeof(milliseconds)/sizeof(milliseconds[0]); index++) { + options.snmp_timeout = milliseconds[index]; + handle = snmp_host_init(&options); + #if LONG_MAX / 1000L < INT_MAX + if (milliseconds[index] > LONG_MAX / 1000L) { + assert(handle == NULL); + continue; + } + #endif + assert(handle != NULL); + session = snmp_sess_session(handle); + assert(session != NULL && session->timeout == (long)milliseconds[index] * 1000L); + snmp_host_cleanup(handle); + } + options.snmp_timeout = 0; + assert(snmp_host_init(&options) == NULL); + options.snmp_timeout = -1; + assert(snmp_host_init(&options) == NULL); + options.snmp_timeout = INT_MIN; + assert(snmp_host_init(&options) == NULL); + set = previous; + puts("production SNMPv3 key and timeout boundary regressions passed"); +} + +static void assert_v3_response(const snmp_connection_t *options, bool admitted, bool access_denied, int level) { + /* Net-SNMP has a process-global USM user cache. Isolate every credential + * case so a previous admitted session cannot satisfy a later rejected one. */ + pid_t child = fork(); + assert(child >= 0); + if (child == 0) { + alarm(4); + host_t host = {0}; + host.id = options->host_id; + host.snmp.session = snmp_host_init(options); + assert(host.snmp.session != NULL); + const struct snmp_session *session = snmp_sess_session(host.snmp.session); + assert(session != NULL && session->securityLevel == level); + char *response = snmp_get(&host, ".1.3.6.1.2.1.1.6.0"); + assert(response != NULL); + if (admitted) { + assert(strcmp(response, "isolated-regression-v3-agent") == 0); + assert(!host.ignore_host); + } else if (access_denied) { + /* Authenticated VACM refusal is a transport-success agent error. + * Preserve the existing empty-data availability contract. */ + assert(response[0] == '\0'); + assert(host.snmp.status == STAT_SUCCESS && !host.ignore_host); + } else { + assert(IS_UNDEFINED(response)); + assert(host.ignore_host); + } + free(response); + snmp_host_cleanup(host.snmp.session); + snmp_spine_close(); + _exit(0); + } + int status; + assert(waitpid(child, &status, 0) == child); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 0); +} + +void test_snmpv3_agent_contracts(void) { + char *agent = getenv("SPINE_TEST_SNMPV3_HOST"); + assert(agent != NULL && agent[0] != '\0'); + config_t previous = set; + set.snmp.snmp_retries = 0; + snmp_spine_init(); + snmp_connection_t options = session_options(agent, "regression-v3-noauth"); + options.snmp_password = ""; + assert_v3_response(&options, TRUE, FALSE, SNMP_SEC_LEVEL_NOAUTH); + options.snmp_username = "regression-v3-auth"; + options.snmp_password = "maplesyrup"; + assert_v3_response(&options, TRUE, FALSE, SNMP_SEC_LEVEL_AUTHNOPRIV); + options.snmp_password = "regression-wrong-auth"; + assert_v3_response(&options, FALSE, FALSE, SNMP_SEC_LEVEL_AUTHNOPRIV); + options.snmp_username = "regression-v3-priv"; + options.snmp_password = "maplesyrup"; + options.snmp_priv_protocol = "AES"; + options.snmp_priv_passphrase = "regression-privacy"; + assert_v3_response(&options, TRUE, FALSE, SNMP_SEC_LEVEL_AUTHPRIV); + options.snmp_priv_passphrase = "regression-wrong-privacy"; + assert_v3_response(&options, FALSE, FALSE, SNMP_SEC_LEVEL_AUTHPRIV); + options.snmp_priv_protocol = "[None]"; + options.snmp_priv_passphrase = ""; + assert_v3_response(&options, FALSE, TRUE, SNMP_SEC_LEVEL_AUTHNOPRIV); /* VACM requires privacy. */ + snmp_spine_close(); + set = previous; + puts("production live SNMPv3 authentication and privacy regressions passed"); +} diff --git a/tests/test_spine_stream_contracts.c b/tests/test_spine_stream_contracts.c new file mode 100644 index 0000000..ec97a76 --- /dev/null +++ b/tests/test_spine_stream_contracts.c @@ -0,0 +1,82 @@ +/* + * Copyright (C) 2004-2026 The Cacti Group + * + * This program is free software; you can redistribute it and/or modify it + * under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License + * for more details. + */ +#include "common.h" +#include "spine.h" + +/* Dispatch before regression initialization: this is a real child producer, + * not a replacement implementation of exec_poll or its pipe reader. */ +int run_script_stream_fixture(const char *scenario) { + const char *bytes; + int status = 0; + if (strcmp(scenario, "empty") == 0) bytes = ""; + else if (strcmp(scenario, "partial") == 0) bytes = "partial-7"; + else if (strcmp(scenario, "lines") == 0) bytes = "7\n8\n"; + else if (strcmp(scenario, "failure-output") == 0) { bytes = "9"; status = 23; } + else if (strcmp(scenario, "failure-empty") == 0) { bytes = ""; status = 23; } + else return 99; + size_t length = strlen(bytes); + assert(length <= _POSIX_PIPE_BUF); + if (length != 0) { + ssize_t sent; + do { sent = write(STDOUT_FILENO, bytes, length); } while (sent < 0 && errno == EINTR); + assert(sent == (ssize_t)length); + } + return status; +} + +static void assert_stream_result(host_t *host, const char *scenario, const char *type, const char *expected) { + char command[BUFSIZE]; + spine_snprintf(command, sizeof(command), "./test_spine_regressions --script-stream %s", scenario); + char *result = exec_poll(host, command, 991, type); + assert(result != NULL && strcmp(result, expected) == 0); + free(result); + assert(spine_permits_available(&available_scripts) == 1); +} + +void test_script_stream_contracts(void) { + int previous_timeout = set.php.script_timeout; + set.php.script_timeout = 5; + assert(spine_permits_init(&available_scripts, 1) == 0); + host_t host = {0}; + host.id = 991; + STRNCOPY(host.hostname, "owned-stream-fixture"); + assert_stream_result(&host, "empty", "DS", "U"); + assert_stream_result(&host, "empty", "DQ", "U"); + assert_stream_result(&host, "partial", "DS", "partial-7"); + assert_stream_result(&host, "lines", "DS", "7\n8\n"); + /* Legacy script results are determined by bytes, not child exit status. + * Preserve this contract while verifying the pipe backend exposes status. */ + assert_stream_result(&host, "failure-output", "DS", "9"); + assert_stream_result(&host, "failure-empty", "DQ", "U"); + assert(spine_permits_destroy(&available_scripts) == 0); + set.php.script_timeout = previous_timeout; + + int descriptor = nft_popen("./test_spine_regressions --script-stream failure-output", "r"); + assert(descriptor >= 0); + pid_t child = nft_pchild(descriptor); + assert(child > 0); + assert(spine_wait_readable(descriptor, spine_monotonic_time() + 5) == 1); + char output[8] = {0}; + ssize_t received; + do { received = read(descriptor, output, sizeof(output)); } while (received < 0 && errno == EINTR); + assert(received == 1 && output[0] == '9'); + do { received = read(descriptor, output, sizeof(output)); } while (received < 0 && errno == EINTR); + assert(received == 0); + int status = nft_pclose(descriptor); + assert(WIFEXITED(status) && WEXITSTATUS(status) == 23); + assert(nft_pchild(descriptor) == -1 && errno == EBADF); + assert(nft_pclose(descriptor) == -1 && errno == EBADF); + assert(waitpid(child, &status, WNOHANG) == -1 && errno == ECHILD); + puts("production script stream regressions passed"); +} diff --git a/tests/tools/coverage_process_exit.h b/tests/tools/coverage_process_exit.h new file mode 100644 index 0000000..1a9db11 --- /dev/null +++ b/tests/tools/coverage_process_exit.h @@ -0,0 +1,15 @@ +/* Copyright (C) 2026 The Cacti Group + * Licensed under the GNU Lesser General Public License, version 2.1 or later. */ +#ifndef SPINE_COVERAGE_PROCESS_EXIT_H +#define SPINE_COVERAGE_PROCESS_EXIT_H +/* Instrumentation only: flush owned subprocess counters before POSIX _exit. + * Preserve _exit's descriptor and stdio behavior; do not replace it with exit. */ +#include +extern void __gcov_dump(void); +static inline __attribute__((noreturn)) void spine_coverage_process_exit(int status) { + __gcov_dump(); + _exit(status); +} +#define _exit(status) spine_coverage_process_exit(status) + +#endif diff --git a/tests/tools/run_coverage.sh b/tests/tools/run_coverage.sh new file mode 100755 index 0000000..be5de40 --- /dev/null +++ b/tests/tools/run_coverage.sh @@ -0,0 +1,182 @@ +#!/usr/bin/env bash +# Copyright (C) 2026 The Cacti Group +# Licensed under the GNU Lesser General Public License, version 2.1 or later. +# Run only against isolated regression services; database tests mutate fixtures. +set -euo pipefail +: "${SPINE_TEST_DB_HOST:?Set the isolated regression database host}" +: "${SPINE_TEST_SNMP_HOST:?Set the isolated regression SNMP host}" +coverage_base="${SPINE_COVERAGE_DIRECTORY:-coverage}" +mkdir -p "$coverage_base" +coverage_base="$(cd "$coverage_base" && pwd -P)" +source_dir="$(pwd -P)" +coverage_dir="$(mktemp -d "$coverage_base/run.XXXXXX")" +printf '%s\n' incomplete > "$coverage_dir/status.txt" +printf '%s\n' "$coverage_dir" > "$coverage_base/latest-run.txt" +coverage_complete=0 +capability_added=0 +finish() { + status=$? + if [[ "$capability_added" -eq 1 ]]; then + if [[ "$(id -u)" -eq 0 ]]; then setcap -r ./test_spine_regressions; else sudo setcap -r ./test_spine_regressions; fi + fi + if [[ "$coverage_complete" -eq 1 ]] && [[ "$status" -eq 0 ]]; then + printf '%s\n' success > "$coverage_dir/status.txt" + else + printf '%s\n' incomplete > "$coverage_dir/status.txt" + fi +} +trap finish EXIT +for tool in gcc gcov lcov sha256sum tar; do command -v "$tool" >/dev/null; done +# Exclude only the owned evidence directory, never an untested production unit. +coverage_find_path='/not-in-source-tree' +[[ "$coverage_base" != "$source_dir" ]] +case "$coverage_base" in + "$source_dir"/*) coverage_find_path="./${coverage_base#"$source_dir"/}";; + *) ;; +esac +find . -path "$coverage_find_path" -prune -o -type f \( -name '*.c' -o -name '*.h' -o -name 'Makefile.am' -o -name 'configure.ac' -o -name 'copyright_year.sh' -o -path './tests/fixtures/*' -o -path './tests/tools/*' -o -path './.github/workflows/*.yml' \) -not -path './.git/*' -not -path './config/config.h' -print0 | sort -z > "$coverage_dir/source-inputs.nul" +xargs -0 sha256sum < "$coverage_dir/source-inputs.nul" > "$coverage_dir/source.sha256" +tar -czf "$coverage_dir/source-inputs.tar.gz" --null -T "$coverage_dir/source-inputs.nul" +# Autoreconf replaces this tracked template. Preserve its original bytes and +# hash the generated template actually used by configure separately below. +cp Makefile.in "$coverage_dir/source-Makefile.in" +if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then + git rev-parse HEAD > "$coverage_dir/revision.txt" + git status --porcelain > "$coverage_dir/worktree-status.txt" +else + : "${SPINE_SOURCE_REVISION:?Set the exact source revision for a copied build tree}" + printf '%s\n' "$SPINE_SOURCE_REVISION" > "$coverage_dir/revision.txt" + printf '%s\n' 'Copied build tree: source.sha256 is authoritative; repository status unavailable.' > "$coverage_dir/worktree-status.txt" +fi +gcc --version > "$coverage_dir/compiler.txt" +gcc -dumpmachine >> "$coverage_dir/compiler.txt" +if command -v dpkg-query >/dev/null; then + dpkg-query -W gcc libc6-dev libmariadb-dev libsnmp-dev libssl-dev snmpd lcov autoconf automake libtool > "$coverage_dir/dependencies.txt" +else + printf '%s\n' 'Package version inventory unavailable: dpkg-query is not installed.' > "$coverage_dir/dependencies.txt" +fi +gcov --version > "$coverage_dir/gcov.txt" +lcov --version > "$coverage_dir/lcov.txt" +mkdir -p config m4 +autoreconf -fi > "$coverage_dir/build.log" 2>&1 +./configure CC=gcc CFLAGS="-std=gnu11 -g -O0 -UNDEBUG --coverage -fprofile-update=atomic -include $source_dir/tests/tools/coverage_process_exit.h" LDFLAGS='--coverage' >> "$coverage_dir/build.log" 2>&1 +make --no-print-directory coverage-source-list > "$coverage_dir/production-sources.txt" +make --no-print-directory coverage-test-source-list > "$coverage_dir/test-sources.txt" +make --no-print-directory coverage-fault-source-list > "$coverage_dir/fault-sources.txt" +mapfile -t production_sources < "$coverage_dir/production-sources.txt" +mapfile -t test_sources < "$coverage_dir/test-sources.txt" +mapfile -t fault_sources < "$coverage_dir/fault-sources.txt" +[[ "${#production_sources[@]}" -gt 0 ]] +[[ "${#test_sources[@]}" -gt "${#production_sources[@]}" ]] +[[ "${#fault_sources[@]}" -gt 0 ]] +# subdir-objects places each object beside its source, so a producer is the +# source directory plus the per-target object prefix and the unit name. +producer_path() { + local directory + directory="$(dirname "$1")" + if [[ "$directory" = . ]]; then printf '%s%s' "$2" "$(basename "${1%.c}")" + else printf '%s/%s%s' "$directory" "$2" "$(basename "${1%.c}")"; fi +} +producer_units=() +for source in "${production_sources[@]}"; do + case "$source" in *.c) ;; *) printf 'Unsupported production source: %s\n' "$source" >&2; exit 1;; esac + [[ -s "$source" ]] + producer_units+=("$(producer_path "$source" "")") + grep -Fxq "$source" "$coverage_dir/test-sources.txt" + # The fault binary reuses the registered renamed spine globals object. + if [[ "$source" != spine.c ]]; then grep -Fxq "$source" "$coverage_dir/fault-sources.txt"; fi +done +for source in "${test_sources[@]}"; do + case "$source" in *.c) ;; *) printf 'Unsupported test source: %s\n' "$source" >&2; exit 1;; esac + [[ -s "$source" ]] + producer_units+=("$(producer_path "$source" test_spine_regressions-)") +done +for source in "${fault_sources[@]}"; do + case "$source" in *.c) ;; *) printf 'Unsupported fault source: %s\n' "$source" >&2; exit 1;; esac + [[ -s "$source" ]] + producer_units+=("$(producer_path "$source" test_spine_faults-)") +done +printf '%s\n' "${producer_units[@]}" > "$coverage_dir/producers.txt" +[[ "$(sort -u "$coverage_dir/producers.txt" | wc -l)" -eq "${#producer_units[@]}" ]] +make clean >> "$coverage_dir/build.log" 2>&1 +find . -path "$coverage_find_path" -prune -o -name '*.gcda' -type f -exec rm -f {} + +make -j2 spine test_spine_regressions test_spine_faults >> "$coverage_dir/build.log" 2>&1 +sha256sum config/config.h Makefile.in > "$coverage_dir/generated-config.sha256" +cp config/config.h "$coverage_dir/generated-config.h" +cp Makefile.in "$coverage_dir/generated-Makefile.in" +sha256sum spine test_spine_regressions test_spine_faults > "$coverage_dir/binaries.sha256" +cp config.log "$coverage_dir/config.log" +notes=() +for producer in "${producer_units[@]}"; do [[ -s "$producer.gcno" ]]; notes+=("$producer.gcno"); done +tests/tools/verify_coverage.sh --producers "$source_dir" "$coverage_dir/producers.txt" gcno +sha256sum "${notes[@]}" > "$coverage_dir/notes.sha256" +./test_spine_regressions > "$coverage_dir/default.log" 2>&1 +grep -Fq 'production regression tests passed' "$coverage_dir/default.log" +grep -Fq 'production SNMPv3 key and timeout boundary regressions passed' "$coverage_dir/default.log" +grep -Fq 'production CLI alias regressions passed' "$coverage_dir/default.log" +grep -Fq 'production script stream regressions passed' "$coverage_dir/default.log" +grep -Fq 'production config structure bindings passed' "$coverage_dir/default.log" +grep -Fq 'production numeric error-list boundary regressions passed' "$coverage_dir/default.log" +grep -Fq 'production additional contracts passed' "$coverage_dir/default.log" +./test_spine_regressions --database > "$coverage_dir/database.log" 2>&1 +grep -Fq 'production settings write outcome regressions passed' "$coverage_dir/database.log" +grep -Fq 'production database configuration regressions passed' "$coverage_dir/database.log" +grep -Fq 'production output write failure and retry regressions passed' "$coverage_dir/database.log" +grep -Fq 'production output recollection retains incomplete historical samples passed' "$coverage_dir/database.log" +grep -Fq 'production simultaneous output failure ordering regressions passed' "$coverage_dir/database.log" +grep -Fq 'production remote output destination failure and recollection regressions passed' "$coverage_dir/database.log" +grep -Fq 'production nullable SNMP profile handoff passed' "$coverage_dir/database.log" +grep -Fq 'production live SNMP and PHP reindex contracts passed' "$coverage_dir/database.log" +./test_spine_regressions --snmp-agent > "$coverage_dir/snmp.log" 2>&1 +grep -Fq 'production SNMP agent regressions passed' "$coverage_dir/snmp.log" +grep -Fq 'production local silent UDP SNMP multi timeout regressions passed' "$coverage_dir/snmp.log" +SPINE_SNMPV3_EVIDENCE_DIR="$coverage_dir" tests/tools/run_snmpv3.sh > "$coverage_dir/snmpv3.log" 2>&1 +grep -Fq 'production live SNMPv3 authentication and privacy regressions passed' "$coverage_dir/snmpv3.log" +./test_spine_faults > "$coverage_dir/fault-default.log" 2>&1 +grep -Fq 'production logger date-format failure regressions passed' "$coverage_dir/fault-default.log" +grep -Fq 'production process creation failure regressions passed' "$coverage_dir/fault-default.log" +grep -Fq 'production linker fault regressions passed' "$coverage_dir/fault-default.log" +./test_spine_faults --database > "$coverage_dir/fault-database.log" 2>&1 +grep -Fq 'production real lost-connection retry regressions passed' "$coverage_dir/fault-database.log" +grep -Fq 'production ping-only SNMP session ownership regressions passed' "$coverage_dir/fault-database.log" +grep -Fq 'production actual main worker launch failure and retry regressions passed' "$coverage_dir/fault-database.log" +grep -Fq 'production linker fault regressions passed' "$coverage_dir/fault-database.log" +profiles=() +for producer in "${producer_units[@]}"; do [[ -s "$producer.gcda" ]]; profiles+=("$producer.gcda"); done +tests/tools/verify_coverage.sh --producers "$source_dir" "$coverage_dir/producers.txt" gcda +if [[ "$(id -u)" -eq 0 ]]; then + chmod a+rw "${profiles[@]}" + setpriv --reuid=65534 --regid=65534 --clear-groups ./test_spine_regressions --icmp-no-capability > "$coverage_dir/icmp-denied.log" 2>&1 + setcap cap_net_raw+ep ./test_spine_regressions + capability_added=1 + setpriv --reuid=65534 --regid=65534 --clear-groups ./test_spine_regressions --raw-icmp > "$coverage_dir/icmp-capability.log" 2>&1 +else + ./test_spine_regressions --icmp-no-capability > "$coverage_dir/icmp-denied.log" 2>&1 + sudo setcap cap_net_raw+ep ./test_spine_regressions + capability_added=1 + ./test_spine_regressions --raw-icmp > "$coverage_dir/icmp-capability.log" 2>&1 +fi +grep -Fq 'production ICMP socket-failure regression passed' "$coverage_dir/icmp-denied.log" +grep -Fq 'production ICMP loopback regression passed' "$coverage_dir/icmp-capability.log" +lcov_args=() +for source in "${production_sources[@]}"; do lcov_args+=(--include "$source_dir/$source"); done +lcov --capture --directory . --output-file "$coverage_dir/production.info" --rc branch_coverage=1 "${lcov_args[@]}" > "$coverage_dir/capture.log" 2>&1 +tests/tools/test_coverage_verifier.sh > "$coverage_dir/verifier-self-test.log" +tests/tools/verify_coverage.sh "$coverage_dir/production.info" "$source_dir" "$coverage_dir/production-sources.txt" +lcov --summary "$coverage_dir/production.info" --rc branch_coverage=1 > "$coverage_dir/summary.txt" 2>&1 +# Bind every producer and scenario to this immutable tested source snapshot. +sha256sum "${profiles[@]}" > "$coverage_dir/profiles.sha256" +mkdir -p "$coverage_dir/profiles" "$coverage_dir/bin" +cp --parents -- "${notes[@]}" "${profiles[@]}" "$coverage_dir/profiles/" +cp -- spine test_spine_regressions test_spine_faults "$coverage_dir/bin/" +(cd "$coverage_dir" && find profiles bin -type f -print0 | sort -z | xargs -0 sha256sum) > "$coverage_dir/saved-producers.sha256" +{ + sha256sum --check "$coverage_dir/source.sha256" + sha256sum --check "$coverage_dir/generated-config.sha256" + sha256sum --check "$coverage_dir/binaries.sha256" + sha256sum --check "$coverage_dir/notes.sha256" +} > "$coverage_dir/source-verification.log" +printf '%s\n' default numeric-error-boundaries config-bindings script-streams cli-aliases additional-contracts database settings-write-outcome output-write-failure-retry output-recollection-new-timestamp simultaneous-output-failure-ordering remote-output-destination-failure-recollection output-sql-batch-boundary nullable-snmp-profile live-reindex snmp local-silent-udp-snmp-multi-timeout snmpv3-key-timeout snmpv3-live fault-default fault-logger fault-process fault-database-retry fault-ping-only-session worker-launch-admitted worker-launch-rejected worker-launch-eagain-retry icmp-denied icmp-capability > "$coverage_dir/scenarios.txt" +(cd "$coverage_dir" && sha256sum default.log database.log snmp.log snmpv3.log snmpv3-agent.log fault-default.log fault-database.log icmp-denied.log icmp-capability.log scenarios.txt production.info summary.txt source.sha256 generated-config.sha256 binaries.sha256 notes.sha256 profiles.sha256 production-sources.txt test-sources.txt fault-sources.txt producers.txt revision.txt worktree-status.txt compiler.txt dependencies.txt gcov.txt lcov.txt source-Makefile.in generated-Makefile.in generated-config.h source-inputs.nul source-inputs.tar.gz saved-producers.sha256) > "$coverage_dir/evidence.sha256" +cat "$coverage_dir/summary.txt" +coverage_complete=1 diff --git a/tests/tools/run_snmpv3.sh b/tests/tools/run_snmpv3.sh new file mode 100755 index 0000000..c2a27b4 --- /dev/null +++ b/tests/tools/run_snmpv3.sh @@ -0,0 +1,27 @@ +#!/usr/bin/env bash +# Copyright (C) 2026 The Cacti Group +# Licensed under the GNU Lesser General Public License, version 2.1 or later. +# Run only against the owned local agent and public regression credentials. +set -euo pipefail +: "${SPINE_SNMPV3_EVIDENCE_DIR:?Set the owned agent evidence directory}" +command -v snmpd >/dev/null +mkdir -p "$SPINE_SNMPV3_EVIDENCE_DIR" +snmpv3_persistent="$(mktemp -d "${TMPDIR:-/tmp}/spine-snmpv3.XXXXXX")" +snmpv3_agent_pid='' +cleanup_snmpv3() { + status=$? + if [[ -n "$snmpv3_agent_pid" ]]; then + kill "$snmpv3_agent_pid" 2>/dev/null || true + wait "$snmpv3_agent_pid" || true + fi + cp "$snmpv3_persistent/agent.log" "$SPINE_SNMPV3_EVIDENCE_DIR/snmpv3-agent.log" || status=1 + rm -rf -- "$snmpv3_persistent" + exit "$status" +} +trap cleanup_snmpv3 EXIT +cp tests/fixtures/snmpv3-users.conf "$snmpv3_persistent/snmpd.conf" +SNMP_PERSISTENT_DIR="$snmpv3_persistent" snmpd -f -Lo -C -c "$snmpv3_persistent/snmpd.conf,tests/fixtures/snmpv3-agent.conf" > "$snmpv3_persistent/agent.log" 2>&1 & +snmpv3_agent_pid=$! +sleep 1 +kill -0 "$snmpv3_agent_pid" +SPINE_TEST_SNMPV3_HOST=127.0.0.1 ./test_spine_regressions --snmpv3-agent diff --git a/tests/tools/test_coverage_verifier.sh b/tests/tools/test_coverage_verifier.sh new file mode 100755 index 0000000..2192ef5 --- /dev/null +++ b/tests/tools/test_coverage_verifier.sh @@ -0,0 +1,49 @@ +#!/usr/bin/env bash +# Copyright (C) 2026 The Cacti Group +# Licensed under the GNU Lesser General Public License, version 2.1 or later. +set -euo pipefail +owned_dir="$(mktemp -d "${TMPDIR:-/tmp}/spine-coverage-verifier.XXXXXX")" +trap 'rm -rf "$owned_dir"' EXIT +verifier="$(dirname "$0")/verify_coverage.sh" +printf '%s\n' one.c two.c > "$owned_dir/sources.txt" +for source in one.c two.c; do + printf 'SF:/source/%s\nLF:2\nLH:1\nBRF:2\nBRH:1\nFNF:1\nFNH:1\nend_of_record\n' "$source" >> "$owned_dir/complete.info" +done +"$verifier" "$owned_dir/complete.info" /source "$owned_dir/sources.txt" +reject() { + local profile="$1" + if "$verifier" "$profile" /source "$owned_dir/sources.txt" > /dev/null 2>&1; then + printf 'Verifier incorrectly admitted %s\n' "$profile" >&2 + exit 1 + fi +} +reject "$owned_dir/absent.info" +sed '/SF:\/source\/one.c/,/end_of_record/d' "$owned_dir/complete.info" > "$owned_dir/omitted.info" +reject "$owned_dir/omitted.info" +sed -e 's/BRF:2/BRF:0/' -e 's/BRH:1/BRH:0/' "$owned_dir/complete.info" > "$owned_dir/no-branches.info" +reject "$owned_dir/no-branches.info" +sed -e 's/LH:1/LH:0/' -e 's/BRH:1/BRH:0/' "$owned_dir/complete.info" > "$owned_dir/unexecuted.info" +reject "$owned_dir/unexecuted.info" +{ printf 'SF:/source/one.c\n'; sed '/SF:\/source\/one.c/,/end_of_record/d' "$owned_dir/complete.info"; } > "$owned_dir/borrowed-metrics.info" +reject "$owned_dir/borrowed-metrics.info" +sed '$d' "$owned_dir/complete.info" > "$owned_dir/dangling.info" +reject "$owned_dir/dangling.info" +cat "$owned_dir/complete.info" "$owned_dir/complete.info" > "$owned_dir/duplicate.info" +reject "$owned_dir/duplicate.info" +sed '/^LH:/d' "$owned_dir/complete.info" > "$owned_dir/missing-metric.info" +reject "$owned_dir/missing-metric.info" +sed 's/LH:1/LH:3/' "$owned_dir/complete.info" > "$owned_dir/overcount.info" +reject "$owned_dir/overcount.info" +printf '%s\n' one two > "$owned_dir/producers.txt" +printf 'fixture\n' > "$owned_dir/one.gcno" +printf 'fixture\n' > "$owned_dir/two.gcno" +"$verifier" --producers "$owned_dir" "$owned_dir/producers.txt" gcno +if "$verifier" --producers "$owned_dir" "$owned_dir/producers.txt" gcda; then exit 1; fi +printf '%s\n' one missing > "$owned_dir/missing-producer.txt" +if "$verifier" --producers "$owned_dir" "$owned_dir/missing-producer.txt" gcno; then exit 1; fi +printf '%s\n' one one > "$owned_dir/duplicate-producer.txt" +if "$verifier" --producers "$owned_dir" "$owned_dir/duplicate-producer.txt" gcno; then exit 1; fi +printf '%s\n' zero > "$owned_dir/zero-producer.txt" +: > "$owned_dir/zero.gcno" +if "$verifier" --producers "$owned_dir" "$owned_dir/zero-producer.txt" gcno; then exit 1; fi +printf 'Coverage verifier rejects absent, omitted, branchless, unexecuted, malformed, dangling, duplicate and inconsistent reports\n' diff --git a/tests/tools/verify_coverage.sh b/tests/tools/verify_coverage.sh new file mode 100755 index 0000000..71ec69b --- /dev/null +++ b/tests/tools/verify_coverage.sh @@ -0,0 +1,63 @@ +#!/usr/bin/env bash +# Copyright (C) 2026 The Cacti Group +# Licensed under the GNU Lesser General Public License, version 2.1 or later. +set -euo pipefail +if [[ "${1:-}" = '--producers' ]]; then + : "${2:?Usage: verify_coverage.sh --producers BUILD_DIRECTORY MANIFEST gcno|gcda}" + : "${3:?Usage: verify_coverage.sh --producers BUILD_DIRECTORY MANIFEST gcno|gcda}" + : "${4:?Usage: verify_coverage.sh --producers BUILD_DIRECTORY MANIFEST gcno|gcda}" + case "$4" in gcno|gcda) ;; *) exit 1;; esac + [[ -s "$3" ]] + [[ -z "$(sort "$3" | uniq -d)" ]] + while IFS= read -r producer; do + case "$producer" in ''|/*|*..*|*[!a-zA-Z0-9_/-]*) exit 1;; *) ;; esac + [[ -s "$2/$producer.$4" ]] + done < "$3" + exit 0 +fi +: "${1:?Usage: verify_coverage.sh PROFILE SOURCE_DIRECTORY PRODUCTION_MANIFEST}" +: "${2:?Usage: verify_coverage.sh PROFILE SOURCE_DIRECTORY PRODUCTION_MANIFEST}" +: "${3:?Usage: verify_coverage.sh PROFILE SOURCE_DIRECTORY PRODUCTION_MANIFEST}" +[[ -s "$1" ]] +[[ -s "$3" ]] +awk -F: -v source_dir="$2" ' + function fail(message) { print message > "/dev/stderr"; failed=1; exit 1 } + FNR==NR { + if ($0=="" || $0 !~ /\.c$/) fail("Invalid production source manifest") + identity="SF:" source_dir "/" $0 + if (identity in expected) fail("Duplicate production source in manifest") + expected[identity]=1 + next + } + /^SF:/ { + if (open_record) fail("Previous coverage record was not terminated") + identity=$0 + if (!(identity in expected)) fail("Unexpected coverage source: " identity) + if (identity in produced) fail("Duplicate coverage source: " identity) + open_record=1 + delete fields + next + } + /^(LF|LH|BRF|BRH|FNF|FNH):/ { + if (!open_record) fail("Coverage metrics outside a source record") + if (NF!=2 || $2 !~ /^[0-9]+$/ || ($1 in fields)) fail("Invalid or duplicate coverage metric") + fields[$1]=$2+0 + next + } + /^end_of_record$/ { + if (!open_record) fail("Coverage terminator without a source record") + if (!("LF" in fields) || !("LH" in fields) || !("BRF" in fields) || !("BRH" in fields) || !("FNF" in fields) || !("FNH" in fields)) fail("Incomplete coverage metrics") + if (fields["LF"]<=0 || fields["FNF"]<=0 || fields["LH"]>fields["LF"] || fields["BRH"]>fields["BRF"] || fields["FNH"]>fields["FNF"]) fail("Inconsistent coverage metrics") + total_lines_hit+=fields["LH"] + total_branches_hit+=fields["BRH"] + produced[identity]=1 + open_record=0 + next + } + END { + if (failed) exit 1 + if (open_record) fail("Unterminated final coverage record") + for (identity in expected) if (!(identity in produced)) fail("Absent production coverage: " identity) + if (total_lines_hit<=0 || total_branches_hit<=0) fail("No measured line/branch execution") + } +' "$3" "$1" diff --git a/tests/unit/test_linked.c b/tests/unit/test_linked.c index e288083..78f5433 100644 --- a/tests/unit/test_linked.c +++ b/tests/unit/test_linked.c @@ -424,6 +424,8 @@ static void build_ip_icmp(unsigned char *buf, size_t len, uint16_t id, uint16_t iph = (struct ip *) buf; iph->ip_hl = sizeof(struct ip) / 4; iph->ip_v = 4; + /* a raw ICMP socket only delivers ICMP datagrams; the classifier checks it */ + iph->ip_p = IPPROTO_ICMP; pkt = (struct icmp *) (buf + sizeof(struct ip)); pkt->icmp_type = type; @@ -542,10 +544,10 @@ static void test_config_defaults_populates_the_set(void **state) { memset(&set, 0, sizeof set); config_defaults(); - assert_int_equal(set.threads, DEFAULT_THREADS); - assert_int_equal(set.db_port, DEFAULT_DB_PORT); - assert_string_equal(set.db_host, DEFAULT_DB_HOST); - assert_string_equal(set.db_db, DEFAULT_DB_DB); + assert_int_equal(set.poller.threads, DEFAULT_THREADS); + assert_int_equal(set.database.port, DEFAULT_DB_PORT); + assert_string_equal(set.database.host, DEFAULT_DB_HOST); + assert_string_equal(set.database.database, DEFAULT_DB_DB); } static void test_read_spine_config_rejects_a_missing_file(void **state) { @@ -572,10 +574,10 @@ static void test_read_spine_config_reads_settings(void **state) { config_defaults(); assert_int_equal(read_spine_config(path), 0); - assert_string_equal(set.db_host, "testhost"); - assert_string_equal(set.db_db, "testdb"); - assert_string_equal(set.db_user, "testuser"); - assert_int_equal(set.db_port, 3399); + assert_string_equal(set.database.host, "testhost"); + assert_string_equal(set.database.database, "testdb"); + assert_string_equal(set.database.user, "testuser"); + assert_int_equal(set.database.port, 3399); remove(path); } @@ -602,15 +604,15 @@ static void test_set_date_format_clamps_an_out_of_range_format(void **state) { (void) state; config_defaults(); - set.log_datetime_format = GD_MAX + 10; - set.log_datetime_separator = GDC_MAX + 10; + set.logging.log_datetime_format = GD_MAX + 10; + set.logging.log_datetime_separator = GDC_MAX + 10; set_date_format(); fmt = get_date_format(); assert_non_null(fmt); - assert_int_equal(set.log_datetime_format, GD_DEFAULT); - assert_int_equal(set.log_datetime_separator, GDC_DEFAULT); + assert_int_equal(set.logging.log_datetime_format, GD_DEFAULT); + assert_int_equal(set.logging.log_datetime_separator, GDC_DEFAULT); } static void test_get_date_format_covers_each_supported_format(void **state) { @@ -623,8 +625,8 @@ static void test_get_date_format_covers_each_supported_format(void **state) { for (fmt_value = GD_MIN; fmt_value <= GD_MAX; fmt_value++) { for (sep_value = GDC_MIN; sep_value <= GDC_MAX; sep_value++) { - set.log_datetime_format = fmt_value; - set.log_datetime_separator = sep_value; + set.logging.log_datetime_format = fmt_value; + set.logging.log_datetime_separator = sep_value; set_date_format(); fmt = get_date_format(); diff --git a/tests/unit/test_php_runtime.c b/tests/unit/test_php_runtime.c index 071c0ee..d1165ec 100644 --- a/tests/unit/test_php_runtime.c +++ b/tests/unit/test_php_runtime.c @@ -170,11 +170,11 @@ static int php_setup(void **state) { assert_non_null(php_processes); php_processes_initialize(php_processes, MAX_PHP_SERVERS); - set.php_servers = 2; - set.script_timeout = 1; + set.php.php_servers = 2; + set.php.script_timeout = 1; set.cacti_version = 1300; - set.poller_id = 1; - set.log_destination = LOGDEST_STDOUT; + set.poller.poller_id = 1; + set.logging.log_destination = LOGDEST_STDOUT; block_shell_spawn = FALSE; block_php_spawn = FALSE; shell_spawn_reached = FALSE; @@ -192,8 +192,8 @@ static int php_setup(void **state) { track_write = FALSE; write_calls = 0; restore_nofile = FALSE; - snprintf(set.path_php, sizeof(set.path_php), "%s", PHP_TEST_SERVER_PATH); - snprintf(set.path_php_server, sizeof(set.path_php_server), "%s", "normal"); + snprintf(set.php.path_php, sizeof(set.php.path_php), "%s", PHP_TEST_SERVER_PATH); + snprintf(set.php.path_php_server, sizeof(set.php.path_php_server), "%s", "normal"); return 0; } @@ -233,7 +233,7 @@ static void test_round_robin_wraps_at_server_count(void **state) { php_processes[1].php_pid = 12; php_processes[1].php_read_fd = 12; php_processes[1].php_write_fd = 13; - set.php_current_server = set.php_servers; + set.php.php_current_server = set.php.php_servers; assert_int_equal(php_get_process(), 0); assert_int_equal(php_get_process(), 1); assert_int_equal(php_get_process(), 0); @@ -257,7 +257,7 @@ static void test_process_slot_initialization_invalidates_zero_descriptors(void * static void test_round_robin_rejects_failed_slots(void **state) { (void) state; - snprintf(set.path_php, sizeof(set.path_php), "%s", "/does/not/exist/spine-php-test"); + snprintf(set.php.path_php, sizeof(set.php.path_php), "%s", "/does/not/exist/spine-php-test"); assert_int_equal(php_get_process(), -1); } @@ -265,7 +265,7 @@ static void test_round_robin_returns_a_healthy_slot_when_all_are_contended(void int i; (void) state; - for (i = 0; i < set.php_servers; i++) { + for (i = 0; i < set.php.php_servers; i++) { php_processes[i].php_state = PHP_READY; php_processes[i].php_pid = 100 + i; php_processes[i].php_read_fd = 20 + (i * 2); @@ -273,10 +273,10 @@ static void test_round_robin_returns_a_healthy_slot_when_all_are_contended(void thread_mutex_lock(LOCK_PHP_PROC_0 + i); } - set.php_current_server = 0; + set.php.php_current_server = 0; assert_int_equal(php_get_process(), 0); - for (i = 0; i < set.php_servers; i++) { + for (i = 0; i < set.php.php_servers; i++) { thread_mutex_unlock(LOCK_PHP_PROC_0 + i); php_processes[i].php_pid = -1; php_processes[i].php_read_fd = -1; @@ -309,7 +309,7 @@ static void test_dead_slot_is_recovered_while_another_slot_is_contended(void **s php_processes[0].php_pid = 100; php_processes[0].php_read_fd = 20; php_processes[0].php_write_fd = 21; - set.php_current_server = 0; + set.php.php_current_server = 0; assert_int_equal(pthread_create(&holder, NULL, hold_php_slot, &held_slot), 0); pthread_mutex_lock(&spawn_barrier_mutex); while (!shell_spawn_reached) { @@ -341,8 +341,8 @@ static void test_failed_recovery_falls_back_to_the_contended_slot(void **state) php_processes[0].php_pid = 100; php_processes[0].php_read_fd = 20; php_processes[0].php_write_fd = 21; - set.php_current_server = 0; - snprintf(set.path_php, sizeof(set.path_php), "%s", "/does/not/exist/spine-php-test"); + set.php.php_current_server = 0; + snprintf(set.php.path_php, sizeof(set.php.path_php), "%s", "/does/not/exist/spine-php-test"); assert_int_equal(pthread_create(&holder, NULL, hold_php_slot, &held_slot), 0); pthread_mutex_lock(&spawn_barrier_mutex); while (!shell_spawn_reached) { @@ -506,7 +506,7 @@ static void test_legacy_cacti_version_uses_positional_poller_id(void **state) { static void test_remote_poller_id_uses_environ_argument(void **state) { (void) state; - set.poller_id = 2; + set.poller.poller_id = 2; assert_int_equal(php_init(0), TRUE); assert_int_equal(php_processes[0].php_state, PHP_READY); php_close(0); @@ -514,7 +514,7 @@ static void test_remote_poller_id_uses_environ_argument(void **state) { static void test_php_child_restores_sigpipe_default(void **state) { (void) state; - snprintf(set.path_php_server, sizeof(set.path_php_server), "%s", "check-sigpipe"); + snprintf(set.php.path_php_server, sizeof(set.php.path_php_server), "%s", "check-sigpipe"); assert_int_equal(php_init(0), TRUE); assert_int_equal(php_processes[0].php_state, PHP_READY); } @@ -629,7 +629,7 @@ static void test_php_spawn_does_not_inherit_an_nft_collision_descriptor(void **s static void test_init_marks_an_unexpected_handshake_busy(void **state) { (void) state; - snprintf(set.path_php_server, sizeof(set.path_php_server), "%s", "bad-start"); + snprintf(set.php.path_php_server, sizeof(set.php.path_php_server), "%s", "bad-start"); assert_int_equal(php_init(0), TRUE); assert_int_equal(php_processes[0].php_state, PHP_BUSY); } @@ -640,7 +640,7 @@ static void test_busy_handshake_is_recovered_on_the_next_poll(void **state) { pid_t failed_pid; (void) state; - snprintf(set.path_php_server, sizeof(set.path_php_server), "%s", "bad-start"); + snprintf(set.php.path_php_server, sizeof(set.php.path_php_server), "%s", "bad-start"); assert_int_equal(php_init(0), TRUE); assert_int_equal(php_processes[0].php_state, PHP_BUSY); failed_pid = php_processes[0].php_pid; @@ -648,7 +648,7 @@ static void test_busy_handshake_is_recovered_on_the_next_poll(void **state) { assert_true(php_processes[0].php_read_fd >= 0); assert_true(php_processes[0].php_write_fd >= 0); - snprintf(set.path_php_server, sizeof(set.path_php_server), "%s", "normal"); + snprintf(set.php.path_php_server, sizeof(set.php.path_php_server), "%s", "normal"); process = php_get_process(); assert_int_equal(process, 0); assert_true(php_processes[0].php_pid > 1); @@ -692,7 +692,7 @@ static void test_recovery_and_command_share_the_slot_lock(void **state) { pthread_t command_thread; (void) state; - set.php_servers = 1; + set.php.php_servers = 1; block_php_spawn = TRUE; command_finished = FALSE; assert_int_equal(pthread_create(&recovery_thread, NULL, get_php_process_thread, &recovery), 0); @@ -728,9 +728,9 @@ static void test_recovery_attempts_only_one_failed_slot_per_call(void **state) { double elapsed; (void) state; - set.php_servers = 3; - set.script_timeout = 1; - snprintf(set.path_php_server, sizeof(set.path_php_server), "%s", "silent"); + set.php.php_servers = 3; + set.php.script_timeout = 1; + snprintf(set.php.path_php_server, sizeof(set.php.path_php_server), "%s", "silent"); started = get_time_as_double(); assert_int_equal(php_get_process(), -1); elapsed = get_time_as_double() - started; @@ -740,7 +740,7 @@ static void test_recovery_attempts_only_one_failed_slot_per_call(void **state) { static void test_child_exit_during_startup_fails_closed_without_restart(void **state) { (void) state; - snprintf(set.path_php_server, sizeof(set.path_php_server), "%s", "exit-before-start"); + snprintf(set.php.path_php_server, sizeof(set.php.path_php_server), "%s", "exit-before-start"); assert_int_equal(php_init(0), TRUE); assert_int_equal(php_processes[0].php_state, PHP_BUSY); } @@ -751,7 +751,7 @@ static void test_php_init_later_failure_preserves_earlier_server(void **state) { int first_write_fd; (void) state; - set.php_servers = 2; + set.php.php_servers = 2; fail_php_spawn_call = 2; assert_int_equal(php_init(PHP_INIT), FALSE); @@ -846,8 +846,8 @@ static void test_cloexec_pipe_failure_releases_both_descriptors(void **state) { static void test_init_timeout_does_not_recurse(void **state) { (void) state; - set.script_timeout = 0; - snprintf(set.path_php_server, sizeof(set.path_php_server), "%s", "silent"); + set.php.script_timeout = 0; + snprintf(set.php.path_php_server, sizeof(set.php.path_php_server), "%s", "silent"); assert_int_equal(php_init(0), TRUE); assert_int_equal(php_processes[0].php_state, PHP_BUSY); } @@ -857,7 +857,7 @@ static void test_spawn_failure_releases_every_resource(void **state) { /* calloc-like zeroes reproduce the process-wide initialization that used to * let a failed slot masquerade as stdin. php_init() must replace them. */ memset(&php_processes[0], 0, sizeof(php_processes[0])); - snprintf(set.path_php, sizeof(set.path_php), "%s", "/does/not/exist/spine-php-test"); + snprintf(set.php.path_php, sizeof(set.php.path_php), "%s", "/does/not/exist/spine-php-test"); assert_int_equal(php_init(0), FALSE); assert_int_equal(php_processes[0].php_pid, -1); assert_int_equal(php_processes[0].php_read_fd, -1); @@ -931,7 +931,7 @@ static void test_readpipe_rejects_fd_at_fd_setsize(void **state) { oversized_fd = duplicate_at_fdsetsize(pdes[0]); close(pdes[0]); close(pdes[1]); - snprintf(set.path_php, sizeof(set.path_php), "%s", "/does/not/exist/spine-php-test"); + snprintf(set.php.path_php, sizeof(set.php.path_php), "%s", "/does/not/exist/spine-php-test"); php_processes[0].php_state = PHP_READY; php_processes[0].php_read_fd = oversized_fd; result = php_readpipe(0, command); @@ -985,8 +985,8 @@ static void test_command_retires_fd_at_fd_setsize(void **state) { pause(); _exit(0); } - set.php_servers = 1; - snprintf(set.path_php, sizeof(set.path_php), "%s", "/does/not/exist/spine-php-test"); + set.php.php_servers = 1; + snprintf(set.php.path_php, sizeof(set.php.path_php), "%s", "/does/not/exist/spine-php-test"); php_processes[0].php_state = PHP_READY; php_processes[0].php_read_fd = oversized_fd; php_processes[0].php_write_fd = pdes[1]; @@ -1050,7 +1050,7 @@ static void test_failed_write_stops_when_restart_handshake_is_not_ready(void **s (void) state; prepare_broken_ready_slot(); - snprintf(set.path_php_server, sizeof(set.path_php_server), "%s", "bad-start"); + snprintf(set.php.path_php_server, sizeof(set.php.path_php_server), "%s", "bad-start"); result = php_cmd("poll 9", 0); track_write = FALSE; assert_non_null(result); @@ -1095,7 +1095,7 @@ static void test_command_gives_up_after_three_failed_writes(void **state) { } php_processes[0].php_read_fd = pdes[0]; php_processes[0].php_write_fd = pdes[1]; - snprintf(set.path_php, sizeof(set.path_php), "%s", "/does/not/exist/spine-php-test"); + snprintf(set.php.path_php, sizeof(set.php.path_php), "%s", "/does/not/exist/spine-php-test"); assert_int_equal(sigaction(SIGPIPE, NULL, &saved_sigpipe), 0); memset(&default_sigpipe, 0, sizeof(default_sigpipe)); default_sigpipe.sa_handler = SIG_DFL; diff --git a/tests/unit/test_ping_icmp.c b/tests/unit/test_ping_icmp.c index 5b40467..9071ce0 100644 --- a/tests/unit/test_ping_icmp.c +++ b/tests/unit/test_ping_icmp.c @@ -248,11 +248,11 @@ static void make_host(host_t *host, const char *addr) { memset(host, 0, sizeof(*host)); host->id = 1; snprintf(host->hostname, sizeof(host->hostname), "%s", addr); - host->ping_timeout = 400; - host->ping_retries = 1; - host->ping_port = 33439; - host->availability_method = AVAIL_PING; - host->ping_method = PING_ICMP; + host->availability.timeout = 400; + host->availability.retries = 1; + host->availability.port = 33439; + host->availability.method = AVAIL_PING; + host->availability.ping_method = PING_ICMP; } static int ping_reset(void **state) { @@ -263,8 +263,8 @@ static int ping_reset(void **state) { /* is_debug_device() walks this global unguarded and ping_icmp() calls it */ memset(pi_debug_table, 0, sizeof(pi_debug_table)); debug_devices = pi_debug_table; - set.ping_timeout = 400; - set.ping_retries = 1; + set.availability.ping_timeout = 400; + set.availability.ping_retries = 1; use_controlled_socket = 0; controlled_socket_fd = -1; controlled_socket_type = 0; @@ -275,7 +275,7 @@ static int ping_reset(void **state) { packet_size = ICMP_HDR_SIZE + strlen("cacti-monitoring-system"); packet_allocation = NULL; packet_released = 0; - set.icmp_uses_caps = FALSE; + set.availability.icmp_uses_caps = FALSE; controlled_pair[0] = -1; controlled_pair[1] = -1; resolver_mode = 0; @@ -377,8 +377,8 @@ static void test_timeout_releases_packet_and_socket(void **state) { use_owned_controlled_socket(); track_packet = 1; make_host(&host, "127.0.0.1"); - host.ping_timeout = 1; - host.ping_retries = 0; + host.availability.timeout = 1; + host.availability.retries = 0; memset(&ping, 0, sizeof(ping)); assert_int_equal(ping_icmp(&host, &ping), HOST_DOWN); @@ -424,7 +424,7 @@ static void test_cached_capability_path_releases_resources(void **state) { (void) state; use_controlled_socket = 1; controlled_socket_fd = FD_SETSIZE; - set.icmp_uses_caps = TRUE; + set.availability.icmp_uses_caps = TRUE; make_host(&host, "127.0.0.1"); memset(&ping, 0, sizeof(ping)); diff --git a/tests/unit/test_snmpv3_session.c b/tests/unit/test_snmpv3_session.c index d4bb510..d3543d7 100644 --- a/tests/unit/test_snmpv3_session.c +++ b/tests/unit/test_snmpv3_session.c @@ -70,7 +70,7 @@ static int session_reset(void **state) { (void) state; config_defaults(); init_mutexes(); - set.snmp_retries = 1; + set.snmp.snmp_retries = 1; captured_security_level = -1; captured_auth_key_len = 0; captured_priv_key_len = 0; @@ -133,8 +133,14 @@ static int level_for(char *auth_protocol, char *auth_password, char eid[] = ""; void *sessp; - sessp = snmp_host_init(1, host, 3, NULL, user, auth_password, auth_protocol, - priv_passphrase, priv_protocol, ctx, eid, 161, 500); + const snmp_connection_t options = { + .host_id = 1, .hostname = host, .snmp_version = 3, .snmp_community = NULL, + .snmp_username = user, .snmp_password = auth_password, .snmp_auth_protocol = auth_protocol, + .snmp_priv_passphrase = priv_passphrase, .snmp_priv_protocol = priv_protocol, + .snmp_context = ctx, .snmp_engine_id = eid, .snmp_port = 161, .snmp_timeout = 500 + }; + + sessp = snmp_host_init(&options); if (sessp == NULL) { return -1; diff --git a/tests/unit/test_util_paths.c b/tests/unit/test_util_paths.c index 682c562..1d8dcf2 100644 --- a/tests/unit/test_util_paths.c +++ b/tests/unit/test_util_paths.c @@ -45,13 +45,19 @@ static char *item_inserts[8]; static int host_insert_count; static int item_insert_count; static int total_insert_count; +static int host_insert_failures; #define mysql_num_rows test_mysql_num_rows #define mysql_get_server_version test_mysql_get_server_version #define mysql_fetch_row test_mysql_fetch_row +#define mysql_num_fields test_mysql_num_fields +#define mysql_errno test_mysql_errno my_ulonglong test_mysql_num_rows(MYSQL_RES *res) { (void) res; return (my_ulonglong) rows_to_report; } unsigned long test_mysql_get_server_version(MYSQL *mysql) { (void) mysql; return 80020; } +/* The remote push checks the column count against its transfer plan. */ +unsigned int test_mysql_num_fields(MYSQL_RES *res) { (void) res; return fake_query_kind == FAKE_QUERY_ITEMS ? 5 : 21; } +unsigned int test_mysql_errno(MYSQL *mysql) { (void) mysql; return 0; } MYSQL_ROW test_mysql_fetch_row(MYSQL_RES *res) { static char *cells[21]; @@ -129,13 +135,18 @@ int db_insert(MYSQL *mysql, int type, const char *query) { assert_true(host_insert_count < 8); host_inserts[host_insert_count++] = strdup(query); assert_non_null(host_inserts[host_insert_count - 1]); + if (host_insert_failures > 0) { + host_insert_failures--; + return 0; + } } else if (strncmp(query, "INSERT INTO poller_item ", strlen("INSERT INTO poller_item ")) == 0) { assert_true(item_insert_count < 8); item_inserts[item_insert_count++] = strdup(query); assert_non_null(item_inserts[item_insert_count - 1]); } - return 0; + /* TRUE: the remote push stops on the first failed statement. */ + return 1; } void db_escape(MYSQL *mysql, char *output, int max_size, const char *input) { size_t in = 0; @@ -152,7 +163,7 @@ void db_escape(MYSQL *mysql, char *output, int max_size, const char *input) { } output[out] = '\0'; } -int append_hostrange(char *obuf, const char *colname) { return 0; } +int append_hostrange(char *obuf, size_t capacity, const char *colname) { return 0; } int parse_logdest(const char *res, int default_dest) { return 0; } const char *printable_logdest(int dest) { return ""; } void php_close(int php_process) {} @@ -162,10 +173,12 @@ void php_close(int php_process) {} #undef mysql_num_rows #undef mysql_get_server_version #undef mysql_fetch_row +#undef mysql_num_fields +#undef mysql_errno static char *build(int sep_code, int fmt_code) { - set.log_datetime_separator = sep_code; - set.log_datetime_format = fmt_code; + set.logging.log_datetime_separator = sep_code; + set.logging.log_datetime_format = fmt_code; set_date_format(); return get_date_format(); @@ -221,8 +234,8 @@ static void test_out_of_range_codes_clamp_to_defaults(void **state) { from_default = build(GDC_MAX + 7, GD_MAX + 7); assert_string_equal(from_default, expected); - assert_int_equal(set.log_datetime_separator, GDC_DEFAULT); - assert_int_equal(set.log_datetime_format, GD_DEFAULT); + assert_int_equal(set.logging.log_datetime_separator, GDC_DEFAULT); + assert_int_equal(set.logging.log_datetime_format, GD_DEFAULT); from_default = build(GDC_MIN - 3, GD_MIN - 3); assert_string_equal(from_default, expected); @@ -369,33 +382,12 @@ static int count_occurrences(const char *haystack, const char *needle) { return count; } -static void test_push_flush_drops_suffix_overflow(void **state) { - char *sqlbuf; - char *cursor; - MYSQL mysql; - int before; - - (void) state; - sqlbuf = malloc(HUGE_BUFSIZE); - assert_non_null(sqlbuf); - memset(sqlbuf, 'x', HUGE_BUFSIZE - 1); - sqlbuf[HUGE_BUFSIZE - 1] = '\0'; - cursor = sqlbuf + HUGE_BUFSIZE - 1; - before = total_insert_count; - - push_flush_batch(&mysql, sqlbuf, &cursor, " suffix"); - - assert_int_equal(total_insert_count, before); - assert_ptr_equal(cursor, sqlbuf + HUGE_BUFSIZE - 1); - free(sqlbuf); -} - static void test_remote_push_keeps_batch_boundary_rows(void **state) { int i; (void) state; memset(&set, 0, sizeof(set)); - set.poller_id = 2; + set.poller.poller_id = 2; row_is_null = 0; fake_query_kind = FAKE_QUERY_DEFAULT; byte_boundary_mode = 0; @@ -406,14 +398,14 @@ static void test_remote_push_keeps_batch_boundary_rows(void **state) { assert_int_equal(host_insert_count, 2); assert_int_equal(item_insert_count, 2); - assert_int_equal(count_occurrences(host_inserts[0], " (500, "), 1); - assert_int_equal(count_occurrences(host_inserts[0], " (501, "), 0); - assert_int_equal(count_occurrences(host_inserts[1], " (500, "), 0); - assert_int_equal(count_occurrences(host_inserts[1], " (501, "), 1); - assert_int_equal(count_occurrences(item_inserts[0], " (10000, "), 1); - assert_int_equal(count_occurrences(item_inserts[0], " (10001, "), 0); - assert_int_equal(count_occurrences(item_inserts[1], " (10000, "), 0); - assert_int_equal(count_occurrences(item_inserts[1], " (10001, "), 1); + assert_int_equal(count_occurrences(host_inserts[0], " ('500', "), 1); + assert_int_equal(count_occurrences(host_inserts[0], " ('501', "), 0); + assert_int_equal(count_occurrences(host_inserts[1], " ('500', "), 0); + assert_int_equal(count_occurrences(host_inserts[1], " ('501', "), 1); + assert_int_equal(count_occurrences(item_inserts[0], " ('10000', "), 1); + assert_int_equal(count_occurrences(item_inserts[0], " ('10001', "), 0); + assert_int_equal(count_occurrences(item_inserts[1], " ('10000', "), 0); + assert_int_equal(count_occurrences(item_inserts[1], " ('10001', "), 1); assert_null(strstr(host_inserts[0], " AS rs ")); assert_non_null(strstr(host_inserts[0], "VALUES(snmp_sysDescr)")); assert_null(strstr(item_inserts[0], " AS rs ")); @@ -433,7 +425,7 @@ static void test_remote_push_flushes_wide_rows_before_overflow(void **state) { (void) state; memset(&set, 0, sizeof(set)); - set.poller_id = 2; + set.poller.poller_id = 2; row_is_null = 0; fake_query_kind = FAKE_QUERY_DEFAULT; byte_boundary_mode = 1; @@ -458,6 +450,33 @@ static void test_remote_push_flushes_wide_rows_before_overflow(void **state) { fake_query_kind = FAKE_QUERY_DEFAULT; } +/* A failed batch must end the push: the row that triggered the flush is not + * queued for a later batch, and poller items are never attempted. */ +static void test_remote_push_stops_on_failed_batch(void **state) { + int before; + + (void) state; + memset(&set, 0, sizeof(set)); + set.poller.poller_id = 2; + row_is_null = 0; + fake_query_kind = FAKE_QUERY_DEFAULT; + byte_boundary_mode = 0; + host_insert_count = 0; + item_insert_count = 0; + host_insert_failures = 1; + before = frees_seen; + + assert_false(poller_transfer_status(&fake_mysql, &fake_mysql)); + + assert_int_equal(host_insert_failures, 0); + assert_int_equal(host_insert_count, 1); + assert_int_equal(item_insert_count, 0); + assert_int_equal(frees_seen - before, 1); + assert_int_equal(count_occurrences(host_inserts[0], " ('501', "), 0); + free(host_inserts[0]); + host_inserts[0] = NULL; + fake_query_kind = FAKE_QUERY_DEFAULT; +} /* ---- issue#565: spine_log() appends the newline without overrunning ---- */ @@ -467,11 +486,11 @@ static void route_log_to_a_file(void) { snprintf(log_path, sizeof(log_path), "/tmp/spine_log_test_%d.log", (int) getpid()); unlink(log_path); - set.log_destination = LOGDEST_FILE; - set.log_level = POLLER_VERBOSITY_DEBUG; - set.logfile_processed = TRUE; - set.poller_id = 1; - snprintf(set.path_logfile, sizeof(set.path_logfile), "%s", log_path); + set.logging.log_destination = LOGDEST_FILE; + set.logging.log_level = POLLER_VERBOSITY_DEBUG; + set.logging.logfile_processed = TRUE; + set.poller.poller_id = 1; + snprintf(set.logging.path_logfile, sizeof(set.logging.path_logfile), "%s", log_path); } static char *read_log(size_t *len) { @@ -531,7 +550,9 @@ static void test_spine_log_survives_a_full_line(void **state) { out = read_log(&n); assert_non_null(out); assert_int_equal(n, LOGSIZE - 1); - assert_int_equal(out[n - 1], 'y'); + /* The message is shortened so the record still ends its line. */ + assert_int_equal(out[n - 1], '\n'); + assert_int_equal(out[n - 2], 'y'); free(big); unlink(log_path); @@ -571,7 +592,7 @@ int main(void) { cmocka_unit_test(test_success_path_frees_once), cmocka_unit_test(test_remote_push_keeps_batch_boundary_rows), cmocka_unit_test(test_remote_push_flushes_wide_rows_before_overflow), - cmocka_unit_test(test_push_flush_drops_suffix_overflow), + cmocka_unit_test(test_remote_push_stops_on_failed_batch), cmocka_unit_test(test_spine_log_appends_a_newline), cmocka_unit_test(test_spine_log_survives_a_full_line), cmocka_unit_test(test_spine_log_does_not_double_an_existing_newline), diff --git a/tests/unit/test_util_strings.c b/tests/unit/test_util_strings.c index f8c4661..17d8ad9 100644 --- a/tests/unit/test_util_strings.c +++ b/tests/unit/test_util_strings.c @@ -23,7 +23,7 @@ MYSQL_RES *db_query(MYSQL *mysql, int type, const char *query) { return NULL; } void db_free_result(MYSQL_RES *result) {} int db_insert(MYSQL *mysql, int type, const char *query) { return 0; } void db_escape(MYSQL *mysql, char *output, int max_size, const char *input) {} -int append_hostrange(char *obuf, const char *colname) { return 0; } +int append_hostrange(char *obuf, size_t capacity, const char *colname) { return 0; } int parse_logdest(const char *res, int default_dest) { return 0; } const char *printable_logdest(int dest) { return ""; } void php_close(int php_process) {} diff --git a/util.c b/util.c index 4b8ada0..fdef09c 100644 --- a/util.c +++ b/util.c @@ -34,12 +34,174 @@ #include "common.h" #include "spine.h" #include "regex.h" - +#include #include #define SPINE_STRINGIFY_INNER(value) #value #define SPINE_STRINGIFY(value) SPINE_STRINGIFY_INNER(value) +/* Bounded formatting must reject truncation before a partial SQL statement or + * command can be used. Return the number actually written, never the size that + * would have been needed, so append callers cannot advance past the buffer. */ +int spine_snprintf(char *output, size_t capacity, const char *format, ...) { + va_list args; + int length; + + va_start(args, format); + length = vsnprintf(output, capacity, format, args); + va_end(args); + if (length < 0 || (size_t) length >= capacity) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Formatted output exceeds its destination buffer"); + } + return length; +} + +int spine_count_to_int(unsigned long long count) { + if (count > INT_MAX) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Result count exceeds supported integer range"); + } + return (int)count; +} + +void spine_sleep_usec(unsigned int microseconds) { + struct timespec requested = { + (time_t)(microseconds / 1000000), (long)(microseconds % 1000000) * 1000 + }; + while (nanosleep(&requested, &requested) != 0) { + if (errno != EINTR) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Unable to wait for retry delay"); + } + } +} + +double spine_monotonic_time(void) { + struct timespec now; + if (clock_gettime(CLOCK_MONOTONIC, &now) != 0) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Unable to read monotonic clock"); + } + return (double)now.tv_sec + (double)now.tv_nsec / 1000000000; +} + +static int spine_wait_fd(int fd, double deadline, bool writable) { + if (fd < 0 || fd >= FD_SETSIZE) { + errno = EBADF; + return -1; + } + if (!isfinite(deadline)) { + errno = EINVAL; + return -1; + } + for (;;) { + double remaining = deadline - spine_monotonic_time(); + if (remaining <= 0) return 0; + if (remaining > INT_MAX) { + errno = EINVAL; + return -1; + } + struct timeval timeout; + timeout.tv_sec = (time_t)remaining; + timeout.tv_usec = (suseconds_t)((remaining - (double)timeout.tv_sec) * 1000000); + fd_set fds; + FD_ZERO(&fds); + FD_SET(fd, &fds); + int status = select(fd + 1, writable ? NULL : &fds, writable ? &fds : NULL, NULL, &timeout); + if (status < 0 && errno == EINTR) continue; + return status; + } +} + +int spine_wait_readable(int fd, double deadline) { + return spine_wait_fd(fd, deadline, FALSE); +} + +int spine_wait_writable(int fd, double deadline) { + return spine_wait_fd(fd, deadline, TRUE); +} + + +int spine_permits_init(spine_permits_t *permits, int count) { + if (count < 0) return EINVAL; + int status = pthread_mutex_init(&permits->mutex, NULL); + if (status == 0) permits->available = count; + return status; +} + +int spine_permits_destroy(spine_permits_t *permits) { + return pthread_mutex_destroy(&permits->mutex); +} + +static void spine_permits_lock(spine_permits_t *permits) { + if (pthread_mutex_lock(&permits->mutex) != 0) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Unable to lock process permits"); + } +} + +static void spine_permits_unlock(spine_permits_t *permits) { + if (pthread_mutex_unlock(&permits->mutex) != 0) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Unable to unlock process permits"); + } +} + +int spine_permits_try_acquire(spine_permits_t *permits) { + spine_permits_lock(permits); + int status = EAGAIN; + if (permits->available > 0) { + permits->available--; + status = 0; + } + spine_permits_unlock(permits); + return status; +} + +int spine_permits_release(spine_permits_t *permits) { + spine_permits_lock(permits); + int status = EOVERFLOW; + if (permits->available < INT_MAX) { + permits->available++; + status = 0; + } + spine_permits_unlock(permits); + return status; +} + +int spine_permits_available(spine_permits_t *permits) { + spine_permits_lock(permits); + int available = permits->available; + spine_permits_unlock(permits); + return available; +} + +void spine_clear_sensitive(void *buffer, size_t length) { + volatile unsigned char *bytes = buffer; + while (length > 0) { + *bytes++ = 0; + length--; + } +} + +/* Preserve printable device output while keeping each log record on one line. + * Also remove terminal controls and Unicode line/paragraph separators. */ +void spine_sanitize_log_message(char *message) { + for (unsigned char *p = (unsigned char *)message; *p != 0; p++) { + if (*p < 0x20 || *p == 0x7f) { + *p = ' '; + } else if (p[0] == 0xc2 && p[1] == 0x85) { + p[0] = ' '; + p[1] = ' '; + } else if (p[0] == 0xe2 && p[1] == 0x80 && (p[2] == 0xa8 || p[2] == 0xa9)) { + p[0] = ' '; + p[1] = ' '; + p[2] = ' '; + } + } +} + static int nopts = 0; /*! Override Options Structure @@ -65,10 +227,9 @@ static struct { * */ void set_option(const char *option, const char *value) { - if (nopts >= (int)(sizeof(opttable) / sizeof(opttable[0]))) { - die("ERROR: too many --option overrides"); + if (option == NULL || value == NULL || nopts >= (int)(sizeof(opttable) / sizeof(opttable[0]))) { + die("ERROR: Invalid or excessive command-line setting overrides"); } - opttable[nopts ].opt = option; opttable[nopts++].val = value; } @@ -174,13 +335,12 @@ static char *getsetting(MYSQL *psql, int mode, const char *setting) { char *retval; MYSQL_RES *result; MYSQL_ROW mysql_row; - int i; assert(psql != 0); assert(setting != 0); /* see if it's in the option table */ - for (i=0; i 0) { - mysql_row = mysql_fetch_row(result); - - if (mysql_row != NULL) { - retval = strdup(mysql_row[0]); - db_free_result(result); - return retval; - }else{ - db_free_result(result); - return strdup(""); - } - }else{ - db_free_result(result); - return strdup(""); - } - }else{ - return strdup(""); - } + if (result == NULL) return strdup(""); + mysql_row = mysql_num_rows(result) > 0 ? mysql_fetch_row(result) : NULL; + retval = mysql_row != NULL && mysql_row[0] != NULL ? strdup(mysql_row[0]) : strdup(""); + db_free_result(result); + return retval; } /*! \fn int putsetting(MYSQL *psql, const char *setting, const char *value) @@ -233,7 +378,7 @@ static char *getsetting(MYSQL *psql, int mode, const char *setting) { * \return true for successful or false for failed * */ -static int putsetting(MYSQL *psql, int mode, const char *mysetting, const char *myvalue) { +int putsetting(MYSQL *psql, int mode, const char *mysetting, const char *myvalue) { char qstring[BUFSIZE]; int result = 0; @@ -241,23 +386,19 @@ static int putsetting(MYSQL *psql, int mode, const char *mysetting, const char * assert(mysetting != 0); assert(myvalue != 0); - if (set.dbonupdate == 0) { - snprintf(qstring, sizeof(qstring), "INSERT INTO settings (name, value) " + if (set.database.onupdate == 0) { + spine_snprintf(qstring, sizeof(qstring), "INSERT INTO settings (name, value) " "VALUES ('%s', '%s') " "ON DUPLICATE KEY UPDATE value = VALUES(value)", mysetting, myvalue); } else { - snprintf(qstring, sizeof(qstring), "INSERT INTO settings (name, value) " + spine_snprintf(qstring, sizeof(qstring), "INSERT INTO settings (name, value) " "VALUES ('%s', '%s') AS rs " "ON DUPLICATE KEY UPDATE value = rs.value", mysetting, myvalue); } result = db_insert(psql, mode, qstring); - if (result == 0) { - return TRUE; - } else { - return FALSE; - } + return result; } /*! \fn static char *getpsetting(MYSQL *psql, const char *setting) @@ -278,13 +419,12 @@ static char *getpsetting(MYSQL *psql, int mode, const char *setting) { char *retval; MYSQL_RES *result; MYSQL_ROW mysql_row; - int i; assert(psql != 0); assert(setting != 0); /* see if it's in the option table */ - for (i=0; i 0) { - mysql_row = mysql_fetch_row(result); - - if (mysql_row != NULL) { - retval = strdup(mysql_row[0]); - db_free_result(result); - return retval; - } else { - db_free_result(result); - return 0; - } - } else { - db_free_result(result); - return 0; - } - } else { - return 0; - } + if (result == NULL) return NULL; + mysql_row = mysql_num_rows(result) > 0 ? mysql_fetch_row(result) : NULL; + retval = mysql_row != NULL && mysql_row[0] != NULL ? strdup(mysql_row[0]) : NULL; + db_free_result(result); + return retval; } /*! \fn static int getboolsetting(MYSQL *psql, int mode, const char *setting, int dflt) @@ -374,43 +499,26 @@ static char *getglobalvariable(MYSQL *psql, int mode, const char *setting) { char *retval; MYSQL_RES *result; MYSQL_ROW mysql_row; - int i; assert(psql != 0); assert(setting != 0); /* see if it's in the option table */ - for (i=0; i 0) { - mysql_row = mysql_fetch_row(result); - - if (mysql_row != NULL) { - retval = strdup(mysql_row[1]); - db_free_result(result); - return retval; - } else { - db_free_result(result); - return 0; - } - } else { - db_free_result(result); - return 0; - } - } else { - return 0; - } + if (result == NULL) return NULL; + mysql_row = mysql_num_rows(result) > 0 ? mysql_fetch_row(result) : NULL; + retval = mysql_row != NULL && mysql_row[1] != NULL ? strdup(mysql_row[1]) : NULL; + db_free_result(result); + return retval; } /*! \fn int is_debug_device(int device_id) @@ -421,7 +529,10 @@ int is_debug_device(int device_id) { extern int *debug_devices; int i = 0; - while (i < 100) { + /* main() releases the list during shutdown; later logging must not read it. */ + if (debug_devices == NULL) return FALSE; + + while (i < MAX_DEBUG_DEVICES) { if (debug_devices[i] == '\0') break; if (debug_devices[i] == device_id) { return TRUE; @@ -433,219 +544,67 @@ int is_debug_device(int device_id) { return FALSE; } -/*! \fn static void read_availability_settings(MYSQL *psql) - * \brief load the device availability and ping settings - * - * Split out of read_config_options() unchanged; the settings and their - * fallbacks are exactly as they were. - */ -static void read_availability_settings(MYSQL *psql) { +void parse_debug_devices(char *device_list, int *devices, size_t capacity) { + if (capacity == 0) return; + devices[0] = 0; + char *saveptr = NULL; + const char *token = strtok_r(device_list, ",", &saveptr); + for (size_t i = 0; token != NULL && i < capacity - 1; i++, token = strtok_r(NULL, ",", &saveptr)) { + devices[i] = atoi(token); + devices[i + 1] = 0; + } +} + +static void read_logging_options(MYSQL *mysql) { char *res; - - /* set availability_method */ - if ((res = getsetting(psql, LOCAL, "availability_method")) != 0) { - set.availability_method = atoi(res); - free(res); - } - - /* log the availability_method variable */ - SPINE_LOG_DEBUG(("DEBUG: The availability_method variable is %i", set.availability_method)); - - /* set ping_recovery_count */ - if ((res = getsetting(psql, LOCAL, "ping_recovery_count")) != 0) { - set.ping_recovery_count = atoi(res); - free(res); - } - - /* log the ping_recovery_count variable */ - SPINE_LOG_DEBUG(("DEBUG: The ping_recovery_count variable is %i", set.ping_recovery_count)); - - /* set ping_failure_count */ - if ((res = getsetting(psql, LOCAL, "ping_failure_count")) != 0) { - set.ping_failure_count = atoi(res); - free(res); - } - - /* log the ping_failure_count variable */ - SPINE_LOG_DEBUG(("DEBUG: The ping_failure_count variable is %i", set.ping_failure_count)); - - /* set ping_method */ - if ((res = getsetting(psql, LOCAL, "ping_method")) != 0) { - set.ping_method = atoi(res); - free(res); - } - - /* log the ping_method variable */ - SPINE_LOG_DEBUG(("DEBUG: The ping_method variable is %i", set.ping_method)); - - /* set ping_retries */ - if ((res = getsetting(psql, LOCAL, "ping_retries")) != 0) { - set.ping_retries = atoi(res); - free(res); - } - - /* log the ping_retries variable */ - SPINE_LOG_DEBUG(("DEBUG: The ping_retries variable is %i", set.ping_retries)); - - /* set ping_timeout */ - if ((res = getsetting(psql, LOCAL, "ping_timeout")) != 0) { - set.ping_timeout = atoi(res); - free(res); - } else { - set.ping_timeout = 400; - } - - /* log the ping_timeout variable */ - SPINE_LOG_DEBUG(("DEBUG: The ping_timeout variable is %i", set.ping_timeout)); - - /* set snmp_retries */ - if ((res = getsetting(psql, LOCAL, "snmp_retries")) != 0) { - set.snmp_retries = atoi(res); - free(res); - } else { - set.snmp_retries = 3; - } - - /* log the snmp_retries variable */ - SPINE_LOG_DEBUG(("DEBUG: The snmp_retries variable is %i", set.snmp_retries)); -} - -/*! \fn void read_config_options(void) - * \brief Reads the default Spine runtime parameters from the database and set's the global array - * - * load default values from the database for poller processing - * - */ -int db_row_alias_upsert_supported(const char *version, unsigned long version_number) { - const char *p; - - if (version == NULL) { - return FALSE; - } - - for (p = version; *p != '\0'; p++) { - if (strncasecmp(p, "mariadb", 7) == 0) { - return FALSE; - } - } - - return version_number >= 80020 && version_number < 100000; -} - -void read_config_options(void) { - MYSQL mysql; - MYSQL mysqlr; - MYSQL_RES *result; - int num_rows; - int mode; - char web_root[BUFSIZE]; - char sqlbuf[HUGE_BUFSIZE]; - char *sqlp; - size_t remaining; - char *res; - char spine_priv[BUFSIZE]; - char spine_auth[BUFSIZE]; - char spine_capabilities[BUFSIZE]; - - web_root[0] = '\0'; - - /* publish spine snmpv3 capabilities to the database */ - memset(spine_capabilities, 0, sizeof(spine_capabilities)); - memset(spine_priv, 0, sizeof(spine_priv)); - memset(spine_auth, 0, sizeof(spine_auth)); - - db_connect(LOCAL, &mysql); - - /* one round trip instead of one per setting */ - settings_cache_load(&mysql, LOCAL); - - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { - db_connect(REMOTE, &mysqlr); - mode = REMOTE; - } else { - mode = LOCAL; - } - - /* get the mysql server version */ - if ((res = getglobalvariable(&mysql, LOCAL, "version")) != 0) { - snprintf(set.dbversion, BUFSIZE, "%s", res); - free(res); - } - - /* The row alias form, INSERT ... AS rs, is a syntax error before - MySQL 8.0.20 and is not MariaDB syntax. */ - set.dbonupdate = db_row_alias_upsert_supported(set.dbversion, mysql_get_server_version(&mysql)); - - /* get the cacti version from the database */ - set.cacti_version = get_cacti_version(&mysql, LOCAL); - - /* log the path_webroot variable */ - SPINE_LOG_DEBUG(("DEBUG: The binary Cacti version is %d", set.cacti_version)); - - /* Warn when Spine and Cacti come from different release lines. Point - * releases are expected to drift, so only major.minor is compared; - * cacti_version is encoded as major*1000 + minor*100 + point. */ - if (set.cacti_version > 0) { - int spine_major = 0, spine_minor = 0, spine_point = 0; - - if (sscanf(VERSION, "%d.%d.%d", &spine_major, &spine_minor, &spine_point) >= 2) { - int spine_line = (spine_major * 1000) + (spine_minor * 100); - int cacti_line = (set.cacti_version / 100) * 100; - - if (spine_line != cacti_line) { - SPINE_LOG(("WARNING: Spine %s does not match the Cacti release line (Cacti reports %d.%d). Use the Spine built for this Cacti version.", - VERSION, set.cacti_version / 1000, (set.cacti_version / 100) % 10)); - } - } - } - + char web_root[BUFSIZE] = {0}; /* get logging level from database - overrides spine.conf */ - if ((res = getsetting(&mysql, LOCAL, "log_verbosity")) != 0) { + if ((res = getsetting(mysql, LOCAL, "log_verbosity")) != 0) { const int n = atoi(res); free(res); - if (n != 0) set.log_level = n; + if (n != 0) set.logging.log_level = n; } /* determine script server path operation and default log file processing */ - if ((res = getsetting(&mysql, LOCAL, "path_webroot")) != 0) { - snprintf(set.path_php_server, BUFSIZE, "%s/script_server.php", res); + if ((res = getsetting(mysql, LOCAL, "path_webroot")) != 0) { + snprintf(set.php.path_php_server, BUFSIZE, "%s/script_server.php", res); snprintf(web_root, BUFSIZE, "%s", res); free(res); } /* determine logfile path */ - if ((res = getsetting(&mysql, LOCAL, "path_cactilog")) != 0) { + if ((res = getsetting(mysql, LOCAL, "path_cactilog")) != 0) { if (strlen(res) != 0) { - snprintf(set.path_logfile, DBL_BUFSIZE, "%s", res); + snprintf(set.logging.path_logfile, DBL_BUFSIZE, "%s", res); } else { if (strlen(web_root) != 0) { - snprintf(set.path_logfile, DBL_BUFSIZE, "%s/log/cacti.log", web_root); + snprintf(set.logging.path_logfile, DBL_BUFSIZE, "%s/log/cacti.log", web_root); } else { - set.path_logfile[0] ='\0'; + set.logging.path_logfile[0] ='\0'; } } free(res); } else { - snprintf(set.path_logfile, DBL_BUFSIZE, "%s/log/cacti.log", web_root); + snprintf(set.logging.path_logfile, DBL_BUFSIZE, "%s/log/cacti.log", web_root); } /* get log separator */ - if ((res = getsetting(&mysql, LOCAL, "default_datechar")) != 0) { - set.log_datetime_separator = atoi(res); + if ((res = getsetting(mysql, LOCAL, "default_datechar")) != 0) { + set.logging.log_datetime_separator = atoi(res); free(res); - if (set.log_datetime_separator < GDC_MIN || set.log_datetime_separator > GDC_MAX) { - set.log_datetime_separator = GDC_DEFAULT; + if (set.logging.log_datetime_separator < GDC_MIN || set.logging.log_datetime_separator > GDC_MAX) { + set.logging.log_datetime_separator = GDC_DEFAULT; } } /* get log date format */ - if ((res = getsetting(&mysql, LOCAL, "default_date_format")) != 0) { - set.log_datetime_format = atoi(res); + if ((res = getsetting(mysql, LOCAL, "default_date_format")) != 0) { + set.logging.log_datetime_format = atoi(res); free(res); - if (set.log_datetime_format < GD_MIN || set.log_datetime_format > GD_MAX) { - set.log_datetime_format = GD_DEFAULT; + if (set.logging.log_datetime_format < GD_MIN || set.logging.log_datetime_format > GD_MAX) { + set.logging.log_datetime_format = GD_DEFAULT; } } @@ -654,191 +613,235 @@ void read_config_options(void) { set_date_format(); /* determine log file, syslog or both, default is 1 or log file only */ - if ((res = getsetting(&mysql, LOCAL, "log_destination")) != 0) { - set.log_destination = parse_logdest(res, LOGDEST_FILE); + if ((res = getsetting(mysql, LOCAL, "log_destination")) != 0) { + set.logging.log_destination = parse_logdest(res, LOGDEST_FILE); free(res); } else { - set.log_destination = LOGDEST_FILE; + set.logging.log_destination = LOGDEST_FILE; } /* log the path_webroot variable */ - SPINE_LOG_DEBUG(("DEBUG: The path_php_server variable is %s", set.path_php_server)); + SPINE_LOG_DEBUG(("DEBUG: The path_php_server variable is %s", set.php.path_php_server)); /* log the path_cactilog variable */ - SPINE_LOG_DEBUG(("DEBUG: The path_cactilog variable is %s", set.path_logfile)); + SPINE_LOG_DEBUG(("DEBUG: The path_cactilog variable is %s", set.logging.path_logfile)); /* the version variable */ - SPINE_LOG_DEBUG(("DEBUG: The version variable is %s", set.dbversion)); + SPINE_LOG_DEBUG(("DEBUG: The version variable is %s", set.database.version)); /* log the log_destination variable */ SPINE_LOG_DEBUG(("DEBUG: The log_destination variable is %i (%s)", - set.log_destination, - printable_logdest(set.log_destination))); + set.logging.log_destination, + printable_logdest(set.logging.log_destination))); - set.logfile_processed = TRUE; + set.logging.logfile_processed = TRUE; - /* get PHP Path Information for Scripting */ - if ((res = getsetting(&mysql, LOCAL, "path_php_binary")) != 0) { - STRNCOPY(set.path_php, res); +} + +static void read_ping_options(MYSQL *mysql) { + char *res; + /* set availability_method */ + if ((res = getsetting(mysql, LOCAL, "availability_method")) != 0) { + set.availability.availability_method = atoi(res); free(res); } - /* log the path_php variable */ - SPINE_LOG_DEBUG(("DEBUG: The path_php variable is %s", set.path_php)); + /* log the availability_method variable */ + SPINE_LOG_DEBUG(("DEBUG: The availability_method variable is %i", set.availability.availability_method)); - read_availability_settings(&mysql); + /* set ping_recovery_count */ + if ((res = getsetting(mysql, LOCAL, "ping_recovery_count")) != 0) { + set.availability.ping_recovery_count = atoi(res); + free(res); + } - /* set logging option for errors */ - set.log_perror = getboolsetting(&mysql, LOCAL, "log_perror", FALSE); + /* log the ping_recovery_count variable */ + SPINE_LOG_DEBUG(("DEBUG: The ping_recovery_count variable is %i", set.availability.ping_recovery_count)); - /* log the log_perror variable */ - SPINE_LOG_DEBUG(("DEBUG: The log_perror variable is %i", set.log_perror)); + /* set ping_failure_count */ + if ((res = getsetting(mysql, LOCAL, "ping_failure_count")) != 0) { + set.availability.ping_failure_count = atoi(res); + free(res); + } - /* set logging option for errors */ - set.log_pwarn = getboolsetting(&mysql, LOCAL, "log_pwarn", FALSE); + /* log the ping_failure_count variable */ + SPINE_LOG_DEBUG(("DEBUG: The ping_failure_count variable is %i", set.availability.ping_failure_count)); - /* log the log_pwarn variable */ - SPINE_LOG_DEBUG(("DEBUG: The log_pwarn variable is %i", set.log_pwarn)); + /* set ping_method */ + if ((res = getsetting(mysql, LOCAL, "ping_method")) != 0) { + set.availability.ping_method = atoi(res); + free(res); + } - /* set option to increase insert performance */ - set.boost_redirect = getboolsetting(&mysql, LOCAL, "boost_redirect", FALSE); + /* log the ping_method variable */ + SPINE_LOG_DEBUG(("DEBUG: The ping_method variable is %i", set.availability.ping_method)); - /* log the boost_redirect variable */ - SPINE_LOG_DEBUG(("DEBUG: The boost_redirect variable is %i", set.boost_redirect)); + /* set ping_retries */ + if ((res = getsetting(mysql, LOCAL, "ping_retries")) != 0) { + set.availability.ping_retries = atoi(res); + free(res); + } - /* set option for determining if boost is enabled */ - set.boost_enabled = getboolsetting(&mysql, LOCAL, "boost_rrd_update_enable", FALSE); + /* log the ping_retries variable */ + SPINE_LOG_DEBUG(("DEBUG: The ping_retries variable is %i", set.availability.ping_retries)); - /* log the boost_rrd_update_enable variable */ - SPINE_LOG_DEBUG(("DEBUG: The boost_rrd_update_enable variable is %i", set.boost_enabled)); + /* set ping_timeout */ + if ((res = getsetting(mysql, LOCAL, "ping_timeout")) != 0) { + set.availability.ping_timeout = atoi(res); + free(res); + } else { + set.availability.ping_timeout = 400; + } - /* set logging option for statistics */ - set.log_pstats = getboolsetting(&mysql, LOCAL, "log_pstats", FALSE); + /* log the ping_timeout variable */ + SPINE_LOG_DEBUG(("DEBUG: The ping_timeout variable is %i", set.availability.ping_timeout)); - /* log the log_pstats variable */ - SPINE_LOG_DEBUG(("DEBUG: The log_pstats variable is %i", set.log_pstats)); + /* set snmp_retries */ + if ((res = getsetting(mysql, LOCAL, "snmp_retries")) != 0) { + set.snmp.snmp_retries = atoi(res); + free(res); + } else { + set.snmp.snmp_retries = 3; + } + /* log the snmp_retries variable */ + SPINE_LOG_DEBUG(("DEBUG: The snmp_retries variable is %i", set.snmp.snmp_retries)); + +} + +static void read_process_options(MYSQL *mysql, int mode) { + char *res; /* get Cacti defined max threads override spine.conf */ - if (set.threads_set == FALSE) { - if ((res = getpsetting(&mysql, mode, "threads")) != 0) { - set.threads = atoi(res); - free(res); - if (set.threads > MAX_THREADS) { - set.threads = MAX_THREADS; - } + res = set.poller.threads_set == FALSE ? getpsetting(mysql, mode, "threads") : NULL; + if (res != NULL) { + set.poller.threads = atoi(res); + free(res); + if (set.poller.threads > MAX_THREADS) { + set.poller.threads = MAX_THREADS; } } /* log the threads variable */ - SPINE_LOG_DEBUG(("DEBUG: The threads variable is %i", set.threads)); + SPINE_LOG_DEBUG(("DEBUG: The threads variable is %i", set.poller.threads)); /* get the poller_interval for those who have elected to go with a 1 minute polling interval */ - if ((res = getsetting(&mysql, LOCAL, "poller_interval")) != 0) { - set.poller_interval = atoi(res); + if ((res = getsetting(mysql, LOCAL, "poller_interval")) != 0) { + set.poller.poller_interval = atoi(res); free(res); } else { - set.poller_interval = 0; + set.poller.poller_interval = 0; } /* log the poller_interval variable */ - if (set.poller_interval == 0) { + if (set.poller.poller_interval == 0) { SPINE_LOG_DEBUG(("DEBUG: The polling interval is the system default")); } else { - SPINE_LOG_DEBUG(("DEBUG: The polling interval is %i seconds", set.poller_interval)); + SPINE_LOG_DEBUG(("DEBUG: The polling interval is %i seconds", set.poller.poller_interval)); } /* get the concurrent_processes variable to determine thread sleep values */ - if ((res = getsetting(&mysql, LOCAL, "concurrent_processes")) != 0) { - set.num_parent_processes = atoi(res); + if ((res = getsetting(mysql, LOCAL, "concurrent_processes")) != 0) { + set.poller.num_parent_processes = atoi(res); free(res); } else { - set.num_parent_processes = 1; + set.poller.num_parent_processes = 1; } /* log the concurrent processes variable */ - SPINE_LOG_DEBUG(("DEBUG: The number of concurrent processes is %i", set.num_parent_processes)); + SPINE_LOG_DEBUG(("DEBUG: The number of concurrent processes is %i", set.poller.num_parent_processes)); /* get the script timeout to establish timeouts */ - if ((res = getsetting(&mysql, LOCAL, "script_timeout")) != 0) { - set.script_timeout = atoi(res); + if ((res = getsetting(mysql, LOCAL, "script_timeout")) != 0) { + set.php.script_timeout = atoi(res); free(res); - if (set.script_timeout < 5) { - set.script_timeout = 5; + if (set.php.script_timeout < 5) { + set.php.script_timeout = 5; } } else { - set.script_timeout = 25; + set.php.script_timeout = 25; } /* log the script timeout value */ - SPINE_LOG_DEBUG(("DEBUG: The script timeout is %i", set.script_timeout)); + SPINE_LOG_DEBUG(("DEBUG: The script timeout is %i", set.php.script_timeout)); +} + +static void read_script_options(MYSQL *mysql) { + char *res; /* get selective_device_debug string */ - if ((res = getsetting(&mysql, LOCAL, "selective_device_debug")) != 0) { - STRNCOPY(set.selective_device_debug, res); + if ((res = getsetting(mysql, LOCAL, "selective_device_debug")) != 0) { + STRNCOPY(set.logging.selective_device_debug, res); free(res); } /* log the selective_device_debug variable */ - SPINE_LOG_DEBUG(("DEBUG: The selective_device_debug variable is %s", set.selective_device_debug)); + SPINE_LOG_DEBUG(("DEBUG: The selective_device_debug variable is %s", set.logging.selective_device_debug)); /* get spine_log_level */ - if ((res = getsetting(&mysql, LOCAL, "spine_log_level")) != 0) { - set.spine_log_level = atoi(res); + if ((res = getsetting(mysql, LOCAL, "spine_log_level")) != 0) { + set.logging.spine_log_level = atoi(res); free(res); } /* log the spine_log_level variable */ - SPINE_LOG_DEBUG(("DEBUG: The spine_log_level variable is %i", set.spine_log_level)); + SPINE_LOG_DEBUG(("DEBUG: The spine_log_level variable is %i", set.logging.spine_log_level)); /* get the number of script server processes to run */ - if ((res = getsetting(&mysql, LOCAL, "php_servers")) != 0) { - set.php_servers = atoi(res); + if ((res = getsetting(mysql, LOCAL, "php_servers")) != 0) { + set.php.php_servers = atoi(res); free(res); - if (set.php_servers > MAX_PHP_SERVERS) { - set.php_servers = MAX_PHP_SERVERS; + if (set.php.php_servers > MAX_PHP_SERVERS) { + set.php.php_servers = MAX_PHP_SERVERS; } - if (set.php_servers <= 0) { - set.php_servers = 1; + if (set.php.php_servers <= 0) { + set.php.php_servers = 1; } } else { - set.php_servers = 2; + set.php.php_servers = 2; } /* log the script timeout value */ - SPINE_LOG_DEBUG(("DEBUG: The number of php script servers to run is %i", set.php_servers)); + SPINE_LOG_DEBUG(("DEBUG: The number of php script servers to run is %i", set.php.php_servers)); /* get the number of active profiles on the system run */ - if ((res = getsetting(&mysql, LOCAL, "active_profiles")) != 0) { - set.active_profiles = atoi(res); + if ((res = getsetting(mysql, LOCAL, "active_profiles")) != 0) { + set.poller.active_profiles = atoi(res); free(res); - if (set.active_profiles <= 0) { - set.active_profiles = 0; + if (set.poller.active_profiles <= 0) { + set.poller.active_profiles = 0; } } else { - set.active_profiles = 0; + set.poller.active_profiles = 0; } /* log the script timeout value */ - SPINE_LOG_DEBUG(("DEBUG: The number of active data source profiles is %i", set.active_profiles)); + SPINE_LOG_DEBUG(("DEBUG: The number of active data source profiles is %i", set.poller.active_profiles)); /* get the number of snmp_ports in use */ - if ((res = getsetting(&mysql, LOCAL, "total_snmp_ports")) != 0) { - set.total_snmp_ports = atoi(res); + if ((res = getsetting(mysql, LOCAL, "total_snmp_ports")) != 0) { + set.snmp.total_snmp_ports = atoi(res); free(res); - if (set.total_snmp_ports <= 0) { - set.total_snmp_ports = 0; + if (set.snmp.total_snmp_ports <= 0) { + set.snmp.total_snmp_ports = 0; } } else { - set.total_snmp_ports = 0; + set.snmp.total_snmp_ports = 0; } /* log the script timeout value */ - SPINE_LOG_DEBUG(("DEBUG: The number of snmp ports on the system is %i", set.total_snmp_ports)); + SPINE_LOG_DEBUG(("DEBUG: The number of snmp ports on the system is %i", set.snmp.total_snmp_ports)); +} + +static void read_php_requirement(MYSQL *mysql) { + MYSQL_RES *result; + int num_rows; + char sqlbuf[HUGE_BUFSIZE]; + char *sqlp; /*---------------------------------------------------------------- * determine if the php script server is required by searching for * all the host records for an action of POLLER_ACTION_PHP_SCRIPT_SERVER. @@ -846,68 +849,76 @@ void read_config_options(void) { * server. * */ - set.php_required = FALSE; /* assume no */ + set.php.php_required = FALSE; /* assume no */ /* log the requirement for the script server */ - if (!strlen(set.host_id_list)) { + if (!strlen(set.hosts.host_id_list)) { sqlp = sqlbuf; - remaining = sizeof(sqlbuf); - spine_appendf(&sqlp, &remaining, "SELECT SQL_NO_CACHE action FROM poller_item"); - spine_appendf(&sqlp, &remaining, " WHERE action=%d", POLLER_ACTION_PHP_SCRIPT_SERVER); - sqlp += append_hostrange(sqlp, "host_id"); - remaining = sizeof(sqlbuf) - (size_t) (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, " AND poller_id=%i", set.poller_id); - spine_appendf(&sqlp, &remaining, " LIMIT 1"); + sqlp += spine_snprintf(sqlp, sizeof(sqlbuf) - (size_t)(sqlp - sqlbuf), "SELECT SQL_NO_CACHE action FROM poller_item"); + sqlp += spine_snprintf(sqlp, sizeof(sqlbuf) - (size_t)(sqlp - sqlbuf), " WHERE action=%d", POLLER_ACTION_PHP_SCRIPT_SERVER); + sqlp += append_hostrange(sqlp, sizeof(sqlbuf) - (size_t)(sqlp - sqlbuf), "host_id"); + sqlp += spine_snprintf(sqlp, sizeof(sqlbuf) - (size_t)(sqlp - sqlbuf), " AND poller_id=%i", set.poller.poller_id); + spine_snprintf(sqlp, sizeof(sqlbuf) - (size_t)(sqlp - sqlbuf), " LIMIT 1"); - result = db_query(&mysql, LOCAL, sqlbuf); - num_rows = mysql_num_rows(result); + result = db_query(mysql, LOCAL, sqlbuf); + num_rows = spine_count_to_int(mysql_num_rows(result)); db_free_result(result); - if (num_rows > 0) set.php_required = TRUE; + if (num_rows > 0) set.php.php_required = TRUE; SPINE_LOG_DEBUG(("DEBUG: StartDevice='%i', EndDevice='%i', TotalPHPScripts='%i'", - set.start_host_id, - set.end_host_id, + set.hosts.start_host_id, + set.hosts.end_host_id, num_rows)); } else { sqlp = sqlbuf; - remaining = sizeof(sqlbuf); - spine_appendf(&sqlp, &remaining, "SELECT SQL_NO_CACHE action FROM poller_item"); - spine_appendf(&sqlp, &remaining, " WHERE action=%d", POLLER_ACTION_PHP_SCRIPT_SERVER); - spine_appendf(&sqlp, &remaining, " AND host_id IN(%s)", set.host_id_list); - spine_appendf(&sqlp, &remaining, " AND poller_id=%i", set.poller_id); - spine_appendf(&sqlp, &remaining, " LIMIT 1"); + sqlp += spine_snprintf(sqlp, sizeof(sqlbuf) - (size_t)(sqlp - sqlbuf), "SELECT SQL_NO_CACHE action FROM poller_item"); + sqlp += spine_snprintf(sqlp, sizeof(sqlbuf) - (size_t)(sqlp - sqlbuf), " WHERE action=%d", POLLER_ACTION_PHP_SCRIPT_SERVER); + sqlp += spine_snprintf(sqlp, sizeof(sqlbuf) - (size_t)(sqlp - sqlbuf), " AND host_id IN(%s)", set.hosts.host_id_list); + sqlp += spine_snprintf(sqlp, sizeof(sqlbuf) - (size_t)(sqlp - sqlbuf), " AND poller_id=%i", set.poller.poller_id); + spine_snprintf(sqlp, sizeof(sqlbuf) - (size_t)(sqlp - sqlbuf), " LIMIT 1"); - result = db_query(&mysql, LOCAL, sqlbuf); - num_rows = mysql_num_rows(result); + result = db_query(mysql, LOCAL, sqlbuf); + num_rows = spine_count_to_int(mysql_num_rows(result)); db_free_result(result); - if (num_rows > 0) set.php_required = TRUE; + if (num_rows > 0) set.php.php_required = TRUE; SPINE_LOG_DEBUG(("DEBUG: Device List to be polled='%s', TotalPHPScripts='%i'", - set.host_id_list, + set.hosts.host_id_list, num_rows)); } SPINE_LOG_DEBUG(("DEBUG: The PHP Script Server is %sRequired", - set.php_required + set.php.php_required ? "" : "Not ")); +} + +static void read_snmp_batch_size(MYSQL *mysql) { + char *res; /* determine the maximum oid's to obtain in a single get request */ - if ((res = getsetting(&mysql, LOCAL, "max_get_size")) != 0) { - set.snmp_max_get_size = atoi(res); + if ((res = getsetting(mysql, LOCAL, "max_get_size")) != 0) { + set.snmp.snmp_max_get_size = atoi(res); free(res); - if (set.snmp_max_get_size > 128) { - set.snmp_max_get_size = 128; + if (set.snmp.snmp_max_get_size > 128) { + set.snmp.snmp_max_get_size = 128; } } else { - set.snmp_max_get_size = 25; + set.snmp.snmp_max_get_size = 25; } /* log the snmp_max_get_size variable */ - SPINE_LOG_DEBUG(("DEBUG: The Maximum SNMP OID Get Size is %i", set.snmp_max_get_size)); + SPINE_LOG_DEBUG(("DEBUG: The Maximum SNMP OID Get Size is %i", set.snmp.snmp_max_get_size)); + +} + +static void publish_snmp_capabilities(MYSQL *mysql) { + char spine_auth[BUFSIZE] = {0}; + char spine_priv[BUFSIZE] = {0}; + char spine_capabilities[BUFSIZE] = {0}; #ifndef NETSNMP_DISABLE_MD5 strcat(spine_auth, "MD5"); @@ -950,317 +961,282 @@ void read_config_options(void) { spine_capabilities[0] = '\0'; } - if (set.poller_id == 1 && spine_capabilities[0] != '\0') { - putsetting(&mysql, LOCAL, "spine_capabilities", spine_capabilities); + if (set.poller.poller_id == 1 && spine_capabilities[0] != '\0') { + putsetting(mysql, LOCAL, "spine_capabilities", spine_capabilities); } +} + +/*! \fn void read_config_options(void) + * \brief Reads the default Spine runtime parameters from the database and set's the global array + * + * load default values from the database for poller processing + * + */ +/*! \fn int db_row_alias_upsert_supported(const char *version, unsigned long version_number) + * \brief whether the server accepts INSERT ... AS alias ON DUPLICATE KEY UPDATE + */ +int db_row_alias_upsert_supported(const char *version, unsigned long version_number) { + if (version == NULL) { + return FALSE; + } + + for (const char *p = version; *p != '\0'; p++) { + if (strncasecmp(p, "mariadb", 7) == 0) { + return FALSE; + } + } + + return version_number >= 80020 && version_number < 100000; +} + +void read_config_options() { + MYSQL mysql; + MYSQL mysqlr; + int mode; + char *res; + + db_connect(LOCAL, &mysql); + + /* one round trip instead of one per setting */ + settings_cache_load(&mysql, LOCAL); + + if (set.poller.poller_id > 1 && set.poller.mode == REMOTE_ONLINE) { + db_connect(REMOTE, &mysqlr); + mode = REMOTE; + } else { + mode = LOCAL; + } + + /* get the mysql server version */ + if ((res = getglobalvariable(&mysql, LOCAL, "version")) != 0) { + snprintf(set.database.version, BUFSIZE, "%s", res); + free(res); + } + + /* The row alias form, INSERT ... AS rs, is a syntax error before + MySQL 8.0.20 and is not MariaDB syntax. */ + set.database.onupdate = db_row_alias_upsert_supported(set.database.version, mysql_get_server_version(&mysql)); + + /* get the cacti version from the database */ + set.cacti_version = get_cacti_version(&mysql, LOCAL); + + /* log the path_webroot variable */ + SPINE_LOG_DEBUG(("DEBUG: The binary Cacti version is %d", set.cacti_version)); + + /* Warn when Spine and Cacti come from different release lines. Point + * releases are expected to drift, so only major.minor is compared; + * cacti_version is encoded as major*1000 + minor*100 + point. */ + if (set.cacti_version > 0) { + int spine_major = 0, spine_minor = 0, spine_point = 0; + + if (sscanf(VERSION, "%d.%d.%d", &spine_major, &spine_minor, &spine_point) >= 2) { + int spine_line = (spine_major * 1000) + (spine_minor * 100); + int cacti_line = (set.cacti_version / 100) * 100; + + if (spine_line != cacti_line) { + SPINE_LOG(("WARNING: Spine %s does not match the Cacti release line (Cacti reports %d.%d). Use the Spine built for this Cacti version.", + VERSION, set.cacti_version / 1000, (set.cacti_version / 100) % 10)); + } + } + } + + read_logging_options(&mysql); + + /* get PHP Path Information for Scripting */ + if ((res = getsetting(&mysql, LOCAL, "path_php_binary")) != 0) { + STRNCOPY(set.php.path_php, res); + free(res); + } + + /* log the path_php variable */ + SPINE_LOG_DEBUG(("DEBUG: The path_php variable is %s", set.php.path_php)); + + read_ping_options(&mysql); + + /* set logging option for errors */ + set.logging.log_perror = getboolsetting(&mysql, LOCAL, "log_perror", FALSE); + + /* log the log_perror variable */ + SPINE_LOG_DEBUG(("DEBUG: The log_perror variable is %i", set.logging.log_perror)); + + /* set logging option for errors */ + set.logging.log_pwarn = getboolsetting(&mysql, LOCAL, "log_pwarn", FALSE); + + /* log the log_pwarn variable */ + SPINE_LOG_DEBUG(("DEBUG: The log_pwarn variable is %i", set.logging.log_pwarn)); + + /* set option to increase insert performance */ + set.boost.boost_redirect = getboolsetting(&mysql, LOCAL, "boost_redirect", FALSE); + + /* log the boost_redirect variable */ + SPINE_LOG_DEBUG(("DEBUG: The boost_redirect variable is %i", set.boost.boost_redirect)); + + /* set option for determining if boost is enabled */ + set.boost.boost_enabled = getboolsetting(&mysql, LOCAL, "boost_rrd_update_enable", FALSE); + + /* log the boost_rrd_update_enable variable */ + SPINE_LOG_DEBUG(("DEBUG: The boost_rrd_update_enable variable is %i", set.boost.boost_enabled)); + + /* set logging option for statistics */ + set.logging.log_pstats = getboolsetting(&mysql, LOCAL, "log_pstats", FALSE); + + /* log the log_pstats variable */ + SPINE_LOG_DEBUG(("DEBUG: The log_pstats variable is %i", set.logging.log_pstats)); + + read_process_options(&mysql, mode); + + read_script_options(&mysql); + + read_php_requirement(&mysql); + + read_snmp_batch_size(&mysql); + + publish_snmp_capabilities(&mysql); + db_disconnect(&mysql); - if (set.poller_id > 1 && set.mode == REMOTE_ONLINE) { + if (set.poller.poller_id > 1 && set.poller.mode == REMOTE_ONLINE) { db_disconnect(&mysqlr); } settings_cache_free(); } -/* An upper bound on one row of either batch below. Every column is a number or - a db_escape() result bounded by sizeof(tmpstr), plus separators. The row - count alone is not a size bound: 500 rows of wide escaped SNMP strings - overrun sqlbuf, and spine_appendf() then reports a truncation nothing was - checking, so a statement cut inside a quoted value reached the server. */ -#define PUSH_ROW_MAX (24 * DBL_BUFSIZE) +typedef struct { + const char *table; + const char *columns; + const char *filter_column; + const char *order_column; + const char *const *updates; + size_t update_count; + size_t field_count; + size_t row_limit; +} poller_transfer_t; -/*! \fn static void push_flush_batch(MYSQL *mysqlr, const char *sqlbuf, char **sqlp, const char *suffix) - * \brief terminate the accumulated batch and send it to the main server - * - * Refuses to send a statement that did not fit rather than shipping a - * truncated one. - */ -static void push_flush_batch(MYSQL *mysqlr, const char *sqlbuf, char **sqlp, const char *suffix) { - size_t remaining = HUGE_BUFSIZE - (*sqlp - sqlbuf); - - if (!spine_appendf(sqlp, &remaining, "%s", suffix)) { - SPINE_LOG(("ERROR: The remote push statement overflowed its buffer; this batch was dropped")); - return; +static void transfer_queries(const poller_transfer_t *plan, char *query, size_t query_capacity, char *prefix, size_t prefix_capacity, char *suffix, size_t suffix_capacity) { + size_t used = (size_t)spine_snprintf(query, query_capacity, "SELECT SQL_NO_CACHE %s FROM %s WHERE poller_id = %d", plan->columns, plan->table, set.poller.poller_id); + if (set.hosts.host_id_list[0] != '\0') used += (size_t)spine_snprintf(query + used, query_capacity - used, " AND %s IN (%s)", plan->filter_column, set.hosts.host_id_list); + spine_snprintf(query + used, query_capacity - used, " ORDER BY %s", plan->order_column); + spine_snprintf(prefix, prefix_capacity, "INSERT INTO %s (%s) VALUES ", plan->table, plan->columns); + /* These rows go to the main server. The cached upsert capability describes the + * local server, so the row-alias form could reach a MariaDB main server + * that rejects it. VALUES() is accepted by both (#590). */ + used = (size_t)spine_snprintf(suffix, suffix_capacity, " ON DUPLICATE KEY UPDATE "); + for (size_t index = 0; index < plan->update_count; index++) { + const char *column = plan->updates[index]; + used += (size_t)spine_snprintf(suffix + used, suffix_capacity - used, "%s%s=VALUES(%s)", index == 0 ? "" : ", ", column, column); } +} - db_insert(mysqlr, REMOTE, sqlbuf); +static size_t transfer_row(MYSQL *destination, MYSQL_ROW row, size_t field_count, char *output, size_t capacity) { + /* The selected columns are at most 300 UTF-8 characters: allow 4 bytes per + * character and doubling for SQL escaping. Preserve SQL NULL explicitly. */ + char escaped[DBL_BUFSIZE * 2]; + size_t used = (size_t)spine_snprintf(output, capacity, "("); + for (size_t index = 0; index < field_count; index++) { + if (row[index] == NULL) { + used += (size_t)spine_snprintf(output + used, capacity - used, "%sNULL", index == 0 ? "" : ", "); + } else { + db_escape(destination, escaped, sizeof(escaped), row[index]); + used += (size_t)spine_snprintf(output + used, capacity - used, "%s'%s'", index == 0 ? "" : ", ", escaped); + } + } + used += (size_t)spine_snprintf(output + used, capacity - used, ")"); + return used; +} + +static bool transfer_batch(MYSQL *destination, char *buffer, size_t used, const char *suffix) { + spine_snprintf(buffer + used, HUGE_BUFSIZE - used, "%s", suffix); + return db_insert(destination, REMOTE, buffer); +} + +static bool transfer_table(MYSQL *source, MYSQL *destination, const poller_transfer_t *plan) { + char query[MEGA_BUFSIZE]; + char prefix[BUFSIZE]; + char suffix[BUFSIZE]; + transfer_queries(plan, query, sizeof(query), prefix, sizeof(prefix), suffix, sizeof(suffix)); + MYSQL_RES *result = db_query(source, LOCAL, query); + if (result == NULL) return FALSE; + if (mysql_num_fields(result) != plan->field_count) { + db_free_result(result); + return FALSE; + } + /* Size one row for every column at its escaped maximum plus separators; + * a fixed few kilobytes could not hold a host row with wide SNMP strings. */ + size_t row_capacity = plan->field_count * (DBL_BUFSIZE * 2 + 4) + 2; + char *row_sql = malloc(row_capacity); + char *buffer = malloc(HUGE_BUFSIZE); + if (buffer == NULL || row_sql == NULL) { + free(row_sql); + free(buffer); + db_free_result(result); + return FALSE; + } + size_t used = 0; + size_t rows = 0; + size_t suffix_length = strlen(suffix); + MYSQL_ROW row; + bool success = TRUE; + while ((row = mysql_fetch_row(result)) != NULL) { + size_t length = transfer_row(destination, row, plan->field_count, row_sql, row_capacity); + if (rows > 0 && (rows == plan->row_limit || length + suffix_length + 3 > HUGE_BUFSIZE - used)) { + if (!transfer_batch(destination, buffer, used, suffix)) { + success = FALSE; + break; + } + rows = 0; + } + if (rows == 0) used = (size_t)spine_snprintf(buffer, HUGE_BUFSIZE, "%s", prefix); + else used += (size_t)spine_snprintf(buffer + used, HUGE_BUFSIZE - used, ", "); + used += (size_t)spine_snprintf(buffer + used, HUGE_BUFSIZE - used, "%s", row_sql); + rows++; + } + if (mysql_errno(source) != 0) success = FALSE; + if (success && rows > 0) success = transfer_batch(destination, buffer, used, suffix); + free(row_sql); + free(buffer); + db_free_result(result); + return success; +} + +bool poller_transfer_status(MYSQL *source, MYSQL *destination) { + static const char *const host_updates[] = { + "snmp_sysDescr", "snmp_sysObjectID", "snmp_sysUpTimeInstance", "snmp_sysContact", "snmp_sysName", "snmp_sysLocation", + "status", "status_event_count", "status_fail_date", "status_rec_date", "status_last_error", "min_time", "max_time", + "cur_time", "avg_time", "polling_time", "total_polls", "failed_polls", "availability", "last_updated" + }; + static const char *const item_updates[] = {"rrd_next_step"}; + static const poller_transfer_t host_plan = { + "host", "id, snmp_sysDescr, snmp_sysObjectID, snmp_sysUpTimeInstance, snmp_sysContact, snmp_sysName, snmp_sysLocation, status, status_event_count, status_fail_date, status_rec_date, status_last_error, min_time, max_time, cur_time, avg_time, polling_time, total_polls, failed_polls, availability, last_updated", + "id", "id", host_updates, sizeof(host_updates) / sizeof(host_updates[0]), 21, 500 + }; + static const poller_transfer_t item_plan = { + "poller_item", "local_data_id, host_id, rrd_name, rrd_step, rrd_next_step", + "host_id", "local_data_id, rrd_name", item_updates, sizeof(item_updates) / sizeof(item_updates[0]), 5, 10000 + }; + SPINE_LOG_MEDIUM(("Pushing Host Status to Main Server")); + if (!transfer_table(source, destination, &host_plan)) return FALSE; + SPINE_LOG_MEDIUM(("Pushing Poller Item RRD Next Step to Main Server")); + return transfer_table(source, destination, &item_plan); } void poller_push_data_to_main(void) { - MYSQL mysql; - MYSQL mysqlr; - MYSQL_RES *result; - MYSQL_ROW row; - int num_rows; - int rows; - char sqlbuf[HUGE_BUFSIZE]; - char *sqlp = sqlbuf; - size_t remaining; - char query[MEGA_BUFSIZE]; - char prefix[BUFSIZE]; - char suffix[BUFSIZE]; - // tmpstr needs to be greater than 2 * the maximum column size being processed below - char tmpstr[DBL_BUFSIZE]; - - db_connect(LOCAL, &mysql); - db_connect(REMOTE, &mysqlr); - - /* Since MySQL 5.7 the sql_mode defaults are too strict for cacti */ - db_insert(&mysql, LOCAL, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'NO_ZERO_DATE', ''))"); - db_insert(&mysql, LOCAL, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY', ''))"); - db_insert(&mysqlr, REMOTE, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'NO_ZERO_DATE', ''))"); - db_insert(&mysqlr, REMOTE, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY', ''))"); - - SPINE_LOG_MEDIUM(("Pushing Host Status to Main Server")); - - if (strlen(set.host_id_list)) { - snprintf(query, MEGA_BUFSIZE, "SELECT SQL_NO_CACHE id, snmp_sysDescr, snmp_sysObjectID, " - "snmp_sysUpTimeInstance, snmp_sysContact, snmp_sysName, snmp_sysLocation, " - "status, status_event_count, status_fail_date, status_rec_date, " - "status_last_error, min_time, max_time, cur_time, avg_time, polling_time, " - "total_polls, failed_polls, availability, last_updated " - "FROM host " - "WHERE poller_id = %d " - "AND id IN (%s)", set.poller_id, set.host_id_list); - } else { - snprintf(query, MEGA_BUFSIZE, "SELECT SQL_NO_CACHE id, snmp_sysDescr, snmp_sysObjectID, " - "snmp_sysUpTimeInstance, snmp_sysContact, snmp_sysName, snmp_sysLocation, " - "status, status_event_count, status_fail_date, status_rec_date, " - "status_last_error, min_time, max_time, cur_time, avg_time, polling_time, " - "total_polls, failed_polls, availability, last_updated " - "FROM host " - "WHERE poller_id = %d", set.poller_id); + MYSQL source; + MYSQL destination; + db_connect(LOCAL, &source); + db_connect(REMOTE, &destination); + /* Preserve the supported zero-date and GROUP BY session policies. */ + bool configured = db_insert(&source, LOCAL, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'NO_ZERO_DATE', ''))") && + db_insert(&source, LOCAL, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY', ''))") && + db_insert(&destination, REMOTE, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'NO_ZERO_DATE', ''))") && + db_insert(&destination, REMOTE, "SET SESSION sql_mode = (SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY', ''))"); + if (!configured || !poller_transfer_status(&source, &destination)) { + SPINE_LOG(("ERROR: Collector synchronization incomplete; earlier batches may have reached the main server. Local rows are retained for retry.")); + set.exit.exit_code = EXIT_FAILURE; } - - snprintf(prefix, BUFSIZE, "INSERT INTO host (id, snmp_sysDescr, snmp_sysObjectID, " - "snmp_sysUpTimeInstance, snmp_sysContact, snmp_sysName, snmp_sysLocation, " - "status, status_event_count, status_fail_date, status_rec_date, " - "status_last_error, min_time, max_time, cur_time, avg_time, polling_time, " - "total_polls, failed_polls, availability, last_updated) VALUES "); - - /* Every INSERT below goes to the main server on the REMOTE connection, and - spine.c only calls this function when poller_id > 1 and mode is - REMOTE_ONLINE. set.dbonupdate describes the LOCAL server, so branching on - it here picks the syntax of the wrong database: a MySQL 8 remote poller - would emit the row-alias form at a MariaDB main server, which rejects it, - losing the whole batch silently. VALUES() is deprecated on MySQL 8 but - accepted by both, so it stays until rdbonupdate is populated. See #590. */ - snprintf(suffix, BUFSIZE, " ON DUPLICATE KEY UPDATE " - "snmp_sysDescr=VALUES(snmp_sysDescr), " - "snmp_sysObjectID=VALUES(snmp_sysObjectID), " - "snmp_sysUpTimeInstance=VALUES(snmp_sysUpTimeInstance), " - "snmp_sysContact=VALUES(snmp_sysContact), " - "snmp_sysName=VALUES(snmp_sysName), " - "snmp_sysLocation=VALUES(snmp_sysLocation), " - "status=VALUES(status), " - "status_event_count=VALUES(status_event_count), " - "status_fail_date=VALUES(status_fail_date), " - "status_rec_date=VALUES(status_rec_date), " - "status_last_error=VALUES(status_last_error), " - "min_time=VALUES(min_time), " - "max_time=VALUES(max_time), " - "cur_time=VALUES(cur_time), " - "avg_time=VALUES(avg_time), " - "polling_time=VALUES(polling_time), " - "total_polls=VALUES(total_polls), " - "failed_polls=VALUES(failed_polls), " - "availability=VALUES(availability), " - "last_updated=VALUES(last_updated)"); - - if ((result = db_query(&mysql, LOCAL, query)) != 0) { - num_rows = mysql_num_rows(result); - rows = 0; - - if (num_rows > 0) { - while ((row = mysql_fetch_row(result))) { - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - - /* Flush before the row, never in place of it. The old shape reset - the batch when it filled and dropped the row that triggered the - reset, so every 501th device silently missed the push. The byte - test is the other half: the count is not a size bound once the - escaped SNMP strings are wide. */ - if (rows > 0 && (rows >= 500 || remaining < PUSH_ROW_MAX)) { - push_flush_batch(&mysqlr, sqlbuf, &sqlp, suffix); - rows = 0; - } - - { - if (rows == 0) { - sqlp = sqlbuf; - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s", prefix); - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, " ("); - } else { - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, ", ("); - } - - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[0]); // id mediumint - - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[1]); // snmp_sysDescr varchar(300) - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[2]); // snmp_sysObjectID varchar(128) - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[3]); // snmp_sysUpTimeInstance bigint - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[4]); // snmp_sysContact varchar(300) - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[5]); // snmp_sysName varchar(300) - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[6]); // snmp_sysLocation varchar(300) - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[7]); // status tinyint - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[8]); // status_event_count mediumint - - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[9]); // status_fail_date timestamp - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[10]); // status_rec_date timestamp - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[11]); // status_last_error varchar(255) - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[12]); // min_time decimal(10,5) - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[13]); // max_time decimal(10,5) - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[14]); // cur_time decimal(10,5) - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[15]); // avg_time decimal(10,5) - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[16]); // polling_time double - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[17]); // total_polls int - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[18]); // failed_polls int - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[19]); // availability decimal(8,5) - - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[20]); // last_updated timestamp - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s'", tmpstr); - - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, ")"); - - rows++; - } - } - } - - if (rows > 0) { - push_flush_batch(&mysqlr, sqlbuf, &sqlp, suffix); - } - } - - db_free_result(result); - - SPINE_LOG_MEDIUM(("Pushing Poller Item RRD Next Step to Main Server")); - - if (strlen(set.host_id_list)) { - snprintf(query, MEGA_BUFSIZE, "SELECT SQL_NO_CACHE local_data_id, host_id, rrd_name, rrd_step, rrd_next_step " - "FROM poller_item " - "WHERE poller_id = %d " - "AND host_id IN (%s)", set.poller_id, set.host_id_list); - } else { - snprintf(query, MEGA_BUFSIZE, "SELECT SQL_NO_CACHE local_data_id, host_id, rrd_name, rrd_step, rrd_next_step " - "FROM poller_item " - "WHERE poller_id = %d ", - set.poller_id); - } - - snprintf(prefix, BUFSIZE, "INSERT INTO poller_item (local_data_id, host_id, rrd_name, rrd_step, rrd_next_step) VALUES "); - - /* Every INSERT below goes to the main server on the REMOTE connection, and - spine.c only calls this function when poller_id > 1 and mode is - REMOTE_ONLINE. set.dbonupdate describes the LOCAL server, so branching on - it here picks the syntax of the wrong database: a MySQL 8 remote poller - would emit the row-alias form at a MariaDB main server, which rejects it, - losing the whole batch silently. VALUES() is deprecated on MySQL 8 but - accepted by both, so it stays until rdbonupdate is populated. See #590. */ - snprintf(suffix, BUFSIZE, " ON DUPLICATE KEY UPDATE " - "rrd_next_step=VALUES(rrd_next_step)"); - - if ((result = db_query(&mysql, LOCAL, query)) != 0) { - num_rows = mysql_num_rows(result); - rows = 0; - - if (num_rows > 0) { - while ((row = mysql_fetch_row(result))) { - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - - /* Flush before the row, never in place of it. The old shape reset - the batch when it filled and dropped the row that triggered the - reset, so every 10001th device silently missed the push. The byte - test is the other half: the count is not a size bound once the - escaped SNMP strings are wide. */ - if (rows > 0 && (rows >= 10000 || remaining < PUSH_ROW_MAX)) { - push_flush_batch(&mysqlr, sqlbuf, &sqlp, suffix); - rows = 0; - } - - { - if (rows == 0) { - sqlp = sqlbuf; - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s", prefix); - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, " ("); - } else { - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, ", ("); - } - - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[0]); // local_data_id - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[1]); // host_id - - db_escape(&mysql, tmpstr, sizeof(tmpstr), row[2]); // rrd_name - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "'%s', ", tmpstr); - - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s, ", row[3]); // rrd_step - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, "%s", row[4]); // rrd_next_step - - remaining = HUGE_BUFSIZE - (sqlp - sqlbuf); - spine_appendf(&sqlp, &remaining, ")"); - - rows++; - } - } - } - - if (rows > 0) { - push_flush_batch(&mysqlr, sqlbuf, &sqlp, suffix); - - rows = 0; - } - } - - db_free_result(result); - - db_disconnect(&mysql); - db_disconnect(&mysqlr); + db_disconnect(&source); + db_disconnect(&destination); } /*! \fn int read_spine_config(const char *file) @@ -1269,74 +1245,82 @@ void poller_push_data_to_main(void) { * * \return 0 if successful or -1 if the file could not be opened */ -int read_spine_config(const char *file) { - FILE *fp; - char buff[BUFSIZE]; - char p1[BUFSIZE]; - char p2[BUFSIZE]; - char *chars; +static int apply_config_directive(const char *name, const char *value) { + /* Immutable metadata cannot be changed by configuration values written + * into set. Offsets and capacities describe the actual destination fields. */ + static const struct { + const char *name; + size_t offset; + size_t capacity; + } strings[] = { + {"RDB_Host", offsetof(config_t, remote_database.host), sizeof(set.remote_database.host)}, + {"RDB_Database", offsetof(config_t, remote_database.database), sizeof(set.remote_database.database)}, + {"RDB_User", offsetof(config_t, remote_database.user), sizeof(set.remote_database.user)}, + {"RDB_Pass", offsetof(config_t, remote_database.password), sizeof(set.remote_database.password)}, + {"RDB_SSL_Key", offsetof(config_t, remote_database.ssl_key), sizeof(set.remote_database.ssl_key)}, + {"RDB_SSL_Cert", offsetof(config_t, remote_database.ssl_cert), sizeof(set.remote_database.ssl_cert)}, + {"RDB_SSL_CA", offsetof(config_t, remote_database.ssl_ca), sizeof(set.remote_database.ssl_ca)}, + {"DB_Host", offsetof(config_t, database.host), sizeof(set.database.host)}, + {"DB_Database", offsetof(config_t, database.database), sizeof(set.database.database)}, + {"DB_User", offsetof(config_t, database.user), sizeof(set.database.user)}, + {"DB_Pass", offsetof(config_t, database.password), sizeof(set.database.password)}, + {"DB_SSL_Key", offsetof(config_t, database.ssl_key), sizeof(set.database.ssl_key)}, + {"DB_SSL_Cert", offsetof(config_t, database.ssl_cert), sizeof(set.database.ssl_cert)}, + {"DB_SSL_CA", offsetof(config_t, database.ssl_ca), sizeof(set.database.ssl_ca)}, + {"SNMP_Clientaddr", offsetof(config_t, snmp.snmp_clientaddr), sizeof(set.snmp.snmp_clientaddr)}, + }; + for (size_t i = 0; i < sizeof(strings) / sizeof(strings[0]); i++) { + if (STRIMATCH(name, strings[i].name)) { + strncopy((char *)&set + strings[i].offset, value, strings[i].capacity); + return TRUE; + } + } + if (STRIMATCH(name, "RDB_Port")) set.remote_database.port = atoi(value); + else if (STRIMATCH(name, "RDB_UseSSL")) set.remote_database.ssl = atoi(value); + else if (STRIMATCH(name, "DB_Port")) set.database.port = atoi(value); + else if (STRIMATCH(name, "DB_UseSSL")) set.database.ssl = atoi(value); + else if (STRIMATCH(name, "Poller")) set.poller.poller_id = atoi(value); + else if (STRIMATCH(name, "DB_PreG")) { + if (!set.console.stderr_notty) fprintf(stderr, "WARNING: DB_PreG is no longer supported\n"); + } else if (STRIMATCH(name, "Cacti_Log")) { + STRNCOPY(set.logging.path_logfile, value); + set.logging.logfile_processed = 1; + set.logging.log_destination = LOGDEST_BOTH; + } else { + return FALSE; + } + return TRUE; +} - if ((fp = fopen(file, "rb")) == NULL) { - if (set.log_level == POLLER_VERBOSITY_DEBUG) { - if (!set.stderr_notty) { - fprintf(stderr, "ERROR: Could not open config file [%s]\n", file); - } +int read_spine_config(const char *file) { + FILE *fp = fopen(file, "rb"); + char buff[BUFSIZE]; + char name[BUFSIZE]; + char value[BUFSIZE]; + char display_file[BUFSIZE]; + strncopy(display_file, file, sizeof(display_file)); + spine_sanitize_log_message(display_file); + + if (fp == NULL) { + if (set.logging.log_level == POLLER_VERBOSITY_DEBUG && !set.console.stderr_notty) { + fprintf(stderr, "ERROR: Could not open config file [%s]\n", display_file); } return -1; - } else { - if (!set.stdout_notty) { - fprintf(stdout, "SPINE: Using spine config file [%s]\n", file); - } - - while (!feof(fp)) { - chars = fgets(buff, BUFSIZE, fp); - - if (chars != NULL && !feof(fp) && *buff != '#' && *buff != ' ' && *buff != '\n') { - sscanf(buff, "%15s %255s", p1, p2); - - if (STRIMATCH(p1, "RDB_Host")) STRNCOPY(set.rdb_host, p2); - else if (STRIMATCH(p1, "RDB_Database")) STRNCOPY(set.rdb_db, p2); - else if (STRIMATCH(p1, "RDB_User")) STRNCOPY(set.rdb_user, p2); - else if (STRIMATCH(p1, "RDB_Pass")) STRNCOPY(set.rdb_pass, p2); - else if (STRIMATCH(p1, "RDB_Port")) set.rdb_port = atoi(p2); - else if (STRIMATCH(p1, "RDB_UseSSL")) set.rdb_ssl = atoi(p2); - else if (STRIMATCH(p1, "RDB_SSL_Key")) STRNCOPY(set.rdb_ssl_key, p2); - else if (STRIMATCH(p1, "RDB_SSL_Cert")) STRNCOPY(set.rdb_ssl_cert, p2); - else if (STRIMATCH(p1, "RDB_SSL_CA")) STRNCOPY(set.rdb_ssl_ca, p2); - else if (STRIMATCH(p1, "DB_Host")) STRNCOPY(set.db_host, p2); - else if (STRIMATCH(p1, "DB_Database")) STRNCOPY(set.db_db, p2); - else if (STRIMATCH(p1, "DB_User")) STRNCOPY(set.db_user, p2); - else if (STRIMATCH(p1, "DB_Pass")) STRNCOPY(set.db_pass, p2); - else if (STRIMATCH(p1, "DB_Port")) set.db_port = atoi(p2); - else if (STRIMATCH(p1, "DB_UseSSL")) set.db_ssl = atoi(p2); - else if (STRIMATCH(p1, "DB_SSL_Key")) STRNCOPY(set.db_ssl_key, p2); - else if (STRIMATCH(p1, "DB_SSL_Cert")) STRNCOPY(set.db_ssl_cert, p2); - else if (STRIMATCH(p1, "DB_SSL_CA")) STRNCOPY(set.db_ssl_ca, p2); - else if (STRIMATCH(p1, "Poller")) set.poller_id = atoi(p2); - else if (STRIMATCH(p1, "DB_PreG")) { - if (!set.stderr_notty) { - fprintf(stderr,"WARNING: DB_PreG is no longer supported\n"); - } - } else if (STRIMATCH(p1, "Cacti_Log")) { - STRNCOPY(set.path_logfile, p2); - set.logfile_processed = 1; - set.log_destination = LOGDEST_BOTH; - } else if (STRIMATCH(p1, "SNMP_Clientaddr")) STRNCOPY(set.snmp_clientaddr, p2); - else if (!set.stderr_notty) { - fprintf(stderr,"WARNING: Unrecognized directive: %s=%s in %s\n", p1, p2, file); - } - - *p1 = '\0'; - *p2 = '\0'; - } - } - - if (strlen(set.db_pass) == 0) *set.db_pass = '\0'; - - fclose(fp); - - return 0; } + if (!set.console.stdout_notty) { + fprintf(stdout, "SPINE: Using spine config file [%s]\n", display_file); + } + while (fgets(buff, sizeof(buff), fp) != NULL) { + if (buff[0] == '#' || buff[0] == ' ' || buff[0] == '\n') continue; + if (sscanf(buff, "%15s %255s", name, value) != 2) continue; + if (!apply_config_directive(name, value) && !set.console.stderr_notty) { + spine_sanitize_log_message(name); + fprintf(stderr, "WARNING: Unrecognized directive: %s in %s\n", name, display_file); + } + } + int failed = ferror(fp); + if (fclose(fp) != 0) failed = TRUE; + return failed ? -1 : 0; } /*! \fn void config_defaults(void) @@ -1344,31 +1328,31 @@ int read_spine_config(const char *file) { * \param *set global runtime parameters * */ -void config_defaults(void) { - set.threads = DEFAULT_THREADS; +void config_defaults() { + set.poller.threads = DEFAULT_THREADS; /* default server */ - set.db_port = DEFAULT_DB_PORT; + set.database.port = DEFAULT_DB_PORT; - STRNCOPY(set.db_host, DEFAULT_DB_HOST); - STRNCOPY(set.db_db, DEFAULT_DB_DB ); - STRNCOPY(set.db_user, DEFAULT_DB_USER); - STRNCOPY(set.db_pass, DEFAULT_DB_PASS); + STRNCOPY(set.database.host, DEFAULT_DB_HOST); + STRNCOPY(set.database.database, DEFAULT_DB_DB ); + STRNCOPY(set.database.user, DEFAULT_DB_USER); + STRNCOPY(set.database.password, DEFAULT_DB_PASS); /* remote default server */ - set.rdb_port = DEFAULT_DB_PORT; + set.remote_database.port = DEFAULT_DB_PORT; - STRNCOPY(set.rdb_host, DEFAULT_DB_HOST); - STRNCOPY(set.rdb_db, DEFAULT_DB_DB ); - STRNCOPY(set.rdb_user, DEFAULT_DB_USER); - STRNCOPY(set.rdb_pass, DEFAULT_DB_PASS); + STRNCOPY(set.remote_database.host, DEFAULT_DB_HOST); + STRNCOPY(set.remote_database.database, DEFAULT_DB_DB ); + STRNCOPY(set.remote_database.user, DEFAULT_DB_USER); + STRNCOPY(set.remote_database.password, DEFAULT_DB_PASS); STRNCOPY(config_paths[0], CONFIG_PATH_1); STRNCOPY(config_paths[1], CONFIG_PATH_2); STRNCOPY(config_paths[2], CONFIG_PATH_3); STRNCOPY(config_paths[3], CONFIG_PATH_4); - set.log_destination = LOGDEST_FILE; + set.logging.log_destination = LOGDEST_FILE; } /*! \fn void die(const char *format, ...) @@ -1385,17 +1369,19 @@ void die(const char *format, ...) { int old_errno = errno; va_start(args, format); - vsnprintf(logmessage, BUFSIZE, format, args); + vsnprintf(logmessage, sizeof(logmessage), format, args); va_end(args); - if (set.log_perror) { + if (set.logging.log_perror) { char perr[BUFSIZE]; snprintf(perr, BUFSIZE, " [%d, %s]", old_errno, strerror(old_errno)); - strncat(logmessage, perr, BUFSIZE - strlen(logmessage) - 1); + size_t used = strlen(logmessage); + snprintf(logmessage + used, sizeof(logmessage) - used, "%s", perr); } + spine_sanitize_log_message(logmessage); - if (set.logfile_processed) { - if (set.parent_fork == SPINE_PARENT) { + if (set.logging.logfile_processed) { + if (set.poller.parent_fork == SPINE_PARENT) { snprintf(flogmessage, DBL_BUFSIZE, "%s (Spine parent)", logmessage); } else { snprintf(flogmessage, DBL_BUFSIZE, "%s (Spine thread)", logmessage); @@ -1406,73 +1392,41 @@ void die(const char *format, ...) { fprintf(stderr, "%s\n", flogmessage); - if (set.parent_fork == SPINE_PARENT) { - if (set.php_initialized) { - php_close(PHP_INIT); - } + if ((set.poller.parent_fork == SPINE_PARENT) && (set.php.php_initialized)) { + php_close(PHP_INIT); } - exit(set.exit_code); + exit(set.exit.exit_code == EXIT_SUCCESS ? EXIT_FAILURE : set.exit.exit_code); } -/* The log timestamp format depends only on two settings that are read once in +/* The log timestamp format depends only on two settings read once in * read_config_options(), so it is built there, while spine is still single - * threaded, and only read afterwards. Rebuilding it per log line cost a - * malloc/free pair and two switches on every message. */ + * threaded, and only read afterwards. Rebuilding it per log line cost a + * malloc/free pair on every message. */ static char log_date_format[GD_FMT_SIZE] = "%Y/%b/%d %H:%M:%S - "; /*! \fn void set_date_format(void) * \brief build the cached log timestamp format from the current settings * - * Call once the log format and separator settings are known. Not safe to - * call after the poller threads have started. + * Not safe to call after the poller threads have started. */ void set_date_format(void) { - char log_sep = '/'; - - if (set.log_datetime_separator < GDC_MIN || set.log_datetime_separator > GDC_MAX) { - set.log_datetime_separator = GDC_DEFAULT; + static const char separators[] = {'-', '/', '.'}; + static const char *const formats[][3] = { + {"%m", "%d", "%Y"}, {"%b", "%d", "%Y"}, + {"%d", "%m", "%Y"}, {"%d", "%b", "%Y"}, + {"%Y", "%m", "%d"}, {"%Y", "%b", "%d"} + }; + if (set.logging.log_datetime_separator < GDC_MIN || set.logging.log_datetime_separator > GDC_MAX) { + set.logging.log_datetime_separator = GDC_DEFAULT; } - - if (set.log_datetime_format < GD_MIN || set.log_datetime_format > GD_MAX) { - set.log_datetime_format = GD_DEFAULT; - } - - switch (set.log_datetime_separator) { - case GDC_DOT: - log_sep = '.'; - break; - case GDC_HYPHEN: - log_sep = '-'; - break; - default: - log_sep = '/'; - break; - } - - switch (set.log_datetime_format) { - case GD_MO_D_Y: - snprintf(log_date_format, GD_FMT_SIZE, "%%m%c%%d%c%%Y %%H:%%M:%%S - ", log_sep, log_sep); - break; - case GD_MN_D_Y: - snprintf(log_date_format, GD_FMT_SIZE, "%%b%c%%d%c%%Y %%H:%%M:%%S - ", log_sep, log_sep); - break; - case GD_D_MO_Y: - snprintf(log_date_format, GD_FMT_SIZE, "%%d%c%%m%c%%Y %%H:%%M:%%S - ", log_sep, log_sep); - break; - case GD_D_MN_Y: - snprintf(log_date_format, GD_FMT_SIZE, "%%d%c%%b%c%%Y %%H:%%M:%%S - ", log_sep, log_sep); - break; - case GD_Y_MO_D: - snprintf(log_date_format, GD_FMT_SIZE, "%%Y%c%%m%c%%d %%H:%%M:%%S - ", log_sep, log_sep); - break; - case GD_Y_MN_D: - snprintf(log_date_format, GD_FMT_SIZE, "%%Y%c%%b%c%%d %%H:%%M:%%S - ", log_sep, log_sep); - break; - default: - snprintf(log_date_format, GD_FMT_SIZE, "%%Y%c%%m%c%%d %%H:%%M:%%S - ", log_sep, log_sep); - break; + if (set.logging.log_datetime_format < GD_MIN || set.logging.log_datetime_format > GD_MAX) { + set.logging.log_datetime_format = GD_DEFAULT; } + char separator = separators[set.logging.log_datetime_separator]; + const char *const *parts = formats[set.logging.log_datetime_format]; + spine_snprintf(log_date_format, sizeof(log_date_format), "%s%c%s%c%s %%H:%%M:%%S - ", + parts[0], separator, parts[1], separator, parts[2]); } char *get_date_format(void) { @@ -1484,202 +1438,112 @@ char *get_date_format(void) { * \param *logmessage a pointer to the pre-formatted log message. * */ +bool spine_should_log_device(int host_id, int verbosity) { + return is_debug_device(host_id) || set.logging.log_level >= verbosity; +} + +static FILE *log_error_stream(void) { + #ifdef DISABLE_STDERR + return stdout; + #else + return stderr; + #endif +} + +static bool log_stream_available(const FILE *stream) { + return (stream == stdout && !set.console.stdout_notty) || (stream == stderr && !set.console.stderr_notty); +} + +static void log_format_error(const char *message) { + FILE *stream = log_error_stream(); + if (log_stream_available(stream)) fprintf(stream, "%s\n", message); +} + +static bool log_format_message(char *output, const char *message) { + char prefix[LOGSIZE]; + snprintf(prefix, sizeof(prefix), "SPINE: Poller[%i] PID[%i] PT[%lu] ", set.poller.poller_id, getpid(), (unsigned long int)pthread_self()); + time_t now = time(NULL); + struct tm local; + const char *date_format = get_date_format(); + size_t date_length = 0; + if (localtime_r(&now, &local) != NULL) date_length = strftime(output, 50, date_format, &local); + if (date_length == 0) { + output[0] = '\0'; + log_format_error("ERROR: Could not get string from strftime()"); + } + int prefix_length = spine_count_to_int(strlen(prefix)); + int message_length = spine_count_to_int(strlen(message)); + int available = LOGSIZE - spine_count_to_int(date_length) - 2; + if (prefix_length > available) prefix_length = available; + if (message_length > available - prefix_length) message_length = available - prefix_length; + snprintf(output + date_length, LOGSIZE - date_length, "%.*s%.*s", prefix_length, prefix, message_length, message); + return date_length != 0; +} + +static void log_to_syslog(const char *message) { + if (!IS_LOGGING_TO_SYSLOG()) return; + openlog("Cacti", LOG_NDELAY | LOG_PID, LOG_SYSLOG); + if ((strstr(message, "ERROR") || strstr(message, "FATAL")) && set.logging.log_perror) syslog(LOG_CRIT, "%s\n", message); + if (strstr(message, "WARNING") && set.logging.log_pwarn) syslog(LOG_WARNING, "%s\n", message); + if (strstr(message, "STATS") && set.logging.log_pstats) syslog(LOG_NOTICE, "%s\n", message); + closelog(); +} + +static bool log_to_file(const char *message) { + if (!IS_LOGGING_TO_FILE() || set.logging.log_level == POLLER_VERBOSITY_NONE || !set.logging.path_logfile[0] || !set.logging.logfile_processed) return TRUE; + /* Serialize complete records and the once-only diagnostic. Append mode + * creates missing files without a stat/open race that can truncate them. */ + static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; + static bool reported_error = FALSE; + if (pthread_mutex_lock(&mutex) != 0) return FALSE; + /* O_APPEND selects the end atomically and one write() keeps another + * process sharing the log from splicing into a record. Reopening per + * message preserves log rotation. */ + int oldstate; + pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &oldstate); + int fd = open(set.logging.path_logfile, O_WRONLY | O_CREAT | O_APPEND, 0666); + bool success = FALSE; + if (fd >= 0) { + size_t length = strlen(message); + ssize_t written = write(fd, message, length); + success = written >= 0 && (size_t) written == length; + if (close(fd) != 0) success = FALSE; + } + pthread_setcancelstate(oldstate, NULL); + if (!success && !reported_error) { + printf("ERROR: Spine Log File Could Not Be Opened/Created or Written\n"); + reported_error = TRUE; + } + if (pthread_mutex_unlock(&mutex) != 0) return FALSE; + return success; +} + int spine_log(const char *format, ...) { - va_list args; - - FILE *fp = NULL; - - /* variables for time display */ - time_t nowbin; - struct tm now_time; - struct tm *now_ptr; - - /* keep track of an errored log file */ - static int log_error = FALSE; - - int of = 20; - char logprefix[LOGSIZE]; /* Formatted Log Prefix */ - char ulogmessage[LOGSIZE]; /* Un-Formatted Log Message */ - char flogmessage[LOGSIZE]; /* Formatted Log Message */ - char stdoutmessage[LOGSIZE+of]; /* Message for stdout */ - - double cur_time; - char * log_fmt; - int prefix_len; - int ulog_len; - int flog_len; - + char message[LOGSIZE]; + va_list args; va_start(args, format); - vsnprintf(ulogmessage, LOGSIZE - 1, format, args); + vsnprintf(message, LOGSIZE - 1, format, args); va_end(args); - - /* default for "console" messages to go to stdout */ - fp = stdout; - - /* log message prefix */ - - snprintf(logprefix, LOGSIZE, "SPINE: Poller[%i] PID[%i] PT[%lu] ", set.poller_id, getpid(), (unsigned long int)pthread_self()); - - /* get time for poller_output table */ - nowbin = time(&nowbin); - - localtime_r(&nowbin,&now_time); - now_ptr = &now_time; - + spine_sanitize_log_message(message); if (IS_LOGGING_TO_STDOUT()) { - cur_time = get_time_as_double(); - snprintf(stdoutmessage, LOGSIZE + of, "Total[%3.4f] %s", cur_time - start_time, ulogmessage); - puts(stdoutmessage); - return TRUE; + char console[LOGSIZE + 20]; + snprintf(console, sizeof(console), "Total[%3.4f] %s", get_time_as_double() - start_time, message); + return puts(console) == EOF ? FALSE : TRUE; } - - log_fmt = get_date_format(); - - if (strlen(log_fmt) == 0) { - #ifdef DISABLE_STDERR - fp = stdout; - #else - fp = stderr; - #endif - - if ((set.stderr_notty) && (fp == stderr)) { - /* do nothing stderr does not exist */ - } else if ((set.stdout_notty) && (fp == stdout)) { - /* do nothing stdout does not exist */ - } else { - fprintf(fp, "ERROR: Could not get format from get_date_format()\n"); - } + char formatted[LOGSIZE]; + bool date_valid = log_format_message(formatted, message); + log_to_syslog(formatted); + if (strchr(formatted, '\n') == NULL) { + size_t used = strlen(formatted); + snprintf(formatted + used, sizeof(formatted) - used, "\n"); } - - prefix_len = strlen(logprefix); - ulog_len = strlen(ulogmessage); - flog_len = 0; - - if ((flog_len = strftime(flogmessage, 50, log_fmt, now_ptr)) == (int) 0) { - #ifdef DISABLE_STDERR - fp = stdout; - #else - fp = stderr; - #endif - - if ((set.stderr_notty) && (fp == stderr)) { - /* do nothing stderr does not exist */ - } else if ((set.stdout_notty) && (fp == stdout)) { - /* do nothing stdout does not exist */ - } else { - fprintf(fp, "ERROR: Could not get string from strftime()\n"); - } + bool success = log_to_file(formatted); + if (set.logging.log_level >= POLLER_VERBOSITY_NONE) { + FILE *stream = stdout; + if (!date_valid || strstr(formatted, "ERROR") || strstr(formatted, "WARNING") || strstr(formatted, "FATAL")) stream = log_error_stream(); + if (log_stream_available(stream) && fprintf(stream, "%s", formatted) < 0) success = FALSE; } - - /* determine how many characters to append */ - if (prefix_len > LOGSIZE - flog_len - 1) { - prefix_len = LOGSIZE - flog_len - 1; - } - - if (ulog_len + prefix_len > LOGSIZE - flog_len - 1) { - ulog_len = LOGSIZE - flog_len - prefix_len - 1; - } - - /* strncat() here trips -Wstringop-truncation: prefix_len/ulog_len are - * runtime-clamped just above to fit, but GCC can't prove that across the - * branches. memcpy() plus an explicit terminator is exactly as safe and - * doesn't trigger the false positive. */ - memcpy(flogmessage + flog_len, logprefix, (size_t) prefix_len); - flog_len += prefix_len; - memcpy(flogmessage + flog_len, ulogmessage, (size_t) ulog_len); - flog_len += ulog_len; - flogmessage[flog_len] = '\0'; - - /* output to syslog/eventlog */ - if (IS_LOGGING_TO_SYSLOG()) { - openlog("Cacti", LOG_NDELAY | LOG_PID, LOG_SYSLOG); - - if ((strstr(flogmessage,"ERROR") || (strstr(flogmessage, "FATAL"))) && (set.log_perror)) { - syslog(LOG_CRIT,"%s\n", flogmessage); - } - - if ((strstr(flogmessage,"WARNING")) && (set.log_pwarn)){ - syslog(LOG_WARNING,"%s\n", flogmessage); - } - - if ((strstr(flogmessage,"STATS")) && (set.log_pstats)){ - syslog(LOG_NOTICE,"%s\n", flogmessage); - } - - closelog(); - } - - /* append a line feed to the log message if needed. The two strncat() - * calls above are allowed to fill flogmessage exactly, so appending - * unconditionally would put the terminator one byte past the end. */ - if (!strstr(flogmessage, "\n")) { - size_t flogmessage_len = strlen(flogmessage); - - if (flogmessage_len < sizeof(flogmessage) - 1) { - flogmessage[flogmessage_len] = '\n'; - flogmessage[flogmessage_len + 1] = '\0'; - } - } - - if ((IS_LOGGING_TO_FILE() && - (set.log_level != POLLER_VERBOSITY_NONE) && - (strlen(set.path_logfile) != 0))) { - if (set.logfile_processed) { - int log_fd; - int oldstate; - size_t message_len; - ssize_t bytes_written; - - /* O_APPEND selects the end atomically, and one write() keeps a message - * from being split into stdio-sized chunks another thread can splice - * into. Reopening per message preserves log rotation. */ - pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &oldstate); - log_fd = open(set.path_logfile, O_WRONLY | O_CREAT | O_APPEND, 0666); - - if (log_fd >= 0) { - message_len = strlen(flogmessage); - bytes_written = write(log_fd, flogmessage, message_len); - close(log_fd); - - if (bytes_written < 0 || (size_t) bytes_written != message_len) { - if (!log_error) { - printf("ERROR: Spine Log File Write Was Incomplete\n"); - log_error = TRUE; - } - } - } else { - if (!log_error) { - printf("ERROR: Spine Log File Could Not Be Opened/Created\n"); - log_error = TRUE; - } - } - - pthread_setcancelstate(oldstate, NULL); - } - } - - if (set.log_level >= POLLER_VERBOSITY_NONE) { - if ((strstr(flogmessage,"ERROR")) || - (strstr(flogmessage,"WARNING")) || - (strstr(flogmessage,"FATAL"))) { - #ifdef DISABLE_STDERR - fp = stdout; - #else - fp = stderr; - #endif - } - - if ((set.stderr_notty) && (fp == stderr)) { - /* do nothing stderr does not exist */ - } else if ((set.stdout_notty) && (fp == stdout)) { - /* do nothing stdout does not exist */ - } else { - fprintf(fp, "%s", flogmessage); - } - } - - - return TRUE; + return success; } /*! \fn int file_exists(const char *filename) @@ -1754,9 +1618,10 @@ int is_ipaddress(const char *string) { * */ int is_numeric(char *string) { - char *end_ptr_long, *end_ptr_double; + char *end_ptr_long; + char *end_ptr_double; int conv_base=10; - int length; + size_t length; length = strlen(trim(string)); @@ -1771,13 +1636,11 @@ int is_numeric(char *string) { if (errno != ERANGE) { if (end_ptr_long == string + length) { /* integer string */ return TRUE; - } else if (end_ptr_long == string) { - if (*end_ptr_long != '\0' && + } else if ((end_ptr_long == string) && (*end_ptr_long != '\0' && *end_ptr_long != '.' && *end_ptr_long != '-' && - *end_ptr_long != '+') { /* ignore partial string matches but doubles can begin with '+', '-', '.' */ - return FALSE; - } + *end_ptr_long != '+')) { /* ignore partial string matches but doubles can begin with '+', '-', '.' */ + return FALSE; } } else { end_ptr_long = NULL; @@ -1828,10 +1691,8 @@ int is_hexadecimal(const char * str, const short ignore_special) { delim_found = TRUE; break; case '\t': - if (ignore_special) { - break; - } - /* fall through */ + if (!ignore_special) return FALSE; + break; default: return FALSE; } @@ -1855,38 +1716,11 @@ int is_hexadecimal(const char * str, const short ignore_special) { * */ char *strip_alpha(char *string) { - int i; - int j; - - i = strlen(string); - j = 0; - - /* trim trailing characters */ - while (i >= 0) { - if (isdigit((int)string[i])) { - break; - } else { - string[i] = '\0'; - } - i--; - } - - /* trim leading characters */ - while (j < i) { - if (isdigit((int)string[j])) { - break; - } else if (string[j] == '-') { - break; - } else if (string[j] == '+') { - j++; - } else { - j++; - } - } - - string = &string[j]; - - return string; + size_t end = strlen(string); + while (end > 0 && !isdigit((unsigned char)string[end - 1])) string[--end] = '\0'; + size_t start = 0; + while (start < end && !isdigit((unsigned char)string[start]) && string[start] != '-') start++; + return string + start; } /*! \fn char *add_slashes(const char *string) @@ -1897,40 +1731,20 @@ char *strip_alpha(char *string) { * */ char *add_slashes(const char *string) { - int length; - int position; - int new_position; - char *return_str; - - length = strlen(string); - - if (!(return_str = (char *) malloc(length * 2 + 1))) { - die("ERROR: Fatal malloc error: util.c add_slashes!"); + size_t length = strlen(string); + if (length > (SIZE_MAX - 1) / 2) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Escaped command exceeds addressable memory"); } - return_str[0] = '\0'; - position = 0; - new_position = 0; - - /* simply return on blank string */ - if (!length) { - return return_str; + char *result = malloc(length * 2 + 1); + if (result == NULL) die("ERROR: Fatal malloc error: util.c add_slashes!"); + size_t used = 0; + for (size_t i = 0; i < length; i++) { + if (string[i] == '\\') result[used++] = '\\'; + result[used++] = string[i]; } - - while (position < length) { - /* backslash detected, change to forward slash */ - if (string[position] == '\\') { - return_str[new_position] = '\\'; - new_position++; - return_str[new_position] = '\\'; - } else { - return_str[new_position] = string[position]; - } - new_position++; - position++; - } - return_str[new_position] = '\0'; - - return(return_str); + result[used] = '\0'; + return result; } /*! \fn char *strncopy(char *dst, const char *src, size_t obuf) @@ -1951,24 +1765,19 @@ char *add_slashes(const char *string) { * */ char *strncopy(char *dst, const char *src, size_t obuf) { - size_t copy_len; - assert(dst != 0); assert(src != 0); + size_t len; + if (obuf == 0) return dst; - - /* Cap the scan at obuf-1: no need to walk past the usable copy capacity, - * and avoids a full strlen when src is large or unterminated near obuf. */ - copy_len = strnlen(src, obuf - 1); - - if (copy_len) { - /* copy_len is the exact byte count and dst is terminated below, so - * memcpy avoids the strncpy truncation diagnostic. */ - memcpy(dst, src, copy_len); + /* Bound the scan by the usable capacity; src may be large or unterminated. */ + len = strnlen(src, obuf - 1); + if (len) { + memcpy(dst, src, len); } - dst[copy_len] = '\0'; + dst[len] = '\0'; return dst; } @@ -1982,7 +1791,7 @@ double get_time_as_double(void) { gettimeofday(&now, NULL); - return (now).tv_sec + ((double) (now).tv_usec / 1000000); + return (double)now.tv_sec + (double)now.tv_usec / 1000000; } /*! \fn trim() @@ -2044,18 +1853,15 @@ char *ltrim(char *str) { * \return the reversed string. */ char *reverse(char* str) { - int end = strlen(str)-1; - int start = 0; - + size_t start = 0; + size_t end = strlen(str); while (start < end) { - str[start] ^= str[end]; - str[end] ^= str[start]; - str[start] ^= str[end]; - - ++start; - --end; + end--; + if (start >= end) break; + char byte = str[start]; + str[start++] = str[end]; + str[end] = byte; } - return str; } @@ -2068,7 +1874,7 @@ int strpos(const char *haystack, const char *needle) { const char *p = strstr(haystack, needle); if (p) { - return p - haystack; + return p - haystack <= INT_MAX ? (int)(p - haystack) : -1; } return -1; @@ -2081,7 +1887,7 @@ int strpos(const char *haystack, const char *needle) { */ int char_count(const char *str, int chr) { const unsigned char *my_str = (const unsigned char *) str; - const unsigned char my_chr = chr; + const unsigned char my_chr = (unsigned char)chr; int count = 0; if (!my_chr) return 1; @@ -2178,7 +1984,7 @@ int hasCaps(void) { } void checkAsRoot(void) { - set.icmp_uses_caps = FALSE; + set.availability.icmp_uses_caps = FALSE; #ifndef __CYGWIN__ #ifdef SOLAR_PRIV priv_set_t *privset; @@ -2215,16 +2021,15 @@ void checkAsRoot(void) { boolean_t pe = priv_ineffect(PRIV_NET_ICMPACCESS); SPINE_LOG_DEBUG(("DEBUG: Privilege PRIV_NET_ICMPACCESS is: '%s'.", pe != 0 ? "Enabled" : "Disabled")); - set.icmp_avail = pe; + set.availability.icmp_avail = pe; /* Free the privset */ priv_freeset(privset); free(p); #else if (hasCaps() != TRUE) { - int ret; SPINE_LOG_DEBUG(("DEBUG: Spine running as %d UID, %d EUID", getuid(), geteuid())); - ret = seteuid(0); + int ret = seteuid(0); if (ret != 0) { SPINE_LOG_DEBUG(("WARNING: Spine NOT able to set effective UID to 0")); } @@ -2240,14 +2045,14 @@ void checkAsRoot(void) { if (probe != -1) { close(probe); SPINE_LOG_DEBUG(("DEBUG: Spine may use unprivileged ICMP sockets.")); - set.icmp_avail = TRUE; + set.availability.icmp_avail = TRUE; } else { SPINE_LOG_DEBUG(("WARNING: Spine NOT running as root. This is required if using ICMP unless net.ipv4.ping_group_range covers this user. Please run \"chown root:root spine;chmod u+s spine\" to resolve.")); - set.icmp_avail = FALSE; + set.availability.icmp_avail = FALSE; } } else { SPINE_LOG_DEBUG(("DEBUG: Spine is running as root.")); - set.icmp_avail = TRUE; + set.availability.icmp_avail = TRUE; if (seteuid(getuid()) == -1) { SPINE_LOG_DEBUG(("WARNING: Spine unable to drop from root to local user.")); @@ -2255,10 +2060,10 @@ void checkAsRoot(void) { } } else { SPINE_LOG_DEBUG(("DEBUG: Spine has cap_net_raw capability.")); - set.icmp_avail = TRUE; - set.icmp_uses_caps = TRUE; + set.availability.icmp_avail = TRUE; + set.availability.icmp_uses_caps = TRUE; } - SPINE_LOG_DEBUG(("DEBUG: Spine has %sgot ICMP", set.icmp_avail?"":"not ")); + SPINE_LOG_DEBUG(("DEBUG: Spine has %sgot ICMP", set.availability.icmp_avail?"":"not ")); #endif #endif } @@ -2272,64 +2077,67 @@ void checkAsRoot(void) { * \return the cacti version * */ +static int parse_cacti_version(const char *value) { + const unsigned long weights[] = {1000, 100, 1}; + unsigned long version = 0; + for (size_t index = 0; index < sizeof(weights) / sizeof(weights[0]); index++) { + if (*value < '0' || *value > '9') return 0; + errno = 0; + char *end; + unsigned long component = strtoul(value, &end, 10); + if (errno == ERANGE || component > ((unsigned long)INT_MAX - version) / weights[index]) return 0; + version += component * weights[index]; + if (index < 2 && *end != '.') return 0; + value = end + (index < 2 ? 1 : 0); + } + /* Preserve suffixes such as develop/beta accepted by the previous three-component scan. */ + return (int)version; +} + int get_cacti_version(MYSQL *psql, int mode) { - char qstring[BUFSIZE]; - char *retval; - MYSQL_RES *result; - MYSQL_ROW mysql_row; - int major, minor, point; - int cacti_version; + assert(psql != NULL); + MYSQL_RES *result = db_query(psql, mode, "SELECT cacti FROM version LIMIT 1"); + if (result == NULL) return 0; + MYSQL_ROW row = mysql_fetch_row(result); + int version = row != NULL && row[0] != NULL ? parse_cacti_version(row[0]) : 0; + db_free_result(result); + return version; +} - assert(psql != 0); +/* pthread storage keeps the borrowed match valid without requiring C11 TLS. + * Each polling thread owns its buffer, which is freed when the thread exits. */ +static pthread_key_t regex_result_key; +static pthread_once_t regex_result_once = PTHREAD_ONCE_INIT; - snprintf(qstring, sizeof(qstring), "SELECT cacti FROM version LIMIT 1"); - - result = db_query(psql, mode, qstring); - - if (result != 0) { - if (mysql_num_rows(result) > 0) { - mysql_row = mysql_fetch_row(result); - - if (mysql_row != NULL) { - retval = strdup(mysql_row[0]); - db_free_result(result); - - if (STRIMATCH(retval, "new_install")) { - SPINE_FREE(retval); - - return 0; - } else { - sscanf(retval, "%d.%d.%d", &major, &minor, &point); - cacti_version = (major * 1000) + (minor * 100) + (point * 1); - - SPINE_FREE(retval); - - return cacti_version; - } - }else{ - db_free_result(result); - return 0; - } - }else{ - db_free_result(result); - return 0; - } - }else{ - return 0; +static void initialize_regex_result_key(void) { + if (pthread_key_create(®ex_result_key, free) != 0) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Unable to create thread-local regex storage"); } } -const char *regex_replace(const char *exp, const char *value) { +static char *regex_result_buffer(void) { + if (pthread_once(®ex_result_once, initialize_regex_result_key) != 0) { + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Unable to initialize thread-local regex storage"); + } + char *buffer = pthread_getspecific(regex_result_key); + if (buffer == NULL) { + buffer = malloc(RESULTS_BUFFER); + if (buffer == NULL) die("ERROR: Fatal malloc error: regex result buffer!"); + if (pthread_setspecific(regex_result_key, buffer) != 0) { + free(buffer); + set.exit.exit_code = EXIT_FAILURE; + die("ERROR: Unable to retain thread-local regex storage"); + } + } + return buffer; +} + +char *regex_replace(const char *exp, char *value) { regex_t regex; int reti; - /* Thread-local storage: each polling thread gets its own buffer, so - * concurrent callers do not race. Callers must consume the result before - * making another call on the same thread (all current call sites do this). - * C11 _Thread_local is equivalent; __thread is used here since GCC/Clang - * (the only supported compilers for this codebase) treat them identically. */ - static __thread char msgbuf[SMALL_BUFSIZE]; - regmatch_t matches[MAX_MATCHES]; - size_t match_len; + char *msgbuf = NULL; /* Stored output_regex values use the historical basic-regex dialect. */ reti = regcomp(®ex, exp, 0); @@ -2338,20 +2146,24 @@ const char *regex_replace(const char *exp, const char *value) { } /* Execute regular expression */ + regmatch_t matches[MAX_MATCHES]; reti = regexec(®ex, value, MAX_MATCHES, matches, 0); if (!reti) { - match_len = (size_t)(matches[0].rm_eo - matches[0].rm_so); - if (match_len >= SMALL_BUFSIZE) { - match_len = SMALL_BUFSIZE - 1; + // regex matched + size_t length = (size_t) (matches[0].rm_eo - matches[0].rm_so); + if (length >= RESULTS_BUFFER) { + regfree(®ex); + return value; } - memcpy(msgbuf, value + matches[0].rm_so, match_len); - msgbuf[match_len] = '\0'; + msgbuf = regex_result_buffer(); + memcpy(msgbuf, value + matches[0].rm_so, length); + msgbuf[length] = '\0'; } /* Free memory allocated to the pattern buffer by regcomp() */ regfree(®ex); - return (reti) ? value : msgbuf; + return reti ? value : msgbuf; } int format_spine_capabilities(char *output, size_t output_size, diff --git a/util.h b/util.h index b3c5165..95112c1 100644 --- a/util.h +++ b/util.h @@ -37,6 +37,8 @@ extern void read_config_options(void); extern int read_spine_config(const char *file); extern void config_defaults(void); +extern bool poller_transfer_status(MYSQL *source, MYSQL *destination); +extern void poller_push_data_to_main(void); /* cacti logging function */ extern int spine_log(const char *format, ...) @@ -57,17 +59,23 @@ extern int is_hexadecimal(const char * str, const short ignore_special); /* determine if a device is a debug device */ extern int is_debug_device(int device_id); +extern void parse_debug_devices(char *device_list, int *devices, size_t capacity); +extern bool spine_should_log_device(int host_id, int verbosity); /* string and file functions */ extern char *add_slashes(const char *string); extern int file_exists(const char *filename); extern char *strip_alpha(char *string); extern char *strncopy(char *dst, const char *src, size_t n); +extern int spine_snprintf(char *output, size_t capacity, const char *format, ...) + __attribute__((format(printf, 3, 4))); +extern void spine_clear_sensitive(void *buffer, size_t length); +extern void spine_sanitize_log_message(char *message); extern char *trim(char *str); extern char *rtrim(char *str); extern char *ltrim(char *str); extern char *reverse(char *str); -extern int strpos(const char *haystack, const char *needle); +extern int strpos(const char *haystack, const char *needle) ; extern int char_count(const char *str, int chr); /* convert a delimited hexadecimal value without conflating overflow with 0 */ @@ -77,7 +85,7 @@ int hex2dec(const char *str, unsigned long long *result); #define MAX_MATCHES 5 #define REGEX_NUMBER "([-+]*)([0-9]*)([.][0-9]+)" #define CAPABILITY_PROTOCOL_LIST_MAX 480 -const char *regex_replace(const char *exp, const char *value); +char *regex_replace(const char *exp, char *value); int format_spine_capabilities(char *output, size_t output_size, const char *auth_protocols, const char *priv_protocols); @@ -118,6 +126,19 @@ extern double start_time; /* the version of Cacti as a decimal */ int get_cacti_version(MYSQL *psql, int mode); +extern void spine_sleep_usec(unsigned int microseconds); +extern double spine_monotonic_time(void); + +extern int spine_count_to_int(unsigned long long count); + +extern int spine_wait_readable(int fd, double deadline); +extern int spine_wait_writable(int fd, double deadline); + +extern int spine_permits_init(spine_permits_t *permits, int count); +extern int spine_permits_destroy(spine_permits_t *permits); +extern int spine_permits_try_acquire(spine_permits_t *permits); +extern int spine_permits_release(spine_permits_t *permits); +extern int spine_permits_available(spine_permits_t *permits); /*! \fn int spine_appendf(char **cursor, size_t *remaining, const char *fmt, ...) * \brief append to a bounded buffer without walking off the end *