/******************************************************************************* Author ......... Larry Adams Contact ........ larryjadams@comcast.net Home Site ...... http://www.cacti.net Program ........ Cacti Performance Booster Version ........ 4.2 Purpose ........ Large Site Performance Booster for Cacti *******************************************************************************/ ----[ Purpose This plugin boost's Cacti performance especially for Large Sites. It does this by introducing three new features to Cacti. First, it caches recently viewed images to a public Cache folder for all users to share. Second, it introduces an "on demand" RRD update feaure to Cacti. This feature will only update the RRD files when there is demand on the system to view a graph. The RRD's will be updated just before the Graph is rendered by the web server. Then, on a predetermined schedule, it conducts a batch update's of all remaining RRD's. Lastly, it introduces an RRD update service to Cacti. This service allows you to add multiple Cacti servers to your web farm allowing all the servers to participate in the "on demand" RRD update and viewing process. ----[ Features On Demand RRD Updates Timed and Mass RRD Updates by Number of Records RRD Update Service PNG Caching ----[ Prerequisites Before you install Boost, you need to have met the following pre-requisites: 1) Cacti 0.8.7 and above. 2) Cacti Plugin Architecture v2.x 3) MySQL 5.0 or above. Versions of MySQL less than 5.0 are not supported. 4) Have quite a bit of system memory available to hold your memory resident database and/or your sql results during major rrd update cycles. Before enabling boost, please consider very carefully how you plan to use it long term. Carefull thought should be given to how often you update RRD's at what point you need to either increase your MySQL max heap table size, or increase your update frequency. You should also consider carefully how much memory to allocate to PHP when retrieving records during major updates. If you are unsure of what I am talking about, you should not use Boost!!! ----[ Installation Install is similar to other plugins with the following notes: 0) Check permissions, the tar file was made on Windows. My next PC will be Linux. :) 1) You must first choose your RRD update cache database format. There are two options: - Memory Resident (MySQL 5.x and Above) - MyISAM (All Supported MySQL Versions) The benefit of the Memory Resident impmenentation is that database exchanges will be very fast and will not delay the system. However, if your system is rebooted between major update cycles, you can loose some graph data. The MyISAM version will increase MySQL load during polling, but does store the information in a lossless (well almost anyway) format. IMPORTANT: If you plan on using a memory resident database format, make sure you set the mysql environment variable "max_heap_table_size" large enough to accomodate all of your records, else, you will loose data. This environment must go into either your my.cnf or my.ini file. To install either format, you must follow the instructions below: mysql -u -p cacti < "my_chosen_format.sql" Options here are as follows: boost_sql_memory.sql, boost_sql_myisam.sql NOTE: You may skip this step. If you do, the tables will be initialized as memory tables. Note: if you choose the memory table option, you may want to limit the size of the output field to your maximum output length from all polled items as MySQL varchar(512) take 512 bytes of memory even if your output is simply a 'U'. 2) Once you have the SQL in place, you must enable you plugin by going to the Plugin Management interface and installing and enabling the plugin. Please note, that if you are running a prior version of Boost, it might be a good idea to clear out the boost cache before doing any of this. 3) Then, you must decide on a directory to hold your image cache, and verify permissions both their and in the rra directory. If you are not using the boost server, the web server account will need R/W into both directories and be able to create files in those directories as well. 4) Next you have to enable any of the three features mentioned. You can do this by going to your Cacti website, logging in as an adminnistrator and going to the Settings->Boost tab. 6) If your Web server is not going to have access to create/update RRD files, you must start the "boost_server.php" file as a service either as an init.d service, or more reliably as an inittab service. NOTE: The Inittab Process has the benefit of being able to restart the boost server if it crashes and therefore is preferred. In the UNIX/LINUX environment, to use the init.d process, use a script attached cacti_rrdsvc script and follow the instructions below. ------------------------------------------------------------------------ Init.d Process: ------------------------------------------------------------------------ a) Place the cacti_rrdsvc in the /etc/init.d folder. b) Edit cacti_rrdsvc and modify the variable PROGPATH to poing to the location of boost_server.php, and the DEBUBLOG entry. c) Edit both boost_server.php and boost_rrdupdate.php and verify the PHP path in the first line of the file. d) Mark the boost_server.php and boost_rrdupdate.php files as executable: chmod +x boost_server.php chmod +x boost_rrdupdate.php e) If using an id other than root to update the rrdfiles mark the boost_server.php and boost_rrdupdate.php files setiud: chown root:root boost_server.php chmod +s boost_server.php chown root:root boost_rrdupdate.php chmod +s boost_rrdupdate.php f) Edit the boost_server.php and make sure the embedded path to PHP is correct. g) Run the following commands. On Most Linux Variants: dos2unix cacti_rrdsvc (sorry, I'm still writing in Windows) chkconfig --add cacti_rrdsvc chkconfig cacti_rrdsvc on h) Then either: service cacti_rrdsvc start [debug] or: /etc/init.d/cacti_rrdsvc start [debug] The "debug" option is optional i) Make sure it's running by issuing the following command: ps -ef | grep cacti_rrdsvc ------------------------------------------------------------------------ Inittab Process: ------------------------------------------------------------------------ NOTE: this process varies by Operating system. For this example, we will assume a Linux install. On Windows, you should create a service. I will not be covering how to do that here. a) Edit /etc/inittab and Place a new entry in the file as such: # Cacti Boost Server bsts:345:respawn:/var/www/html/cacti/plugins/boost/boost_server.php You may replace bsts with the unique ID of your choice. b) Edit both boost_server.php and boost_rrdupdate.php and verify the PHP path in the first line of the file. c) Mark the boost_server.php and boost_rrdupdate.php files as executable: chmod +x boost_server.php chmod +x boost_rrdupdate.php d) If using an id other than root to update the rrdfiles mark the boost_server.php and boost_rrdupdate.php files setiud: chown root:root boost_server.php chmod +s boost_server.php chown root:root boost_rrdupdate.php chmod +s boost_rrdupdate.php e) Refresh/Reload the inittab using the following command: init q f) Make sure it's running by issuing the following command: ps -ef | grep boost_server.php 7) Finally, you can poll the server using telnet. To do so, start a telnet session on the port you choose to use (default 9050) and type the "status" command followd by a carriage return. You should receive a favorable response. ----[ Additional Help? If you need additional help, please goto forums.cacti.net. ----[ Possible Bugs? If you figure out this problem, let me know!!! ----[ Special Thanks Ernest Wilson (N3NCY) - For allowing me to break and then fix his system repeatedly perfecting this Plugin. Jimmy Connor (cigamit) - For bringing the plugin architecture to the world of Cacti and provding continual support of my development efforts. dononeil (dononeil) - For his help with the inittab process setup documentation for Linux. Boris Lytochkin (BorisL) - For his idea of using a lock file to detect process overruns correctly and inspiring me to complete Boost 2.0. ----[ Future Changes Got any ideas or complaints, please e-mail me! ----[ Changelog --- SVN --- bug: Correct misspelling feature: Set the permissions on the Cache files so that they can be managed by the poller feature#0001853: Boost v4.2 does not check graph size before pushing cached image --- 4.2 --- compat: Don't override Cacti log level bug: eliminate harmless temporary table errors bug: when disabled, this plugin should not run bug: warn if you are attempting to use boost redirect and spine isn't in use --- 4.1 --- bug: Major issue with BTREE PRIMARY KEYS and MEMORY TABLES in MySQL causing gaps feature: When boost redirect is not in use, do not update to the second to prevent graph breakage --- 4.0 --- feature: Add support for Cacti's new rrdtool segfault detection bug: Setting logging to MEDIUM and higher causes boost images to break --- 3.1 --- bug: Boost poller statistics logging missing global variable declaration bug#0001643: When boost 3.0 plugin is installed and then uninstalled it doesn't cleans database entries properly. bug#0001555: Boost errors generated in log when another user has graphs open feature: Allow boost to recover from rrdtool crashes compat: Make the error handler not require PHP 5.2 compat: Remove PIA 1.x support --- 3.0 --- feature: Deep performance tuning and code cleanup feature: Add logging to boost updates for performance tuning feature: Add hooks to support 95th Percentile and Bandwidth Summation --- 2.6 --- bug: Initial RRDfile update taks two boost cycles and not one bug: Cacti RRDtool fetch command bypasses boost causing issues bug: Don't optimize boost table if it's in MEMORY storage engine --- 2.5 --- feature: Integration with DSStats feature: Support boost redirect for large systems feature: Properly support structured paths --- 2.4 --- bug: [Boost 2.3] Include line referring to thold plugin bug: Should be db_fetch_row and not db_fetch_cell feature: Allow specification of memory limits --- 2.3 --- feature: Better max memory reporting for older PHP installs feature: Reduce the load on the database server when displaying boost stats feature: Integrate better with tholds vrule display (image caching off) feature: Upgrade Automatically when either viewing the console or the plugin management page. --- 2.2 --- bug: If running the boost server, mysql can timeout causing 2006 errors bug: Typo relative to Linux Kernel versions in relative to command length bug: Logging did not work on all occasions due to an undefined variable error bug: Improve the counting algorythm to remove some bogus RRDtool update errors bug: Check the validity of Multivalue updates prior to allowing updates bug: Correct documentation relative to the the inittab process so that it works bug: Prevent more than one Boost Server from running at a time by validating Socket bind --- 2.1 --- bug: Correcting overrun issues bug: Rename maximum rows to maximum data sources bug: RRDupdate batch process was not working as designed causing excessive memory to be consumed in some cases feature: Log memory performance and make available via UI --- 2.0 --- bug: remove the base start time, it complicates things bug: make version, force, and debug options consistent bug: rrdupdates oftentimes caused gaps in graphs due to limit issues feature: add an rrdtool output logging option feature: support inittab for restarting the boost server feature: support process locking control methodology --- 1.8 --- bug: correct issue with multiple temporary tables using the same name are encountered bug: significant bug from 1.6 that prevents the poller_output_boost table from emptying completely --- 1.7 --- feature: Add Cacti 0.8.7 compatibility --- 1.6 --- feature: make the delete process a high performance one by batching the deletes bug: set a good memory limit and runtime in poller_boost.php --- 1.5 --- bug: Under some circumstanced the poller_boost.php could not be fored to run bug: Scripts that do not return the correct number of DS' did not update properly bug: New forcerun option introduced in version 1.3 was causing boost not disable properly when you unselected it from the user interface. --- 1.4 --- bug: Fixed an issue with data queries where the rrdupdate template was not formated correctly --- 1.3 --- bug: Fixed an issue where multi part responses with a "0" would return "U" feature: Added better logging functions to help in debugging feature: Allow you to increase the MySQL Insert string length to improve performance feature: Allow you to increase the RRDtool update string length to improve performance --- 1.2 --- bug: Added boost_server.php and poller_boost.php to the no session array bug: Made slight change to rrd-update functions to accomodate for abarant output in the poller_output_boost table. This would cause the boost plugin to loose data if improperly formatted data made it into the table. --- 1.1 --- bug: Fix issues with Multiprocess RRDupdating of RRD Files --- 1.0 --- Initial release --- Known Issues --- None reported