This commit includes metrics forecasting for VMs and Hosts. It provides
basic algorithms for linear regression.
* VM prediction includes the following metrics:
- cpu
- memory
- netrx
- nettx
- diskrdbytes
- diskwrbytes
- diskrdiops
- diskwriops
* Host prediction includes:
- usedcpu
- freecpu
- usedmemory
- freememory
- netrx
- nettx
The prediction framework uses a time series DB per VM (stored in the VM
folder) to compute the forecast.
co-authored-by: Marco Mancini <mmancini@opennebula.io>
co-authored-by: Jakub Walczak <jwalczak@opennebula.io>
co-authored-by: dmingolla <dmingolla@opennebula.io>
This commits introduces a new framework for OpenNebula, it is based on
standard the OpenNebula middleware access driver (mad) model. Benefits
compared to the current implementation:
* Less overhead on API server and streamlined communications
* Ease integration with third party schedulers
* New "optimize" action to balance the workload of a cluster
* Embedded Plan Manager to apply optimization and deployment plans with
custom parameters for the scheduling window
* Custom optimization policies per cluster
This commits introduces 2 schedulers:
* Rank scheduler based on the match-making algorithm present in 6.x
series
* OpenNebula DRS an optimization algorithm based on an ILP solver for
placement and cluster optimization
Also new API calls has been added for onecluster as well as the CLI
updates to expose them.
co-authored-by: MarioRobres <mrobres@opennebula.io>
co-authored-by: Pavel Czerny <pczerny@opennebula.io>
co-authored-by: Mirko Stojiljkovic <mstojiljkovic@opennebula.io>
co-authored-by: Valentyn Bohdan <vbohdan@opennebula.io>
This feature auto mounts NFS volumes for datastores. The behavior can be controlled with the following template attributes:
- ANFS_ENABLE: YES/NO to enable the automatic NFS management
- ANFS_HOST: hostname/IP of the NFS server
- ANFS_PATH: of the NFS share for this datastore
- ANFS_OPTS: Comma separated options (fstab-like) used for mounting the NFS shares
The shared drivers also performs a "garbage-collector" like to unmount NFS volumes on the hypervisors when the datastores are deleted. This operation is performed in a "lazy" way, when VMs are deployed on the hypervisor.
* oneimage create: mount/umount DS on demand
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Move sh mount/umount logic to libfs.sh
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* oneimage delete: mount/umount DS on demand
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Limit NFS mount/umount from SH code to 30s
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* oneimage clone: mount/umount DS on demand
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* onevm deploy: mount DS on demand
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Temporal mount only when needed
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* onevm deploy: mount DS on demand (persisten)
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Linter
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* onevm deploy: umount unneeded DS
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Allow mount/umount/sed from sudoers
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Temporal mount: use NFS_AUTO_OPTS
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Do AutoNFS cleanup even during non-related ops
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Final refactor and improvements
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* ro values for Datastore object
* Wording for AutoNFS RO attributes
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* NFS_AUTO_ENABLE: change 'true' to 'yes'
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* sudoers: merge everythng into ONE_NFS
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Quote NFS_AUTO_OPTS
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Fix difference between sudoers specs
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Touch .automounted in AutoNFS mounts
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Update sudoers.rb
* Update rm
* Quote bash variable expansion
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
---------
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
Co-authored-by: Ruben S. Montero <rsmontero@opennebula.org>
This feature adds support for VM quotas at cluster level. CLUSTER_IDS
can be set for a VM quota so the admin can limit the number of resources (e.g. VMs) a
user/group can create in a given cluster.
This commit also fixes:
- B #2226
- B #6823
Co-authored-by: Pavel Czerny <pczerny@opennebula.io>
This commit recovers the "enforce" argument for the resize operation onevm resize request. In this case the flag is only applied to the capacity, the NUMA topology is always evaluated in the resize operation.
Signed-off-by: Valentyn Bohdan <vbohdan@opennebula.io>
* Remove vm_import table from DB
* Remove imported vms actions
* Fix fsck for image and network
* onedb fsck fix running_vms only for non-backup images
* M #-: Replace raw pointers with references/smart pointers.
- Change PoolSQL::allocate method's signature to accept reference instead of pointer and fix all method calls.
- Remove some of the error-handling goto, to be able initialize some variables during declaration.
Signed-off-by: Valentyn Bohdan <vbohdan@opennebula.io>
Implementation overview:
- Incremental points are saved as dedicated rbd snapshots under the
"one_backup_<increment_id>" namespace. This snapshots are used to
generate delta files in rbdiff format.
- The rbdiff formats are stored in the backup server to restore the rbd volumes.
- The restore process is performed directly on the Ceph cluster
importing the base image (first full backup in the chain, rbd import)
and then applying the increments (rbd import-diff) up to the target
increment.
- Two new pseudo-protocols has been implemented to adopt the restore
pattern above (restic+rbd, rsync+rbd). This protocols bundle of the
rbdiff files in a tarball for transfer from the backup server. Note:
reconstruct process uses the Ceph BRIDGE_LIST and not the backup
server (as opposed to qcow2 backups)
Other bug fixes
- This commit also fixes#6741, resetting the backup chain after a
restore
- The original ceph drivers do not receive the full action information,
this now has been fixed by including VM information in the STDIN string sent
to the driver.
Compatibility note.
- backup actions should return now the backup format used raw, rbd, ...
If not provided oned (6.10.x) will use raw as a default to accommodate any
third party driver implementation. It is recommended to include this
third argument.
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
Co-authored-by: Guillermo Ramos <gramos@opennebula.io>
- oned load only last 2 history records (not the full list)
- Dump all history records only if needed in VirtualMachine::to_xml.
- Dump conforms XML schecam and removes VM template from history records.
Speed up of onevm show command:
- for small SQLite DB is for VM with 500 histories: 130 ms down to 5 ms
- for big MySQL DB VM with 687 histories: 1000 ms down to 200 ms
* Cleanup oned in case of initialization error
* Fix monitoring after onemonitord restart. The code includes a "hook" point in case a driver is re-started so custom code can be executed. InformationManager sends the list of hosts and raft status in this case.
* B #5801: Update error msg, in case of duplicated drivers
* Fix body_json for history record
* Update the body_json on every VM update to keep the data synced with the xml body
* Remove few duplicated VM updates
* Refactor release vnc port
* Fix VNC port for recover --recreate in stopped state
* Improve error messages
* Update recover --recreate description
(cherry picked from commit 9ed1fba8b95c1ea5d3984f136e638b973577ad69)
The Template::remove could crash if user assign Single value to attribute, which is expected to be a Vector, e.g. SCHED_ACTION, PCI, ...
Other minor changes:
* refactor the 'set' method - no need to call private _set
* Simplify some callers of the Template::remove method
This commit implements the in-place restore of VM backups. Selected VM disks will
be replaced with the specified backup:
* A new API call has been added to the XML-RPC API (`one.vm.restore`) with
the following arguments:
- VM ID to be restored, needs to be in **poweroff** state
- IMAGE ID of the backup to restore
- INCREMENT ID, only for incremental backups, the increment to use
(defults to -1 to use the last increment available)
- DISK ID of the disk to restore (defaults to -1 to restore all VM
disks)
* Datastore drivers needs to implemente a new operation `ls`. This new
operation takes the VM, image information of the backup and datastore
information and returns the restore URL for the disks in the backup.
* This commit includes the implementation for qcow2 and ssh drivers,
ceph will be implemented in a separated PR. The new driver action is
`restore host:vm_dir vm_id img_id inc_id disk_id`
* The restore operation is performed in a new state `PROLOG_RESTORE`
rendered as `RESTORE` and `rest` in short form. State in in RSuntone.
TODO:
- Remove any existing VM snapshot (system/disk) in the VM. Note that
snapshots are not included in a backup.
- Ceph drivers
- JAVA, GO Lang API bindings
- Sunstone interface, new state and new operation. Review new state in
RSuntone.
co-authored-by: Pavel Czerny <pczerny@opennebula.io>
When a disk with snapshots is resized and a subsequent revert operation
is performed, the following actions are now done:
* DISK/SIZE is updated to reflect the active snapshot size
* Quotas are updated to account the space freed after the resize
operation
(cherry picked from commit 178142105dea30c7deffd1842d0f0ad23d7dc93b)
(cherry picked from commit d2aaf2c8823d4172b1be673f959c4c922266534f)
Valentyn Bohdan