This commit reverts the backup_incremental test. Now we check for not supported
configurations:
- qcow2 disks with snapshots
- raw disks in local/shared TM's
- lvm with lvm thin disabled
Add LAST_BRIDGE To the backup operation. OpenNebula oned selects and stores the bridge in the BACKUP metadata. The drivers can use the selected bridge to perform the different backup phases (on the same bridge)
(cherry picked from commit 18e251659d)
This commits adds support for generating a NUMA PCIe hierarchy by adding
a pcie-expander-bus in each virtual node. When a PCI passthrough or
SR-IOV device is added is attached to the same virtual node as the
physical server. Changes include:
- Generate VM_* device parameters when the allocation is performed
- Generate CONTEXT PCIi_ADDRESS parameters at deployment time
- Refactor of the HostSharePCI to simplify argument passing.
The hierarchy created is as follows (example 2 NUMA nodes):
+-----------+
| |
| PCIe Root |
| |
+--+-----+--+
| ... |
+-----+ +-----+
NUMA node 0 | | NUMA node 1
+----+-----+ +----+-----+
| expander | | expander |
| bus | | bus |
+----+-----+ +----+-----+
| |
| ...
|
+------> port (0)
| ...
+------> port (3)
|
| +-----------+
| up | PCIe +-----> downstream port (0)
+----+ Switch | ...
| +-----> downstream port (8)
+-----------+
Two new attributes are supported for the Host/Cluster template:
- PCI_FILTER. Format is vendor:device:class with "*" as wild card. Example PCI_FILTER="10de:*,386e:0edc"
- PCI_SHORT_ADDRESS. Comma separated list of short PCI address. Example
PCI_SHORT_ADDRESS="c1:00.2,c1:00.3,01:00.5"
When any of these attributes are defined any PCI device returned by the monitor system is filtered using this attributes. If the device does not match PCI_FILTER and PCI_SHORT_ADDRESS it is discarded.
Signed-off-by: Valentyn Bohdan <vbohdan@opennebula.io>
Co-authored-by: Ruben S. Montero <rsmontero@opennebula.org>
This commit includes metrics forecasting for VMs and Hosts. It provides
basic algorithms for linear regression.
* VM prediction includes the following metrics:
- cpu
- memory
- netrx
- nettx
- diskrdbytes
- diskwrbytes
- diskrdiops
- diskwriops
* Host prediction includes:
- usedcpu
- freecpu
- usedmemory
- freememory
- netrx
- nettx
The prediction framework uses a time series DB per VM (stored in the VM
folder) to compute the forecast.
co-authored-by: Marco Mancini <mmancini@opennebula.io>
co-authored-by: Jakub Walczak <jwalczak@opennebula.io>
co-authored-by: dmingolla <dmingolla@opennebula.io>
This commits introduces a new framework for OpenNebula, it is based on
standard the OpenNebula middleware access driver (mad) model. Benefits
compared to the current implementation:
* Less overhead on API server and streamlined communications
* Ease integration with third party schedulers
* New "optimize" action to balance the workload of a cluster
* Embedded Plan Manager to apply optimization and deployment plans with
custom parameters for the scheduling window
* Custom optimization policies per cluster
This commits introduces 2 schedulers:
* Rank scheduler based on the match-making algorithm present in 6.x
series
* OpenNebula DRS an optimization algorithm based on an ILP solver for
placement and cluster optimization
Also new API calls has been added for onecluster as well as the CLI
updates to expose them.
co-authored-by: MarioRobres <mrobres@opennebula.io>
co-authored-by: Pavel Czerny <pczerny@opennebula.io>
co-authored-by: Mirko Stojiljkovic <mstojiljkovic@opennebula.io>
co-authored-by: Valentyn Bohdan <vbohdan@opennebula.io>
This feature auto mounts NFS volumes for datastores. The behavior can be controlled with the following template attributes:
- ANFS_ENABLE: YES/NO to enable the automatic NFS management
- ANFS_HOST: hostname/IP of the NFS server
- ANFS_PATH: of the NFS share for this datastore
- ANFS_OPTS: Comma separated options (fstab-like) used for mounting the NFS shares
The shared drivers also performs a "garbage-collector" like to unmount NFS volumes on the hypervisors when the datastores are deleted. This operation is performed in a "lazy" way, when VMs are deployed on the hypervisor.
* oneimage create: mount/umount DS on demand
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Move sh mount/umount logic to libfs.sh
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* oneimage delete: mount/umount DS on demand
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Limit NFS mount/umount from SH code to 30s
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* oneimage clone: mount/umount DS on demand
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* onevm deploy: mount DS on demand
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Temporal mount only when needed
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* onevm deploy: mount DS on demand (persisten)
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Linter
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* onevm deploy: umount unneeded DS
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Allow mount/umount/sed from sudoers
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Temporal mount: use NFS_AUTO_OPTS
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Do AutoNFS cleanup even during non-related ops
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Final refactor and improvements
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* ro values for Datastore object
* Wording for AutoNFS RO attributes
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* NFS_AUTO_ENABLE: change 'true' to 'yes'
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* sudoers: merge everythng into ONE_NFS
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Quote NFS_AUTO_OPTS
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Fix difference between sudoers specs
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Touch .automounted in AutoNFS mounts
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
* Update sudoers.rb
* Update rm
* Quote bash variable expansion
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
---------
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
Co-authored-by: Ruben S. Montero <rsmontero@opennebula.org>
This feature adds support for VM quotas at cluster level. CLUSTER_IDS
can be set for a VM quota so the admin can limit the number of resources (e.g. VMs) a
user/group can create in a given cluster.
This commit also fixes:
- B #2226
- B #6823
Co-authored-by: Pavel Czerny <pczerny@opennebula.io>
This commit recovers the "enforce" argument for the resize operation onevm resize request. In this case the flag is only applied to the capacity, the NUMA topology is always evaluated in the resize operation.
Signed-off-by: Valentyn Bohdan <vbohdan@opennebula.io>
* Remove vm_import table from DB
* Remove imported vms actions
* Fix fsck for image and network
* onedb fsck fix running_vms only for non-backup images
* M #-: Replace raw pointers with references/smart pointers.
- Change PoolSQL::allocate method's signature to accept reference instead of pointer and fix all method calls.
- Remove some of the error-handling goto, to be able initialize some variables during declaration.
Signed-off-by: Valentyn Bohdan <vbohdan@opennebula.io>
Implementation overview:
- Incremental points are saved as dedicated rbd snapshots under the
"one_backup_<increment_id>" namespace. This snapshots are used to
generate delta files in rbdiff format.
- The rbdiff formats are stored in the backup server to restore the rbd volumes.
- The restore process is performed directly on the Ceph cluster
importing the base image (first full backup in the chain, rbd import)
and then applying the increments (rbd import-diff) up to the target
increment.
- Two new pseudo-protocols has been implemented to adopt the restore
pattern above (restic+rbd, rsync+rbd). This protocols bundle of the
rbdiff files in a tarball for transfer from the backup server. Note:
reconstruct process uses the Ceph BRIDGE_LIST and not the backup
server (as opposed to qcow2 backups)
Other bug fixes
- This commit also fixes#6741, resetting the backup chain after a
restore
- The original ceph drivers do not receive the full action information,
this now has been fixed by including VM information in the STDIN string sent
to the driver.
Compatibility note.
- backup actions should return now the backup format used raw, rbd, ...
If not provided oned (6.10.x) will use raw as a default to accommodate any
third party driver implementation. It is recommended to include this
third argument.
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
Co-authored-by: Guillermo Ramos <gramos@opennebula.io>
Jan Orel