Implementation overview:
- Incremental points are saved as dedicated rbd snapshots under the
"one_backup_<increment_id>" namespace. This snapshots are used to
generate delta files in rbdiff format.
- The rbdiff formats are stored in the backup server to restore the rbd volumes.
- The restore process is performed directly on the Ceph cluster
importing the base image (first full backup in the chain, rbd import)
and then applying the increments (rbd import-diff) up to the target
increment.
- Two new pseudo-protocols has been implemented to adopt the restore
pattern above (restic+rbd, rsync+rbd). This protocols bundle of the
rbdiff files in a tarball for transfer from the backup server. Note:
reconstruct process uses the Ceph BRIDGE_LIST and not the backup
server (as opposed to qcow2 backups)
Other bug fixes
- This commit also fixes#6741, resetting the backup chain after a
restore
- The original ceph drivers do not receive the full action information,
this now has been fixed by including VM information in the STDIN string sent
to the driver.
Compatibility note.
- backup actions should return now the backup format used raw, rbd, ...
If not provided oned (6.10.x) will use raw as a default to accommodate any
third party driver implementation. It is recommended to include this
third argument.
Signed-off-by: Guillermo Ramos <gramos@opennebula.io>
Co-authored-by: Guillermo Ramos <gramos@opennebula.io>
(cherry picked from commit 5f7b370c52)
- oned load only last 2 history records (not the full list)
- Dump all history records only if needed in VirtualMachine::to_xml.
- Dump conforms XML schecam and removes VM template from history records.
Speed up of onevm show command:
- for small SQLite DB is for VM with 500 histories: 130 ms down to 5 ms
- for big MySQL DB VM with 687 histories: 1000 ms down to 200 ms
(cherry picked from commit e13c329db4)
* Fix Host NUMA nodes after VM migration
* Move template parsing to HostShareCapacity
+ add warning about unsafe pointer usage
(cherry picked from commit 0cde7d96a2)
* Cleanup oned in case of initialization error
* Fix monitoring after onemonitord restart. The code includes a "hook" point in case a driver is re-started so custom code can be executed. InformationManager sends the list of hosts and raft status in this case.
* B #5801: Update error msg, in case of duplicated drivers
(cherry picked from commit 7b07def90e)
* Fix body_json for history record
* Update the body_json on every VM update to keep the data synced with the xml body
* Remove few duplicated VM updates
* Refactor release vnc port
* Fix VNC port for recover --recreate in stopped state
* Improve error messages
* Update recover --recreate description
(cherry picked from commit 9ed1fba8b95c1ea5d3984f136e638b973577ad69)
The Template::remove could crash if user assign Single value to attribute, which is expected to be a Vector, e.g. SCHED_ACTION, PCI, ...
Other minor changes:
* refactor the 'set' method - no need to call private _set
* Simplify some callers of the Template::remove method
This commit implements the in-place restore of VM backups. Selected VM disks will
be replaced with the specified backup:
* A new API call has been added to the XML-RPC API (`one.vm.restore`) with
the following arguments:
- VM ID to be restored, needs to be in **poweroff** state
- IMAGE ID of the backup to restore
- INCREMENT ID, only for incremental backups, the increment to use
(defults to -1 to use the last increment available)
- DISK ID of the disk to restore (defaults to -1 to restore all VM
disks)
* Datastore drivers needs to implemente a new operation `ls`. This new
operation takes the VM, image information of the backup and datastore
information and returns the restore URL for the disks in the backup.
* This commit includes the implementation for qcow2 and ssh drivers,
ceph will be implemented in a separated PR. The new driver action is
`restore host:vm_dir vm_id img_id inc_id disk_id`
* The restore operation is performed in a new state `PROLOG_RESTORE`
rendered as `RESTORE` and `rest` in short form. State in in RSuntone.
TODO:
- Remove any existing VM snapshot (system/disk) in the VM. Note that
snapshots are not included in a backup.
- Ceph drivers
- JAVA, GO Lang API bindings
- Sunstone interface, new state and new operation. Review new state in
RSuntone.
co-authored-by: Pavel Czerny <pczerny@opennebula.io>
When a disk with snapshots is resized and a subsequent revert operation
is performed, the following actions are now done:
* DISK/SIZE is updated to reflect the active snapshot size
* Quotas are updated to account the space freed after the resize
operation
(cherry picked from commit 178142105dea30c7deffd1842d0f0ad23d7dc93b)
(cherry picked from commit d2aaf2c8823d4172b1be673f959c4c922266534f)
If q35 machine type is detected the slot of the pci device is set to 0
and the bus to pci_id + 1.
Q35 models uses pcie-root-ports to attach PCI devices. Each PCI port is
selected by the bus parameter of the PCI address and it that does not accept a
slot number greater than 0.
Example:
A VM with 2 X710 VFs is defined OpenNebula as:
PCI=[
ADDRESS="0000:44:0a:0",
BUS="44",
CLASS="0200",
DEVICE="154c",
DOMAIN="0000",
FUNCTION="0",
NUMA_NODE="0",
PCI_ID="0",
SHORT_ADDRESS="44:0a.0",
SLOT="0a",
VENDOR="8086",
VM_ADDRESS="01:00.0",
VM_BUS="0x01",
VM_DOMAIN="0x0000",
VM_FUNCTION="0",
VM_SLOT="0000" ]
PCI=[
ADDRESS="0000:44:0a:1",
BUS="44",
CLASS="0200",
DEVICE="154c",
DOMAIN="0000",
FUNCTION="1",
NUMA_NODE="0",
PCI_ID="1",
SHORT_ADDRESS="44:0a.1",
SLOT="0a",
VENDOR="8086",
VM_ADDRESS="02:00.0",
VM_BUS="0x02",
VM_DOMAIN="0x0000",
VM_FUNCTION="0",
VM_SLOT="0000" ]
Each PCI VFs is attached to different pcie-root-port, selected with the
VM_BUS parameter:
00:02.0 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
00:02.1 PCI bridge: Red Hat, Inc. QEMU PCIe Root port
The PCI topology is:
-[0000:00]-+-00.0 Intel Corporation 82G33/G31/P35/P31 Express DRAM Controller
+-01.0 Cirrus Logic GD 5446
+-02.0-[01]----00.0 Intel Corporation Ethernet Virtual Function 700 Series
+-02.1-[02]----00.0 Intel Corporation Ethernet Virtual Function 700 Series
+-02.2-[03-04]----00.0-[04]--
+-02.3-[05]----00.0 Red Hat, Inc. Virtio network device
+-02.4-[06]----00.0 Red Hat, Inc. Virtio SCSI
+-02.5-[07]----00.0 Red Hat, Inc. QEMU XHCI Host Controller
+-02.6-[08]----00.0 Red Hat, Inc. Virtio console
+-02.7-[09]----00.0 Red Hat, Inc. Virtio memory balloon
+-03.0-[0a]--
+-03.1-[0b]--
+-03.2-[0c]--
+-03.3-[0d]--
+-03.4-[0e]--
+-03.5-[0f]--
+-03.6-[10]--
+-03.7-[11]--
+-1f.0 Intel Corporation 82801IB (ICH9) LPC Interface Controller
+-1f.2 Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]
\-1f.3 Intel Corporation 82801I (ICH9 Family) SMBus Controller
* PCI (de)attach go and java api
* Remove unused VirtualMachinePCI.h
* Use vm_authorization method to check Request authorization
* Fix typos
* L #-: C++ linting + onevm
* Adds 2 new API calls one.vm.attachpci and one.vm.detachpci
* The operation is only for POWEROFF VMs
* Adds 2 new actions to register the event in the VM history pci-detach/pci-attach
* Ruby OCA bindings
* onevm cli commands
TODO:
- JAVA and GO bindings
- Sunstone implementation
(cherry picked from commit 5c4331a4dd)
- Generic quotas are defined and read from oned.conf (QUOTA_VM_ATTRIBUTE)
- Generic quotas react to the running mode (i.e. account only when the VM is
running). For each quota oned also adds a RUNNIN_* metric
- one.system.config API call has been updated so every user can read
quota configuration:
. Sensitive information is hidden (***)
. Oneadmin group has no longer access to sensitive information
through the API
- CLI has been updated to render generic quotas
- onedb fsck fixes generic quotas reading the configuration from
oned.conf
Other changes in this PR:
- Refactor Quota metrics to use std::vector instead of C array
Squashed commit of the following:
- New methods to Template and Attribute classes to render hidden
attributes as "***"
- Update Quota calls to not include unneeded quotas
co-authored-by:Ruben S. Montero <rsmontero@opennebula.org>
The new attribute does not allocate extra IPs on NICs with FLOATING_IP =
yes and FLOATING_ONLY = yes. The NIC will only containg a random MAC
address and VROUTER_IP/MAC.
This commit also checks that network mode auto is not used for NIC of a
VROUTER
(cherry picked from commit 18c7816074)
* Remove useless default values from VM quotas, adapt chown and chgrp to it
* Fix VM transition from poweroff, stopped, suspended, undeployed to done when quotas are tmp exceeded
* Fix VM transition from poweroff to undeployed
* Fix VM transition from suspended to stopped
(cherry picked from commit 1e25904148)
- New attribute VIDEO, that includes
- TYPE: Custom video device (none, vga, cirrus, virtio)
 - IOMMU: Enable IOMMU for the video device
 - ATS: Enable ATS (Address Translation Services) for the video device
 - VRAM: VRAM allocated to the video device (Megabytes)
 - RESOLUTION: Video resolution, in format like: 1280x720 or 1920x1080
- Support for onevm CLI command
- Updated VM xsd file.
* Set priority on Backup Job create
* Fix a bug when running backup jobs in sequential mode
* Change the update semantics to support replace mode
* Update Ruby, Golang and Java API accordingly
* F #6063: Adress PR comments
The connection to an external scheduler module is configured in sched.conf:
EXTERNAL_SCHEDULER = [
SERVER = "http://localhost:4567",
PROXY = "",
TIMEOUT = 10
]
The API post on '/' the list of VMs, their pre-selected list of
candidate hosts based on REQUIREMENTS along with the VM information
(CAPACITY, TEMPLATE and USER_TEMPLATE).
Example:
{
"VMS": [
{
"CAPACITY": {
"CPU": 1.5,
"DISK_SIZE": 1024,
"MEMORY": 131072
},
"HOST_IDS": [
3,
4,
5
],
"ID": 32,
"STATE": "PENDING",
"TEMPLATE": {
"AUTOMATIC_DS_REQUIREMENTS": "(\"CLUSTERS/ID\" @> 0)",
"AUTOMATIC_NIC_REQUIREMENTS": "(\"CLUSTERS/ID\" @> 0)",
"AUTOMATIC_REQUIREMENTS": "(CLUSTER_ID = 0) & !(PUBLIC_CLOUD = YES) & !(PIN_POLICY = PINNED)",
"CPU": "1.5",
"MEMORY": "128",
...
},
"USER_TEMPLATE": {}
},
{
"CAPACITY": {
"CPU": 1.5,
"DISK_SIZE": 1024,
"MEMORY": 131072
},
"HOST_IDS": [
3,
4,
5
],
"ID": 33,
"STATE": "PENDING",
"TEMPLATE": {
...
},
"USER_TEMPLATE": {}
}
]
}
The scheduler needs to respond to this post action with a simple list of
the allocation for each VM:
{
"VMS": [
{
"ID": 32,
"HOST_ID": 2
},
{
"ID": 33,
"HOST_ID": 0
}
]
}
This commits vendorize Vendorize nlohmann-json (MIT license)
* Schedule Action Manager executes only on RAFT leader in HA configurations.
* Relative time for Scheduled Action Backup Jobs returns error.
* Store Backup Job ID to VM and associated backup Image.
* VM can be only in one Backup Job at a time.
* Deny single Backup if the VM is already part of BackupJob to prevent
inconsistent backup states.
Other fixes:
- Fix crash on scheduled backup for VM in pending state
- Fix error messages for Scheduled Actions
- Remove SCHED_ACTION from Backup Job template on update
co-authored-by: Pavel Czerný <pczerny@opennebula.io>
- Add suport for delete snapshots in the middle of the chain in qcow2
(ALLOW_ORPHANS=NO). Example, snapshot 2 below:
0 <- 1 <- 2 <- 3
- It supports removing the last snapshot and active (blockpull) and
first one (snapshot 0). The first snapshot cannot be deleted in
persistent images because of the underlying link setting.
- Snapshots are deleted by blockcommit. For example to delete 2, 3 is
committed into 2:
0 <- 1 <- 2: (2+3)
A new file 3.current is created in the filesystem to look for the
actual file name supporting snapshot 3.
- qcow2/shared operations snap_delete, snap_revert and cpds
(disk-saveas) has been updated to react to the .current files
- The kvm/deploy action has been updated to resolve links in source file
attributes. This is needed so the <backingStore> of the libvirt domain
does not contain the sysmlinks that changes on the revert/create
operations.
- The kvm/migrate action also includes a minor modification to resolv
the VM disk
- Snapshots with more than 1 child (or their relatives) cannot be deleted to not
break backing chains.
- It has been found that the snap-delete operation in suspend state is
insecure as the guest caches may not be updated accordingly. The
operation is now not allowed in this state.
- The associated state has been removed from oned. It needs to be
removed in CLI/Sunstone and APIs
- The changes are for VM disk-snapshots. A similar update can be made
for images in the datastore/fs snap_delete action. This operation is
not implemented in this commit.
Note: Output of xmllint may or may not include new lines when multiple matches
of an xpath expression occurs. Parsing of xmllint output adds additional new lines
to deal with different versions.
Includes the following changes:
- xml-schema for Backup Job and Scheduled Actions
- GO, Java api
- Deprecate onevm update-chart, delete-chart
* The commands are replaced by sched-update and sched-delete
* Refactor method deprecate_command, it's still possible to run the
command
* Delete 'shutdown' and 'delete' commands deprecated years ago
* Fix --verbose option for sched-update and sched-delete
- Re-implementation of scheduled actions, now are managed and executed
by oned
- Backup Job objects, API, and CLI commands
* Add Sched actions and sg (de)attach to VM_*_OPERATION
* Add Backup cancel action
* For consistency add poweroff migrate, alias (de)attach and nic update to set_auth_ops
Ruben S. Montero