Proxmox
Links
- Proxmox VE Administration Guide
- https://pve.proxmox.com/wiki Wiki
- Monitoring Proxmox with Zabbix
- Proxmox Backup Server
- Backup and Restore
Commands
qm Qemu Manager
pvesm Storage manager
pveperf
pvecm
Documentation
Directory structure
/etc/pve
/var/lib/vz
/var/lib/vz/template/iso
Proxmox cluster
https://pve.proxmox.com/wiki/Cluster_Manager
Cluster manager
pvecm status pvecm nodes
HA status
ha-manager status
Monitoring proxmox with zabbix
https://github.com/takala-jp/zabbix-proxmox
FAQ
Proxmox API call failed: Couldn't authenticate user: zabbix@pve
Funky characters in password string?
Failed to establish a new connection: [Errno -2] Name or service not known
Just that, check your DNS
FAQ
Clustering
Show cluster status
pvecm status
It seems relatively safe to restart corosync
https://blog.jenningsga.com/proxmox-keeping-quorum-with-qdevices/
https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_corosync_external_vote_support
corosync-qdevice[11695]: Can't read quorum.device.model cmap key
On the qdevice node
Check corosync-cmapctl ?
also see https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_corosync_external_vote_support
"Quorum: 2 Activity blocked"
In my case this meant boot up second real node first
On working node:
corosync-cmapctl | grep quorum.device
quorum.device.model (str) = net
quorum.device.net.algorithm (str) = ffsplit
quorum.device.net.host (str) = 192.168.178.2
quorum.device.net.tls (str) = on
quorum.device.votes (u32) = 1
https://bugs.launchpad.net/ubuntu/+source/corosync-qdevice/+bug/1733889
Shutting down a node
Should just work. Takes guests down with it when they're not in HA
Cores, sockets and vCPUs
vCPUs is what the vm uses, equals sockets*cores
Migrating
VM is locked (create) (500)
Not always clear why, but try
qm unlock 111
Replication
missing replicate feature on volume 'local-lvm
looks like replication of lvm isn't supported
Check if qemu agent is running
See if IP is shown under Summary, also
qm agent 105 ping
Move to unused disk
If you moved disk, and decided to move back to the old one:
- detach current disk
- select the unused disk
- click Add
Stop all proxmox services
systemctl stop pve-cluster systemctl stop pvedaemon systemctl stop pveproxy systemctl stop pvestatd
Storage (xx) not available on selected target
probably some storage mounted only on one node, so not clustered
switch to community repository
cat /etc/apt/sources.list.d/pve-enterprise.list #deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise
echo "deb http://download.proxmox.com/debian/pve buster pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list
apt update
W: (pve-apt-hook) You are attempting to remove the meta-package 'proxmox-ve'!
cehck sources.list :)
Backups
PBS GC & Prune scheduling
https://pbs.proxmox.com/docs/prune-simulator/
qmp command 'backup' failed - got timeout
https://github.com/proxmox/qemu/blob/master/qmp-commands.hx
proxmox-backup-client
export PBS_REPOSITORY="backup@pbs@pbs-server:backuprepo"
proxmox-backup-client snapshot list
proxmox-backup-client prune vm/101 --dry-run --keep-daily 7 --keep-weekly 3
proxmox-backup-client garbage-collect
dirty-bitmap status: existing bitmap was invalid and has been cleared
unexpected property 'prune-backups' (500)
When for example Add: iSCSI Uncheck "Keep all backups" in "Backup retention"
FAILED 00:00:02 unable to activate storage
TODO
VM 101 Backup failed: VM is locked (snapshot)
Check if there's no snapshot running (how?)
qm unlock 101
can't acquire lock '/var/run/vzdump.lock' - got timeout
Check if vzdump is running, otherwise kill it (cluster?)
VM 101 Backup failed::= VM is locked (snapshot-delete)
Check /etc/pve/qemu-server/101.conf for 'snapstate'
If that says 'delete' for a snapshot try deleting the snapshot:
qm delsnapshot 101 snapname
If that throws like Failed to find logical volume 'pve/snap_vm-101-disk-0_saving'
qm delsnapshot 101 snapname --force
to get it out of 101.conf
Error: VM quit/powerdown failed - got timeout
qm stop VMID
if that complains about lock, remove the lock and try again
a used vhost backend has no free memory slots left
You have not turned on protection against thin pools running out of space.
serial console from command line
qm terminal <id}
enable serial console in guest
looks like this is not needed:
systemctl enable serial-getty@ttyS0.service
in /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0 console=tty0"
ttyS0 is for qm terminal, tty0 is for the "console" buttion in UI
- debian based
update-grub
- redhat based
grub2-mkconfig --output=/boot/grub2/grub.cfg
add
serial0: socket
to /etc/pve/qemu-server/[vmid].conf and restart
agetty: /dev/ttyS0: not a device
systemctl status useless again, means the serial bit is missing from <vmid>.conf
TASK ERROR: command 'apt-get update' failed: exit code 100
subtle way of telling you to get subscription of at least change the sources list
Import vmdk to lvm
https://pve.proxmox.com/wiki/Qemu/KVM_Virtual_Machines#_importing_virtual_machines_and_disk_images
Can't apply changes to memory allocation
Maybe try enabling NUMA in CPU settings
Adding hardware shows orange
something is not supported (Options->Hotplug)
"Connection error 401: no ticket"
Login session expired?
can't lock file '/var/lock/qemu-server/lock-102.conf' - got timeout (500)
Maybe someone else has/had webui open
== TASK ERROR: Can't use string ("keep-all=0,keep-last=3") as a HASH ref while "strict refs" in use at /usr/share/perl5/PVE/VZDump.pm line 502. ==
Classic, means incorrect syntax in your /etc/pve/storage.cfg
The current guest configuration does not support taking new snapshots
You're using raw instead of qcow2. Convert: Hardware->Hard disk "Move Disk"
WARNING: Device /dev/dm-21 not initialized in udev database even after waiting 10000000 microseconds.
Until someone fixes it:
udevadm trigger
Also look for link to dm-21 in /dev/disk/by-id
"connection error - server offline?"
Try reconnection browser
Find vm name by id
grep name: /etc/pve/nodes/*/qemu-server/101.conf |head -n 1 | cut -d ' ' -f 2
Started Proxmox VE replication runner.
??
Find ID by name
grep -l "name: <NAME>" /etc/pve/nodes/*/qemu-server/*conf| sed 's/^.*\/\([0-9]*\)\.conf/\1/g'
Can't migrate VM with local CD/DVD
Remove the CD :)
Memory allocated to VMs
qm list|grep -v VM| awk '{ sum+=$4 } END { print sum }'
Ceph
Got timeout(500)
Check
pveceph status
Possibly problem with ceph mgr
