Proxmox: Difference between revisions

From DWIKI
mNo edit summary
mNo edit summary
(47 intermediate revisions by the same user not shown)
Line 1: Line 1:


 
 


= Links =
= Links =
Line 5: Line 8:
*[https://pve.proxmox.com/pve-docs/pve-admin-guide.html Proxmox VE Administration Guide]  
*[https://pve.proxmox.com/pve-docs/pve-admin-guide.html Proxmox VE Administration Guide]  
*[https://pve.proxmox.com/wiki https://pve.proxmox.com/wiki Wiki]  
*[https://pve.proxmox.com/wiki https://pve.proxmox.com/wiki Wiki]  
*[https://www.zabbix.com/integrations/proxmox Monitoring Proxmox with Zabbix]
*[https://www.proxmox.com/en/proxmox-backup-server Proxmox Backup Server]
*[https://pve.proxmox.com/wiki/Backup_and_Restore Backup and Restore]
 


 
 


=Documentation=
= Commands =
 
== qm Qemu Manager ==
 
== pvesm Storage manager ==
 
= Documentation =
 
== Directory structure ==
== Directory structure ==
===/etc/pve===


=== /etc/pve ===


===/var/lib/vz===
=== /var/lib/vz ===


/var/lib/vz/template/iso
/var/lib/vz/template/iso
Line 19: Line 34:




== Proxmox cluster ==
=== Cluster manager ===
pvecm status
pvecm nodes
 
=== HA status ===
ha-manager status
= Monitoring proxmox with zabbix =
[https://github.com/takala-jp/zabbix-proxmox https://github.com/takala-jp/zabbix-proxmox]
== FAQ ==
=== Proxmox API call failed: Couldn't authenticate user: zabbix@pve ===
Funky characters in password string?
 
=== Failed to establish a new connection: [Errno -2] Name or service not known ===
Just that, check your DNS


= FAQ =
= FAQ =
== Clustering ==
=== Show cluster status ===
pvecm status
 
=== [https://blog.jenningsga.com/proxmox-keeping-quorum-with-qdevices/ https://blog.jenningsga.com/proxmox-keeping-quorum-with-qdevices/] ===
[https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_corosync_external_vote_support https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_corosync_external_vote_support]
=== corosync-qdevice[11695]: Can't read quorum.device.model cmap key ===
On the qdevice node
Check corosync-cmapctl ?
also see [https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_corosync_external_vote_support https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_corosync_external_vote_support]
=== "Quorum: 2 Activity blocked" ===
In my case this meant boot up second real node first
 
 
On working node:
corosync-cmapctl | grep quorum.device<br/> quorum.device.model (str) = net<br/> quorum.device.net.algorithm (str) = ffsplit<br/> quorum.device.net.host (str) = 192.168.178.2<br/> quorum.device.net.tls (str) = on<br/> quorum.device.votes (u32) = 1<br/> <br/> [https://bugs.launchpad.net/ubuntu/+source/corosync-qdevice/+bug/1733889 https://bugs.launchpad.net/ubuntu/+source/corosync-qdevice/+bug/1733889]
&nbsp;
&nbsp;
&nbsp;
&nbsp;
=== Shutting down a node ===
Should just work. Takes guests down with it when they're not in HA
== Cores, sockets and vCPUs ==
vCPUs is what the vm uses, equals sockets*cores
&nbsp;
&nbsp;
== Migrating ==
=== VM is locked (create) (500) ===
Not always clear why, but try
qm unlock 111
&nbsp;
&nbsp;
&nbsp;
== Replication ==
=== missing replicate feature on volume 'local-lvm ===
looks like replication of lvm isn't supported
== Check if qemu agent is running ==
See if IP is shown under Summary, also
qm agent 105 ping
== Move to unused disk ==
If you moved disk, and decided to move back to the old one:
*detach current disk
*select the unused disk
*click Add
== Stop all proxmox services ==
systemctl stop pve-cluster systemctl stop pvedaemon systemctl stop pveproxy systemctl stop pvestatd
== Storage (xx) not available on selected target ==
probably some storage mounted only on one node, so not clustered
&nbsp;
== switch to community repository ==
cat /etc/apt/sources.list.d/pve-enterprise.list
#deb [https://enterprise.proxmox.com/debian/pve https://enterprise.proxmox.com/debian/pve] buster pve-enterprise
echo "deb [http://download.proxmox.com/debian/pve http://download.proxmox.com/debian/pve] buster pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list
apt update
== W: (pve-apt-hook) You are attempting to remove the meta-package 'proxmox-ve'! ==
cehck sources.list&nbsp;:)
&nbsp;
&nbsp;
&nbsp;
&nbsp;
&nbsp;
&nbsp;
== Backups ==
=== PBS GC & Prune scheduling [https://pbs.proxmox.com/docs/prune-simulator/ https://pbs.proxmox.com/docs/prune-simulator/]
=== proxmox-backup-client ===
export PBS_REPOSITORY="backup@pbs@pbs-server:backuprepo"
proxmox-backup-client snapshot list
proxmox-backup-client prune vm/101 --dry-run --keep-daily 7 --keep-weekly 3
proxmox-backup-client garbage-collect
=== dirty-bitmap status: existing bitmap was invalid and has been cleared ===
*[https://qemu-project.gitlab.io/qemu/interop/bitmaps.html https://qemu-project.gitlab.io/qemu/interop/bitmaps.html]
&nbsp;
=== unexpected property 'prune-backups' (500) ===
When for example Add: iSCSI Uncheck "Keep all backups" in "Backup retention"
&nbsp;
=== FAILED 00:00:02 unable to activate storage ===
TODO
&nbsp;
=== VM 101 Backup failed: VM is locked (snapshot) ===
Check if there's no snapshot running (how?)
qm unlock 101
=== can't acquire lock '/var/run/vzdump.lock' - got timeout ===
Check if vzdump is running, otherwise kill it (cluster?)
&nbsp;
=== VM 101 Backup failed::= VM is locked (snapshot-delete) ===
Check /etc/pve/qemu-server/101.conf for 'snapstate'
If that says 'delete' for a snapshot try deleting the snapshot:
qm delsnapshot 101 snapname
If that throws like Failed to find logical volume 'pve/snap_vm-101-disk-0_saving'
  qm delsnapshot 101 snapname --force
to get it out of 101.conf


== Error: VM quit/powerdown failed - got timeout ==
== Error: VM quit/powerdown failed - got timeout ==


  qm stop VMID
  qm stop VMID
if that complains about lock, remove the lock and try again


&nbsp;
&nbsp;


&nbsp;
== a used vhost backend has no free memory slots left ==
 
== You have not turned on protection against thin pools running out of space. ==


== serial console from command line ==
== serial console from command line ==
Line 81: Line 307:


Maybe try enabling NUMA in CPU settings
Maybe try enabling NUMA in CPU settings
&nbsp;


&nbsp;
&nbsp;
Line 86: Line 314:
== Adding hardware shows orange ==
== Adding hardware shows orange ==


something is not supported
something is not supported (Options->Hotplug)
 
== "Connection error 401: no ticket" ==


=="Connection error 401: no ticket"==
Login session expired?
Login session expired?
== can't lock file '/var/lock/qemu-server/lock-102.conf' - got timeout (500) ==
Maybe someone else has/had webui open
== TASK ERROR: Can't use string ("keep-all=0,keep-last=3") as a HASH ref while "strict refs" in use at /usr/share/perl5/PVE/VZDump.pm line 502. ==
Classic, means incorrect syntax in your /etc/pve/storage.cfg
&nbsp;
== The current guest configuration does not support taking new snapshots ==
You're using raw instead of qcow2. Convert: Hardware->Hard disk "Move Disk"
&nbsp;
&nbsp;
== WARNING: Device /dev/dm-21 not initialized in udev database even after waiting 10000000 microseconds. ==
Until someone fixes it:
udevadm trigger
     
Also look for link to dm-21 in /dev/disk/by-id
== "connection error - server offline?" ==
Try reconnection browser
&nbsp;
== Find vm name by id ==
grep name /etc/pve/nodes/*/qemu-server/101.conf |head -n 1
     
&nbsp;
== Started Proxmox VE replication runner. ==
??
== Find ID by name ==
grep -l "name: <NAME>"  /etc/pve/nodes/*/qemu-server/*conf| sed 's/^.*\/\([0-9]*\)\.conf/\1/g'
     
&nbsp;
== Can't migrate VM with local CD/DVD ==
Remove the CD&nbsp;:)
&nbsp;
== Memory allocated to VMs ==
qm list|grep -v VM| awk '{ sum+=$4 } END { print sum }'

Revision as of 10:31, 8 December 2021

 

 

Links

 

 

Commands

qm Qemu Manager

pvesm Storage manager

Documentation

Directory structure

/etc/pve

/var/lib/vz

/var/lib/vz/template/iso


Proxmox cluster

Cluster manager

pvecm status
pvecm nodes

 

HA status

ha-manager status

Monitoring proxmox with zabbix

https://github.com/takala-jp/zabbix-proxmox

FAQ

Proxmox API call failed: Couldn't authenticate user: zabbix@pve

Funky characters in password string?

 

Failed to establish a new connection: [Errno -2] Name or service not known

Just that, check your DNS

FAQ

Clustering

Show cluster status

pvecm status

 

https://blog.jenningsga.com/proxmox-keeping-quorum-with-qdevices/

https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_corosync_external_vote_support

corosync-qdevice[11695]: Can't read quorum.device.model cmap key

On the qdevice node

Check corosync-cmapctl ?

also see https://pve.proxmox.com/pve-docs/chapter-pvecm.html#_corosync_external_vote_support

"Quorum: 2 Activity blocked"

In my case this meant boot up second real node first

 

 

On working node:

corosync-cmapctl | grep quorum.device
quorum.device.model (str) = net
quorum.device.net.algorithm (str) = ffsplit
quorum.device.net.host (str) = 192.168.178.2
quorum.device.net.tls (str) = on
quorum.device.votes (u32) = 1

https://bugs.launchpad.net/ubuntu/+source/corosync-qdevice/+bug/1733889

 

 

 

 

Shutting down a node

Should just work. Takes guests down with it when they're not in HA

Cores, sockets and vCPUs

vCPUs is what the vm uses, equals sockets*cores

 

 

Migrating

VM is locked (create) (500)

Not always clear why, but try

qm unlock 111

 

 

 

Replication

missing replicate feature on volume 'local-lvm

looks like replication of lvm isn't supported

Check if qemu agent is running

See if IP is shown under Summary, also

qm agent 105 ping

Move to unused disk

If you moved disk, and decided to move back to the old one:

  • detach current disk
  • select the unused disk
  • click Add

Stop all proxmox services

systemctl stop pve-cluster systemctl stop pvedaemon systemctl stop pveproxy systemctl stop pvestatd

Storage (xx) not available on selected target

probably some storage mounted only on one node, so not clustered

 

switch to community repository

cat /etc/apt/sources.list.d/pve-enterprise.list 
#deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise
echo "deb http://download.proxmox.com/debian/pve buster pve-no-subscription" > /etc/apt/sources.list.d/pve-no-subscription.list 
apt update

W: (pve-apt-hook) You are attempting to remove the meta-package 'proxmox-ve'!

cehck sources.list :)

 

 

 

 

 

 

Backups

=== PBS GC & Prune scheduling https://pbs.proxmox.com/docs/prune-simulator/

proxmox-backup-client

export PBS_REPOSITORY="backup@pbs@pbs-server:backuprepo"
proxmox-backup-client snapshot list
proxmox-backup-client prune vm/101 --dry-run --keep-daily 7 --keep-weekly 3
proxmox-backup-client garbage-collect

dirty-bitmap status: existing bitmap was invalid and has been cleared

 

unexpected property 'prune-backups' (500)

When for example Add: iSCSI Uncheck "Keep all backups" in "Backup retention"

 

FAILED 00:00:02 unable to activate storage

TODO

 

VM 101 Backup failed: VM is locked (snapshot)

Check if there's no snapshot running (how?)

qm unlock 101

can't acquire lock '/var/run/vzdump.lock' - got timeout

Check if vzdump is running, otherwise kill it (cluster?)

 

VM 101 Backup failed::= VM is locked (snapshot-delete)

Check /etc/pve/qemu-server/101.conf for 'snapstate'

If that says 'delete' for a snapshot try deleting the snapshot:

qm delsnapshot 101 snapname

If that throws like Failed to find logical volume 'pve/snap_vm-101-disk-0_saving'

 qm delsnapshot 101 snapname --force

to get it out of 101.conf

Error: VM quit/powerdown failed - got timeout

qm stop VMID

if that complains about lock, remove the lock and try again

 

a used vhost backend has no free memory slots left

You have not turned on protection against thin pools running out of space.

serial console from command line

qm terminal <id}

enable serial console in guest

looks like this is not needed:

systemctl enable serial-getty@ttyS0.service

in /etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyS0 console=tty0"

ttyS0 is for qm terminal, tty0 is for the "console" buttion in UI

  1. debian based

update-grub

  1. redhat based

grub2-mkconfig --output=/boot/grub2/grub.cfg

 

add

serial0: socket

to /etc/pve/qemu-server/[vmid].conf and restart

 

agetty: /dev/ttyS0: not a device

systemctl status useless again, means the serial bit is missing from <vmid>.conf

TASK ERROR: command 'apt-get update' failed: exit code 100

subtle way of telling you to get subscription of at least change the sources list

Import vmdk to lvm

https://pve.proxmox.com/wiki/Qemu/KVM_Virtual_Machines#_importing_virtual_machines_and_disk_images

Can't apply changes to memory allocation

Maybe try enabling NUMA in CPU settings

 

 

Adding hardware shows orange

something is not supported (Options->Hotplug)

"Connection error 401: no ticket"

Login session expired?

can't lock file '/var/lock/qemu-server/lock-102.conf' - got timeout (500)

Maybe someone else has/had webui open

== TASK ERROR: Can't use string ("keep-all=0,keep-last=3") as a HASH ref while "strict refs" in use at /usr/share/perl5/PVE/VZDump.pm line 502. ==

Classic, means incorrect syntax in your /etc/pve/storage.cfg

 

The current guest configuration does not support taking new snapshots

You're using raw instead of qcow2. Convert: Hardware->Hard disk "Move Disk"

 

 

WARNING: Device /dev/dm-21 not initialized in udev database even after waiting 10000000 microseconds.

Until someone fixes it:

udevadm trigger
      

Also look for link to dm-21 in /dev/disk/by-id

"connection error - server offline?"

Try reconnection browser

 

Find vm name by id

grep name /etc/pve/nodes/*/qemu-server/101.conf |head -n 1
      

 

Started Proxmox VE replication runner.

??

Find ID by name

grep -l "name: <NAME>"  /etc/pve/nodes/*/qemu-server/*conf| sed 's/^.*\/\([0-9]*\)\.conf/\1/g'
      

 

Can't migrate VM with local CD/DVD

Remove the CD :)

 

Memory allocated to VMs

qm list|grep -v VM| awk '{ sum+=$4 } END { print sum }'