Table of Contents

Proxmox Installation

This guide outlines how to install Proxmox on a new cluster node. The node will be named pve{n}.

Pre-Installation

Reset the BIOS

Remove the CMOS battery, wait 30 sec and put it back. On some models, it is necessary to short circuit the two pins where it is inserted the battery or put a jumper elsewhere on the motherboard (all to power supply detached). Please consult your specific motherboard manual for instructions.

Bios settings

The location of these options in the bios varies from machine to machine. For the computer

  • Settings > Restore defaults
  • Settings > Advanced > Power Management Set UP > Restore After AC Power Loss = (Power On)
  • Boot mode = UEFI
  • Exit > Save and Exit

Installing Proxmox

Follow the proxmox installation setup

General configuration:

  • Keyboard Layout: U.S. English
  • Mail: admstaff at cs.unibo.it
  • Password: (the same as other PVE)

Network configuration:

  • FQDN: pve{n}.students.cs.unibo.it
  • IP: 130.136.3.{2+n}/24
  • Gateway: 130.136.3.254
  • DNS: 130.136.1.110 1.1.1.1

Post installation

  1. Log in as root

  2. To check the internet connection you can ping other nodes of the clusters and external machines (for example 1.1.1.1).

  3. Check if the web interface is working from your machine To connect you need to do an SSH forward of port 8006 on the PVE. You can use the command:

    $ ssh -L 8006:localhost:8006 root@130.136.3.{2+n}

    On your machine, navigating to https://localhost:8006 you should see the Proxmox login.

  4. You need to delete the paid Proxmox mirror and the message it reminds you of to pay the subscription every login. We can use proxmox-nag. Follow the guide on their repository for how to install it. One should suffice:

    $ bash <(curl -s https://raw.githubusercontent.com/foundObjects/pve-nag-buster/master/install.sh)

  5. Do an initial system update with:

    $ apt update
$ sudo apt upgrade

Insert the new node into the cluster

To insert the new pve{n} into the cluster you must run the command (on the new node):

$ pvecm add <ip> --use_ssh 1

Where <ip> is the address of a node already present in the cluster

NOTE: Half of the nodes in the cluster must be online for the addition to proceed.

ProxmoxManagement

ID Policy

Virtual Machines

To simplify we made a web page to simply calculate the vm id: https://vm.students.cs.unibo.it

Virtual machines on proxmox are identified using 9-digit numeric IDs.

The ids for new machines must respect this scheme (starting from the left):

Digit position Description Notes
0 VM critical level The value 10 takes precedence over the value 90. Refer to the table below to set the level correctly
1 VM state Refer to the table below to set the state correctly
2 Network
3 VM category Refer to the table below to set the category correctly
4-5 LDAP uidNumber of the user owning the vm Refer to the ldap wiki
6-7-8 Last octet of the machine’s IP address Refer to netbox.students.cs.unibo.it to choose a free ip
Critical Level
Value Description
1 Reserved to VMs for admstaff core services
2 Reserved for csunibo projects hosted on admstaff servers
3 Reserved for students projects hosted on admstaff servers
4 Kubernetes Cluster
7 Other
9 Reserved for templates. See more below

If you don’t know what to choose, level 70 “Other” is probably for you.

State
Value Description
0 Public IPv4
1 Private IPv4
Network
Value Description
0 130.136.3.0/24
1 130.136.201.0/24
Categories
Category Value
Networking 1
Security 2
General Purpose 5
Analytics 6
Testing 8
Templates 9

A possible ID could therefore be: 100102055 or a networking VM created by the user fil with public IP 130.136.3.55

Templates Specific
Digit position Description Notes
0 VM critical level 9 as defined in the table above
1-2 <unused>
3 VM category 9 as defined in the table above
4 Is container template? 1 true, 0 false
5-6 Operating System Refer to the table below to set the category correctly
7-8 Incremental number In case same OS template have different purpose.
Operating System
Value Description
00 Debian 11 Bullseye
01 Debian 12 Bookworm
10 ArchLinux
20 Fedora 38

Create Virtual Machine

To create a virtual machine you need to clone one of the present templates identified by the id 900x (currently in pve1). It is necessary to change the IP of the machine once cloned.

  • For VMs check the Cloud-Init section of the vm edit the IP Config section.
  • For Containers check Network section of the container

To choose a valid and not already used IP, use netbox.students.cs.unibo.it

Failed Migration

If you see something like this error:

task started by HA resource agent
2024-03-25 14:27:56 # /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=pve1' root@130.136.3.3 /bin/true
2024-03-25 14:27:56 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2024-03-25 14:27:56 @    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
2024-03-25 14:27:56 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2024-03-25 14:27:56 IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
2024-03-25 14:27:56 Someone could be eavesdropping on you right now (man-in-the-middle attack)!
2024-03-25 14:27:56 It is also possible that a host key has just been changed.
2024-03-25 14:27:56 The fingerprint for the RSA key sent by the remote host is
2024-03-25 14:27:56 SHA256:T5aK223sjFaD6zJhs2SbqAxU4pNm5R/y1potIgzBBd4.
2024-03-25 14:27:56 Please contact your system administrator.
2024-03-25 14:27:56 Add correct host key in /root/.ssh/known_hosts to get rid of this message.
2024-03-25 14:27:56 Offending RSA key in /etc/ssh/ssh_known_hosts:1
2024-03-25 14:27:56   remove with:
2024-03-25 14:27:56   ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "pve1"
2024-03-25 14:27:56 RSA host key for pve1 has changed and you have requested strict checking.
2024-03-25 14:27:56 Host key verification failed.
2024-03-25 14:27:56 ERROR: migration aborted (duration 00:00:00): Can't connect to destination address using public key
TASK ERROR: migration aborted

From A [origin] to B [destination] you need to do:

  1. On the A: ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "pveB"
  2. On the A: ssh -o "HostKeyAlias=pveB" root@<IP_pveB>