Proxmox
Table of Contents
Proxmox Installation
This guide outlines how to install Proxmox on a new cluster node. The node will be named pve{n}.
Pre-Installation
Reset the BIOS
Remove the CMOS battery, wait 30 sec and put it back. On some models, it is necessary to short circuit the two pins where it is inserted the battery or put a jumper elsewhere on the motherboard (all to power supply detached). Please consult your specific motherboard manual for instructions.
Bios settings
The location of these options in the bios varies from machine to machine. For the computer
- Settings > Restore defaults
- Settings > Advanced > Power Management Set UP > Restore After AC Power Loss = (Power On)
- Boot mode = UEFI
- Exit > Save and Exit
Installing Proxmox
Follow the proxmox installation setup
General configuration:
- Keyboard Layout: U.S. English
- Mail: admstaff at cs.unibo.it
- Password: (the same as other PVE)
Network configuration:
- FQDN:
pve{n}.students.cs.unibo.it - IP:
130.136.3.{2+n}/24 - Gateway:
130.136.3.254 - DNS:
130.136.1.110 1.1.1.1
Post installation
-
Log in as root
-
To check the internet connection you can ping other nodes of the clusters and external machines (for example
1.1.1.1). -
Check if the web interface is working from your machine To connect you need to do an SSH forward of port 8006 on the PVE. You can use the command:
$ ssh -L 8006:localhost:8006 root@130.136.3.{2+n}On your machine, navigating to https://localhost:8006 you should see the Proxmox login.
-
You need to delete the paid Proxmox mirror and the message it reminds you of to pay the subscription every login. We can use proxmox-nag. Follow the guide on their repository for how to install it. One should suffice:
$ bash <(curl -s https://raw.githubusercontent.com/foundObjects/pve-nag-buster/master/install.sh) -
Do an initial system update with:
$ apt update $ sudo apt upgrade
Insert the new node into the cluster
To insert the new pve{n} into the cluster you must run the command (on the new node):
$ pvecm add <ip> --use_ssh 1
Where <ip> is the address of a node already present in the cluster
NOTE: Half of the nodes in the cluster must be online for the addition to proceed.
ProxmoxManagement
ID Policy
Virtual Machines
To simplify we made a web page to simply calculate the vm id: https://vm.students.cs.unibo.it
Virtual machines on proxmox are identified using 9-digit numeric IDs.
The ids for new machines must respect this scheme (starting from the left):
| Digit position | Description | Notes |
|---|---|---|
| 0 | VM critical level | The value 10 takes precedence over the value 90. Refer to the table below to set the level correctly |
| 1 | VM state | Refer to the table below to set the state correctly |
| 2 | Network | |
| 3 | VM category | Refer to the table below to set the category correctly |
| 4-5 | LDAP uidNumber of the user owning the vm | Refer to the ldap wiki |
| 6-7-8 | Last octet of the machine’s IP address | Refer to netbox.students.cs.unibo.it to choose a free ip |
Critical Level
| Value | Description |
|---|---|
| 1 | Reserved to VMs for admstaff core services |
| 2 | Reserved for csunibo projects hosted on admstaff servers |
| 3 | Reserved for students projects hosted on admstaff servers |
| 4 | Kubernetes Cluster |
| 7 | Other |
| 9 | Reserved for templates. See more below |
If you don’t know what to choose, level 70 “Other” is probably for you.
State
| Value | Description |
|---|---|
| 0 | Public IPv4 |
| 1 | Private IPv4 |
Network
| Value | Description |
|---|---|
| 0 | 130.136.3.0/24 |
| 1 | 130.136.201.0/24 |
Categories
| Category | Value |
|---|---|
| Networking | 1 |
| Security | 2 |
| General Purpose | 5 |
| Analytics | 6 |
| Testing | 8 |
| Templates | 9 |
A possible ID could therefore be: 100102055 or a networking VM created by the user fil with public IP 130.136.3.55
Templates Specific
| Digit position | Description | Notes |
|---|---|---|
| 0 | VM critical level | 9 as defined in the table above |
| 1-2 | <unused> | |
| 3 | VM category | 9 as defined in the table above |
| 4 | Is container template? | 1 true, 0 false |
| 5-6 | Operating System | Refer to the table below to set the category correctly |
| 7-8 | Incremental number | In case same OS template have different purpose. |
Operating System
| Value | Description |
|---|---|
| 00 | Debian 11 Bullseye |
| 01 | Debian 12 Bookworm |
| 10 | ArchLinux |
| 20 | Fedora 38 |
| 30 | WIndows XP Pro |
| 33 | WIndows 7 Pro |
| 35 | WIndows 10 Pro |
Create Virtual Machine
To create a virtual machine you need to clone one of the present templates identified by the id 900x (currently in pve1).
Please use the ID convention written above.
It is necessary to change the IP of the machine once cloned.
- For VMs check the
Cloud-Initsection of the vm edit theIP Configsection. - For Containers check
Networksection of the container
To choose a valid and not already used IP, use NetBox
After the VM is up please update NetBox to reflect all the changes
Failed Migration
If you see something like this error:
task started by HA resource agent
2024-03-25 14:27:56 # /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=pve1' root@130.136.3.3 /bin/true
2024-03-25 14:27:56 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2024-03-25 14:27:56 @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
2024-03-25 14:27:56 @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
2024-03-25 14:27:56 IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
2024-03-25 14:27:56 Someone could be eavesdropping on you right now (man-in-the-middle attack)!
2024-03-25 14:27:56 It is also possible that a host key has just been changed.
2024-03-25 14:27:56 The fingerprint for the RSA key sent by the remote host is
2024-03-25 14:27:56 SHA256:T5aK223sjFaD6zJhs2SbqAxU4pNm5R/y1potIgzBBd4.
2024-03-25 14:27:56 Please contact your system administrator.
2024-03-25 14:27:56 Add correct host key in /root/.ssh/known_hosts to get rid of this message.
2024-03-25 14:27:56 Offending RSA key in /etc/ssh/ssh_known_hosts:1
2024-03-25 14:27:56 remove with:
2024-03-25 14:27:56 ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "pve1"
2024-03-25 14:27:56 RSA host key for pve1 has changed and you have requested strict checking.
2024-03-25 14:27:56 Host key verification failed.
2024-03-25 14:27:56 ERROR: migration aborted (duration 00:00:00): Can't connect to destination address using public key
TASK ERROR: migration aborted
From A [origin] to B [destination] you need to do:
- On the A:
ssh-keygen -f "/etc/ssh/ssh_known_hosts" -R "pveB" - On the A:
ssh -o "HostKeyAlias=pveB" root@<IP_pveB>