{"id":6391,"date":"2021-12-16T21:11:40","date_gmt":"2021-12-16T21:11:40","guid":{"rendered":"https:\/\/www.geekdecoder.com\/?p=6391"},"modified":"2021-12-16T21:11:40","modified_gmt":"2021-12-16T21:11:40","slug":"install-proxmox-ve-on-debian-11-bullseye","status":"publish","type":"post","link":"https:\/\/www.qbytes.cloud\/index.php\/2021\/12\/16\/install-proxmox-ve-on-debian-11-bullseye\/","title":{"rendered":"Install Proxmox VE on Debian 11 Bullseye 3 Node Cluster with Ceph"},"content":{"rendered":"

The installation of a supported Proxmox VE server should be done via bare-metal ISO installer<\/a>. In some cases it makes sense to install Proxmox VE on top of a running Debian Bullseye 64-bit, especially if you want a custom partition layout. For this How-To any official Bullseye installation medium should work.<\/p>\n

For this Demo, I am installing 3 KVM VM’s. As you can do VM nesting. Also, you can do this on Vmware or Hyper V. Here is the Architecture.<\/p>\n

<\/p>\n

3 Nodes
\nHardware: 1 CPU, 2048 RAM, 2 Drives, (10GB install OS, 100GB for Storage), 2 Network Cards
\npx-01 px-01.ipgw.io 192.168.0.50 10.10.10.1
\npx-02 px-02.ipgw.io 192.168.0.51 10.10.10.2
\npx-03 px-03.ipgw.io 192.168.0.52 10.10.10.3<\/p>\n

Install a standard Debian Bullseye. Consider the Debian installation guide, and configure a static IP. It is recommended to only install the “standard system utilities” and “SSH server” package selection, as Proxmox VE brings its own packages for QEMU and LXC. A desktop environment is not necessary.<\/p>\n

Once completed, ssh to the server. Add an \/etc\/hosts entry for your IP address. The hostname of your machine must be resolvable via \/etc\/hosts. This means that in \/etc\/hosts you need one of the following entries for your hostname. Note: This also means removing the address 127.0.1.1 that is present as default.<\/p>\n

For instance, if your IP address is 192.168.0.52, and your hostname px-03.ipgw.io, then your \/etc\/hosts file could look like:<\/p>\n

\n127.0.0.1       localhost\n192.168.0.52    px-03.ipgw.io px-03\n# The following lines are desirable for IPv6 capable hosts\n::1     localhost ip6-localhost ip6-loopback\nff02::1 ip6-allnodes\nff02::2 ip6-allrouters\n<\/pre>\n
You can test if your setup is ok using the hostname command:<\/p>\n
\n# hostname --ip-address\n192.168.0.52\n<\/pre>\n
Install Proxmox VE
\nAdapt your sources.list<\/p>\n
Add the Proxmox VE repository:<\/p>\n
\necho "deb [arch=amd64] http:\/\/download.proxmox.com\/debian\/pve bullseye pve-no-subscription" > \/etc\/apt\/sources.list.d\/pve-install-repo.list\n<\/pre>\n
Add the Proxmox VE repository key as root (or use sudo):<\/p>\n
\nwget https:\/\/enterprise.proxmox.com\/debian\/proxmox-release-bullseye.gpg -O \/etc\/apt\/trusted.gpg.d\/proxmox-release-bullseye.gpg \n<\/pre>\n
Verify the download:<\/p>\n
\nsha512sum \/etc\/apt\/trusted.gpg.d\/proxmox-release-bullseye.gpg\n<\/pre>\n
Output:<\/p>\n
\n7fb03ec8a1675723d2853b84aa4fdb49a46a3bb72b9951361488bfd19b29aab0a789a4f8c7406e71a69aabbc727c936d3549731c4659ffa1a08f44db8fdcebfa  \/etc\/apt\/trusted.gpg.d\/proxmox-release-bullseye.gpg\n<\/pre>\n
Update your repository and system by running:<\/p>\n
\napt update && apt full-upgrade\n<\/pre>\n
Install the Proxmox VE packages<\/p>\n
\napt install proxmox-ve postfix open-iscsi\n<\/pre>\n
Mail Server<\/p>\n
If you have a mail server in your network, you should configure postfix as a satellite system. Your existing mail server will then be the relay host which will route the emails sent by Proxmox VE to their final recipient. If you don’t know what to enter here, choose local only and leave the system name as is.<\/p>\n
\"\"<\/a><\/p>\n
Postfix Configuration<\/p>\n
Verify the hostname here.
\n\"\"<\/a><\/p>\n
Once finished, reboot your system.<\/p>\n
Login the ssh and change the password for root<\/p>\n
\nroot@px-03:~# passwd\nNew password:\nRetype new password:\npasswd: password updated successfully\n<\/pre>\n
Next, remove the os-prober package. The os-prober package scans all the partitions of your host, including those assigned to guests VMs, to create dual-boot GRUB entries. If you didn’t install Proxmox VE as dual boot beside another Operating System, you can safely remove the os-prober package.<\/p>\n
\napt remove os-prober\n<\/pre>\n
Now, connect to the Proxmox VE web interface. Connect to the admin web interface (https:\/\/youripaddress:8006). If you have a fresh install and haven not added any users yet, you should use the root account with your linux root password, and select “PAM Authentication” to log in.<\/p>\n
Create a Linux Bridge<\/p>\n
Once logged in, create a Linux Bridge called vmbr0, and add your first network interface to it.<\/p>\n
Here are the settings before we make changes:<\/p>\n
Primary Network Interface ( The one that connects to the Internet)
\nenp1s0
\nName:enp1s0
\nIPv4\/CIDR: 192.168.0.52\/24
\nGateway(IPv4):192.168.0.1
\nAutostart: On<\/p>\n
Secondary Network Adaptor (The One that is Private)
\nenp7s0<\/p>\n
\"\"<\/a><\/p>\n
Change enp1s0 to below:<\/p>\n
enp1s0
\nName:enp1s0
\nIPv4\/CIDR:
\nGateway(IPv4):
\nAutostart: Off<\/p>\n
\"\"<\/a><\/p>\n
Add the Linux Bridge<\/p>\n
Name:vmbr0
\nIPv4\/CIDR:192.168.0.52\/24
\nGateway(IPv4):192.168.0.1
\nBridge ports:enp1s0
\nAutostart:On<\/p>\n
\"\"<\/a><\/p>\n
Change Secondary Network Adaptor to this:
\nName:enp7s0
\nIPv4\/CIDR:10.10.10.1\/24
\nGateway(IPv4):
\nBridge ports:
\nAutostart:On<\/p>\n
Click “Apply Configuration” to save changes. If there are edit, edit and click Apply again.<\/p>\n
Here are the settings after the changes.<\/p>\n
\"\"<\/a><\/p>\n
You see that the IP is now on the Linux Bridge. Check ssh and GUI access.<\/p>\n
Repeat these steps for nodes 2 and 3 and change the IP’s in increments as above.<\/p>\n
Now, lets change the repositories. Disable The Enterprise Repo. And click “Reload”. Click on “Updates” on the Left and Refresh all 3 nodes.<\/p>\n
\"\"<\/a><\/p>\n
After all 3 nodes are installed and online, Create the cluster on node 1.<\/p>\n
Click the DATACENTER on the Left side. Click “Cluster” > “Create Cluster”.
\n\"\"<\/a>
\nThen create as below. Give the cluster a name and choose the private IP for it.<\/p>\n
\"\"<\/a><\/p>\n
Once completed, go to node 1 and go to “Join Information”. Click Copy the info.<\/p>\n
\"\"<\/a><\/p>\n
Go the nodes 2 and 3 and click “Join the cluster”. Be sure to select the private IP and enter the root password for it.<\/p>\n
\"\"<\/a><\/p>\n
Once completed, you should see 3 nodes on px-01.<\/p>\n
\"\"<\/a><\/p>\n
Install Ceph on All 3 Nodes. On each node, on the Left Side – Click “Ceph” > Click “Install Ceph”.<\/p>\n
\"\"<\/a><\/p>\n
You will see a popup for Pacific. Click “Start Pacific Installation”. Once Completed, Click “Next”. The finish the configuration.<\/p>\n
\"\"<\/a><\/p>\n
\"\"<\/a><\/p>\n
Next, create the Ceph Monitors. From the GUI.From px-01. click on Monitor > Create.<\/p>\n
\"\"<\/a><\/p>\n
Do this 4 times to add 4 monitors for each node.<\/p>\n
\"\"<\/a><\/p>\n
Now create the OSD’s. This is where the data will be stored. For this we use the second 100GB disks on each node. Clock on px-01. Under OSD, click “Create OSD”. To create the other OSD’s, click on each node to create it. Click “Reload” to view the OSD’s.<\/p>\n
\"\"<\/a><\/p>\n
Here are the finished OSD’s.<\/p>\n
\"\"<\/a><\/p>\n
Next we create 2 Pools for storage. 1 For Virtual Machines and 1 for Containers.<\/p>\n
Click on Pools on px-01. The “Create”. Create the Vm pool.<\/p>\n
\"\"<\/a><\/p>\n
Next Create the lxc Pool.<\/p>\n
\"\"<\/a><\/p>\n
Next create a test Pool.<\/p>\n
\"\"<\/a><\/p>\n
Now run a redos write benchmark. Click “Shell”. Run the following in the shell:<\/p>\n
\nrados -p test1 bench 10 write --no-cleanup\n<\/pre>\n
\"\"<\/a><\/p>\n
Bandwidth is 10.8956<\/p>\n
Next do a Read Test Type, Run:<\/p>\n
\nrados -p test1 bench 10 seq\n<\/pre>\n
Write is 53.5305<\/p>\n
Now we can remove the pool. Select pool “test1” and click “Destroy”.<\/p>\n
Now lets edit our Pools. Click on “Datacenter” > “Storage”. Edit as follows:<\/p>\n
ceph-vm<\/p>\n
\"\"<\/a><\/p>\n
ceph-lxc<\/p>\n
\"\"<\/a><\/p>\n
Next, lets copy the keyrings.<\/p>\n
\ncd \/etc\/pve\/priv\/\nroot@px-01:\/etc\/pve\/priv# cp \/etc\/ceph\/ceph.client.admin.keyring ceph\/ceph-vm.keyring\nroot@px-01:\/etc\/pve\/priv# cp \/etc\/ceph\/ceph.client.admin.keyring ceph\/ceph-lxc.keyring\n<\/pre>\n
Now we can see the storage in the GUI summary.<\/p>\n
\"\"<\/a><\/p>\n
Now we are complete.<\/p>\n","protected":false},"excerpt":{"rendered":"
The installation of a supported Proxmox VE server should be done via bare-metal ISO installer. In some cases it makes sense to install Proxmox VE on top of a running Debian Bullseye 64-bit, especially if you want a custom partition layout. For this How-To any official Bullseye installation medium should work. For this Demo, I … Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[90],"tags":[],"class_list":["post-6391","post","type-post","status-publish","format-standard","hentry","category-proxmox"],"_links":{"self":[{"href":"https:\/\/www.qbytes.cloud\/index.php\/wp-json\/wp\/v2\/posts\/6391","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.qbytes.cloud\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.qbytes.cloud\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.qbytes.cloud\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.qbytes.cloud\/index.php\/wp-json\/wp\/v2\/comments?post=6391"}],"version-history":[{"count":0,"href":"https:\/\/www.qbytes.cloud\/index.php\/wp-json\/wp\/v2\/posts\/6391\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.qbytes.cloud\/index.php\/wp-json\/wp\/v2\/media?parent=6391"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.qbytes.cloud\/index.php\/wp-json\/wp\/v2\/categories?post=6391"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.qbytes.cloud\/index.php\/wp-json\/wp\/v2\/tags?post=6391"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}