{"id":4453,"date":"2018-06-11T21:26:55","date_gmt":"2018-06-11T21:26:55","guid":{"rendered":"https:\/\/geekdecoder.com\/?p=4453"},"modified":"2018-06-11T21:26:55","modified_gmt":"2018-06-11T21:26:55","slug":"installation-for-proxmox","status":"publish","type":"post","link":"https:\/\/www.qbytes.cloud\/index.php\/2018\/06\/11\/installation-for-proxmox\/","title":{"rendered":"Installation for Proxmox"},"content":{"rendered":"

Install 3 nodes. Each node specs:
\n\"\"<\/a><\/p>\n

Since our host acts as a router we have to make sure it\u2019s kernel has all IP packet forwarding features activated. Take a look at \u2018\/etc\/sysctl.conf\u2019 and make sure that the following two lines aren\u2019t commented out:<\/p>\n

\n\nnet.ipv4.ip_forward=1\nnet.ipv6.conf.all.forwarding=1\n\n<\/pre>\n
Lastly make sure your host won\u2019t send ICPM \u201credirect\u201d messages to guests, telling them to find the gateway by themselves. This won\u2019t work with our particular network setup. Add the following to \u2018\/etc\/sysctl.conf\u2019:<\/p>\n
\n\nnet.ipv4.conf.all.send_redirects=0\n\n<\/pre>\n
Reboot<\/p>\n
Networking for nodes Incomplete – this is not currently working):
\nSet up 3 NIC’s<\/p>\n
ens192 – primary for Public IP. This is the public IP for the installation.
\nens224 – proxmox VE Cluster Network
\nens256 – Ceph Install<\/p>\n
Once installed, edit \/etc\/network\/interfaces on all 3 nodes. Change the IP addresses accordingly. Remeber to backup the files before editing.
\nExample \/etc\/network\/interfaces for px1 node 1.<\/p>\n
\nauto lo\niface lo inet loopback\n\nauto ens192\niface ens192 inet static\n        address  216.55.xxx.xxx\n        netmask  255.255.255.0\n        gateway  216.xx.xxx.1\n        post-up echo 1 &gt; \/proc\/sys\/net\/ipv4\/conf\/ens192\/proxy_arp\n#Public IP\n\nauto vmbr0\niface vmbr0 inet static\n        address  192.168.1.151\n        netmask  255.255.255.0\n        bridge_ports none\n        bridge_stp off\n        bridge_fd 0\n\tpost-up echo 1 &gt; \/proc\/sys\/net\/ipv4\/ip_forward\n        post-up iptables -t nat -A POSTROUTING -s '192.168.1.0\/24' -o ens192 -j MASQUERADE\n        post-down iptables -t nat -D POSTROUTING -s '192.168.1.0\/24' -o ens192 -j MASQUERADE\n<\/pre>\n
\n#Private Network for VM Creation\n\nauto ens224\niface ens224 inet static\n        address  10.0.0.151\n        netmask  255.255.255.0\n#Proxmox VE Cluster Network\n\nauto ens256\niface ens256 inet static\n  address  10.10.10.151\n  netmask  255.255.255.0\n#ceph network\n\n<\/pre>\n
Setting up the cluster<\/p>\n
Hosts file edits. Backup hosts file and change to this on all nodes. This is not the hostname but an identifier.<\/p>\n
# corosync network hosts\n10.0.0.151 one-corosync.geekdecoder.com one-corosync\n10.0.0.152 two-corosync.geekdecoder.com two-corosync\n10.0.0.153 three-corosync.geekdecoder.com three-corosync\n<\/pre>\n
Adding Nodes With Separated Cluster Network
\nWhen adding a node to a cluster with a separated cluster network you need to use the ringX_addr parameters to set the nodes address on those networks:<\/p>\n
pvecm add IP-ADDRESS-CLUSTER -ring0_addr IP-ADDRESS-RING0
\nIf you want to use the Redundant Ring Protocol you will also want to pass the ring1_addr parameter.<\/p>\n
Creating the cluster after network and host file setup. Log into node 1.<\/p>\n
node1# pvecm create clustername -bindnet0_addr 10.0.0.151 -ring0_addr one-corosync\n<\/pre>\n
Output<\/p>\n
Corosync Cluster Engine Authentication key generator.\nGathering 1024 bits for key from \/dev\/urandom.\nWriting corosync key to \/etc\/corosync\/authkey.\nWriting corosync config to \/etc\/pve\/corosync.conf\nRestart corosync and cluster filesystem\n<\/pre>\n
Reboot<\/p>\n
# reboot\n<\/pre>\n
Adding nodes.<\/p>\n
From man corosync.conf<\/p>\n
ringnumber – This specifies the ring number for the interface. When using the redundant ring protocol, each interface should specify separate ring numbers to uniquely identify to the membership protocol which interface to use for which redundant ring. The ringnumber must start at 0.<\/p>\n
Adding nodes. Login to a node you want to add, node2, node3, node4, etc. In this example, login to node 2 (10.0.0.152)and the IP for node1 is 10.0.0.151.
\nRun the following where”<\/p>\n
node2# pvecm add 10.0.0.151 -ring0_addr 10.0.0.152\n\n<\/pre>\n
Status<\/p>\n
 pvecm status\nQuorum information\n------------------\nDate:             Tue Jun 12 15:17:10 2018\nQuorum provider:  corosync_votequorum\nNodes:            2\nNode ID:          0x00000001\nRing ID:          1\/12\nQuorate:          Yes\n\nVotequorum information\n----------------------\nExpected votes:   2\nHighest expected: 2\nTotal votes:      2\nQuorum:           2\nFlags:            Quorate\n\nMembership information\n----------------------\n    Nodeid      Votes Name\n0x00000001          1 10.0.0.151 (local)\n0x00000002          1 10.0.0.152\n\n<\/pre>\n
Nodes<\/p>\n
 pvecm nodes\n\nMembership information\n----------------------\n    Nodeid      Votes Name\n         1          1 one-corosync (local)\n         2          1 10.0.0.152\n         3          1 10.0.0.153\n\n<\/pre>\n
log into the node 1 web GUI.
\n\"\"<\/a><\/p>\n
Installation of Ceph packages<\/p>\n
Login to each node and run:<\/p>\n
~# pveceph install --version luminous\nupdate available package list\nReading package lists... Done\nBuilding dependency tree\nReading state information... Done\ngdisk is already the newest version (1.0.1-1).\nThe following additional packages will be installed:\n  binutils ceph-base ceph-mgr ceph-mon ceph-osd cryptsetup-bin libcephfs2 libcurl3 libgoogle-perftools4 libjs-jquery\n  libjs-sphinxdoc libjs-underscore libleveldb1v5 liblttng-ust-ctl2 liblttng-ust0 libparted2 librados2 libradosstriper1\n  librbd1 librgw2 libtcmalloc-minimal4 libunwind8 parted python-bs4 python-cephfs python-cffi-backend python-cherrypy3\n  python-click python-colorama python-cryptography python-dnspython python-enum34 python-flask python-formencode\n  python-idna python-ipaddress python-itsdangerous python-jinja2 python-logutils python-mako python-markupsafe\n  python-openssl python-paste python-pastedeploy python-pastedeploy-tpl python-pecan python-prettytable python-pyasn1\n  python-rados python-rbd python-repoze.lru python-rgw python-routes python-setuptools python-simplegeneric\n  python-singledispatch python-tempita python-waitress python-webob python-webtest python-werkzeug\nSuggested packages:\n  binutils-doc ceph-mds libparted-dev libparted-i18n parted-doc python-cryptography-doc python-cryptography-vectors\n  python-enum34-doc python-flask-doc python-egenix-mxdatetime python-jinja2-doc python-beaker python-mako-doc\n  python-openssl-doc python-openssl-dbg httpd-wsgi libapache2-mod-python libapache2-mod-scgi python-pastescript\n  python-pastewebkit doc-base python-setuptools-doc python-waitress-doc python-webob-doc python-webtest-doc ipython\n  python-genshi python-lxml python-greenlet python-redis python-pylibmc | python-memcache python-werkzeug-doc\nRecommended packages:\n  ceph-mds ntp | time-daemon javascript-common python-lxml | python-html5lib python-blinker python-simplejson\n  libjs-mochikit python-openid python-scgi python-pastescript python-lxml python-pyquery python-pyinotify\nThe following NEW packages will be installed:\n  binutils ceph ceph-base ceph-mgr ceph-mon ceph-osd cryptsetup-bin libcephfs2 libcurl3 libgoogle-perftools4 libjs-jquery\n  libjs-sphinxdoc libjs-underscore libleveldb1v5 liblttng-ust-ctl2 liblttng-ust0 libparted2 libtcmalloc-minimal4\n  libunwind8 parted python-bs4 python-cffi-backend python-cherrypy3 python-click python-colorama python-cryptography\n  python-dnspython python-enum34 python-flask python-formencode python-idna python-ipaddress python-itsdangerous\n  python-jinja2 python-logutils python-mako python-markupsafe python-openssl python-paste python-pastedeploy\n  python-pastedeploy-tpl python-pecan python-prettytable python-pyasn1 python-repoze.lru python-rgw python-routes\n  python-setuptools python-simplegeneric python-singledispatch python-tempita python-waitress python-webob python-webtest\n  python-werkzeug\nThe following packages will be upgraded:\n  ceph-common librados2 libradosstriper1 librbd1 librgw2 python-cephfs python-rados python-rbd\n8 upgraded, 55 newly installed, 0 to remove and 27 not upgraded.\nNeed to get 54.5 MB of archives.\nAfter this operation, 179 MB of additional disk space will be used.\n\n<\/pre>\n
Create initial Ceph configuration<\/p>\n
node1# pveceph init --network 10.10.10.0\/24\n<\/pre>\n
After that you can create the first Ceph monitor service using:<\/p>\n
node1# pveceph createmon\n<\/pre>\n
Go to video or site to create other items.
\n