= Installation =

== Install the OS ==

Detailed instructions for installing Fedora are available at
http://docs.fedoraproject.org/en-US/Fedora/21/html/Installation_Guide/ in a number of
languages. The abbreviated version is as follows:

Point your browser to https://getfedora.org/,
choose a flavor (Server is an appropriate choice),
and download the installation image appropriate to your hardware.

Burn the installation image to a DVD or USB drive
footnote:[http://docs.fedoraproject.org/en-US/Fedora/21/html/Installation_Guide/sect-preparing-boot-media.html]
and boot from it, or use the image to boot a virtual machine.

After starting the installation, select your language and keyboard layout at
the welcome screen.
footnote:[http://docs.fedoraproject.org/en-US/Fedora/21/html/Installation_Guide/sect-installation-graphical-mode.html]

At this point, you get a chance to tweak the default installation options.

In the *NETWORK & HOSTNAME* section you'll want to:

- Assign your machine a host name.
  I happen to control the clusterlabs.org domain name, so I will use
  pcmk-1.clusterlabs.org here.
- Assign a fixed IPv4 address. In this example, I'll use 192.168.122.101.

[IMPORTANT]
===========
Do not accept the default network settings.
Cluster machines should never obtain an IP address via DHCP, because
DHCP's periodic address renewal will interfere with corosync.

If you miss this step during installation, it can easily be fixed later. You will have
to navigate to *system settings* and select *network*.  From there, you can select
what device to configure.
===========

In the *Software Selection* section (try saying that 10 times
quickly), leave all *Add-Ons* unchecked so that we see everything that gets
installed. We'll install any extra software we need later.

[IMPORTANT]
===========

By default Fedora uses LVM for partitioning which allows us to
dynamically change the amount of space allocated to a given partition.

However, by default it also allocates all free space to the +/+
(aka. *root*) partition, which cannot be dynamically _reduced_ in size
(dynamic increases are fine, by the way).

So if you plan on following the DRBD or GFS2 portions of this guide,
you should reserve at least 1GiB of space on each machine from which to
create a shared volume.  To do so, enter the *Installation
Destination* section where you are be given an opportunity to reduce
the size of the *root* partition (after choosing which hard drive you
wish to install to). If you want the reserved space to be available
within an LVM volume group, be sure to select *Modify...* next to
the volume group name and change the *Size policy:* to *Fixed*
or *As large as possible*.

===========

It is highly recommended to enable NTP on your cluster nodes. Doing so
ensures all nodes agree on the current time and makes reading log files
significantly easier. You can do this in the *DATE & TIME* section. 
footnote:[http://docs.fedoraproject.org/en-US/Fedora/21/html/Installation_Guide/sect-installation-gui-date-and-time.html]

Once you've completed the installation, set a root password as instructed.
For the purposes of this document, it is not necessary to create any additional
users. After the node reboots, you'll see a (possibly mangled) login prompt on
the console. Login using *root* and the password you created earlier.

image::images/Console.png["Initial Console",align="center",scaledwidth="65%"]

[NOTE]
======

From here on, we're going to be working exclusively from the terminal.

======

== Configure the OS ==

=== Verify Networking ===

Ensure that the machine has the static IP address you configured earlier.

-----
[root@pcmk-1 ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:d7:d6:08 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.101/24 brd 192.168.122.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fed7:d608/64 scope link
       valid_lft forever preferred_lft forever
-----

[NOTE]
=====
If you ever need to change the node's IP address from the command line, follow these instructions:

....
[root@pcmk-1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-${device} # manually edit as desired
[root@pcmk-1 ~]# nmcli dev disconnect ${device}
[root@pcmk-1 ~]# nmcli con reload ${device}
[root@pcmk-1 ~]# nmcli con up ${device}
....

This makes *NetworkManager* aware that a change was made on the config file.

=====

Next, ensure that the routes are as expected:

-----
[root@pcmk-1 ~]# ip route
default via 192.168.122.1 dev eth0  proto static  metric 1024
192.168.122.0/24 dev eth0  proto kernel  scope link  src 192.168.122.101
-----

If there is no line beginning with *default via*, then you may need to add a line such as

[source,Bash]
GATEWAY=192.168.122.1

to +/etc/sysconfig/network+ and restart the network.

Now, check for connectivity to the outside world. Start small by
testing whether we can reach the gateway we configured.

-----
[root@pcmk-1 ~]# ping -c 1 192.168.122.1
PING 192.168.122.1 (192.168.122.1) 56(84) bytes of data.
64 bytes from 192.168.122.1: icmp_req=1 ttl=64 time=0.249 ms

 --- 192.168.122.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.249/0.249/0.249/0.000 ms
-----

Now try something external; choose a location you know should be available.

-----
[root@pcmk-1 ~]# ping -c 1 www.google.com
PING www.l.google.com (173.194.72.106) 56(84) bytes of data.
64 bytes from tf-in-f106.1e100.net (173.194.72.106): icmp_req=1 ttl=41 time=167 ms

 --- www.l.google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 167.618/167.618/167.618/0.000 ms
-----

=== Login Remotely ===

The console isn't a very friendly place to work from, so we will now
switch to accessing the machine remotely via SSH where we can
use copy and paste, etc.

From another host, check whether we can see the new host at all:

-----
beekhof@f16 ~ # ping -c 1 192.168.122.101
PING 192.168.122.101 (192.168.122.101) 56(84) bytes of data.
64 bytes from 192.168.122.101: icmp_req=1 ttl=64 time=1.01 ms

--- 192.168.122.101 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 1.012/1.012/1.012/0.000 ms
-----

Next, login as root via SSH.

-----
beekhof@f16 ~ # ssh -l root 192.168.122.11
root@192.168.122.11's password:
Last login: Fri Mar 30 19:41:19 2012 from 192.168.122.1
[root@pcmk-1 ~]#
-----

=== Apply Updates ===

Apply any package updates released since your installation image was created:
----
[root@pcmk-1 ~]# yum update
----

=== Disable Security During Testing ===

To simplify this guide and focus on the aspects directly connected to
clustering, we will now disable the machine's firewall and SELinux
installation.

[WARNING]
===========
These actions create significant security issues and should not be performed on
machines that will be exposed to the outside world.
===========

////
 TODO: Create an Appendix that deals with (at least) re-enabling the firewall.
////

----
[root@pcmk-1 ~]# setenforce 0
[root@pcmk-1 ~]# sed -i.bak "s/SELINUX=enforcing/SELINUX=permissive/g" /etc/selinux/config
[root@pcmk-1 ~]# systemctl disable firewalld.service
[root@pcmk-1 ~]# systemctl stop firewalld.service
[root@pcmk-1 ~]# iptables --flush
----

[NOTE]
===========
If you are using Fedora 17 or earlier or are using the iptables
service for your firewall, the commands would be:

----
[root@pcmk-1 ~]# setenforce 0
[root@pcmk-1 ~]# sed -i.bak "s/SELINUX=enforcing/SELINUX=permissive/g" /etc/selinux/config
[root@pcmk-1 ~]# systemctl disable iptables.service
[root@pcmk-1 ~]# rm -f /etc/systemd/system/basic.target.wants/iptables.service
[root@pcmk-1 ~]# systemctl stop iptables.service
[root@pcmk-1 ~]# iptables --flush
----
===========


=== Use Short Node Names ===

During installation, we filled in the machine's fully qualified domain
name (FQDN), which can be rather long when it appears in cluster logs and
status output. See for yourself how the machine identifies itself:
(((Nodes, short name)))

----
[root@pcmk-1 ~]# uname -n
pcmk-1.clusterlabs.org
[root@pcmk-1 ~]# dnsdomainname
clusterlabs.org
----
(((Nodes, Domain name (Query))))

The output from the second command is fine, but we really don't need the
domain name included in the basic host details. To address this, we need
to use the `hostnamectl` tool to strip off the domain name.
----
[root@pcmk-1 ~]# hostnamectl set-hostname $(uname -n | sed s/\\..*//)
----
(((Nodes, Domain name (Remove from host name))))

Now check the machine is using the correct names

----
[root@pcmk-1 ~]# uname -n
pcmk-1
[root@pcmk-1 ~]# dnsdomainname
clusterlabs.org
----

If it concerns you that the shell prompt has not been updated, simply
log out and back in again.

== Repeat for Second Node ==

Repeat the Installation steps so far, so that you have two Fedora
nodes ready to have the cluster software installed.

For the purposes of this document, the additional node is called
pcmk-2 with address 192.168.122.102.

== Configure Communication Between Nodes ==

=== Configure Host Name Resolution ===

Confirm that you can communicate between the two new nodes:

----
[root@pcmk-1 ~]# ping -c 3 192.168.122.102
PING 192.168.122.102 (192.168.122.102) 56(84) bytes of data.
64 bytes from 192.168.122.102: icmp_seq=1 ttl=64 time=0.343 ms
64 bytes from 192.168.122.102: icmp_seq=2 ttl=64 time=0.402 ms
64 bytes from 192.168.122.102: icmp_seq=3 ttl=64 time=0.558 ms

--- 192.168.122.102 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.343/0.434/0.558/0.092 ms
----

Now we need to make sure we can communicate with the machines by their
name. If you have a DNS server, add additional entries for the two
machines. Otherwise, you'll need to add the machines to +/etc/hosts+
on both nodes. Below are the entries for my cluster nodes:

----
[root@pcmk-1 ~]# grep pcmk /etc/hosts
192.168.122.101 pcmk-1.clusterlabs.org pcmk-1
192.168.122.102 pcmk-2.clusterlabs.org pcmk-2
----

We can now verify the setup by again using ping:

----
[root@pcmk-1 ~]# ping -c 3 pcmk-2
PING pcmk-2.clusterlabs.org (192.168.122.101) 56(84) bytes of data.
64 bytes from pcmk-1.clusterlabs.org (192.168.122.101): icmp_seq=1 ttl=64 time=0.164 ms
64 bytes from pcmk-1.clusterlabs.org (192.168.122.101): icmp_seq=2 ttl=64 time=0.475 ms
64 bytes from pcmk-1.clusterlabs.org (192.168.122.101): icmp_seq=3 ttl=64 time=0.186 ms

--- pcmk-2.clusterlabs.org ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.164/0.275/0.475/0.141 ms
----

=== Configure SSH ===

SSH is a convenient and secure way to copy files and perform commands
remotely. For the purposes of this guide, we will create a key without a
password (using the -N option) so that we can perform remote actions
without being prompted.

(((SSH)))

[WARNING]
=========
Unprotected SSH keys (those without a password) are not recommended for servers exposed to the outside world.
We use them here only to simplify the demo.
=========

Create a new key and allow anyone with that key to log in:

.Creating and Activating a new SSH Key
----
[root@pcmk-1 ~]# ssh-keygen -t dsa -f ~/.ssh/id_dsa -N ""
Generating public/private dsa key pair.
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
91:09:5c:82:5a:6a:50:08:4e:b2:0c:62:de:cc:74:44 root@pcmk-1.clusterlabs.org

The key's randomart image is:
+--[ DSA 1024]----+
|==.ooEo..        |
|X O + .o o       |
| * A    +        |
|  +      .       |
| .      S        |
|                 |
|                 |
|                 |
|                 |
+-----------------+

[root@pcmk-1 ~]# cp ~/.ssh/id_dsa.pub ~/.ssh/authorized_keys
----
(((Creating and Activating a new SSH Key)))

Install the key on the other node and test that you can now run commands
remotely, without being prompted.

.Installing the SSH Key on Another Host
----
[root@pcmk-1 ~]# scp -r ~/.ssh pcmk-2:
The authenticity of host 'pcmk-2 (192.168.122.102)' can't be established.
RSA key fingerprint is b1:2b:55:93:f1:d9:52:2b:0f:f2:8a:4e:ae:c6:7c:9a.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'pcmk-2,192.168.122.102' (RSA) to the list of known hosts.root@pcmk-2's password:
id_dsa.pub                           100%  616     0.6KB/s   00:00
id_dsa                               100%  672     0.7KB/s   00:00
known_hosts                          100%  400     0.4KB/s   00:00
authorized_keys                      100%  616     0.6KB/s   00:00
[root@pcmk-1 ~]# ssh pcmk-2 -- uname -n
pcmk-2
----

== Install the Cluster Software ==

Fedora 17 and later comes with everything you need, so simply fire up a shell
on both nodes and run the following to install pacemaker and command-line
cluster management software:

----
# yum install -y pacemaker pcs psmisc
----

[IMPORTANT]
===========
This document will show commands that need to be executed on both nodes
with a simple `#` prompt. Be sure to run them on each node individually.
===========

[NOTE]
===========
This document uses pcs for cluster management. Other alternatives,
such as crmsh, are available, but their syntax
will differ from the examples used here.
===========

== Configure the Cluster Software ==

=== Enable pcs Daemon ===

Before the cluster can be configured, the pcs daemon must be started and enabled
to start at boot time on each node. This daemon works with the pcs command-line interface
to manage synchronizing the corosync configuration across all nodes in the cluster.

Start and enable the daemon by issuing the following commands on each node:

----
# systemctl start pcsd.service
# systemctl enable pcsd.service
----

The installed packages will create a *hacluster* user with a disabled password.
While this is fine for running `pcs` commands locally,
the account needs a login password in order to perform such tasks as syncing
the corosync configuration, or starting and stopping the cluster on other nodes.

This tutorial will make use of such commands,
so now we will set a password for the *hacluster* user, using the same password
on both nodes:

----
# passwd hacluster
password:
----

[NOTE]
===========
Alternatively, to script this process or set the password on a
different machine from the one you're logged into, you can use 
the `--stdin` option for `passwd`:

----
[root@pcmk-1 ~]# ssh pcmk-2 -- 'echo redhat1 | passwd --stdin hacluster'
----
===========

=== Configure Corosync ===

On either node, use `pcs cluster auth` to authenticate as the *hacluster* user:

----
[root@pcmk-1 ~]# pcs cluster auth pcmk-1 pcmk-2
Username: hacluster
Password: 
pcmk-1: Authorized
pcmk-2: Authorized
----

[IMPORTANT]
===========
The version of pcs shipped with Fedora 21 will bind only to
the host's IPv6 address in some circumstances. If you get errors
with `pcs cluster auth`, add this line before the first *server.run* line in
+/usr/lib/pcsd/ssl.rb+ to bind to IPv4 only:

----
webrick_options[:BindAddress] = '0.0.0.0'
----

And restart pcsd:
----
[root@pcmk-1 ~]# systemctl restart pcsd
----

This is a temporary workaround that will get removed if the pcsd
package is later updated.
===========

Next, use `pcs cluster setup` to generate and synchronize the corosync
configuration:
----
[root@pcmk-1 ~]# pcs cluster setup --name mycluster pcmk-1 pcmk-2
Shutting down pacemaker/corosync services...
Redirecting to /bin/systemctl stop  pacemaker.service
Redirecting to /bin/systemctl stop  corosync.service
Killing any remaining services...
Removing all cluster configuration files...
pcmk-1: Succeeded
pcmk-2: Succeeded
----

If you received an authorization error for either of those commands, make
sure you configured the *hacluster* user account on each node
with the same password.

[NOTE]
======
Early versions of pcs, such as the one shipped with Fedora 20 and earlier,
require that `--name` be omitted from the above command.

If using a different cluster shell such as crmsh rather than pcs, you must
manually create a corosync.conf and copy it to all nodes.

The pcs command will configure corosync to use UDP unicast transport; if you
choose to use multicast instead, choose a multicast address carefully.
footnote:[For some subtle issues, see the now-defunct http://web.archive.org/web/20101211210054/http://29west.com/docs/THPM/multicast-address-assignment.html or the more detailed treatment in
http://www.cisco.com/c/dam/en/us/support/docs/ip/ip-multicast/ipmlt_wp.pdf[Cisco's
Guidelines for Enterprise IP Multicast Address Allocation] paper.]
======

The final /etc/corosync.conf configuration on each node should look
something like the sample in Appendix B, Sample Corosync Configuration.

[NOTE]
======
With versions of Corosync before 2.0, Pacemaker could obtain membership and
quorum from a custom Corosync plugin. This plugin also had the capability to
start Pacemaker automatically when Corosync was started.
Neither behavior is possible with Corosync 2.0 and later, as support for
plugins was removed.

Because Pacemaker made use of the plugin for message routing, a cluster node
using an older Corosync cannot talk to one using Corosync 2.0 or later.
Rolling upgrades between these versions are therefore not possible, and an
alternate strategy
footnote:[http://www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Pacemaker_Explained/ap-upgrade.html]
must be used.
======
