In Ambari Register Hosts Screen Should We Include Ambari Node Also ?

Pre-requisites

Hadoop can be installed on the following operating systems
1. Red Chapeau Enterprise Linux (RHEL) v6.x
2. Carmine Hat Enterprise Linux (RHEL) v5.ten (deprecated)
3. CentOS v6.x
4. CentOS v5.x (deprecated)
5. Oracle Linux v6.10
6. Oracle Linux v5.x (deprecated)
7. SUSE Linux Enterprise Server (SLES) v11, SP1 and SP3
8. Ubuntu Precise v12.04
Ensure that the nodes have full hostname.
Command to check hostname: "hostname -f"
Ensure that you have the following LINUX software'south
1. yum and rpm (RHEL/CentOS/Oracle Linux)
2. scp, curl, unzip, tar, and wget
3. OpenSSL (v1.01, build 16 or afterward)
4. python v2.6
Ensure that you lot have installed Coffee. Run command "yum install java-1.vii.0-openjdk"
Database & Memory requirements - Ambari requires a relational database to store information about the cluster configuration and topology. If y'all install HDP Stack with Hive or Oozie, they also crave a relational database.
1. Ambari : By default, install an instance of PostgreSQL on the Ambari Server host.
2. Hive : By default (on RHEL/CentOS/Oracle Linux six), Ambari will install an instance of MySQL on the Hive Metastore host.
3. Oozie : By default, Ambari will install an instance of Derby on the Oozie Server host Y'all tin can also employ existing case of PostgreSQL, MySQL or Oracle. For the Ambari database, if you utilize an existing Oracle database, make certain the Oracle listener runs on a port other than 8080 to avoid disharmonize with the default Ambari port.
Also ensure that you have atleast 8 GB of ram on each host. Apache recommends that you have atleast 1 GB of ram available.

Hosts Training:

	Host Proper name	IP Accost
Node i	hdp1.stratapps.com	172.16.16.202
Node 2	hdp2.stratapps.com	172.16.16.203

On NODE1 .i.e hdp1.stratapps.com

Checking Kernel Versions & Centos Release Versions
Step -1:
Ensure the centos installed is 64 Bit using the command

Ensure the Operating system installed is centos 6.seven using the control

Disabling IP Tables & Selinux

Step-ii:
Configuring iptables:During cluster installation we need to ensure the connections are open between cluster hosts without any restrictions.

In order to achieve that we need to Turn Off the IP tables (Firewall).

The easiest way to exercise this is by disabling the iptables, equally follows:

#Service iptables stop
              #Service ip6tables stop
              #chkconfig iptables off
              #chkconfig  ip6tables off

Pace-3:
Disable SELinux and PackageKit. Forthis the post-obit file needs to exist inverse.

Vi /etc/sysconfig/selinux

To permanently disable SELinux prepare SELINUX=disabled

Adding Host Names to Cluster Nodes

Stride -4:
Ensure that the nodes have fully qualified domain name (full hostname).
Employ control "hostname -f" to bank check the full hostname or FQDN.

Use the /etc/hosts file to add both the node hostnames.

172.16.16.202 hdp1.stratapps.com hdp1 172.sixteen.16.203 hdp2.stratapps.com hdp2

On NODE2i.e hdp2.stratapps.com

Follow the steps from step1 to step4 on Node2 i.due east hdp2.stratapps.com equally shown above.

In one case all the steps are completed. Ensure the communication happens between the nodes using the below command on both nodes and vice versa.

On Node1 issue the command

#ping hdp2.stratapps.com #ping hdp1.stratapps.com

On Node 2 result the command

#ping hdp1.stratapps.com #ping hdp2.stratapps.com

SSH Configuration On Cluster Nodes

Step-5:
Create countersign -less SSH. To have Ambari Server automatically install Ambari Agents on all your cluster hosts, you must set password-less SSH connections betwixt the Ambari Server host and all other hosts in the cluster. The Ambari Server host uses SSH public fundamental hallmark to remotely admission and install the Ambari Agent.

Generate public and individual SSH keys on the Ambari Server host ("Command ssh-keygen")

Copy the SSH Public Key(id_rsa.pub) to the root account on your target hosts.

.ssh/id_rsa .ssh/id_rsa.pub    # ssh-re-create-id -i $HOME/.ssh/id_rsa.pub root@hdp1.stratapps.com # ssh-copy-id -i $Dwelling house/.ssh/id_rsa.pub root@hdp2.stratapps.com

Need to examination the ssh connection to the host without countersign.

and also connect to node2 using ssh without password

Step-six:
Needs to exist performed on Node2 i.e hdp2.stratapps.com

# ssh-copy-id -i $HOME/.ssh/id_rsa.pub root@hdp1.stratapps.com # ssh-copy-id -i $Abode/.ssh/id_rsa.pub root@hdp2.stratapps.com

Need to test the ssh connexion to the host without password.

And as well connect to node1 using ssh without password

Installation of httpd & ntp Packages

Step -7:
Configure an HTTP server. Ensure to installhttpd using the command

#yum install httpd on both node1 and node2

Http server is required for webbrowser access.

Enable NTP on the Node1.i.e hdp1.stratapps.com and on the Node2 hdp2.stratapps.com

Download Ambari and hdp Repositories for Cluster Installation

Step -8:

Setting up a local repository: If your cluster is backside a firewall that prevents or limits Cyberspace access, you can install Ambari and a Stack using local repositories else you can utilize internet to admission Hortonwork repositories.

Obtaining the Repositories: Ambari Repositories : If y'all do non have Internet access for setting up the Ambari repository, use the link advisable for your Os family to download a tarball that contains the software.
wget -nvhttp://public-repo-1.hortonworks.com/HDP/centos6/two.x/GA/two.two.0.0/hdp.repo -O /etc/yum.repos.d/HDP.repo

HDP Stack Repositories : If you lot practice not have Cyberspace access to fix the Stack repositories, utilise the link advisable for your Bone family unit to download a tarball that contains the HDP Stack version you plan to install.
Wget–nvhttp://public-repo-1.hortonworks.com/ambari/centos6/ane.ten/updates/ane.7.0/ambari.repo -O/etc/yum.repos.d/ambari.repo

Installation and Configuration of Ambari Server

Step-9: Installing Ambari
Issue the command "yum install ambari-server"on node1 hdp1.stratapps.com

Once ambari is installed nosotros need to run the setup using the command.

Select a JDK version to download. Enter 1 to download Oracle JDK 1.7.By default, Ambari Server setup downloads and installs Oracle JDK1.7 by accompanying Coffee Cryptography Extension (JCE) Policy Files.

Select due north at Enter advanced database configuration to use the default, embedded PostgreSQL database for Ambari. The default PostgreSQL database name is ambari. The default user name and countersign are ambari/bigdata. Else, to use an existing PostgreSQL, MySQL or Oracle database with Ambari, select y.

Once the setup is completed - Showtime the Ambari Server

Run the post-obit command on the Ambari Server host hdp1.stratapps.com

To kickoff the server:

To stop the Ambari Server:

To know status:

Installing, Configuring, and Deploying a HDP Cluster

We volition use the Ambari Install Wizard running in your browser to install, configure, and deploy our cluster.

After starting the Ambari service, open Ambari Web using a web browser & Indicate your browser to http://:8080, where is the name of yourambari server host. i.e http:// hdp1.stratapps.com:8080/#/login

You lot can alter these credentials later.

Post Installation using Ambari Server UI (User Interface)

Step-ten:

Step-11: From the Ambari Welcome page, cull Launch Install Sorcerer.

Then press "Launch Install Wizard" for creating new cluster.

Step-12: In Proper noun for your cluster, blazon a name for the cluster you lot desire to create. There should be no white spaces or special characters in the proper noun

Give your ain name for cluster and Press "Adjacent".

Step-xiii: The Service Stack (the Stack) is a coordinated and tested set of HDP components. Employ a radio button to select the Stack version you want to install. To install an HDP 2x stack, select the HDP ii.two, HDP two.1, or HDP 2.0 radio button.

press "Next".

Step-xiv: Need to provide the fully qulaified domain host names hither for cluster creation.

We need to access the individual primal file created earlier between the hosts.

For that first nosotros need to open up node1. i.eastward hdp1.stratapps.com with winscp. Follow above screen.

And and so re-create id_rsa file(individual key) into your desktop. Detect the above screen.

In club to build upward the cluster, the install magician prompts you for full general information near how you want to prepare it upward. You need to supply the FQDN of each of your hosts. The wizard also needs to access the private key file you created earlier. Using the host names and key file information, the wizard tin locate, admission, and collaborate securely with all hosts in the cluster. If you want to let Ambari automatically install the Ambari Agent on all your hosts using SSH, select Provide your SSH Private Key and either use the Cull File push in the Host Registration Information department to detect the individual central file that matches the public key you installed earlier on all your hosts or cut and paste the cardinal into the text box manually.

And then press "Annals and Ostend".

Stride-xv: Confirm Hosts

Here it volition check ssh authentication with provided private key.Bank check to a higher place screen.

Step-16: Cull Services you want to deploy

Step-17:
The Ambari install magician assigns the master components for selected services to appropriate hosts in your cluster and displays the assignments in Assign Masters.

Hive server,Metastore Web Hcat should be Hosted in same machine. Here nosotros hosted on node1 i.e hdp1.stratapps.com.

Step-18:
The Ambari installation wizard assigns the slave components (DataNodes, NodeManagers, and RegionServers) to appropriate hosts in your cluster. Information technology besides attempts to select hosts for installing the appropriate set of clients.

All the clients were installed on node2 i.e hdp2.stratapps.com

Pace-19:
Customize Services, here nosotros need to provide usernames and passwords in the required fields.

Here nosotros demand to provide password and mail ID for nagios service.

Review all the settings and services before starting the installation.

Stride-20:
Starting time the installation by clicking the Deploy button.

Step-21:

Here, the dashboard shows that all services are upwards and runing.

This completes our Hadoop Server Cluster deployment.

Note: Difference between single node installation and multinode installation

In unmarried node hdp installation, nosotros take all hadoop daemons on unmarried node with separate jvms. When comes to the multinode hadoop installation all hadoop daemonsare sharedon multiple nodes.For example here we have 2 nodes, in this two nodes we had shared hadoop daemons based on our hardware requirements.