修改/etc/hosts文件
sudo gedit /etc/hosts127.0.0.1 localhost192.168.222.129 master下面分别对应三个机器的IP地址,用ifconfig查看192.168.222.130 slave1192.168.222.131 slave2# The following lines are desirable for IPv6 capable hosts::1ip6-localhost ip6-loopbackfe00::0 ip6-localnetff00::0 ip6-mcastprefixff02::1 ip6-allnodesff02::2 ip6-allrouters测试是否能ping通ping master/slave1/slave2
==========
分别给3台机器都创建hadoop用户,方便进行管理
二、 建立hadoop运行帐号即为hadoop集群专门设置一个用户组及用户(1)创建hadoop用户组(2)创建hadoop用户sudo adduser -ingroup hadoop hadoop回车后会提示输入新的UNIX密码,这是新建用户hadoop的密码,输入回车即可。最后确认信息是否正确,如果没问题,输入 Y,回车即可。(3)为hadoop用户添加权限(给hadoop用户赋予和root用户同样的权限)输入:sudo gedit /etc/sudoers~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
================================================
用新增加的hadoop用户登录Ubuntu系统,进行SSH的配置
================================================
配置ssh免密码登录hadoop工作时,各节点要相互通讯,正常情况下linux之间通讯要提供用户名、密码(目的是保证通讯安全),如果需要人工干预输入密码,显然不方便,做这一步的目的,是让各节点能自动通过安全认证,不影响正常通讯。安装ssh: sudo apt-get install openssh-server安装完成后,启动服务:sudo /etc/init.d/ssh start查看服务是否正确启动:ps -e | grep ssh
一.使用SSH使每一台机器免密码登录自己本机
//下面以mster机器为例:1.使登录身份始终是hadoop,操作目录始终是/home/mastermaster@master:~$ su hadoop#切换成hadoop(以hadoop身份登录到系统)hadoop@master:~$cd /home/hadoop(进入个人主目录,默认为/home/hadoop)2.生成公钥私钥id_rsa id_rsa.pubhadoop@master:~$ ssh-keygen -t rsa -P '' (注:最后是二个单引号,生成公钥私钥)Generating public/private rsa key pair.Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): Created directory '/home/hadoop/.ssh'.Your identification has been saved in /home/hadoop/.ssh/id_rsa.Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.The key fingerprint is:cd:67:f5:0f:15:a1:42:ad:7a:29:78:3b:ca:8c:3a:b7 hadoop@slave1The key's randomart image is:+--[ RSA 2048]----+| .. o.||. .. .|| .... .|| o ... o || S + + . .|| . + = ..|| . +.|| . .+ o || .+E.+. .|+-----------------+解释:以rsa算法,生成公钥、私钥对,-P ''表示空密码。该命令运行完后,会在个人主目录/home/hadoop下生成.ssh目录,里面会有二个文件id_rsa(私钥) ,id_rsa.pub(公钥),即:hadoop@master:~$ ls .ssh #查看是否生成公钥私钥id_rsa id_rsa.pub3.导入公钥hadoop@master:cat .ssh/id_rsa.pub >> .ssh/authorized_keys解释:会在个人主目录/home/hadoop/.ssh目录下生成authorized_keys,即hadoop@master:~$ ls .ssh #查看是否生成authorized_keysauthorized_keys id_rsa id_rsa.pub4.修改文件权限hadoop@master:chmod 600 .ssh/authorized_keys5.进行SSH测试是否能免密码连接本机执行完以后,可以在本机上测试下,用ssh连接自己,即:ssh localhost (或ssh master)【注解】废话不多说,直接上下面中3个中括号中的命令【hadoop@master:~$ ssh localhost】 #连接自己The authenticity of host 'localhost (127.0.0.1)' can't be established.ECDSA key fingerprint is d5:f7:47:44:7b:9f:55:ac:e7:ed:6c:8d:f6:98:68:72.Are you sure you want to continue connecting (yes/no)? yes #输入yesWarning: Permanently added 'localhost' (ECDSA) to the list of known hosts.Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)* Documentation: /New release '16.04.3 LTS' available.Run 'do-release-upgrade' to upgrade to it.The programs included with the Ubuntu system are free software;the exact distribution terms for each program are described in theindividual files in /usr/share/doc/*/copyright.Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted byapplicable law.【hadoop@master:~$ exit】 #连接成功后,退出exit注销Connection to localhost closed.【hadoop@master:~$ ssh localhost】 #重新链接,不需要密码,此时成功配置了master机器的SSHWelcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)* Documentation: /281 packages can be updated.193 updates are security updates.New release '16.04.3 LTS' available.Run 'do-release-upgrade' to upgrade to it.Last login: Fri Aug 11 10:22:27 from localhosthadoop@master:~$ 【终于】到此位置,成功搞定一台机器:master,下面我们再进行slave1,slave2机器的SSH免密码登录的配置。(配置方法同上)
二. 将slave公钥文件复制到master(只操作slave机器,不操作master)
a)用scp命令,把公钥文件发放给master(即:刚才已经搞定的那台机器)#在slave1上:scp .ssh/id_rsa.pub hadoop@master:/home/hadoop/id_rsa_01.pub#在slave2上:scp .ssh/id_rsa.pub hadoop@master:/home/hadoop/id_rsa_02.pubb)这二行执行完后,回到master中,查看下/home/hadoop目录,应该有二个新文件id_rsa_01.pub、id_rsa_02.pub,然后在master上,导入这二个公钥cat id_rsa_01.pub >> .ssh/authorized_keyscat id_rsa_02.pub >> .ssh/authorized_keys这样,master这台机器就有所有3台机器的公钥了。
三. 将master上的“最全”公钥,复制到其它机器slave1.slave2
a) 继续保持在master上,scp .ssh/authorized_keys hadoop@slave1:/home/hadoop/.ssh/authorized_keysscp .ssh/authorized_keys hadoop@slave2:/home/hadoop/.ssh/authorized_keysb) 修改其它机器上authorized_keys文件的权限slave1以及slave2机器上,均执行命令chmod 600 .ssh/authorized_keys
四. 验证能否通过SSH无密码连接其他机器
在每个虚拟机上,均用 ssh 其它机器的hostname 验证下,如果能正常无密码连接成功,表示ok以master机器,连接slave1,slave2机器为例:(1)切换到hadoop用户master@master:~$ su hadoop(2)测试无密码连接slave1hadoop@master:/home/master$ ssh slave1Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)* Documentation: /281 packages can be updated.193 updates are security updates.New release '16.04.3 LTS' available.Run 'do-release-upgrade' to upgrade to it.Last login: Fri Aug 11 11:02:00 from slave2(3)测试无密码连接slave2hadoop@slave1:~$ ssh slave2Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)* Documentation: /281 packages can be updated.193 updates are security updates.New release '16.04.3 LTS' available.Run 'do-release-upgrade' to upgrade to it.Last login: Fri Aug 11 11:02:11 from slave2【发现】master可以无密码连接slave1,slave2,同理,下面分别进行测试slave1,slave2~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~【重点】非常容易犯的错误配置SSH成功后,每次都要"先切换到hadoop用户"下,才可以SSH成功(1)切换到用户hadoopmaster@master:~$ su hadoop 密码: (2)SSH本机器hadoop@master:/home/master$ ssh masterWelcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)* Documentation: /New release '16.04.3 LTS' available.Run 'do-release-upgrade' to upgrade to it.Last login: Fri Aug 11 18:12:54 from masterhadoop@master:~$ (3)SSH slave1机器hadoop@master:~$ ssh slave1Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)* Documentation: /New release '16.04.3 LTS' available.Run 'do-release-upgrade' to upgrade to it.Last login: Fri Aug 11 18:16:14 from masterhadoop@slave1:~$ (3)SSH slave2机器hadoop@slave1:~$ ssh slave2Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 4.4.0-31-generic x86_64)* Documentation: /New release '16.04.3 LTS' available.Run 'do-release-upgrade' to upgrade to it.Last login: Fri Aug 11 17:35:47 from slave1hadoop@slave2:~$
[小结]
小结:该步骤非常重要,主要思路是在各节点上生成公钥、私钥,然后将公钥发放其它所有节点。RSA算法是非对称加密算法,仅公布“公钥”,只要私钥不外泄,还是不能解密的,所以安全性依然有保障。如果本步骤失败,根据我个人经验,多半是权限问题,请检查hadoop是否具有足够权限,建议将hadoop加入sudoers列表及root用户组。另外,这里也整理了一些SSH免密码设置失败的原因,请移步 ssh 免密码设置失败原因总结
Hadoop集群安装配置教程_Hadoop2.6.0_Ubuntu/CentOShttp://dblab./blog/285/
如果觉得《2.修改/etc/hosts文件 /安装SSH 配置SSH无密码登陆(全程切换到hadoop用户下操作)》对你有帮助,请点赞、收藏,并留下你的观点哦!