Adventures in Ubuntu and Hadoop Part 4

Let's recap Rocky and Bullwinkle's adventures with Hadoop and Ubuntu on an old desktop computer. They fairly easily installed Ubuntu, Java and Eclipse, then installed an SSH server and set the ip4 address, and then, in their greatest adventure, finally got X-Windows (and VNC) working. Now we are finally ready to install Hadoop. I plan to follow the blog post by Michael Noll and the O'Reilley Hadoop book by Tom White (esp. Appendix A). According to Michael Noll, I have more that satisfied the prereqs.

Recent Hadoop releases are here or here. Some strange domain names. As of today 0.20.203 was the latest stable release, so I grabbed it and put it in /opt/hadoop. Then, per Michael's instructions

sudo tar xzf hadoop-0.20.203.0rc1.tar.gz and sudo chown -R hduser:hadoop hadoop-0.20.203.0

Then edit /home/hduser/.bashrc (don't forget to type sudo!) to add the following lines at the end. YMMV depending on exactly where your Hadoop and Java are installed.


# Set Hadoop-related environment variables
export HADOOP_HOME=/opt/hadoop/hadoop-0.20.203.0

# Set JAVA_HOME
export JAVA_HOME=/opt/java/32/jdk1.6.0_30

# Add Hadoop bin/ directory to PATH
export PATH=$PATH:$HADOOP_HOME/bin

The Hadoop book suggests that you test if it will run by typing hadoop version. Before this will work,either re-login to run the .bashrc script, or manually do all three exports. If you forget to export JAVA_HOME, you'll see a useful, informative message

Error: JAVA_HOME is not set.

But, once you set all three, you'll see something like

mpc@mpc-desktop:/home/hduser$ hadoop version
Hadoop 0.20.203.0
Subversion http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203 -r 1099333
Compiled by oom on Wed May 4 07:57:50 PDT 2011

Wohoo! Our work is done! Well, not really, there's still a whole bunch to go, like configuring the Hadoop Distributed File System. (HDFS). But, let's declare victory for now and return to that on a later day.