0

I have installed hadoop on ubuntu which running on virtualbox. When I first installed hadoop I can start hdfs and create directories with no problem.

But after I restart the virtual machine I got "Connection refused" error when try to run ls command on the hdfs. Then I added "Port 9000" in sshd_config base on Hadoop cluster setup - java.net.ConnectException: Connection refused now I am getting "ls: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had invalid wire type.; Host Details : local host is: "hadoop/127.0.1.1"; destination host is: "localhost":9000;"

--------------etc/hadoop/core-site.xml -------------

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

--------------etc/hadoop/hdfs-site.xml-----------------

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>
Improve this question
2

1 Answer 1

Reset to default
0

Ok. I solved it. I run jps and found out the namenode failed to start. Then when I try to figure out what's wrong I keep looking at "hadoop-xxx-namenode-hadoop.out" which has pretty much no useful information. So I looked at "hadoop-xxx-namenode-hadoop.log" which clearly show the error "org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-xxx/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible."

Did more research I found by default Hadoop will put namenode and datanode file in system temp directory. And when I restart then directory got removed. So I had to do is change "hdfs-site.xml" and add "dfs.namenode.name.dir" and "dfs.namenode.data.dir" properties. Then when I try start hdfs I got "Address already in use". The port I was using was 9000 and after reboot it is binded with "tcp 0 0 *:9000 " so I changed "fs.defaultFS" in core-site.xml to use 9001 instead. Now everything can start now.

Hope this can help.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.