hadoop - JobTracker doesn't show completed tasks

Question

I am running hadoop-1.1.2 on my laptop in pseudo-distributed mode. I am able to run a simple WordCount program, reading from and writing back to HDFS. I am also able to see JobTracker running at http://localhost:50030/jobtracker.jsp. However when I run the WordCount job from Eclipse, there is no entry, either under running or completed jobs.

Am I missing any additional property setting in one of the configuration files?

Thanks.

Answer 1

发生这种情况是因为当您通过 Eclipse 运行作业时，它开始在自身内部运行作业，而不是将其提交给 JobTracker，因为它不知道去哪里找到 JobTracker。您需要将其告知 Eclipse。在您的代码中添加以下行，它应该可以工作：

Configuration conf = new Configuration();
conf.set("fs.default.name", "hdfs://localhost:9000");
conf.set("mapred.job.tracker", "localhost:9001");

Answer 2

将您的 hdfs-site.xml 和 mapred-site.xml 复制到您的“src”文件夹中（使它们在类路径中可用）。它将拾取所有配置。