写了个简单的mapreduce算法,在伪分布式中执行没问题,搭建了hadoop2.6+zookeeper3.4.5 HA环境,修改了程序的input和output路径,在HA环境中执行后没反应:
main 中的代码是这样的:
public class WordCountReNameApp {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = new Job(conf, "WordCountReNameApp");
// 运行主类
job.setJarByClass(WordCountReNameApp.class);
// 指定Mapper
job.setMapperClass(WordMapper.class);
// 指定Reducer
job.setReducerClass(WordReduce.class);
// 指定Mapper输出类型
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(LongWritable.class);
job.setOutputFormatClass(UserFileOutputFormat.class);
// 指定Reducer输出类型
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(LongWritable.class);
// 指定输入路径
// 指定输出路径
String input = "hdfs://ns1/data/word.txt";
String output = "hdfs://ns1/out";// 必须不存在
FileInputFormat.addInputPath(job, new Path(input));
FileOutputFormat.setOutputPath(job, new Path(output));
// 提交
job.waitForCompletion(true);
}
输出如下,卡着不动了:
hadoop jarmrwcrenamens1.jar 16/04/2203:06:32 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 16/04/2203:06:33 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing notperformed. Implement the Tool interface and execute your application withToolRunner to remedy this. 16/04/2203:06:34 INFO input.FileInputFormat: Total input paths to process : 1 16/04/2203:06:34 INFO mapreduce.JobSubmitter: number of splits:1 16/04/2203:06:35 INFO mapreduce.JobSubmitter: Submitting tokens for job:job_1461261077789_0002 16/04/2203:06:35 INFO impl.YarnClientImpl: Submitted applicationapplication_1461261077789_0002 16/04/2203:06:35 INFO mapreduce.Job: The url to track the job:http://yun01-nn-01:8088/proxy/application_1461261077789_0002/ 16/04/2203:06:35 INFO mapreduce.Job: Running job: job_1461261077789_0002
|