我正在尝试在Amazon Elastic Mapreduce中运行Hadoop作业.我的数据和jar位于aws s3中.当我设置作业流程时,我将JAR参数传递为
s3n://my-hadoop/input s3n://my-hadoop/output
Run Code Online (Sandbox Code Playgroud)
下面是我的hadoop主要功能
public static void main(String[] args) throws Exception
{
Configuration conf = new Configuration();
Job job = new Job(conf, "MyMR");
job.setJarByClass(MyMR.class);
job.setMapperClass(MyMapper.class);
job.setReducerClass(CountryReducer.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(Text.class);
job.setInputFormatClass(TextInputFormat.class);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
Run Code Online (Sandbox Code Playgroud)
但是,我的作业流程失败,并在stderr中显示以下日志
Exception in thread "main" java.lang.ClassNotFoundException: s3n://my-hadoop/input
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at org.apache.hadoop.util.RunJar.main(RunJar.java:180)
Run Code Online (Sandbox Code Playgroud)
那么如何在aws emr中指定输入和输出路径?