小编cev*_*ira的帖子

我在分析日志文件的数据管道中运行EMR活动,当我的管道发生故障时,我收到以下错误:

Exception in thread "main" org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://10.208.42.127:9000/home/hadoop/temp-output-s3copy already exists
    at org.apache.hadoop.mapred.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:121)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:944)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:905)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:905)
    at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:879)
    at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1316)
    at com.valtira.datapipeline.stream.CloudFrontStreamLogProcessors.main(CloudFrontStreamLogProcessors.java:216)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:187)

如何从Hadoop中删除该文件夹？

我试图从日志文件中获取域名(http://www.example.com),如下所示:

http://maps.google.com/maps?hl=en&tab=wl
http://l.macys.com/simi-valley-ca?cm_mmc=macys_     
https://www.google.co.in/       
https://www.google.ca/  

我想要的

http://maps.google.com/   
http://l.macys.com/    
https://www.google.co.in/   
https://www.google.ca/  

有任何想法吗？