嗨,我想在亚马逊的EMR上运行Apache Nutch 1.2.
为此,我指定了S3的输入目录.我收到以下错误:
Fetcher: java.lang.IllegalArgumentException:
This file system object (hdfs://ip-11-202-55-144.ec2.internal:9000)
does not support access to the request path
's3n://crawlResults2/segments/20110823155002/crawl_fetch'
You possibly called FileSystem.get(conf) when you should have called
FileSystem.get(uri, conf) to obtain a file system supporting your path.
我明白之间的差别FileSystem.get(uri, conf),和FileSystem.get(conf).如果我自己写这个,我会FileSystem.get(uri, conf)尝试使用现有的Nutch代码.
我问这个问题,有人告诉我,我需要修改hadoop-site.xml,以包括以下属性:fs.default.name,fs.s3.awsAccessKeyId,fs.s3.awsSecretAccessKey.我更新了这些属性core-site.xml(hadoop-site.xml不存在),但这没有什么区别.有没有人有任何其他想法？谢谢您的帮助.