小编rby*_*oor的帖子

Oozie s3 as job folder

Oozie is failing with following error when workflow.xml is provided from s3, But the same worked provided workflow.xml from HDFS. Same has worked with earlier versions of oozie, Is there anything changed from 4.3 version of oozie.?

Env:

  • HDP 3.1.0
  • Oozie 4.3.1
  • oozie.service.HadoopAccessorService.supported.filesystems=*

Job.properties

nameNode=hdfs://ambari-master-1a.xdata.com:8020
jobTracker=ambari-master-2a.xdata.com:8050
queue=default
#OOZIE job details
basepath=s3a://mybucket/test/oozie
oozie.use.system.libpath=true
oozie.wf.application.path=${basepath}/jobs/test-hive?
Run Code Online (Sandbox Code Playgroud)

#(works with this change in Job.properties)

basepath=hdfs://ambari-master-1a.xdata.com:8020/test/oozie

workflow.xml

?<workflow-app xmlns="uri:oozie:workflow:0.5" name="test-hive">
    <start to="hive-query"/>
    <action name="hive-query" retry-max="2" retry-interval="10">
        <hive xmlns="uri:oozie:hive-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <script>test_hive.sql</script>
        </hive>
        <ok to="end"/>
        <error to="kill"/> …
Run Code Online (Sandbox Code Playgroud)

hadoop hive amazon-s3 bigdata oozie

1
推荐指数
1
解决办法
211
查看次数

标签 统计

amazon-s3 ×1

bigdata ×1

hadoop ×1

hive ×1

oozie ×1