Oozie:Sqoopdynamic目标目录

我从Oozie工作stream执行sqoop作业。 我可以在sqoop命令中使用静态名称创build目标目录,如下所示。

 <action name="table1" cred=""> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <command>job --exec EMPLOYEE --meta-connect jdbc:hsqldb:hsql://<host>:<port>/sqoop -- --target-dir /user/test/Employee/20150126</command> </sqoop> <ok to="end" /> <error to="kill" /> </action> 

我需要用date创builddynamic目标目录。 我尝试跟随,但不工作。

 <action name="table1" cred=""> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <command>job --exec EMPLOYEE --meta-connect jdbc:hsqldb:hsql://<host>:<port>/sqoop -- --target-dir /user/test/Employee/$(date +%Y%m%d)</command> </sqoop> <ok to="end" /> <error to="kill" /> </action> 

运行时显示以下错误。

  3622 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version: 1.4.5-cdh5.2.0 3957 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Error parsing arguments for import: 3957 [main] ERROR org.apache.sqoop.tool.BaseSqoopTool - Unrecognized argument: +%Y%m%d) Intercepting System.exit(1) 

您可以将coordina时间从coordinator.xml传递到workflow.xml。 所以在工作流程中,你可以喜欢这个

 /user/test/Employee/${timePassedFromCoordinator} 

您可以使用shell操作通过以下方法将env-var指定为日期。

“variable.sh”

 #!/bin/sh outputDir =$(date +%Y%m%d) 

Workflow.xml

 <action name='shell1'> <shell xmlns="uri:oozie:shell-action:0.1"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <exec>variable.sh</exec> (or) <env-var>[outputDir=$(date +%Y%m%d)]</env-var> </shell> <ok to="table1" /> <error to="fail" /> </action> <action name="table1" cred=""> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <command>job --exec EMPLOYEE --meta-connect jdbc:hsqldb:hsql://<host>:<port>/sqoop -- --target-dir /user/test/Employee/$(outputDir)</command> </sqoop> <ok to="end" /> <error to="kill" /> </action> 

在协调员你可以得到日期和格式所需的格式下

 <action> <workflow> <app-path>${WF_Maig_1}</app-path> <configuration> <property><name>currentbatchtime</name><value>${coord:formatTime(coord:dateOffset(coord:nominalTime(),0,'DAY'),"yyyy-MM-dd")}</value></property> <property><name>nextbatchtime</name><value>${coord:formatTime(coord:dateOffset(coord:nominalTime(),1,'DAY'),"yyyy-MM-dd")}</value></property> </configuration> </workflow> </action> 

现在你可以在workflow.xml和properties文件中使用/ user / test / Employee / $ {currentbatchtime}了

捕获输出帮助我!

 <action name='custom-var'> <shell xmlns="uri:oozie:shell-action:0.1"> ... <exec>set_variable.sh</exec> <file>set_variable.sh</file> <capture-output/> </shell> </action> <action name='sqoop-test'> <sqoop xmlns="uri:oozie:sqoop-action:0.2"> ... <command> --target-dir /test/${wf:actionData('custom-var')['var1']} --m 1 </command> </sqoop> </action> set_variable.sh echo "var1=$(date +%Y/%m/%d)"