Alpine mapreduce operators fail

Alpine mapreduce operators fail

Problem

When running a mapreduce based Alpine operator, the job gets submitted to the Hadoop cluster, but it fails with a generic error message (see the attached screenshot).

Resolution

If all the datasource parameters are configured correctly and the Alpine required HDFS directories are created (with the correct ownership and permissions), sometimes adding these 2 parameters to the datasource helps to resolve this issue:

mapreduce.application.classpath
yarn.application.classpath

In order to get the value of "mapreduce.application.classpath" parameter, go to the command line of a hadoop node and run "hadoop classpath". 

In order to get the value of "yarn.application.classpath" parameter, go to the command line of a hadoop node and run "yarn classpath". 

Then add these parameters with their respective values to the datasource, refresh the flow and try running the same operators to see if they succeed.