如何解决Zeppelin 0.9 在 YARN 客户端模式下运行 Spark 笔记本，而不是 YARN 集群？

我刚刚设置了 Zeppelin（版本 0.9）以与 Hadoop (3.3.0) + Spark (3.1.2) 一起运行。我正在尝试运行 Zeppelin 网站中给出的示例代码：

val bankText = sc.textFile("file:///path/to/bank/bank-full.csv")

case class Bank(age:Integer,job:String,marital : String,education : String,balance : Integer)

// split each line,filter out header (starts with "age"),and map it into Bank case class
val bank = bankText.map(s=>s.split(";")).filter(s=>s(0)!="\"age\"").map(
    s=>Bank(s(0).toInt,s(1).replaceAll("\"",""),s(2).replaceAll("\"",s(3).replaceAll("\"",s(5).replaceAll("\"","").toInt
        )
)

// convert to DataFrame and create temporal table
bank.toDF().registerTempTable("bank")

然后：

%sql select age,count(1) from bank where age < 30 group by age order by age

当我在 YARN 客户端模式下运行它时，它工作正常并产生预期的输出。但是当我尝试在 YARN 集群模式下运行时，它给了我以下错误：

 INFO [2021-07-23 06:05:49,623] ({SchedulerFactory6} ProcessLauncher.java[transition]:109) - Process state is transitioned to LAUNCHED
 INFO [2021-07-23 06:05:49,624] ({SchedulerFactory6} ProcessLauncher.java[launch]:96) - Process is launched: [/opt/zeppelin/bin/interpreter.sh,-d,/opt/zeppelin/interpreter/spark,-c,10.0.15.21,-p,37921,-r,:,-i,spark-shared_process,-l,/opt/zeppelin/local-repo/spark,-g,spark]
 WARN [2021-07-23 06:05:59,666] ({Exec Default Executor} RemoteInterpreterManagedProcess.java[onProcessComplete]:255) - Process is exited with exit value 0
 INFO [2021-07-23 06:05:59,667] ({Exec Default Executor} ProcessLauncher.java[transition]:109) - Process state is transitioned to COMPLETED
 INFO [2021-07-23 06:06:06,192] ({pool-7-thread-5} RemoteInterpreterEventServer.java[registerInterpreterProcess]:176) - Register interpreter process: 10.0.15.21:43289,interpreterGroup: spark-shared_process
 INFO [2021-07-23 06:06:06,193] ({pool-7-thread-5} ProcessLauncher.java[transition]:109) - Process state is transitioned to RUNNING
 INFO [2021-07-23 06:06:06,193] ({SchedulerFactory6} RemoteInterpreterManagedProcess.java[start]:132) - Detected yarn app: application_1627000352599_0011,add it to YarnAppMonitor
ERROR [2021-07-23 06:06:06,194] ({SchedulerFactory6} Job.java[run]:174) - Job Failed
java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess.start(RemoteInterpreterManagedProcess.java:133)
    at org.apache.zeppelin.interpreter.ManagedInterpreterGroup.getorCreateInterpreterProcess(ManagedInterpreterGroup.java:68)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getorCreateInterpreterProcess(RemoteInterpreter.java:104)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:154)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:271)
    at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:444)
    at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:72)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:172)
    at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:132)
    at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:182)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.classNotFoundException: org.apache.hadoop.conf.Configuration
    at java.net.urlclassloader.findClass(urlclassloader.java:382)
    at java.lang.classLoader.loadClass(ClassLoader.java:418)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.classLoader.loadClass(ClassLoader.java:351)
    ... 18 more
ERROR [2021-07-23 06:06:06,195] ({SchedulerFactory6} NotebookServer.java[onStatusChange]:1920) - Error
java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreterManagedProcess.start(RemoteInterpreterManagedProcess.java:133)
    at org.apache.zeppelin.interpreter.ManagedInterpreterGroup.getorCreateInterpreterProcess(ManagedInterpreterGroup.java:68)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getorCreateInterpreterProcess(RemoteInterpreter.java:104)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:154)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:126)
    at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:271)
    at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:444)
    at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:72)
    at org.apache.zeppelin.scheduler.Job.run(Job.java:172)
    at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:132)
    at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:182)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.classNotFoundException: org.apache.hadoop.conf.Configuration
    at java.net.urlclassloader.findClass(urlclassloader.java:382)
    at java.lang.classLoader.loadClass(ClassLoader.java:418)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.classLoader.loadClass(ClassLoader.java:351)

我真的不知道该怎么做。我尝试通过 hadoop classpath 获取 Hadoop 类路径并返回：

/opt/hadoop/etc/hadoop:/opt/hadoop/share/hadoop/common/lib/*:/opt/hadoop/share/hadoop/common/*:/opt/hadoop/share/hadoop/hdfs:/opt/hadoop/share/hadoop/hdfs/lib/*:/opt/hadoop/share/hadoop/hdfs/*:/opt/hadoop/share/hadoop/mapreduce/*:/opt/hadoop/share/hadoop/yarn:/opt/hadoop/share/hadoop/yarn/lib/*:/opt/hadoop/share/hadoop/yarn/*

然后我将其添加到 spark-defaults.conf 和 spark.driver.extraClasspath 中的 spark.executor.extraClasspath，因为 org.apache.hadoop.conf.Configuration 在 hadoop-commons-3.3.0.jar 内。我还确保在 zeppelin-env.sh 中包含适当的信息：

export HADOOP_HOME=/opt/hadoop
export HADOOP_CONF_DIR=/opt/hadoop/etc/hadoop/
export USE_HADOOP=true
export SPARK_HOME=/opt/spark

但由于某种原因，我似乎无法通过 Zeppelin 以 YARN 集群模式进行部署（如果我自己使用 spark-submit，我可以。）

注意：我禁用了 zeppelin.spark.enableSupportedVersionCheck，因为我使用的是由 Zeppelin 官方支持的 Spark 3.1.2 与 Spark 3.0 版本。

解决方法

暂无找到可以解决该程序问题的有效方法，小编努力寻找整理中！

如果你已经找到好的解决方法，欢迎将解决方案带上本链接一起发送给小编。

小编邮箱:dio#foxmail.com (将#修改为@）

case class People(name: String,age: Int)
var df = spark.createDataFrame(List(People("jeff",23),People("andy",20)))
df.createOrReplaceTempView("people")

%sql
select * from people

Table or view not found: people; line 2 pos 14;
''Project [*]
+- ''UnresolvedRelation [people]