文章目录
摘要情况1:JSON解析异常情况2:java.lang.InstantiationException spark.sql.driver情况3: 中kafka:java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Callback情况4:idea启动报错:Connection to node -1 could not be established. Broker may not be available情况5:中kafka: Caused by: java.nio.channels.UnresolvedAddressException master:8080摘要
情况1:JSON解析异常情况2:java.lang.InstantiationException spark.sql.driver情况3 中kafka:java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Callback情况4 idea启动报错:Connection to node -1 could not be established. Broker may not be available情况5中kafka: Caused by: java.nio.channels.UnresolvedAddressException master:8080情况1:JSON解析异常
出错原因:spark命令提交参数json,到另一个jar发现{{或者}}消失了,导致解析异常
解决方案:/u010814849/article/details/78752074 双括号间+空格
情况2:java.lang.InstantiationException spark.sql.driver
出错原因:spark操作mysql数据库缺少驱动
解决方案:
Properties对象设置props.put(“driver”, “com.mysql.jdbc.Driver”);
情况3: 中kafka:java.lang.NoClassDefFoundError: org/apache/kafka/clients/producer/Callback
出错原因:运行期缺少jar包,问题出在maven程序打包没把依赖打进去
解决方案:添加打包插件指定打入依赖jar
<plugin><artifactId>maven-assembly-plugin</artifactId><configuration><descriptorRefs><descriptorRef>jar-with-dependencies</descriptorRef></descriptorRefs></configuration><executions><execution><id>make-assembly</id><phase>package</phase><goals><goal>single</goal></goals></execution></executions></plugin>
情况4:idea启动报错:Connection to node -1 could not be established. Broker may not be available
出错原因:指定bootstrap-servers前面多了空格,导致层级目录出错
解决方案:
情况5:中kafka: Caused by: java.nio.channels.UnresolvedAddressException master:8080
出错原因:ip映射没修改对,导致不认识master
解决方案:如果是ambari安装的kafka修改cinfig下面的,如果是自己linux搭建的,需改动kafka下的cinfig下的server.properties,把PLAINTEXT://localhost:6667 -》 改为PLAINTEXT://192.168.20.91:6667
如果觉得《Linux环境下 java程序提交spark任务到Yarn报错》对你有帮助,请点赞、收藏,并留下你的观点哦!