Local模式下Spark程序只输出关键信息

使用spark-submit提交local任务时，会输出很多Info信息：

-------------------------------------------Time: 1495788633000 ms-------------------------------------------17/05/26 08:50:33 INFO scheduler.JobScheduler: Finished job streaming job 1495788633000 ms.2 from job set of time 1495788633000 ms17/05/26 08:50:33 INFO scheduler.JobScheduler: Total delay: 0.054 s for time 1495788633000 ms (execution: 0.046 s)17/05/26 08:50:33 INFO rdd.MapPartitionsRDD: Removing RDD 177 from persistence list17/05/26 08:50:33 INFO storage.BlockManager: Removing RDD 17717/05/26 08:50:33 INFO rdd.MapPartitionsRDD: Removing RDD 176 from persistence list17/05/26 08:50:33 INFO storage.BlockManager: Removing RDD 17617/05/26 08:50:33 INFO rdd.BlockRDD: Removing RDD 175 from persistence list17/05/26 08:50:33 INFO dstream.SocketInputDStream: Removing blocks of RDD BlockRDD[175] at socketTextStream at CoGroupTest.scala:21 of time 1495788633000 ms17/05/26 08:50:33 INFO storage.BlockManager: Removing RDD 17517/05/26 08:50:33 INFO rdd.MapPartitionsRDD: Removing RDD 178 from persistence list17/05/26 08:50:33 INFO storage.BlockManager: Removing RDD 17817/05/26 08:50:33 INFO rdd.MapPartitionsRDD: Removing RDD 180 from persistence list17/05/26 08:50:33 INFO storage.BlockManager: Removing RDD 18017/05/26 08:50:33 INFO scheduler.ReceivedBlockTracker: Deleting batches: 1495788631000 ms17/05/26 08:50:33 INFO scheduler.InputInfoTracker: remove old batch metadata: 1495788631000 ms17/05/26 08:50:34 INFO scheduler.JobScheduler: Added jobs for time 1495788634000 ms17/05/26 08:50:34 INFO scheduler.JobScheduler: Starting job streaming job 1495788634000 ms.0 from job set of time 1495788634000 ms

可以修改log4j的日志级别，只输出关键信息：

1.修改$SPARK_HOME/conf/log4j.properties

如果有log4j.properties.template，则复制一份为log4j.properties

cplog4j.properties.templatelog4j.properties

2.修改第一行

log4j.rootCategory=INFO, console

为

log4j.rootCategory=ERROR, console

3.再次提交任务，只会输出关键信息：

-------------------------------------------Time: 1495788788000 ms-------------------------------------------(helloworld,helloworld_one)(hello,hello_one)(join,join_one)

如果觉得《Local模式下Spark程序只输出关键信息》对你有帮助，请点赞、收藏，并留下你的观点哦！