Initializing the pyspark on Jupyter Python Notebooks in Team Studio fails with an error
Article ID: KB0076963
Updated On:
| Products | Versions |
|---|---|
| Spotfire Data Science Team Studio | 6.5.0 |
Description
Initializing the pyspark on Jupyter Python Notebooks in Team Studio fails with the following error:
===============================================An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext: java.lang.IllegalStateException: Spark context stopped while waiting for backend at org.apache.spark.scheduler.TaskSchedulerImpl.waitBackendReady(TaskSchedulerImpl.scala:818)
================================================
Issue/Introduction
Initializing the pyspark on Jupyter Python Notebooks in Team Studio fails with a very generic java related error
Environment
Linux
Resolution
Make sure that the java version on the Cloudera datasource is greater than 1.7. By default, the Cloudera datasource is shipped with Java 1.7 so we have to manually update the java version to 1.8 or higher.
NOTE: TIBCO Data Science Team Studio supports Oracle jdk and doesn't support Open jdk yet.
NOTE: TIBCO Data Science Team Studio supports Oracle jdk and doesn't support Open jdk yet.
Feedback
Yes
No
Powered by
