How does Pyspark work in TIBCO Data Science Team Studio?
Article ID: KB0074243
Updated On:
| Products | Versions |
|---|---|
| Spotfire Data Science | 6.5.0 |
Description
How does Pyspark work in TIBCO Data Science Team Studio?
Issue/Introduction
How does Pyspark work in TIBCO Data Science Team Studio?
Environment
Operating system: Linux
Resolution
Pyspark can be initialized in the python notebooks by clicking the button "Initialize Pyspark for Cluster". A code is automatically generated by Team Studio to initialize spark context. This code will run on the Hadoop cluster as the python notebooks server doesn't understand spark code.
NOTE: Make sure that there are no connection and name to Ip address resolution issues between the Python Notebooks server and the Hadoop cluster.
NOTE: Make sure that there are no connection and name to Ip address resolution issues between the Python Notebooks server and the Hadoop cluster.
Feedback
Yes
No
Powered by
