Update Custom Operators With New SDK Changes

Products	Versions
Spotfire Data Science	6.x

Description

How to Update Custom Operators With New SDK Changes

Resolution

When you upgrade your Tibco Spotfire Data Science installation, you will need to make a few changes to any custom operators you have developed in order for them to work correctly.

You will not have to change your code. Changing a value in the pom.xml file will trigger a download of any updates or dependency changes when you rebuild the operator.

Alpine Chorus 5.8 and Above

Navigate to the pom.xml file associated with your project.
Find the property called alpine.version.
Using the table at the bottom of this page, replace the value for alpine.version with the SDK version that matches with the version of Tibco Spotfire Data Science you are upgrading to. For example, if you are upgrading from Tibco Spotfire Data Science 6.1 to 6.3, you would put 1.7 as the alpine.version value because that is the custom operator SDK version.

<properties>
        <alpine.version>1.7</alpine.version>
</properties>

Save the project and recompile your operator(s).

Navigate to the 'Manage Custom Operators' dialog in Tibco Spotfire Data Science and delete the old JAR for your operator. We will be replacing it with the new JAR you just created.
Upload the new JAR. See How to Upload Custom Operators for more instructions, if needed. Your operator has been updated and should work with the new version of Tibco Spotfire Data Science.

Older Versions (5.6.1 - 5.7)

If you have upgraded to Tibco Spotfire Data Science 5.6.1, 5.7, or 5.7.1 and need to upgrade your custom operators, follow this process:

Navigate to the pom.xml file associated with your project.
Find the property called version within the <dependency> section. Change that value to the SDK version that corresponds to the version of Tibco Spotfire Data Science you are upgrading to. See the table below to find the proper value. For example:
```
<dependency>
    <groupId>com.alpine</groupId>
    <artifactId>plugin-core</artifactId>
    <version>1.3</version> <-- change this value to the correct SDK version
    <scope>provided</scope>
</dependency>
```
You must update the version number in all Tibco Spotfire modules. These are the modules that need to be updated.
- plugin-core
- common
- plugin-test
- plugin-spark
- plugin-model
- alpine-model-api
- alpine-model-pack
- plugin-io-impl

This is what you are looking for:

<dependency>
    <groupId>com.alpinenow</groupId>
    <artifactId>plugin-core</artifactId>
    <version>1.3</version> <-- change these values to the correct SDK version
    <scope>provided</scope>
</dependency>
 
<dependency>
    <groupId>com.alpinenow</groupId>
    <artifactId>common</artifactId>
    <version>1.3</version>
    <scope>provided</scope>
</dependency>

In addition, be sure to update the version number on your operator itself.

<groupId>alpine</groupId>
<artifactId>plugin-example</artifactId>
<version>1.3</version> <-- change this value to the correct SDK version

If a new version of Spark is being used (as in the switch from 1.3.1 to 1.5.1), you will need to update the Spark version numbers in your pom.xml file as well. See the table below for the correct Spark version.

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>1.3.1</version> <-- change all of these to the correct SDK version
    <scope>provided</scope>
</dependency>
  
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-mllib_2.10</artifactId>
    <version>1.3.1</version>
    <scope>provided</scope>
</dependency>
  
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-catalyst_2.10</artifactId>
    <version>1.3.1</version>
    <scope>provided</scope>
</dependency>
  
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.10</artifactId>
    <version>1.3.1</version>
    <scope>provided</scope>
</dependency>
  
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-hive_2.10</artifactId>
    <version>1.3.1</version>
    <scope>provided</scope>
</dependency>

Save the project and recompile your operator(s).
Navigate to the 'Manage Custom Operators' dialog in Tibco Spotfire Data Science and delete the old JAR for your operator. We will be replacing it with the new JAR you just created.
Upload the new JAR. See How to Upload Custom Operators for more instructions, if needed. Your operator has been updated and should work with the new version of Tibco Spotfire Data Science.

Migrating to 5.8+ From an Older Version

If you are currently running Tibco Spotfire Data Science 5.6.1, 5.7, or 5.7.1 and you are upgrading to 5.8 or higher, follow this process:

Navigate to the pom.xml file associated with your project.
Create a new property called alpine.version. Set the value of alpine.version to the correct SDK version that corresponds to the version of Tibco Spotfire Data Science you are upgrading to, according to the table below.
```
<properties>
    <alpine.version>1.4</alpine.version> <-- change to the correct SDK version
</properties>
```
Everywhere that you previously had <version> in your Tibco Spotfire Data Science modules, replace that value with $(alpine.version). This way, you will only need to update one value every time you upgrade an operator. These are the modules you will need to change:
- plugin-core
- common
- plugin-test
- plugin-spark
- plugin-model
- alpine-model-api
- alpine-model-pack
- plugin-io-impl

Here's an example of what that looks like:

<dependency>
    <groupId>com.alpinenow</groupId>
    <artifactId>plugin-core</artifactId>
    <version>1.3</version> <-- change these values to $(alpine.version)
    <scope>provided</scope>
</dependency>
 
<dependency>
    <groupId>com.alpinenow</groupId>
    <artifactId>common</artifactId>
    <version>1.3</version>
    <scope>provided</scope>
</dependency>

Spark dependencies are now included and do not need to be explicitly stated in the file. Thus, you can remove these lines:

<!-- Spark dependency -->
  
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>1.5.1</version>
    <scope>provided</scope>
</dependency>
  
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-mllib_2.10</artifactId>
    <version>1.5.1</version>
    <scope>provided</scope>
</dependency>
  
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-catalyst_2.10</artifactId>
    <version>1.5.1</version>
    <scope>provided</scope>
</dependency>
  
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.10</artifactId>
    <version>1.5.1</version>
    <scope>provided</scope>
</dependency>
  
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-hive_2.10</artifactId>
    <version>1.5.1</version>
    <scope>provided</scope>
</dependency>

Finally, there is a new package for testing operators written in Spark. Add this to your dependencies in the pom.xml file:
Save the project and recompile your operator(s).
Navigate to the 'Manage Custom Operators' dialog in Tibco Spotfire Data Science and delete the old JAR for your operator. We will be replacing it with the new JAR you just created.
Upload the new JAR. See How to Upload Custom Operators for more instructions, if needed. Your operator has been updated and should work with the new version of Tibco Spotfire Data Science.