Can a Statistics Services with open-source R job run on more than one core?
Article ID: KB0080161
Updated On:
| Products | Versions |
|---|---|
| Spotfire Statistics Services | - |
Description
Customer wants to be able to run TSSS jobs on multiple cores on their computer
Issue/Introduction
Can a Statistics Services with open-source R job run on more than one core?
Environment
Product: TIBCO Spotfire Statistics Services
Version: All supported versions
OS: All supported Operating Systems
Resolution
Unfortunately this is not available in general in Statistics Services; one job will run on one core.
When you submit a job to Statistics Services, it will launch an open-source R engine through Java. R's computation is single-threaded, so it will only use one core (from R FAQ for Windows at http://cran.r-project.org/bin/windows/base/rw-FAQ.html). An exception here is that in some support functions the BLAS libraries used may be multi-threaded, but this basically only helpful if you are using some specific linear algebra routines. Doug Bates gives some details here in a reply to this thread:
http://r.789695.n4.nabble.com/Compiling-R-with-multi-threaded-BLAS-math-libraries-why-not-actually-td2252452.html
"You need to be working with large matrices and doing very specific
kinds of operations before the time savings of multiple threads
overcomes the communications overhead. In fact, sometimes the
accelerated BLAS can slow down numerical linear algebra calculations,
such as sparse matrix operations."
The other option if you want to use multiple cores is to make use of the R packages for parallelization like snow, parallel, multicore, etc... However, the drawbacks to this are:
1. that you need to re-write your code to make use of these packages
and
2. not all problems are easily parallelized or do not gain much by parallel processing.
When you submit a job to Statistics Services, it will launch an open-source R engine through Java. R's computation is single-threaded, so it will only use one core (from R FAQ for Windows at http://cran.r-project.org/bin/windows/base/rw-FAQ.html). An exception here is that in some support functions the BLAS libraries used may be multi-threaded, but this basically only helpful if you are using some specific linear algebra routines. Doug Bates gives some details here in a reply to this thread:
http://r.789695.n4.nabble.com/Compiling-R-with-multi-threaded-BLAS-math-libraries-why-not-actually-td2252452.html
"You need to be working with large matrices and doing very specific
kinds of operations before the time savings of multiple threads
overcomes the communications overhead. In fact, sometimes the
accelerated BLAS can slow down numerical linear algebra calculations,
such as sparse matrix operations."
The other option if you want to use multiple cores is to make use of the R packages for parallelization like snow, parallel, multicore, etc... However, the drawbacks to this are:
1. that you need to re-write your code to make use of these packages
and
2. not all problems are easily parallelized or do not gain much by parallel processing.
Feedback
Yes
No
Powered by
