zoomerGP provides a fast and composable language to write, fit, and
explore Gaussian Process models in R. The package allows users to
compose a Gaussian process kernel using a consise formula syntax, and
tune hyperparameters through type-2 ML or Variational Inference (SVGD).
Both Gaussian and non-Gaussian Likelihoods are supported.
ZoomerGP’s flagship feature is a tidy formula interface that allows
users to cleanly specify complex Gaussian-process models. As an example,
you can specify and fit a Gaussian Process with an RBF covariance
function to a dataset with the following formula:
gaussian_process(y ~ rbf(x), data = data)
This syntax also allows you to arbitrarily combine different kernels buy adding and multiplying them. As an example, one can try to fit a periodic trend that changes over time and a long-term trend to the data by using the following code:
gaussian_process(y ~ periodic(x) * rbf(x) + rbf(x), data = data)
Each kernel can also act on multiple variables, as shown below:
gaussian_process(a ~ periodic(b,c) * rbf(d,e,f,g) + rbf(h), data = data)
At present the following kernels are implemented:
| Kernel | Function Name | Notes |
|---|---|---|
| Squared Exponential | rbf() | Results in a smooth, infinitely-differentiable posterior. Too smooth for most applications |
| Spectral Mixture | spectral0-5 | Spectal mixture kernel with 1,2,3,4, or 5 components. Can recover any stationary covariance function but is very expensive to fit. |
| Linear | linear() | Recovers a linear trend |
| Indicator | indicator() | Equal to $sigma^2$ if $x=x'$, zero otherwise. Useful when accounting for clustering. |
| If / mask | mask() | Equal of one if both $x$ and $x'$ are equal to one, zero otherwise. Most useful when ‘masking’ out another kernel so it only acts on certain pairs of inputs |
| Periodic | periodic() | Periodic similarity function. Can accept as a keyword argument a vector of periods associated with each dimension. |
Gaussian Process Regression are computationally intensive, and generally
take $O(n^3)$ computations to fit. To allow the method to scale to large
datasets, this package implements the projected process approximation as
described in Rassmusen and Williams’s (2005) Gaussian Processes for
Machine Learning. To turn this approximation on, use the sparse=T
argument when fitting the gaussian processes please be aware that you
may also need to change the default value of the number of inducing
points (n_points) for numerical stability.
Currently, we the package supports 3 engines for hyperparameter
optimization. bfgs is selected by default, but these can be changed
using the training_method argument to gaussian_process function
bfgs - as implimented in the optim package in R. Will sometimes fail
to coverge, leading to an error.rgenoud - as implimented in the rgenoud package. This is likely
the most robust optimizer as it avoids getting stuck in local minima,
but also the most computationally intensive.coin - a learning-rate free algorithm that is implimented in Rust.
Seems to work well across a variety of problems, but can get stuck in
local minima.zoomerGP also implements a variant of the Stein Variational Gaussian
Processes algorithm to perform
variational inference over hyperparameters and latent function values
when the likelihood is non-Gaussian. Specifically, we use the Coin
Sampling algorithm proposed by the
same authors to provide a robust Variational Inference engine that does
not require tuning by the user.
At the moment, inference for the following non-Gaussian likelihoods are implemented:
the logistic_gaussian_process function).If your operating system or version of R is not installed, you must have the Rust compiler installed to compile this package from sources. After the package is compiled, Rust is no longer required, and can be safely uninstalled.
To install Rust on Linux or Mac, you can simply run the following snippet in your terminal.
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
To install Rust on windows, you can use the Rust installation wizard,
rustup-init.exe, found at this
site.
Depending on your version of Windows, you may see an error that looks
something like this:
error: toolchain 'stable-x86_64-pc-windows-gnu' is not installed
In this case, you should run
rustup install stable-x86_64-pc_windows-gnu to install the missing
toolchain. If you’re missing another toolchain, simply type this in the
place of stable-x86_64-pc_windows-gnu in the command above.
Once you have rust installed Rust, you should be able to install the
package with either the install.packages function as above, or using the
install_github function from the devtools package or with the
pkg_install function from the pak package.
## Install with devtools
# install.packages("devtools")
devtools::install_github("beniaminogreen/GPR")
## Install with pak
# install.packages("pak")
pak::pkg_install("beniaminogreen/GPR")
Once the package is installed, you can load it into memory as usual by typing:
library(zoomerGP)