Cross-Validation Function for Scaled Matrix Frobenius Loss

cvScaledMatrixFrobeniusLoss() evaluates the scaled matrix Frobenius loss over a fold object (from 'origami' (Coyle and Hejazi 2018) ). The squared error loss computed for each entry of the estimated covariance matrix is scaled by the training set's sample variances of the variable associated with that entry's row and column variables. This loss should be used instead of cvMatrixFrobeniusLoss() when a dataset's variables' values are of different magnitudes.

cvScaledMatrixFrobeniusLoss(fold, dat, estimator_funs, estimator_params = NULL)

Arguments

fold

A fold object (from make_folds()) over which the estimation procedure is to be performed.

dat

A data.frame containing the full (non-sample-split) data, on which the cross-validated procedure is performed.

estimator_funs

An expression corresponding to a vector of covariance matrix estimator functions to be applied to the training data.

estimator_params

A named list of arguments corresponding to the hyperparameters of covariance matrix estimators, estimator_funs. The name of each list element should be the name of an estimator passed to estimator_funs. Each element of the estimator_params is itself a named list, with names corresponding to an estimators' hyperparameter(s). These hyperparameters may be in the form of a single numeric or a numeric vector. If no hyperparameter is needed for a given estimator, then the estimator need not be listed.

Value

A tibble providing information on estimators, their hyperparameters (if any), and their scaled matrix Frobenius loss evaluated on a given fold.

References

Coyle J, Hejazi N (2018). “origami: A Generalized Framework for Cross-Validation in R.” Journal of Open Source Software, 3(21), 512. doi: 10.21105/joss.00512 .

Examples

library(MASS)
library(origami)
library(rlang)

# generate 10x10 covariance matrix with unit variances and off-diagonal
# elements equal to 0.5
Sigma <- matrix(0.5, nrow = 10, ncol = 10) + diag(0.5, nrow = 10)

# sample 50 observations from multivariate normal with mean = 0, var = Sigma
dat <- mvrnorm(n = 50, mu = rep(0, 10), Sigma = Sigma)

# generate a single fold using MC-cv
resub <- make_folds(dat,
  fold_fun = folds_vfold,
  V = 2
)[[1]]
cvScaledMatrixFrobeniusLoss(
  fold = resub,
  dat = dat,
  estimator_funs = rlang::quo(c(
    linearShrinkEst, thresholdingEst, sampleCovEst
  )),
  estimator_params = list(
    linearShrinkEst = list(alpha = c(0, 1)),
    thresholdingEst = list(gamma = c(0, 1))
  )
)
#> [[1]]
#> # A tibble: 5 × 4
#>   estimator       hyperparameters       loss  fold
#>   <chr>           <chr>                <dbl> <int>
#> 1 linearShrinkEst alpha = 0             32.4     1
#> 2 linearShrinkEst alpha = 1             10.4     1
#> 3 thresholdingEst gamma = 0             10.4     1
#> 4 thresholdingEst gamma = 1             44.1     1
#> 5 sampleCovEst    hyperparameters = NA  10.4     1
#>