Predict with Matched Fold-Model Probabilities

Applies a fitted cross_venn_abers calibrator to new observations and returns calibrated probabilities, predicted classes or per-fold details. It needs the new probabilities from every retained fold model, as one named column per fold (binary) or one named class matrix per fold (multiclass), with the same observation in each row. Use predict.venn_abers for an Inductive calibrator, which needs only the probabilities of a single model.

Usage

# S3 method for class 'cross_venn_abers'
predict(object, newdata, ..., loss = "log", type = "response")

Arguments

object: A fitted cross_venn_abers object.
newdata: A numeric/integer matrix of finite event probabilities in [0, 1], one row per new observation and one column per corresponding fold model. Column names must be complete, non-empty, non-missing, unique, and match the fitted fold identities exactly. Any column order is accepted. A correctly named zero-row matrix is accepted. For multiclass, a list named uniquely and completely by fitted fold IDs, each element a numeric class matrix with the same number of new rows and all fitted class columns. Classes and folds are aligned by name. Zero-row matrices are accepted.
...: Must be empty; unsupported arguments are errors.
loss: Exactly "log" (default) or "brier".
type: Exactly "response" (default), "class", or "details". Supply both options by their full argument names.

Value

For binary calibration, type = "response" returns a data frame with exactly one double column, probability, containing one finite regularized event probability in [0, 1] per input row, in input row order. A zero-row input returns the same empty schema.

For type = "class", an unnamed logical vector: TRUE when the combined event probability is at least 0.5, otherwise FALSE. This uses the combined probability, not a vote among folds. Zero rows return logical(0).

For type = "details", a list with prediction (the standard data frame) and folds (a data frame with columns row, fold, lower, upper, probability). The integer row is the one-based position in this call's new data, independent of input row names. The character fold is the stored fold ID. The three double columns contain that fold's IVAP endpoints and its probability for the chosen loss. Rows are grouped by stored fold order, then new-row order. There is exactly one record per new row and fold: zero input rows give an empty fold table, and one input row gives one record per fold. These are new-observation results, not out-of-fold calibration probabilities.

For multiclass, the standard output is a double class-probability matrix in stored factor-level order, with row names from the first stored fold's matrix. Class output is a factor with those levels, choosing the highest final probability and the first level on exact ties (also for two classes). Details contain prediction plus pairs for OvO or classes for OvR. The table columns are integer row, character non_event (OvO only), character event, character fold, and double lower, upper, probability. Events are the later pair class for OvO or the literal class against the rest for OvR. Records follow pair/class order, then fold first-appearance order within that subproblem's calibration rows, then new-row order. Each row/subproblem/fold appears once. The points use the chosen loss; endpoints do not change with loss. Zero or one new row retains matrix, factor, and detail-table types and mappings.

Details

New-observation columns are matched to their fold calibrators by identity, and the resulting IVAP results are combined using the chosen equal-weight aggregation.

Character fold identities match literally. Numeric fitting IDs match their Base R character representations. Missing, extra, duplicate, empty, or absent column names fail the entire call, as do invalid probability values and unsupported binary types or shapes (including vectors, lists, data frames, logical matrices and character matrices). A column for an unobserved fold is an error. Values are never dropped, clamped or rounded. For multiclass, invalid list names, unequal row counts, or an invalid class matrix in any fold fail the whole call. The class value, name, and row-sum rules match fit time; accepted row sums within tolerance are normalized. Row names are not alignment keys.

Use each retained fold model that supplied that fold's calibration probabilities; predictions from one all-data refit cannot substitute for the matched columns. The caller ensures the same observation occupies a given row in every column. Row names do not establish that alignment, and valid column names do not prove model provenance or exclusion of calibration folds from training. See cross_venn_abers for pipeline responsibilities.

For log loss, let \(p_0^k, p_1^k\) be the IVAP endpoints from fold \(k\). If \(A\) is the geometric mean of \(1-p_0^k\) and \(B\) is the geometric mean of \(p_1^k\), the result is \(B/(A+B)\). Evaluation uses plogis(mean(log(upper)) - mean(log1p(-lower))) in log space, with equal fold weight and no endpoint clipping. It does not pool calibration rows, weight folds by size, or average log-loss fold point probabilities.

For Brier, each fold probability is upper + (lower^2 - upper^2) / 2. The combined probability is the arithmetic mean of these fold probabilities, with equal weights even for unequal fold sizes. Both aggregates can be reconstructed from type = "details". Loss changes neither the fold endpoints nor stored learning state and requires no new fit.

No aggregate endpoints are returned. The geometric pair \((1-A, B)\) can cross; it is not an ordered interval or confidence interval. Neither that pair nor the combined point inherits the IVAP perfect-calibration guarantee. There is no universal score-improvement guarantee. The output feeds the existing scores and Base R calibration plot directly. For multiclass, apply this binary merger within each pair/class first, then OvO coupling or OvR normalization as described in cross_venn_abers. Reconstruct the final matrix from these per-fold diagnostics; there is no average of already coupled/normalized multiclass fold predictions.

Examples

calibrator <- cross_venn_abers(c(0, 1, 0, 1), c(.1, .8, .2, .9), c(1, 1, 2, 2))
newdata <- cbind("1" = c(.2, .8), "2" = c(.3, .7))
predict(calibrator, newdata)
#>   probability
#> 1   0.5000000
#> 2   0.5857864
stopifnot(identical(predict(calibrator, newdata), predict(calibrator, newdata[, 2:1])))
predict(calibrator, newdata[FALSE, ])
#> [1] probability
#> <0 rows> (or 0-length row.names)
details <- predict(calibrator, newdata, loss = "brier", type = "details")
details$folds
#>   row fold lower upper probability
#> 1   1    1   0.0     1       0.500
#> 2   2    1   0.5     1       0.625
#> 3   1    2   0.0     1       0.500
#> 4   2    2   0.0     1       0.500
reconstructed <- vapply(seq_len(nrow(newdata)), function(i) {
  mean(details$folds$probability[details$folds$row == i])
}, 0)
stopifnot(max(abs(reconstructed - details$prediction$probability)) < 1e-10)
predict(calibrator, newdata, loss = "brier", type = "class")
#> [1] TRUE TRUE

Usage

Arguments

Value

Details

See also

Examples