Predict with a Venn-Abers Calibrator

Applies a fitted venn_abers calibrator to new uncalibrated probabilities from the same model and returns calibrated probabilities or predicted classes, preserving the order of newdata. For binary calibration it returns the Venn-Abers interval of two probabilities together with the regularized event probability that merges them according to loss; multiclass calibration returns a class-probability matrix. Use predict.cross_venn_abers for a Cross calibrator, which needs one probability column per retained fold model.

Usage

# S3 method for class 'venn_abers'
predict(object, newdata, ..., loss = "log", type = "response")

Arguments

object

A fitted venn_abers object.

newdata

A numeric vector of uncalibrated event probabilities with finite values in [0, 1]. numeric(0) is accepted for binary objects. For multiclass, a numeric matrix under the fit-time class, name, range, and row-sum contract; zero rows are accepted.

...

Must be empty; unsupported arguments are errors.

loss

Exactly "log" (default) or "brier".

type

Exactly "response" (default) or "class". Multiclass also accepts "details". Supply both options by their full argument names.

Value

For type = "response", a data frame with one row per value in newdata and three double columns: lower and upper are the Venn-Abers interval bounds, and probability is the regularized event probability using the chosen loss. For type = "class", an unnamed logical vector, TRUE when the final event probability is at least 0.5 and FALSE otherwise. Zero observations return the same empty data-frame schema or logical(0); one observation retains its data-frame/vector shape.

For multiclass, "response" returns a numeric class-probability matrix in fitted factor-level order, retaining new-row order and matrix row names. "class" returns an unnamed factor with the stored levels; the first stored class wins exact ties. This also applies with exactly two classes, whereas binary vector ties select the event. Classes use the final probabilities of the fitted method. venn_abers_one_hot represents binary or multiclass class decisions as a one-hot matrix. For OvO, "details" returns a list with prediction (the response matrix) and pairs, a data frame with row, non_event, event, lower, upper, and probability. Row is the integer one-based new-row position; the two class names are character columns. The remaining columns are doubles. Pairs follow factor-level combination order, then new-row order. For OvR, the list contains prediction and classes, a data frame with integer row, character event, and double lower, upper, probability. The event is the named class against all other classes; no artificial rest label is introduced. Records follow stored class order, then new-row order. Zero or one new rows retain these matrix, factor, list, and table schemas.

Details

Log-loss regularization is upper / (1 - lower + upper). Brier regularization is upper + (lower^2 - upper^2) / 2. Changing loss does not change the interval endpoints or mutate the fitted calibrator. New probabilities are never rounded, even if calibration used precision. Input names are not alignment keys. The endpoints are Venn-Abers bounds, not confidence intervals. The probability column can be passed to the existing scores and calibration plot with either loss choice.

For OvO, class and pair normalization precedes prediction, including the zero-pair replacement described in venn_abers. New pair probabilities remain unrounded. After binary regularization, for class j let q[j,i] be its probability in its pair with class i. Reciprocal coupling is w[j] = 1 / (sum(1 / q[j,i]) - (K - 2)), summed over the other K-1 classes. The response is w / sum(w). For each diagnostic record, probability is the event's q and 1 - probability is the non-event's q.

For OvR, input class normalization precedes prediction. Each class probability enters its binary calibrator unrounded. After log or Brier regularization, divide the class points by their sum for each new row to obtain the response. The classes table supplies those pre-normalization points, so details reconstruct the final matrix for either method and loss. The endpoints describe only the binary subproblems, not multiclass confidence intervals. Binary Brier regularization does not imply optimality for the final multiclass Brier score or guarantee improvement in assessment scores.

See also

vignette("method", package = "vennabers") explains the interval and the merger into one probability; vignette("manual-calibration", package = "vennabers") shows prediction in a worked example.

venn_abers, venn_abers_one_hot, brier_score, log_loss

Examples

calibrator <- venn_abers(c(0, 1, 0), c(0.1, 0.2, 0.3))
predict(calibrator, c(0.05, 0.2, 0.4))
#>       lower     upper probability
#> 1 0.0000000 0.5000000   0.3333333
#> 2 0.3333333 0.6666667   0.5000000
#> 3 0.3333333 1.0000000   0.6000000
predict(calibrator, c(0.05, 0.2, 0.4), loss = "brier")
#>       lower     upper probability
#> 1 0.0000000 0.5000000   0.3750000
#> 2 0.3333333 0.6666667   0.5000000
#> 3 0.3333333 1.0000000   0.5555556
predict(calibrator, c(0.05, 0.2, 0.4), loss = "brier", type = "class")
#> [1] FALSE  TRUE  TRUE