deepluq.metrics_dl

deepluq.metrics_dl#

Classes#

DLMetrics

A class to compute various Uncertainty Quantification (UQ) metrics for Deep Learning,

Module Contents#

class deepluq.metrics_dl.DLMetrics#

A class to compute various Uncertainty Quantification (UQ) metrics for Deep Learning, including variation ratio, entropy, mutual information, total variance, and prediction surface using convex hulls.

variation_ratio = 0.0#
shannon_entropy = 0.0#
mutual_information = 0.0#
total_var_center_point = 0.0#
total_var_bounding_box = 0.0#
prediction_surface = -1.0#
hull = []#
box = []#
cal_vr(events)#

Compute the Variation Ratio (VR). Measures the proportion of non-modal class predictions.

Parameters:

events (array-like) – Model outputs or predictions.

Returns:

Variation ratio.

Return type:

float

calcu_entropy(events, eps=1e-15, base=2)#

Compute Shannon entropy of probabilities.

Parameters:

  • events (array-like) – Probability distribution.

  • eps (float) – Small constant to avoid log(0).

  • base (int) – Logarithm base.

Returns:

Shannon entropy (rounded to 5 decimals).

Return type:

float

calcu_mi(events, eps=1e-15, base=2)#

Compute Mutual Information (MI) between predictions.

Parameters:

  • events (array-like) – Model probability outputs.

  • eps (float) – Small constant to avoid log(0).

  • base (int) – Logarithm base.

Returns:

Mutual information.

Return type:

float

calcu_tv(matrix, tag)#

Compute total variance of a multi-dimensional matrix using covariance.

Parameters:

  • matrix (array-like) – Input data matrix.

  • tag (str) – Either ‘bounding_box’ or ‘center_point’.

Returns:

Total variance.

Return type:

float

calcu_mutual_information(X, Y, Z)#

Compute mutual information between three discrete random variables X, Y, and Z.

Reference:

http://www.scholarpedia.org/article/Mutual_information

Parameters:

  • X (array-like) – Discrete random variables of shape (n_samples,).

  • Y (array-like) – Discrete random variables of shape (n_samples,).

  • Z (array-like) – Discrete random variables of shape (n_samples,).

Returns:

Mutual information.

Return type:

float

calcu_prediction_surface(boxes)#

Compute prediction surface by calculating convex hull areas from bounding box corners.

Parameters:

boxes (array-like) – List of bounding boxes [x1, y1, x2, y2].

Returns:

Prediction surface area (sum of convex hulls).

Return type:

float

Contents