quapy.method package

Submodules

quapy.method.aggregative module

class quapy.method.aggregative.ACC(classifier: BaseEstimator = None, fit_classifier=True, val_split=5, solver: Literal['minimize', 'exact', 'exact-raise', 'exact-cc'] = 'minimize', method: Literal['inversion', 'invariant-ratio'] = 'inversion', norm: Literal['clip', 'mapsimplex', 'condsoftmax'] = 'clip', n_jobs=None)

Bases: AggregativeCrispQuantifier

Adjusted Classify & Count (ACC), the “adjusted” variant of CC that corrects the predictions of CC according to the misclassification rates, originally proposed in Forman, G. (2008). Quantifying counts and costs via classification. Data Mining and Knowledge Discovery, 17, 164-206.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

• method (str) –

  adjustment method to be used:

  • ’inversion’: matrix inversion method based on the matrix equality \(P(C)=P(C|Y)P(Y)\), which tries to invert \(P(C|Y)\) matrix.

  • ’invariant-ratio’: invariant ratio estimator of Vaz et al. 2018, which replaces the last equation with the normalization condition.

• solver (str) –

  indicates the method to use for solving the system of linear equations. Valid options are:

  • ’exact-raise’: tries to solve the system using matrix inversion. Raises an error if the matrix has rank strictly less than n_classes.

  • ’exact-cc’: if the matrix is not of full rank, returns p_c as the estimates, which corresponds to no adjustment (i.e., the classify and count method. See quapy.method.aggregative.CC)

  • ’exact’: deprecated, defaults to ‘exact-cc’

  • ’minimize’: minimizes the L2 norm of \(|Ax-B|\). This one generally works better, and is the default parameter. More details about this can be consulted in Bunse, M. “On Multi-Class Extensions of Adjusted Classify and Count”, on proceedings of the 2nd International Workshop on Learning to Quantify: Methods and Applications (LQ 2022), ECML/PKDD 2022, Grenoble (France).

• norm (str) –

  the method to use for normalization.

  • clip, the values are clipped to the range [0,1] and then L1-normalized.

  • mapsimplex projects vectors onto the probability simplex. This implementation relies on Mathieu Blondel’s projection_simplex_sort

  • condsoftmax, applies a softmax normalization only to prevalence vectors that lie outside the simplex

• n_jobs – number of parallel workers

METHODS = ['inversion', 'invariant-ratio']

NORMALIZATIONS = ['clip', 'mapsimplex', 'condsoftmax', None]

SOLVERS = ['exact', 'minimize', 'exact-raise', 'exact-cc']

aggregate(classif_predictions)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Estimates the misclassification rates. :param classif_predictions: array-like with the predicted labels :param labels: array-like with the true labels associated to each predicted label

classmethod getPteCondEstim(classes, y, y_)

Estimate the matrix with entry (i,j) being the estimate of P(hat_yi|yj), that is, the probability that a document that belongs to yj ends up being classified as belonging to yi

Parameters:

• classes – array-like with the class names

• y – array-like with the true labels

• y – array-like with the estimated labels

Returns:

np.ndarray

classmethod newInvariantRatioEstimation(classifier: BaseEstimator, fit_classifier=True, val_split=5, n_jobs=None)

Constructs a quantifier that implements the Invariant Ratio Estimator of Vaz et al. 2018. This amounts to setting method to ‘invariant-ratio’ and clipping to ‘project’.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

• n_jobs – number of parallel workers

Returns:

an instance of ACC configured so that it implements the Invariant Ratio Estimator

quapy.method.aggregative.AdjustedClassifyAndCount

alias of ACC

class quapy.method.aggregative.AggregativeCrispQuantifier(classifier: None | BaseEstimator, fit_classifier: bool = True, val_split: int | float | tuple | None = 5)

Bases: AggregativeQuantifier, ABC

Abstract class for quantification methods that base their estimations on the aggregation of crisp decisions as returned by a hard classifier. Aggregative crisp quantifiers thus extend Aggregative Quantifiers by implementing specifications about crisp predictions.

class quapy.method.aggregative.AggregativeMedianEstimator(base_quantifier: AggregativeQuantifier, param_grid: dict, random_state=None, n_jobs=None)

Bases: BinaryQuantifier

This method is a meta-quantifier that returns, as the estimated class prevalence values, the median of the estimation returned by differently (hyper)parameterized base quantifiers. The median of unit-vectors is only guaranteed to be a unit-vector for n=2 dimensions, i.e., in cases of binary quantification.

Parameters:

• base_quantifier – the base, binary quantifier

• random_state – a seed to be set before fitting any base quantifier (default None)

• param_grid – the grid or parameters towards which the median will be computed

• n_jobs – number of parallel workers

fit(X, y)

Generates a quantifier.

Parameters:

• X – array-like, the training instances

• y – array-like, the labels

Returns:

self

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

predict(instances)

Generate class prevalence estimates for the sample’s instances

Parameters:

X – array-like, the test instances

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

set_predict_request(*, instances: bool | None | str = '$UNCHANGED$') → AggregativeMedianEstimator

Configure whether metadata should be requested to be passed to the predict method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

instances (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for instances parameter in predict.

Returns:

self – The updated object.

Return type:

object

class quapy.method.aggregative.AggregativeQuantifier(classifier: None | BaseEstimator, fit_classifier: bool = True, val_split: int | float | tuple | None = 5)

Bases: BaseQuantifier, ABC

Abstract class for quantification methods that base their estimations on the aggregation of classification results. Aggregative quantifiers implement a pipeline that consists of generating classification predictions and aggregating them. For this reason, the training phase is implemented by classification_fit() followed by aggregation_fit(), while the testing phase is implemented by classify() followed by aggregate(). Subclasses of this abstract class must provide implementations for these methods. Aggregative quantifiers also maintain a classifier attribute.

The method fit() comes with a default implementation based on classification_fit() and aggregation_fit().

The method quantify() comes with a default implementation based on classify() and aggregate().

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation. Set to None when the method does not require any validation data, in order to avoid that some portion of the training data be wasted.

abstractmethod aggregate(classif_predictions: ndarray)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

abstractmethod aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

property classes_

Class labels, in the same order in which class prevalence values are to be computed. This default implementation actually returns the class labels of the learner.

Returns:

array-like, the class labels

property classifier

Gives access to the classifier

Returns:

the classifier (typically an sklearn’s Estimator)

classifier_fit_predict(X, y)

Trains the classifier if requested (fit_classifier=True) and generate the necessary predictions to train the aggregation function.

Parameters:

• X – array-like of shape (n_samples, n_features), the training instances

• y – array-like of shape (n_samples,), the labels

classify(X)

Provides the label predictions for the given instances. The predictions should respect the format expected by aggregate(), e.g., posterior probabilities for probabilistic quantifiers, or crisp predictions for non-probabilistic quantifiers. The default one is “decision_function”.

Parameters:

X – array-like of shape (n_samples, n_features), the data instances

Returns:

np.ndarray of shape (n_instances,) with classifier predictions

fit(X, y)

Trains the aggregative quantifier. This comes down to training a classifier (if requested) and an aggregation function.

Parameters:

• X – array-like of shape (n_samples, n_features), the training instances

• y – array-like of shape (n_samples,), the labels

Returns:

self

predict(X)

Generate class prevalence estimates for the sample’s instances by aggregating the label predictions generated by the classifier.

Parameters:

X – array-like of shape (n_samples, n_features), the data instances

Returns:

np.ndarray of shape (n_classes) with class prevalence estimates.

class quapy.method.aggregative.AggregativeSoftQuantifier(classifier: None | BaseEstimator, fit_classifier: bool = True, val_split: int | float | tuple | None = 5)

Bases: AggregativeQuantifier, ABC

Abstract class for quantification methods that base their estimations on the aggregation of posterior probabilities as returned by a probabilistic classifier. Aggregative soft quantifiers thus extend Aggregative Quantifiers by implementing specifications about soft predictions.

class quapy.method.aggregative.BinaryAggregativeQuantifier(classifier: None | BaseEstimator, fit_classifier: bool = True, val_split: int | float | tuple | None = 5)

Bases: AggregativeQuantifier, BinaryQuantifier

fit(X, y)

Trains the aggregative quantifier. This comes down to training a classifier (if requested) and an aggregation function.

Parameters:

• X – array-like of shape (n_samples, n_features), the training instances

• y – array-like of shape (n_samples,), the labels

Returns:

self

property neg_label

property pos_label

class quapy.method.aggregative.CC(classifier: BaseEstimator = None, fit_classifier: bool = True)

Bases: AggregativeCrispQuantifier

Classify & Count (CC), the most basic quantification method, one that simply classifies all instances and counts how many have been attributed to each class in order to compute class prevalence estimates. This baseline is the unadjusted estimator discussed, among others, in Forman, G. (2008). Quantifying counts and costs via classification. Data Mining and Knowledge Discovery, 17, 164-206.

Parameters:

classifier – a sklearn’s Estimator that generates a classifier

aggregate(classif_predictions: ndarray)

Computes class prevalence estimates by counting the prevalence of each of the predicted labels.

Parameters:

classif_predictions – array-like with classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Nothing to do here!

Parameters:

• classif_predictions – unused

• labels – unused

quapy.method.aggregative.ClassifyAndCount

alias of CC

class quapy.method.aggregative.DMy(classifier: BaseEstimator = None, fit_classifier=True, val_split=5, nbins=8, divergence: str | Callable = 'HD', cdf=False, search='optim_minimize', n_jobs=None)

Bases: AggregativeSoftQuantifier

Generic Distribution Matching quantifier for binary or multiclass quantification based on the space of posterior probabilities. This implementation takes the number of bins, the divergence, and the possibility to work on CDF as hyperparameters.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

• nbins – number of bins used to discretize the distributions (default 8)

• divergence – a string representing a divergence measure (currently, “HD” and “topsoe” are implemented) or a callable function taking two ndarrays of the same dimension as input (default “HD”, meaning Hellinger Distance)

• cdf – whether to use CDF instead of PDF (default False)

• search – string indicating the search strategy used to estimate the prevalence values. Valid options are optim_minimize (default, works for binary and multiclass problems), linear_search (binary only), and ternary_search (binary only)

• n_jobs – number of parallel workers (default None)

classmethod HDy(classifier: BaseEstimator = None, fit_classifier=True, val_split=5, n_jobs=None)

Historical HDy preset expressed as a configuration of DMy.

This preset reproduces the original HDy setup by using Hellinger distance, PDF matching, linear search, and a median sweep over nbins in [10, 20, …, 110].

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None

• fit_classifier – whether to train the learner

• val_split – validation specification for generating posteriors

• n_jobs – number of parallel workers

Returns:

an instance of AggregativeMedianEstimator configured to reproduce the historical HDy preset

aggregate(posteriors: ndarray)

Searches for the mixture model parameter (the sought prevalence values) that yields a validation distribution (the mixture) that best matches the test distribution, in terms of the divergence measure of choice. In the multiclass case, with n the number of classes, the test and mixture distributions contain n channels (proper distributions of binned posterior probabilities), on which the divergence is computed independently. The matching is computed as an average of the divergence across all channels.

Parameters:

posteriors – posterior probabilities of the instances in the sample

Returns:

a vector of class prevalence estimates

aggregation_fit(classif_predictions, labels)

Trains the aggregation function of a distribution matching method. This comes down to generating the validation distributions out of the training data. The validation distributions have shape (n, ch, nbins), with n the number of classes, ch the number of channels, and nbins the number of bins. In particular, let V be the validation distributions; then di=V[i] are the distributions obtained from training data labelled with class i; while dij = di[j] is the discrete distribution of posterior probabilities P(Y=j|X=x) for training data labelled with class i, and dij[k] is the fraction of instances with a value in the k-th bin.

Parameters:

• classif_predictions – array-like with the posterior probabilities returned by the classifier

• labels – array-like with the true labels associated to each posterior

quapy.method.aggregative.DistributionMatchingY

alias of DMy

class quapy.method.aggregative.DyS(classifier: BaseEstimator = None, fit_classifier=True, val_split=5, n_bins=8, divergence: str | Callable = 'HD', tol=1e-05, n_jobs=None)

Bases: AggregativeSoftQuantifier, BinaryAggregativeQuantifier

DyS framework (DyS). DyS is a generalization of HDy method, using a Ternary Search in order to find the prevalence that minimizes the distance between distributions. Details for the ternary search have been got from <https://dl.acm.org/doi/pdf/10.1145/3219819.3220059>

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

• n_bins – an int with the number of bins to use to compute the histograms.

• divergence – a str indicating the name of divergence (currently supported ones are “HD” or “topsoe”), or a callable function computes the divergence between two distributions (two equally sized arrays).

• tol – a float with the tolerance for the ternary search algorithm.

• n_jobs – number of parallel workers.

aggregate(classif_posteriors)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Trains the aggregation function of DyS.

Parameters:

• classif_predictions – array-like with the posterior probabilities returned by the classifier

• labels – array-like with the true labels associated to each posterior

class quapy.method.aggregative.EDy(classifier: BaseEstimator = None, fit_classifier: bool = True, val_split=5, distance: str | Callable = 'manhattan', n_jobs=None)

Bases: _EnergyDistanceCore, AggregativeSoftQuantifier

Energy Distance y (EDy), a posterior-space distribution-matching quantifier based on energy distance.

The method represents each class by the posterior-probability vectors produced by a probabilistic classifier on validation data, and estimates the test prevalence vector by matching the test posterior distribution against the class-conditional validation distributions through an energy-distance objective solved as a quadratic program. The method is therefore another instance of the general mixture-matching view of quantification, but it operates directly on posterior vectors rather than on histogram summaries.

This implementation works for binary and multiclass single-label quantification and relies on the optional quadprog dependency. It was adapted to QuaPy’s current aggregative API from the original implementation available in quantificationlib, and now shares its numerical core with the classifier-free quapy.method.non_aggregative.EDx variant.

The current implementation follows the energy-distance formulation discussed in:

• Alberto Castaño, Laura Morán-Fernández, Jaime Alonso, Verónica Bolón-Canedo, Amparo Alonso-Betanzos, and Juan José del Coz. An analysis of quantification methods based on matching distributions.

• Hideko Kawakubo, Marthinus Christoffel du Plessis, and Masashi Sugiyama (2016). Computationally efficient class-prior estimation under class balance change using energy distance. IEICE Transactions on Information and Systems, 99(1):176-186.

Parameters:

• classifier – a scikit-learn BaseEstimator, or None to use qp.environ['DEFAULT_CLS']

• fit_classifier – whether to train the learner (default True). Set to False if the learner has already been trained outside the quantifier

• val_split – specification of the data used for generating validation posterior probabilities. This can be an integer (default 5) for k-fold cross-validation, a float in (0, 1) for a held-out split, or a tuple (X, y) with explicit validation data

• distance – distance used to compare posterior vectors. Valid string aliases are 'manhattan' (default) and 'euclidean'; a custom callable compatible with pairwise-distance signatures can also be used

• n_jobs – number of parallel workers (default None, meaning the value is taken from the environment)

aggregate(posteriors: ndarray)

Estimate the prevalence vector for a test sample.

Parameters:

posteriors – posterior probabilities returned by the classifier for the instances in the test sample

Returns:

a prevalence vector of shape (n_classes,)

aggregation_fit(classif_predictions, labels)

Estimate the class-conditional posterior distributions on validation data and pre-compute the quadratic-program parameters that depend only on the training side.

In EDy, the validation posteriors are not discretized into histograms. Instead, each class is represented by the cloud of posterior vectors observed for that class, and these clouds are then compared through the selected pairwise distance.

Parameters:

• classif_predictions – posterior probabilities returned by the classifier on validation data

• labels – true labels associated to each posterior vector

class quapy.method.aggregative.EMQ(classifier: BaseEstimator = None, fit_classifier=True, val_split=None, exact_train_prev=True, calib=None, on_calib_error='raise', n_jobs=None)

Bases: AggregativeSoftQuantifier

Expectation Maximization for Quantification (EMQ), aka Saerens-Latinne-Decaestecker (SLD) algorithm. EMQ consists of using the well-known Expectation Maximization algorithm to iteratively update the posterior probabilities generated by a probabilistic classifier and the class prevalence estimates obtained via maximum-likelihood estimation, in a mutually recursive way, until convergence.

This implementation also gives access to the heuristics proposed by Alexandari et al. paper. These heuristics consist of using, as the training prevalence, an estimate of it obtained via k-fold cross validation (instead of the true training prevalence), and to recalibrate the posterior probabilities of the classifier.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the classifier (default is True). Set to False if the given classifier has already been trained.

• val_split – specifies the data used for generating the classifier predictions on which the aggregation function is to be trained. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation. This hyperparameter is only meant to be used when the heuristics are to be applied, i.e., if a calibration is required. The default value is None (meaning the calibration is not required). In case this hyperparameter is set to a value other than None, but the calibration is not required (calib=None), a warning message will be raised.

• exact_train_prev – set to True (default) for using the true training prevalence as the initial observation; set to False for computing the training prevalence as an estimate of it, i.e., as the expected value of the posterior probabilities of the training instances.

• calib – a string indicating the method of calibration. Available choices include “nbvs” (No-Bias Vector Scaling), “bcts” (Bias-Corrected Temperature Scaling), “ts” (Temperature Scaling), and “vs” (Vector Scaling). Default is None (no calibration).

• on_calib_error – a string indicating the policy to follow in case the calibrator fails at runtime. Options include “raise” (default), in which case a RuntimeException is raised; and “backup”, in which case the calibrator is silently skipped.

• n_jobs – number of parallel workers. Only used for recalibrating the classifier if val_split is set to an integer k –the number of folds.

CALIB_OPTIONS = [None, 'nbvs', 'bcts', 'ts', 'vs']

classmethod EM(tr_prev, posterior_probabilities, epsilon=0.0001)

Computes the Expectation Maximization routine.

Parameters:

• tr_prev – array-like, the training prevalence

• posterior_probabilities – np.ndarray of shape (n_instances, n_classes,) with the posterior probabilities

• epsilon – float, the threshold different between two consecutive iterations to reach before stopping the loop

Returns:

a tuple with the estimated prevalence values (shape (n_classes,)) and the corrected posterior probabilities (shape (n_instances, n_classes,))

classmethod EMQ_BCTS(classifier: BaseEstimator, fit_classifier=True, val_split=5, on_calib_error='raise', n_jobs=None)

Constructs an instance of EMQ using the best configuration found in the Alexandari et al. paper, i.e., one that relies on Bias-Corrected Temperature Scaling (BCTS) as a calibration function, and that uses an estimate of the training prevalence instead of the true training prevalence.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

• on_calib_error – a string indicating the policy to follow in case the calibrator fails at runtime. Options include “raise” (default), in which case a RuntimeException is raised; and “backup”, in which case the calibrator is silently skipped.

• n_jobs – number of parallel workers. Only used for recalibrating the classifier if val_split is set to an integer k –the number of folds.

Returns:

An instance of EMQ with BCTS

EPSILON = 0.0001

MAX_ITER = 1000

ON_CALIB_ERROR_VALUES = ['raise', 'backup']

aggregate(classif_posteriors, epsilon=0.0001)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Trains the aggregation function of EMQ. This comes down to recalibrating the posterior probabilities ir requested.

Parameters:

• classif_predictions – array-like with the raw (i.e., uncalibrated) posterior probabilities returned by the classifier

• labels – array-like with the true labels associated to each classifier prediction

classifier_fit_predict(X, y)

Trains the classifier if requested (fit_classifier=True) and generate the necessary predictions to train the aggregation function.

Parameters:

• X – array-like of shape (n_samples, n_features), the training instances

• y – array-like of shape (n_samples,), the labels

classify(X)

Provides the posterior probabilities for the given instances. The calibration function, if required, has no effect in this step, and is only involved in the aggregate method.

Parameters:

X – array-like of shape (n_instances, n_dimensions,)

Returns:

np.ndarray of shape (n_instances, n_classes,) with posterior probabilities

predict_proba(instances, epsilon=0.0001)

Returns the posterior probabilities updated by the EM algorithm.

Parameters:

• instances – np.ndarray of shape (n_instances, n_dimensions)

• epsilon – error tolerance

Returns:

np.ndarray of shape (n_instances, n_classes)

set_predict_proba_request(*, epsilon: bool | None | str = '$UNCHANGED$', instances: bool | None | str = '$UNCHANGED$') → EMQ

Configure whether metadata should be requested to be passed to the predict_proba method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict_proba if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict_proba.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

• epsilon (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for epsilon parameter in predict_proba.

• instances (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for instances parameter in predict_proba.

Returns:

self – The updated object.

Return type:

object

quapy.method.aggregative.EnergyDistanceY

alias of EDy

quapy.method.aggregative.ExpectationMaximizationQuantifier

alias of EMQ

class quapy.method.aggregative.HDy(classifier: BaseEstimator = None, fit_classifier=True, val_split=5)

Bases: AggregativeSoftQuantifier, BinaryAggregativeQuantifier

Hellinger Distance y (HDy). HDy is a probabilistic method for training binary quantifiers, that models quantification as the problem of minimizing the divergence (in terms of the Hellinger Distance) between two distributions of posterior probabilities returned by the classifier. One of the distributions is generated from the unlabelled examples and the other is generated from a validation set. This latter distribution is defined as a mixture of the class-conditional distributions of the posterior probabilities returned for the positive and negative validation examples, respectively. The parameters of the mixture thus represent the estimates of the class prevalence values.

This dedicated class is kept for backward compatibility as the historical HDy implementation. The same historical preset is also available as DMy.HDy().

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

aggregate(classif_posteriors)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Trains the aggregation function of HDy.

Parameters:

• classif_predictions – array-like with the posterior probabilities returned by the classifier

• labels – array-like with the true labels associated to each posterior

quapy.method.aggregative.HellingerDistanceY

alias of HDy

quapy.method.aggregative.HistoricalHDy(classifier: BaseEstimator = None, fit_classifier=True, val_split=5, n_jobs=None)

Historical HDy preset expressed as a configuration of DMy.

This preset reproduces the original HDy setup by using Hellinger distance, PDF matching, linear search, and a median sweep over nbins in [10, 20, …, 110].

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None

• fit_classifier – whether to train the learner

• val_split – validation specification for generating posteriors

• n_jobs – number of parallel workers

Returns:

an instance of AggregativeMedianEstimator configured to reproduce the historical HDy preset

class quapy.method.aggregative.OneVsAllAggregative(binary_quantifier=None, n_jobs=None, parallel_backend='multiprocessing')

Bases: OneVsAllGeneric, AggregativeQuantifier

Allows any binary quantifier to perform quantification on single-label datasets. The method maintains one binary quantifier for each class, and then l1-normalizes the outputs so that the class prevelences sum up to 1. This variant was used, along with the EMQ quantifier, in Gao and Sebastiani, 2016.

Parameters:

• binary_quantifier – a quantifier (binary) that will be employed to work on multiclass model in a one-vs-all manner (default PACC(LogitsticRegression()))

• n_jobs – number of parallel workers

• parallel_backend – the parallel backend for joblib (default “loky”); this is helpful for some quantifiers (e.g., ELM-based ones) that cannot be run with multiprocessing, since the temp dir they create during fit will is removed and no longer available at predict time.

aggregate(classif_predictions)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

classify(X)

If the base quantifier is not probabilistic, returns a matrix of shape (n,m,) with n the number of instances and m the number of classes. The entry (i,j) is a binary value indicating whether instance i belongs to class j. The binary classifications are independent of each other, meaning that an instance can end up be attributed to 0, 1, or more classes. If the base quantifier is probabilistic, returns a matrix of shape (n,m,2) with n the number of instances and m the number of classes. The entry (i,j,1) (resp. (i,j,0)) is a value in [0,1] indicating the posterior probability that instance i belongs (resp. does not belong) to class j. The posterior probabilities are independent of each other, meaning that, in general, they do not sum up to one.

Parameters:

X – array-like

Returns:

np.ndarray

class quapy.method.aggregative.PACC(classifier: BaseEstimator = None, fit_classifier=True, val_split=5, solver: Literal['minimize', 'exact', 'exact-raise', 'exact-cc'] = 'minimize', method: Literal['inversion', 'invariant-ratio'] = 'inversion', norm: Literal['clip', 'mapsimplex', 'condsoftmax'] = 'clip', n_jobs=None)

Bases: AggregativeSoftQuantifier

Probabilistic Adjusted Classify & Count (PACC), the probabilistic variant of ACC that relies on the posterior probabilities returned by a probabilistic classifier, introduced in Bella, A., Ferri, C., Hernández-Orallo, J., and Ramírez-Quintana, M.J. (2010). Quantification via probability estimators. In Proceedings of the 2010 IEEE International Conference on Data Mining (ICDM 2010).

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

• method (str) –

  adjustment method to be used:

  • ’inversion’: matrix inversion method based on the matrix equality \(P(C)=P(C|Y)P(Y)\), which tries to invert P(C|Y) matrix.

  • ’invariant-ratio’: invariant ratio estimator of Vaz et al., which replaces the last equation with the normalization condition.

• solver (str) –

  the method to use for solving the system of linear equations. Valid options are:

  • ’exact-raise’: tries to solve the system using matrix inversion. Raises an error if the matrix has rank strictly less than n_classes.

  • ’exact-cc’: if the matrix is not of full rank, returns p_c as the estimates, which corresponds to no adjustment (i.e., the classify and count method. See quapy.method.aggregative.CC)

  • ’exact’: deprecated, defaults to ‘exact-cc’

  • ’minimize’: minimizes the L2 norm of \(|Ax-B|\). This one generally works better, and is the default parameter. More details about this can be consulted in Bunse, M. “On Multi-Class Extensions of Adjusted Classify and Count”, on proceedings of the 2nd International Workshop on Learning to Quantify: Methods and Applications (LQ 2022), ECML/PKDD 2022, Grenoble (France).

• norm (str) –

  the method to use for normalization.

  • clip, the values are clipped to the range [0,1] and then L1-normalized.

  • mapsimplex projects vectors onto the probability simplex. This implementation relies on Mathieu Blondel’s projection_simplex_sort

  • condsoftmax, applies a softmax normalization only to prevalence vectors that lie outside the simplex

• n_jobs – number of parallel workers

aggregate(classif_posteriors)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Estimates the misclassification rates

Parameters:

• classif_predictions – array-like with posterior probabilities

• labels – array-like with the true labels associated to each vector of posterior probabilities

classmethod getPteCondEstim(classes, y, y_)

class quapy.method.aggregative.PCC(classifier: BaseEstimator = None, fit_classifier: bool = True, val_split=None)

Bases: AggregativeSoftQuantifier

Probabilistic Classify & Count (PCC), the probabilistic variant of CC that relies on the posterior probabilities returned by a probabilistic classifier, introduced in Bella, A., Ferri, C., Hernández-Orallo, J., and Ramírez-Quintana, M.J. (2010). Quantification via probability estimators. In Proceedings of the 2010 IEEE International Conference on Data Mining (ICDM 2010).

Parameters:

classifier – a sklearn’s Estimator that generates a classifier

aggregate(classif_posteriors)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Nothing to do here!

Parameters:

• classif_predictions – unused

• labels – unused

quapy.method.aggregative.ProbabilisticAdjustedClassifyAndCount

alias of PACC

quapy.method.aggregative.ProbabilisticClassifyAndCount

alias of PCC

class quapy.method.aggregative.RLLS(classifier: BaseEstimator = None, fit_classifier=True, val_split=5, mode: Literal['soft', 'hard'] = 'soft', alpha: float = 0.01, delta: float = 0.05, clip_weights: bool = True, norm: Literal['clip', 'mapsimplex', 'condsoftmax'] = 'clip')

Bases: AggregativeSoftQuantifier

Regularized Learning for Domain Adaptation under Label Shifts, used here as an aggregative quantifier.

This implementation ports the regularized weight-estimation component of RLLS to QuaPy’s aggregative interface. It estimates label-shift ratios from validation posteriors and source labels, then rescales the source prevalence to obtain target prevalence estimates.

This method relies on the optional cvxpy dependency.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner; or as a tuple (X, y) defining the specific set of data to use for validation. This method requires source predictions and therefore needs val_split whenever fit_classifier=True.

• mode – whether source- and target-domain quantities are estimated from posterior probabilities (soft, default) or from argmax predictions (hard)

• alpha – multiplicative factor for the regularization level (default 0.01)

• delta – confidence parameter used in the finite-sample regularizer (default 0.05)

• clip_weights – if True, clips negative importance weights to zero before converting them into prevalence estimates

• norm – the normalization method passed to quapy.functional.normalize_prevalence()

aggregate(classif_posteriors)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

quapy.method.aggregative.SLD

alias of EMQ

class quapy.method.aggregative.SMM(classifier: BaseEstimator = None, fit_classifier=True, val_split=5)

Bases: AggregativeSoftQuantifier, BinaryAggregativeQuantifier

SMM method (SMM). SMM is a simplification of matching distribution methods where the representation of the examples is created using the mean instead of a histogram (conceptually equivalent to PACC).

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

aggregate(classif_posteriors)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Trains the aggregation function of SMM.

Parameters:

• classif_predictions – array-like with the posterior probabilities returned by the classifier

• labels – array-like with the true labels associated to each posterior

quapy.method.aggregative.newELM(svmperf_base=None, loss='01', C=1)

Explicit Loss Minimization (ELM) quantifiers. Quantifiers based on ELM represent a family of methods based on structured output learning; these quantifiers rely on classifiers that have been optimized using a quantification-oriented loss measure. This implementation relies on Joachims’ SVM perf structured output learning algorithm, which has to be installed and patched for the purpose (see this script). This function equivalent to:

>>> CC(SVMperf(svmperf_base, loss, C))

Parameters:

• svmperf_base – path to the folder containing the binary files of SVM perf; if set to None (default) this path will be obtained from qp.environ[‘SVMPERF_HOME’]

• loss – the loss to optimize (see quapy.classification.svmperf.SVMperf.valid_losses)

• C – trade-off between training error and margin (default 0.01)

Returns:

returns an instance of CC set to work with SVMperf (with loss and C set properly) as the underlying classifier

quapy.method.aggregative.newSVMAE(svmperf_base=None, C=1)

SVM(KLD) is an Explicit Loss Minimization (ELM) quantifier set to optimize for the Absolute Error as first used by Moreo and Sebastiani, 2021. Equivalent to:

>>> CC(SVMperf(svmperf_base, loss='mae', C=C))

Quantifiers based on ELM represent a family of methods based on structured output learning; these quantifiers rely on classifiers that have been optimized using a quantification-oriented loss measure. This implementation relies on Joachims’ SVM perf structured output learning algorithm, which has to be installed and patched for the purpose (see this script). This function is a wrapper around CC(SVMperf(svmperf_base, loss, C))

Parameters:

• svmperf_base – path to the folder containing the binary files of SVM perf; if set to None (default) this path will be obtained from qp.environ[‘SVMPERF_HOME’]

• C – trade-off between training error and margin (default 0.01)

Returns:

returns an instance of CC set to work with SVMperf (with loss and C set properly) as the underlying classifier

quapy.method.aggregative.newSVMKLD(svmperf_base=None, C=1)

SVM(KLD) is an Explicit Loss Minimization (ELM) quantifier set to optimize for the Kullback-Leibler Divergence as proposed by Esuli et al. 2015. Equivalent to:

>>> CC(SVMperf(svmperf_base, loss='kld', C=C))

Quantifiers based on ELM represent a family of methods based on structured output learning; these quantifiers rely on classifiers that have been optimized using a quantification-oriented loss measure. This implementation relies on Joachims’ SVM perf structured output learning algorithm, which has to be installed and patched for the purpose (see this script). This function is a wrapper around CC(SVMperf(svmperf_base, loss, C))

Parameters:

• svmperf_base – path to the folder containing the binary files of SVM perf; if set to None (default) this path will be obtained from qp.environ[‘SVMPERF_HOME’]

• C – trade-off between training error and margin (default 0.01)

Returns:

returns an instance of CC set to work with SVMperf (with loss and C set properly) as the underlying classifier

quapy.method.aggregative.newSVMNKLD(svmperf_base=None, C=1)

SVM(NKLD) is an Explicit Loss Minimization (ELM) quantifier set to optimize for the Kullback-Leibler Divergence normalized via the logistic function, as proposed by Esuli et al. 2015. Equivalent to:

>>> CC(SVMperf(svmperf_base, loss='nkld', C=C))

Quantifiers based on ELM represent a family of methods based on structured output learning; these quantifiers rely on classifiers that have been optimized using a quantification-oriented loss measure. This implementation relies on Joachims’ SVM perf structured output learning algorithm, which has to be installed and patched for the purpose (see this script). This function is a wrapper around CC(SVMperf(svmperf_base, loss, C))

Parameters:

• svmperf_base – path to the folder containing the binary files of SVM perf; if set to None (default) this path will be obtained from qp.environ[‘SVMPERF_HOME’]

• C – trade-off between training error and margin (default 0.01)

Returns:

returns an instance of CC set to work with SVMperf (with loss and C set properly) as the underlying classifier

quapy.method.aggregative.newSVMQ(svmperf_base=None, C=1)

SVM(Q) is an Explicit Loss Minimization (ELM) quantifier set to optimize for the Q loss combining a classification-oriented loss and a quantification-oriented loss, as proposed by Barranquero et al. 2015. Equivalent to:

>>> CC(SVMperf(svmperf_base, loss='q', C=C))

Quantifiers based on ELM represent a family of methods based on structured output learning; these quantifiers rely on classifiers that have been optimized using a quantification-oriented loss measure. This implementation relies on Joachims’ SVM perf structured output learning algorithm, which has to be installed and patched for the purpose (see this script). This function is a wrapper around CC(SVMperf(svmperf_base, loss, C))

Parameters:

• svmperf_base – path to the folder containing the binary files of SVM perf; if set to None (default) this path will be obtained from qp.environ[‘SVMPERF_HOME’]

• C – trade-off between training error and margin (default 0.01)

Returns:

returns an instance of CC set to work with SVMperf (with loss and C set properly) as the underlying classifier

quapy.method.aggregative.newSVMRAE(svmperf_base=None, C=1)

SVM(KLD) is an Explicit Loss Minimization (ELM) quantifier set to optimize for the Relative Absolute Error as first used by Moreo and Sebastiani, 2021. Equivalent to:

>>> CC(SVMperf(svmperf_base, loss='mrae', C=C))

Quantifiers based on ELM represent a family of methods based on structured output learning; these quantifiers rely on classifiers that have been optimized using a quantification-oriented loss measure. This implementation relies on Joachims’ SVM perf structured output learning algorithm, which has to be installed and patched for the purpose (see this script). This function is a wrapper around CC(SVMperf(svmperf_base, loss, C))

Parameters:

• svmperf_base – path to the folder containing the binary files of SVM perf; if set to None (default) this path will be obtained from qp.environ[‘SVMPERF_HOME’]

• C – trade-off between training error and margin (default 0.01)

Returns:

returns an instance of CC set to work with SVMperf (with loss and C set properly) as the underlying classifier

class quapy.method._kdey.KDEBase

Bases: object

Common ancestor for KDE-based methods. Implements some common routines.

BANDWIDTH_METHOD = ['scott', 'silverman']

KERNELS = ['gaussian', 'aitchison', 'ilr']

clr_transform(X)

effective_bandwidth(bandwidth, kernel)

get_kde_function(X, bandwidth, kernel)

Wraps the KDE function from scikit-learn.

Parameters:

• X – data for which the density function is to be estimated

• bandwidth – the bandwidth of the kernel

• kernel – the kernel family

Returns:

a scikit-learn’s KernelDensity object

get_mixture_components(X, y, classes, bandwidth, kernel)

Returns an array containing the mixture components, i.e., the KDE functions for each class.

Parameters:

• X – the data containing the covariates

• y – the class labels

• n_classes – integer, the number of classes

• bandwidth – float, the bandwidth of the kernel

• kernel – the kernel family

Returns:

a list of KernelDensity objects, each fitted with the corresponding class-specific covariates

ilr_transform(X)

pdf(kde, X, kernel, log_densities=False)

Wraps the density evalution of scikit-learn’s KDE. Scikit-learn returns log-scores (s), so this function returns \(e^{s}\)

Parameters:

• kde – a previously fit KDE function

• X – the data for which the density is to be estimated

• kernel – the kernel family

Returns:

np.ndarray with the densities

shrink_posteriors(X)

transform_posteriors(X, kernel)

class quapy.method._kdey.KDEyCS(classifier: BaseEstimator = None, fit_classifier=True, val_split=5, bandwidth=0.1)

Bases: AggregativeSoftQuantifier

Kernel Density Estimation model for quantification (KDEy) relying on the Cauchy-Schwarz divergence (CS) as the divergence measure to be minimized. This method was first proposed in the paper Kernel Density Estimation for Multiclass Quantification (arXiv), in which the authors proposed a Monte Carlo approach for minimizing the divergence.

The distribution matching optimization problem comes down to solving:

\(\hat{\alpha} = \arg\min_{\alpha\in\Delta^{n-1}} \mathcal{D}(\boldsymbol{p}_{\alpha}||q_{\widetilde{U}})\)

where \(p_{\alpha}\) is the mixture of class-specific KDEs with mixture parameter (hence class prevalence) \(\alpha\) defined by

\(\boldsymbol{p}_{\alpha}(\widetilde{x}) = \sum_{i=1}^n \alpha_i p_{\widetilde{L}_i}(\widetilde{x})\)

where \(p_X(\boldsymbol{x}) = \frac{1}{|X|} \sum_{x_i\in X} K\left(\frac{x-x_i}{h}\right)\) is the KDE function that uses the datapoints in X as the kernel centers.

In KDEy-CS, the divergence is taken to be the Cauchy-Schwarz divergence given by:

\(\mathcal{D}_{\mathrm{CS}}(p||q)=-\log\left(\frac{\int p(x)q(x)dx}{\sqrt{\int p(x)^2dx \int q(x)^2dx}}\right)\)

The authors showed that this distribution matching admits a closed-form solution

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

• bandwidth – float, the bandwidth of the Kernel

aggregate(posteriors: ndarray)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

gram_matrix_mix_sum(X, Y=None)

class quapy.method._kdey.KDEyHD(classifier: BaseEstimator = None, fit_classifier=True, val_split=5, divergence: str = 'HD', bandwidth=0.1, random_state=None, montecarlo_trials=10000)

Bases: AggregativeSoftQuantifier, KDEBase

Kernel Density Estimation model for quantification (KDEy) relying on the squared Hellinger Disntace (HD) as the divergence measure to be minimized. This method was first proposed in the paper Kernel Density Estimation for Multiclass Quantification (arXiv), in which the authors proposed a Monte Carlo approach for minimizing the divergence.

The distribution matching optimization problem comes down to solving:

\(\hat{\alpha} = \arg\min_{\alpha\in\Delta^{n-1}} \mathcal{D}(\boldsymbol{p}_{\alpha}||q_{\widetilde{U}})\)

where \(p_{\alpha}\) is the mixture of class-specific KDEs with mixture parameter (hence class prevalence) \(\alpha\) defined by

\(\boldsymbol{p}_{\alpha}(\widetilde{x}) = \sum_{i=1}^n \alpha_i p_{\widetilde{L}_i}(\widetilde{x})\)

where \(p_X(\boldsymbol{x}) = \frac{1}{|X|} \sum_{x_i\in X} K\left(\frac{x-x_i}{h}\right)\) is the KDE function that uses the datapoints in X as the kernel centers.

In KDEy-HD, the divergence is taken to be the squared Hellinger Distance, an f-divergence with corresponding f-generator function given by:

\(f(u)=(\sqrt{u}-1)^2\)

The authors proposed a Monte Carlo solution that relies on importance sampling:

\(\hat{D}_f(p||q)= \frac{1}{t} \sum_{i=1}^t f\left(\frac{p(x_i)}{q(x_i)}\right) \frac{q(x_i)}{r(x_i)}\)

where the datapoints (trials) \(x_1,\ldots,x_t\sim_{\mathrm{iid}} r\) with \(r\) the uniform distribution.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

• bandwidth – float, the bandwidth of the Kernel

• random_state – a seed to be set before fitting any base quantifier (default None)

• montecarlo_trials – number of Monte Carlo trials (default 10000)

aggregate(posteriors: ndarray)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

class quapy.method._kdey.KDEyML(classifier: BaseEstimator = None, fit_classifier=True, val_split=5, bandwidth=0.1, kernel='gaussian', shrinkage=0.0, random_state=None)

Bases: AggregativeSoftQuantifier, KDEBase

Kernel Density Estimation model for quantification (KDEy) relying on the Kullback-Leibler divergence (KLD) as the divergence measure to be minimized. This method was first proposed in the paper Kernel Density Estimation for Multiclass Quantification (arXiv), in which the authors show that minimizing the distribution mathing criterion for KLD is akin to performing maximum likelihood (ML).

The distribution matching optimization problem comes down to solving:

\(\hat{\alpha} = \arg\min_{\alpha\in\Delta^{n-1}} \mathcal{D}(\boldsymbol{p}_{\alpha}||q_{\widetilde{U}})\)

where \(p_{\alpha}\) is the mixture of class-specific KDEs with mixture parameter (hence class prevalence) \(\alpha\) defined by

\(\boldsymbol{p}_{\alpha}(\widetilde{x}) = \sum_{i=1}^n \alpha_i p_{\widetilde{L}_i}(\widetilde{x})\)

where \(p_X(\boldsymbol{x}) = \frac{1}{|X|} \sum_{x_i\in X} K\left(\frac{x-x_i}{h}\right)\) is the KDE function that uses the datapoints in X as the kernel centers.

In KDEy-ML, the divergence is taken to be the Kullback-Leibler Divergence. This is equivalent to solving: \(\hat{\alpha} = \arg\min_{\alpha\in\Delta^{n-1}} - \mathbb{E}_{q_{\widetilde{U}}} \left[ \log \boldsymbol{p}_{\alpha}(\widetilde{x}) \right]\)

which corresponds to the maximum likelihood estimate.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

• bandwidth – float, the bandwidth of the Kernel

• kernel – kernel of KDE, valid ones are in KDEBase.KERNELS

• shrinkage – amount of shrinkage towards the uniform distribution to apply before Aitchison/ILR transformations. Must be in [0,1).

• random_state – a seed to be set before fitting any base quantifier (default None)

aggregate(posteriors: ndarray)

Searches for the mixture model parameter (the sought prevalence values) that maximizes the likelihood of the data (i.e., that minimizes the negative log-likelihood)

Parameters:

posteriors – instances in the sample converted into posterior probabilities

Returns:

a vector of class prevalence estimates

aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

class quapy.method._neural.QuaNetModule(doc_embedding_size, n_classes, stats_size, lstm_hidden_size=64, lstm_nlayers=1, ff_layers=[1024, 512], bidirectional=True, qdrop_p=0.5, order_by=0)

Bases: Module

Implements the QuaNet forward pass. See QuaNetTrainer for training QuaNet.

Parameters:

• doc_embedding_size – integer, the dimensionality of the document embeddings

• n_classes – integer, number of classes

• stats_size – integer, number of statistics estimated by simple quantification methods

• lstm_hidden_size – integer, hidden dimensionality of the LSTM cell

• lstm_nlayers – integer, number of LSTM layers

• ff_layers – list of integers, dimensions of the densely-connected FF layers on top of the quantification embedding

• bidirectional – boolean, whether or not to use bidirectional LSTM

• qdrop_p – float, dropout probability

• order_by – integer, class for which the document embeddings are to be sorted

property device

forward(doc_embeddings, doc_posteriors, statistics)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class quapy.method._neural.QuaNetTrainer(classifier, fit_classifier=True, sample_size=None, n_epochs=100, tr_iter_per_poch=500, va_iter_per_poch=100, lr=0.001, lstm_hidden_size=64, lstm_nlayers=1, ff_layers=[1024, 512], bidirectional=True, qdrop_p=0.5, patience=10, checkpointdir='../checkpoint', checkpointname=None, device='cuda')

Bases: BaseQuantifier

Implementation of QuaNet, a neural network for quantification. This implementation uses PyTorch and can take advantage of GPU for speeding-up the training phase.

Example:

>>> import quapy as qp
>>> from quapy.method.meta import QuaNet
>>> from quapy.classification.neural import NeuralClassifierTrainer, CNNnet
>>>
>>> # use samples of 100 elements
>>> qp.environ['SAMPLE_SIZE'] = 100
>>>
>>> # load the Kindle dataset as text, and convert words to numerical indexes
>>> dataset = qp.datasets.fetch_reviews('kindle', pickle=True)
>>> qp.train.preprocessing.index(dataset, min_df=5, inplace=True)
>>>
>>> # the text classifier is a CNN trained by NeuralClassifierTrainer
>>> cnn = CNNnet(dataset.vocabulary_size, dataset.n_classes)
>>> classifier = NeuralClassifierTrainer(cnn, device='cuda')
>>>
>>> # train QuaNet (QuaNet is an alias to QuaNetTrainer)
>>> model = QuaNet(classifier, qp.environ['SAMPLE_SIZE'], device='cuda')
>>> model.fit(*dataset.training.Xy)
>>> estim_prevalence = model.predict(dataset.test.instances)

Parameters:

• classifier – an object implementing fit (i.e., that can be trained on labelled data), predict_proba (i.e., that can generate posterior probabilities of unlabelled examples) and transform (i.e., that can generate embedded representations of the unlabelled instances).

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• sample_size – integer, the sample size; default is None, meaning that the sample size should be taken from qp.environ[“SAMPLE_SIZE”]

• n_epochs – integer, maximum number of training epochs

• tr_iter_per_poch – integer, number of training iterations before considering an epoch complete

• va_iter_per_poch – integer, number of validation iterations to perform after each epoch

• lr – float, the learning rate

• lstm_hidden_size – integer, hidden dimensionality of the LSTM cells

• lstm_nlayers – integer, number of LSTM layers

• ff_layers – list of integers, dimensions of the densely-connected FF layers on top of the quantification embedding

• bidirectional – boolean, indicates whether the LSTM is bidirectional or not

• qdrop_p – float, dropout probability

• patience – integer, number of epochs showing no improvement in the validation set before stopping the training phase (early stopping)

• checkpointdir – string, a path where to store models’ checkpoints

• checkpointname – string (optional), the name of the model’s checkpoint

• device – string, indicate “cpu” or “cuda”

property classes_

clean_checkpoint()

Removes the checkpoint

clean_checkpoint_dir()

Removes anything contained in the checkpoint directory

fit(X, y)

Trains QuaNet.

Parameters:

• X – the training instances on which to train QuaNet. If fit_classifier=True, the data will be split in 40/40/20 for training the classifier, training QuaNet, and validating QuaNet, respectively. If fit_classifier=False, the data will be split in 66/34 for training QuaNet and validating it, respectively.

• y – the labels of X

Returns:

self

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

predict(X)

Generate class prevalence estimates for the sample’s instances

Parameters:

X – array-like, the test instances

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

set_params(**parameters)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

quapy.method._neural.mae_loss(output, target)

Torch-like wrapper for the Mean Absolute Error

Parameters:

• output – predictions

• target – ground truth values

Returns:

mean absolute error loss

class quapy.method._threshold_optim.MAX(classifier: BaseEstimator = None, fit_classifier=True, val_split=5)

Bases: ThresholdOptimization

Threshold Optimization variant for ACC as proposed by Forman 2006 and Forman 2008 that looks for the threshold that maximizes tpr-fpr. The goal is to bring improved stability to the denominator of the adjustment.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

condition(tpr, fpr) → float

Implements the criterion according to which the threshold should be selected. This function should return the (float) score to be minimized.

Parameters:

• tpr – float, true positive rate

• fpr – float, false positive rate

Returns:

float, a score for the given tpr and fpr

class quapy.method._threshold_optim.MS(classifier: BaseEstimator = None, fit_classifier=True, val_split=5)

Bases: ThresholdOptimization

Median Sweep. Threshold Optimization variant for ACC as proposed by Forman 2006 and Forman 2008 that generates class prevalence estimates for all decision thresholds and returns the median of them all. The goal is to bring improved stability to the denominator of the adjustment.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

aggregate(classif_predictions: ndarray)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

condition(tpr, fpr) → float

Implements the criterion according to which the threshold should be selected. This function should return the (float) score to be minimized.

Parameters:

• tpr – float, true positive rate

• fpr – float, false positive rate

Returns:

float, a score for the given tpr and fpr

class quapy.method._threshold_optim.MS2(classifier: BaseEstimator = None, fit_classifier=True, val_split=5)

Bases: MS

Median Sweep 2. Threshold Optimization variant for ACC as proposed by Forman 2006 and Forman 2008 that generates class prevalence estimates for all decision thresholds and returns the median of for cases in which tpr-fpr>0.25 The goal is to bring improved stability to the denominator of the adjustment.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

discard(tpr, fpr) → bool

Indicates whether a combination of tpr and fpr should be discarded

Parameters:

• tpr – float, true positive rate

• fpr – float, false positive rate

Returns:

true if the combination is to be discarded, false otherwise

class quapy.method._threshold_optim.T50(classifier: BaseEstimator = None, fit_classifier=True, val_split=5)

Bases: ThresholdOptimization

Threshold Optimization variant for ACC as proposed by Forman 2006 and Forman 2008 that looks for the threshold that makes tpr closest to 0.5. The goal is to bring improved stability to the denominator of the adjustment.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

condition(tpr, fpr) → float

Implements the criterion according to which the threshold should be selected. This function should return the (float) score to be minimized.

Parameters:

• tpr – float, true positive rate

• fpr – float, false positive rate

Returns:

float, a score for the given tpr and fpr

class quapy.method._threshold_optim.ThresholdOptimization(classifier: BaseEstimator = None, fit_classifier=True, val_split=None, n_jobs=None)

Bases: BinaryAggregativeQuantifier

Abstract class of Threshold Optimization variants for ACC as proposed by Forman 2006 and Forman 2008. The goal is to bring improved stability to the denominator of the adjustment. The different variants are based on different heuristics for choosing a decision threshold that would allow for more true positives and many more false positives, on the grounds this would deliver larger denominators.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

• n_jobs – number of parallel workers

aggregate(classif_predictions: ndarray)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregate_with_threshold(classif_predictions, tprs, fprs, thresholds)

aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

abstractmethod condition(tpr, fpr) → float

Implements the criterion according to which the threshold should be selected. This function should return the (float) score to be minimized.

Parameters:

• tpr – float, true positive rate

• fpr – float, false positive rate

Returns:

float, a score for the given tpr and fpr

discard(tpr, fpr) → bool

Indicates whether a combination of tpr and fpr should be discarded

Parameters:

• tpr – float, true positive rate

• fpr – float, false positive rate

Returns:

true if the combination is to be discarded, false otherwise

class quapy.method._threshold_optim.X(classifier: BaseEstimator = None, fit_classifier=True, val_split=5)

Bases: ThresholdOptimization

Threshold Optimization variant for ACC as proposed by Forman 2006 and Forman 2008 that looks for the threshold that yields tpr=1-fpr. The goal is to bring improved stability to the denominator of the adjustment.

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• fit_classifier – whether to train the learner (default is True). Set to False if the learner has been trained outside the quantifier.

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation.

condition(tpr, fpr) → float

Implements the criterion according to which the threshold should be selected. This function should return the (float) score to be minimized.

Parameters:

• tpr – float, true positive rate

• fpr – float, false positive rate

Returns:

float, a score for the given tpr and fpr

quapy.method.base module

class quapy.method.base.BaseQuantifier

Bases: BaseEstimator

Abstract Quantifier. A quantifier is defined as an object of a class that implements the method fit() on a pair X, y, the method predict(), and the set_params() and get_params() for model selection (see quapy.model_selection.GridSearchQ())

abstractmethod fit(X, y)

Generates a quantifier.

Parameters:

• X – array-like, the training instances

• y – array-like, the labels

Returns:

self

abstractmethod predict(X)

Generate class prevalence estimates for the sample’s instances

Parameters:

X – array-like, the test instances

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

quantify(X)

Alias to predict(), for old compatibility

Parameters:

X – array-like

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

class quapy.method.base.BinaryQuantifier

Bases: BaseQuantifier

Abstract class of binary quantifiers, i.e., quantifiers estimating class prevalence values for only two classes (typically, to be interpreted as one class and its complement).

class quapy.method.base.OneVsAll

Bases: object

class quapy.method.base.OneVsAllGeneric(binary_quantifier: BaseQuantifier, n_jobs=None)

Bases: OneVsAll, BaseQuantifier

Allows any binary quantifier to perform quantification on single-label datasets. The method maintains one binary quantifier for each class, and then l1-normalizes the outputs so that the class prevalence values sum up to 1.

fit(X, y)

Generates a quantifier.

Parameters:

• X – array-like, the training instances

• y – array-like, the labels

Returns:

self

predict(X)

Generate class prevalence estimates for the sample’s instances

Parameters:

X – array-like, the test instances

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

quapy.method.base.newOneVsAll(binary_quantifier: BaseQuantifier, n_jobs=None)

quapy.method.meta module

quapy.method.meta.EACC(classifier, param_grid=None, optim=None, param_mod_sel=None, **kwargs)

Implements an ensemble of quapy.method.aggregative.ACC quantifiers, as used by Pérez-Gállego et al., 2019.

Equivalent to:

>>> ensembleFactory(classifier, ACC, param_grid, optim, param_mod_sel, **kwargs)

See ensembleFactory() for further details.

Parameters:

• classifier – sklearn’s Estimator that generates a classifier

• param_grid – a dictionary with the grid of parameters to optimize for

• optim – a valid quantification or classification error, or a string name of it

• param_model_sel – a dictionary containing any keyworded argument to pass to quapy.model_selection.GridSearchQ

• kwargs – kwargs for the class Ensemble

Returns:

an instance of Ensemble

quapy.method.meta.ECC(classifier, param_grid=None, optim=None, param_mod_sel=None, **kwargs)

Implements an ensemble of quapy.method.aggregative.CC quantifiers, as used by Pérez-Gállego et al., 2019.

Equivalent to:

>>> ensembleFactory(classifier, CC, param_grid, optim, param_mod_sel, **kwargs)

See ensembleFactory() for further details.

Parameters:

• classifier – sklearn’s Estimator that generates a classifier

• param_grid – a dictionary with the grid of parameters to optimize for

• optim – a valid quantification or classification error, or a string name of it

• param_model_sel – a dictionary containing any keyworded argument to pass to quapy.model_selection.GridSearchQ

• kwargs – kwargs for the class Ensemble

Returns:

an instance of Ensemble

quapy.method.meta.EEMQ(classifier, param_grid=None, optim=None, param_mod_sel=None, **kwargs)

Implements an ensemble of quapy.method.aggregative.EMQ quantifiers.

Equivalent to:

>>> ensembleFactory(classifier, EMQ, param_grid, optim, param_mod_sel, **kwargs)

See ensembleFactory() for further details.

Parameters:

• classifier – sklearn’s Estimator that generates a classifier

• param_grid – a dictionary with the grid of parameters to optimize for

• optim – a valid quantification or classification error, or a string name of it

• param_model_sel – a dictionary containing any keyworded argument to pass to quapy.model_selection.GridSearchQ

• kwargs – kwargs for the class Ensemble

Returns:

an instance of Ensemble

quapy.method.meta.EHDy(classifier, param_grid=None, optim=None, param_mod_sel=None, **kwargs)

Implements an ensemble of quapy.method.aggregative.HDy quantifiers, as used by Pérez-Gállego et al., 2019.

Equivalent to:

>>> ensembleFactory(classifier, HDy, param_grid, optim, param_mod_sel, **kwargs)

See ensembleFactory() for further details.

Parameters:

• classifier – sklearn’s Estimator that generates a classifier

• param_grid – a dictionary with the grid of parameters to optimize for

• optim – a valid quantification or classification error, or a string name of it

• param_model_sel – a dictionary containing any keyworded argument to pass to quapy.model_selection.GridSearchQ

• kwargs – kwargs for the class Ensemble

Returns:

an instance of Ensemble

quapy.method.meta.EPACC(classifier, param_grid=None, optim=None, param_mod_sel=None, **kwargs)

Implements an ensemble of quapy.method.aggregative.PACC quantifiers.

Equivalent to:

>>> ensembleFactory(classifier, PACC, param_grid, optim, param_mod_sel, **kwargs)

See ensembleFactory() for further details.

Parameters:

• classifier – sklearn’s Estimator that generates a classifier

• param_grid – a dictionary with the grid of parameters to optimize for

• optim – a valid quantification or classification error, or a string name of it

• param_model_sel – a dictionary containing any keyworded argument to pass to quapy.model_selection.GridSearchQ

• kwargs – kwargs for the class Ensemble

Returns:

an instance of Ensemble

class quapy.method.meta.Ensemble(quantifier: BaseQuantifier, size=50, red_size=25, min_pos=5, policy='ave', max_sample_size=None, val_split: LabelledCollection | float = None, n_jobs=None, verbose=False)

Bases: BaseQuantifier

VALID_POLICIES = {'ave', 'ds', 'mae', 'maqe', 'mkld', 'mnae', 'mnkld', 'mnrae', 'mrae', 'mse', 'msre', 'ptr'}

Implementation of the Ensemble methods for quantification described by Pérez-Gállego et al., 2017 and Pérez-Gállego et al., 2019. The policies implemented include:

• Average (policy=’ave’): computes class prevalence estimates as the average of the estimates returned by the base quantifiers.

• Training Prevalence (policy=’ptr’): applies a dynamic selection to the ensemble’s members by retaining only those members such that the class prevalence values in the samples they use as training set are closest to preliminary class prevalence estimates computed as the average of the estimates of all the members. The final estimate is recomputed by considering only the selected members.

• Distribution Similarity (policy=’ds’): performs a dynamic selection of base members by retaining the members trained on samples whose distribution of posterior probabilities is closest, in terms of the Hellinger Distance, to the distribution of posterior probabilities in the test sample

• Accuracy (policy=’<valid error name>’): performs a static selection of the ensemble members by retaining those that minimize a quantification error measure, which is passed as an argument.

Example:

>>> model = Ensemble(quantifier=ACC(LogisticRegression()), size=30, policy='ave', n_jobs=-1)

Parameters:

• quantifier – base quantification member of the ensemble

• size – number of members

• red_size – number of members to retain after selection (depending on the policy)

• min_pos – minimum number of positive instances to consider a sample as valid

• policy – the selection policy; available policies include: ave (default), ptr, ds, and accuracy (which is instantiated via a valid error name, e.g., mae)

• max_sample_size – maximum number of instances to consider in the samples (set to None to indicate no limit, default)

• val_split – a float in range (0,1) indicating the proportion of data to be used as a stratified held-out validation split, or a quapy.data.base.LabelledCollection (the split itself).

• n_jobs – number of parallel workers (default 1)

• verbose – set to True (default is False) to get some information in standard output

property aggregative

Indicates that the quantifier is not aggregative.

Returns:

False

fit(X, y)

Generates a quantifier.

Parameters:

• X – array-like, the training instances

• y – array-like, the labels

Returns:

self

get_params(deep=True)

This function should not be used within quapy.model_selection.GridSearchQ (is here for compatibility with the abstract class). Instead, use Ensemble(GridSearchQ(q),…), with q a Quantifier (recommended), or Ensemble(Q(GridSearchCV(l))) with Q a quantifier class that has a classifier l optimized for classification (not recommended).

Parameters:

deep – for compatibility with scikit-learn

Returns:

raises an Exception

predict(X)

Generate class prevalence estimates for the sample’s instances

Parameters:

X – array-like, the test instances

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

property probabilistic

Indicates that the quantifier is not probabilistic.

Returns:

False

set_params(**parameters)

This function should not be used within quapy.model_selection.GridSearchQ (is here for compatibility with the abstract class). Instead, use Ensemble(GridSearchQ(q),…), with q a Quantifier (recommended), or Ensemble(Q(GridSearchCV(l))) with Q a quantifier class that has a classifier l optimized for classification (not recommended).

Parameters:

parameters – dictionary

Returns:

raises an Exception

class quapy.method.meta.MCMQ(classifiers, quantifiers: List[AggregativeSoftQuantifier], merge_fun='median', val_split=5)

Bases: BaseQuantifier

fit(data: LabelledCollection)

Generates a quantifier.

Parameters:

• X – array-like, the training instances

• y – array-like, the labels

Returns:

self

quantify(instances)

Alias to predict(), for old compatibility

Parameters:

X – array-like

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

set_fit_request(*, data: bool | None | str = '$UNCHANGED$') → MCMQ

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

data (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for data parameter in fit.

Returns:

self – The updated object.

Return type:

object

class quapy.method.meta.MCSQ(classifiers, quantifier: AggregativeSoftQuantifier, merge_fun='median', val_split=5)

Bases: BaseQuantifier

fit(data: LabelledCollection)

Generates a quantifier.

Parameters:

• X – array-like, the training instances

• y – array-like, the labels

Returns:

self

quantify(instances)

Alias to predict(), for old compatibility

Parameters:

X – array-like

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

set_fit_request(*, data: bool | None | str = '$UNCHANGED$') → MCSQ

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

data (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for data parameter in fit.

Returns:

self – The updated object.

Return type:

object

class quapy.method.meta.MedianEstimator(base_quantifier: BinaryQuantifier, param_grid: dict, random_state=None, n_jobs=None)

Bases: BinaryQuantifier

This method is a meta-quantifier that returns, as the estimated class prevalence values, the median of the estimation returned by differently (hyper)parameterized base quantifiers. The median of unit-vectors is only guaranteed to be a unit-vector for n=2 dimensions, i.e., in cases of binary quantification.

Parameters:

• base_quantifier – the base, binary quantifier

• random_state – a seed to be set before fitting any base quantifier (default None)

• param_grid – the grid or parameters towards which the median will be computed

• n_jobs – number of parallel workers

fit(X, y)

Generates a quantifier.

Parameters:

• X – array-like, the training instances

• y – array-like, the labels

Returns:

self

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

predict(X)

Generate class prevalence estimates for the sample’s instances

Parameters:

X – array-like, the test instances

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

class quapy.method.meta.SCMQ(classifier, quantifiers: List[AggregativeSoftQuantifier], merge_fun='median', val_split=5)

Bases: AggregativeSoftQuantifier

MERGE_FUNCTIONS = ['median', 'mean']

aggregate(classif_predictions: ndarray)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

quapy.method.meta.ensembleFactory(classifier, base_quantifier_class, param_grid=None, optim=None, param_model_sel: dict = None, **kwargs)

Ensemble factory. Provides a unified interface for instantiating ensembles that can be optimized (via model selection for quantification) for a given evaluation metric using quapy.model_selection.GridSearchQ. If the evaluation metric is classification-oriented (instead of quantification-oriented), then the optimization will be carried out via sklearn’s GridSearchCV.

Example to instantiate an Ensemble based on quapy.method.aggregative.PACC in which the base members are optimized for quapy.error.mae() via quapy.model_selection.GridSearchQ. The ensemble follows the policy Accuracy based on quapy.error.mae() (the same measure being optimized), meaning that a static selection of members of the ensemble is made based on their performance in terms of this error.

>>> param_grid = {
>>>     'C': np.logspace(-3,3,7),
>>>     'class_weight': ['balanced', None]
>>> }
>>> param_mod_sel = {
>>>     'sample_size': 500,
>>>     'protocol': 'app'
>>> }
>>> common={
>>>     'max_sample_size': 1000,
>>>     'n_jobs': -1,
>>>     'param_grid': param_grid,
>>>     'param_mod_sel': param_mod_sel,
>>> }
>>>
>>> ensembleFactory(LogisticRegression(), PACC, optim='mae', policy='mae', **common)

Parameters:

• classifier – sklearn’s Estimator that generates a classifier

• base_quantifier_class – a class of quantifiers

• param_grid – a dictionary with the grid of parameters to optimize for

• optim – a valid quantification or classification error, or a string name of it

• param_model_sel – a dictionary containing any keyworded argument to pass to quapy.model_selection.GridSearchQ

• kwargs – kwargs for the class Ensemble

Returns:

an instance of Ensemble

quapy.method.meta.get_probability_distribution(posterior_probabilities, bins=8)

Gets a histogram out of the posterior probabilities (only for the binary case).

Parameters:

• posterior_probabilities – array-like of shape (n_instances, 2,)

• bins – integer

Returns:

np.ndarray with the relative frequencies for each bin (for the positive class only)

quapy.method.meta.merge(prev_predictions, merge_fun)

quapy.method.non_aggregative module

class quapy.method.non_aggregative.DMx(nbins=8, divergence: str | Callable = 'HD', cdf=False, search='optim_minimize', n_jobs=None)

Bases: BaseQuantifier

Generic Distribution Matching quantifier for binary or multiclass quantification based on the space of covariates. This implementation takes the number of bins, the divergence, and the possibility to work on CDF as hyperparameters.

Parameters:

• nbins – number of bins used to discretize the distributions (default 8)

• divergence – a string representing a divergence measure (currently, “HD” and “topsoe” are implemented) or a callable function taking two ndarrays of the same dimension as input (default “HD”, meaning Hellinger Distance)

• cdf – whether to use CDF instead of PDF (default False)

• search – string indicating the search strategy used to estimate the prevalence values. Valid options are optim_minimize (default, works for binary and multiclass problems), linear_search (binary only), and ternary_search (binary only)

• n_jobs – number of parallel workers (default None)

classmethod HDx(n_jobs=None)

Hellinger Distance x (HDx). HDx is a method for training binary quantifiers, that models quantification as the problem of minimizing the average divergence (in terms of the Hellinger Distance) across the feature-specific normalized histograms of two representations, one for the unlabelled examples, and another generated from the training examples as a mixture model of the class-specific representations. The parameters of the mixture thus represent the estimates of the class prevalence values.

The method computes all matchings for nbins in [10, 20, …, 110] and reports the mean of the median. The best prevalence is searched via linear search, from 0 to 1 stepping by 0.01.

Parameters:

n_jobs – number of parallel workers

Returns:

an instance of this class setup to mimick the performance of the HDx as originally proposed by González-Castro, Alaiz-Rodríguez, Alegre (2013)

fit(X, y)

Generates the validation distributions out of the training data (covariates). The validation distributions have shape (n, nfeats, nbins), with n the number of classes, nfeats the number of features, and nbins the number of bins. In particular, let V be the validation distributions; then di=V[i] are the distributions obtained from training data labelled with class i; while dij = di[j] is the discrete distribution for feature j in training data labelled with class i, and dij[k] is the fraction of instances with a value in the k-th bin.

Parameters:

• X – array-like of shape (n_samples, n_features), the training instances

• y – array-like of shape (n_samples,), the labels

predict(X)

Searches for the mixture model parameter (the sought prevalence values) that yields a validation distribution (the mixture) that best matches the test distribution, in terms of the divergence measure of choice. The matching is computed as the average dissimilarity (in terms of the dissimilarity measure of choice) between all feature-specific discrete distributions.

Parameters:

X – instances in the sample

Returns:

a vector of class prevalence estimates

quapy.method.non_aggregative.DistributionMatchingX

alias of DMx

class quapy.method.non_aggregative.EDx(distance: str | Callable = 'manhattan', n_jobs=None)

Bases: _EnergyDistanceCore, BaseQuantifier

Energy Distance x (EDx), a covariate-space distribution-matching quantifier based on energy distance.

EDx is the classifier-free counterpart of quapy.method.aggregative.EDy. Instead of representing each class through posterior-probability vectors, it represents each class by the cloud of raw feature vectors observed in the training set and estimates the test prevalence vector by solving the same energy-distance quadratic program directly in feature space.

This implementation works for binary and multiclass single-label quantification and relies on the optional quadprog dependency. The current QuaPy adaptation shares its numerical core with EDy and keeps credit to the original implementation available in quantificationlib.

The formulation follows the same references as EDy, namely:

• Alberto Castaño, Laura Morán-Fernández, Jaime Alonso, Verónica Bolón-Canedo, Amparo Alonso-Betanzos, and Juan José del Coz. An analysis of quantification methods based on matching distributions.

• Hideko Kawakubo, Marthinus Christoffel du Plessis, and Masashi Sugiyama (2016). Computationally efficient class-prior estimation under class balance change using energy distance. IEICE Transactions on Information and Systems, 99(1):176-186.

Parameters:

• distance – distance used to compare feature vectors. Valid string aliases are 'manhattan' (default) and 'euclidean'; a custom callable compatible with pairwise-distance signatures can also be used

• n_jobs – number of parallel workers (default None, meaning the value is taken from the environment)

fit(X, y)

Fit class-conditional feature-space distributions from training data.

predict(X)

Estimate class prevalences for a test sample of raw instances.

quapy.method.non_aggregative.EnergyDistanceX

alias of EDx

quapy.method.non_aggregative.HDx(n_jobs=None)

Hellinger Distance x (HDx). HDx is a method for training binary quantifiers, that models quantification as the problem of minimizing the average divergence (in terms of the Hellinger Distance) across the feature-specific normalized histograms of two representations, one for the unlabelled examples, and another generated from the training examples as a mixture model of the class-specific representations. The parameters of the mixture thus represent the estimates of the class prevalence values.

The method computes all matchings for nbins in [10, 20, …, 110] and reports the mean of the median. The best prevalence is searched via linear search, from 0 to 1 stepping by 0.01.

Parameters:

n_jobs – number of parallel workers

Returns:

an instance of this class setup to mimick the performance of the HDx as originally proposed by González-Castro, Alaiz-Rodríguez, Alegre (2013)

quapy.method.non_aggregative.HellingerDistanceX(n_jobs=None)

Hellinger Distance x (HDx). HDx is a method for training binary quantifiers, that models quantification as the problem of minimizing the average divergence (in terms of the Hellinger Distance) across the feature-specific normalized histograms of two representations, one for the unlabelled examples, and another generated from the training examples as a mixture model of the class-specific representations. The parameters of the mixture thus represent the estimates of the class prevalence values.

The method computes all matchings for nbins in [10, 20, …, 110] and reports the mean of the median. The best prevalence is searched via linear search, from 0 to 1 stepping by 0.01.

Parameters:

n_jobs – number of parallel workers

Returns:

an instance of this class setup to mimick the performance of the HDx as originally proposed by González-Castro, Alaiz-Rodríguez, Alegre (2013)

class quapy.method.non_aggregative.MaximumLikelihoodPrevalenceEstimation

Bases: BaseQuantifier

The Maximum Likelihood Prevalence Estimation (MLPE) method is a lazy method that assumes there is no prior probability shift between training and test instances (put it other way, that the i.i.d. assumpion holds). The estimation of class prevalence values for any test sample is always (i.e., irrespective of the test sample itself) the class prevalence seen during training. This method is considered to be a lower-bound quantifier that any quantification method should beat.

fit(X, y)

Computes the training prevalence and stores it.

Parameters:

• X – array-like of shape (n_samples, n_features), the training instances

• y – array-like of shape (n_samples,), the labels

Returns:

self

predict(X)

Ignores the input instances and returns, as the class prevalence estimantes, the training prevalence.

Parameters:

X – array-like (ignored)

Returns:

the class prevalence seen during training

class quapy.method.non_aggregative.ReadMe(prob_model='full', bootstrap_trials=300, bagging_trials=300, bagging_range=15, confidence_level=0.95, region='intervals', bonferroni=False, random_state=None, verbose=False)

Bases: BaseQuantifier, WithConfidenceABC

ReadMe is a non-aggregative quantification system proposed by Daniel Hopkins and Gary King, 2007. A method of automated nonparametric content analysis for social science. American Journal of Political Science, 54(1):229–247.. The idea is to estimate Q(Y=i) directly from:

\(Q(X)=\sum_{i=1} Q(X|Y=i) Q(Y=i)\)

via least-squares regression, i.e., without incurring the cost of computing posterior probabilities. However, this poses a very difficult representation in which the vector Q(X) and the matrix Q(X|Y=i) can be of very high dimensions. In order to render the problem tracktable, ReadMe performs bagging in the feature space. ReadMe also combines bagging with bootstrap in order to derive confidence intervals around point estimations.

We use the same default parameters as in the official R implementation.

Parameters:

• prob_model – str (‘naive’, or ‘full’), selects the modality in which the probabilities Q(X) and Q(X|Y) are to be modelled. Options include “full”, which corresponds to the original formulation of ReadMe, in which X is constrained to be a binary matrix (e.g., of term presence/absence) and in which Q(X) and Q(X|Y) are modelled, respectively, as matrices of (2^K, 1) and (2^K, n) values, where K is the number of columns in the data matrix (i.e., bagging_range), and n is the number of classes. Of course, this approach is computationally prohibited for large K, so the authors advised against computing it for matrices with K>25 (although we recommend even smaller values of K). A much faster model is “naive”, which considers the Q(X) and Q(X|Y) be multinomial distributions under the bag-of-words perspective. In this case, bagging_range can be set to much larger values. Default is “full” (i.e., original ReadMe behavior).

• bootstrap_trials – int, number of bootstrap trials (default 300)

• bagging_trials – int, number of bagging trials (default 300)

• bagging_range – int, number of features to keep for each bagging trial (default 15)

• confidence_level – float, a value in (0,1) reflecting the desired confidence level (default 0.95)

• region – str in ‘intervals’, ‘ellipse’, ‘ellipse-clr’; indicates the preferred method for defining the confidence region (see WithConfidenceABC)

• bonferroni – bool (default False), whether to apply Bonferroni correction when region=’intervals’. This parameter has no effect for ellipse-based regions.

• random_state – int or None, allows replicability (default None)

• verbose – bool, whether to display information during the process (default False)

MAX_FEATURES_FOR_EMPIRICAL_ESTIMATION = 25

PROBABILISTIC_MODELS = ['naive', 'full']

fit(X, y)

Generates a quantifier.

Parameters:

• X – array-like, the training instances

• y – array-like, the labels

Returns:

self

predict(X)

Generate class prevalence estimates for the sample’s instances

Parameters:

X – array-like, the test instances

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

predict_conf(X, confidence_level=None) -> (<class 'numpy.ndarray'>, <class 'quapy.method.confidence.ConfidenceRegionABC'>)

Adds the method predict_conf to the interface. This method returns not only the point-estimate, but also the confidence region around it.

Parameters:

• instances – a np.ndarray of shape (n_instances, n_features,)

• confidence_level – float in (0, 1), default is 0.95

Returns:

a tuple (point_estimate, conf_region), where point_estimate is a np.ndarray of shape (n_classes,) and conf_region is an object from ConfidenceRegionABC

quapy.method.composable module

This module allows the composition of quantification methods from loss functions and feature transformations. This functionality is realized through an integration of the qunfold package: mirkobunse/qunfold.

class quapy.method.composable.BlobelLoss

Bases: FunctionLoss

The loss function of RUN (Blobel, 1985).

This loss function models a likelihood function under the assumption of independent Poisson-distributed elements of q with Poisson rates M*p.

class quapy.method.composable.CVClassifier(estimator, n_estimators=5, random_state=None)

Bases: BaseEstimator, ClassifierMixin

An ensemble of classifiers that are trained from cross-validation folds.

All objects of this type have a fixed attribute oob_score = True and, when trained, a fitted attribute self.oob_decision_function_, just like scikit-learn bagging classifiers.

Parameters:

• estimator – A classifier that implements the API of scikit-learn.

• n_estimators (optional) – The number of stratified cross-validation folds. Defaults to 5.

• random_state (optional) – The random state for stratification. Defaults to None.

Examples

Here, we create an instance of ACC that trains a logistic regression classifier with 10 cross-validation folds.

>>> ACC(CVClassifier(LogisticRegression(), 10))

fit(X, y, sample_weight=None)

predict(X)

predict_proba(X)

set_fit_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CVClassifier

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in fit.

Returns:

self – The updated object.

Return type:

object

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → CVClassifier

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_weight (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for sample_weight parameter in score.

Returns:

self – The updated object.

Return type:

object

class quapy.method.composable.ClassRepresentation(classifier: Any, is_probabilistic: bool = False, fit_classifier: bool = True)

Bases: AbstractRepresentation

A classification-based data representation.

This representation can either be probabilistic (using the posterior predictions of a classifier) or crisp (using the class predictions of a classifier). It is used in ACC, PACC, CC, PCC, and SLD.

Parameters:

• classifier – A classifier that implements the API of scikit-learn.

• is_probabilistic (optional) – Whether probabilistic or crisp predictions of the classifier are used to represent the data. Defaults to False.

• fit_classifier (optional) – Whether to fit the classifier when this quantifier is fitted. Defaults to True.

classifier: Any

fit_classifier: bool = True

fit_transform(X, y, sample_weight=None, average=True, n_classes=None)

This abstract method has to fit the representation and to return the transformed input data.

Note

Implementations of this abstract method should check the sanity of labels by calling check_y(y, n_classes) and they must set the property self.p_trn = class_prevalences(y, n_classes).

Parameters:

• X – The feature matrix to which this representation will be fitted.

• y – The labels to which this representation will be fitted.

• sample_weight (optional) – Importance weights for each (X[i], y[i]) pair to use during fitting. Defaults to None.

• average (optional) – Whether to return a transfer matrix M or a transformation f(X). Defaults to True.

• n_classes (optional) – The number of expected classes. Defaults to None.

Returns:

A transfer matrix M if average==True or a transformation f(X) if average==False. f(X) might contain NaN entries if the fitting permits the estimation, e.g., in bootstrapped fitting procedures.

is_probabilistic: bool = False

property n_output_features

The number of output features generated by this representation.

transform(X, sample_weight=None, average=True)

This abstract method has to transform the data X.

Parameters:

• X – The feature matrix that will be transformed.

• sample_weight (optional) – Importance weights for each X[i] to use during averaging if average==True. Defaults to None.

• average (optional) – Whether to return a vector q or a transformation f(X). Defaults to True.

Returns:

A vector q = f(X).average(axis=0, weights=sample_weight) if average==True or a transformation f(X) if average==False.

class quapy.method.composable.CombinedLoss(*losses, weights=None)

Bases: AbstractLoss

The weighted sum of multiple losses.

Parameters:

• *losses – An arbitrary number of losses to be added together.

• weights (optional) – An array of weights which the losses are scaled.

instantiate(q, M, N)

This abstract method has to create a lambda expression p -> loss with JAX.

In particular, your implementation of this abstract method should return a lambda expression

>>> return lambda p: loss_value(q, M, p, N)

where loss_value has to return the result of a JAX expression. The JAX requirement ensures that the loss function can be auto-differentiated. Hence, no derivatives of the loss function have to be provided manually. JAX expressions are easy to implement. Just import the numpy wrapper

>>> import jax.numpy as jnp

and use jnp just as if you would use numpy.

Note

p is a vector of class-wise probabilities. This vector will already be the result of our soft-max trick, so that you don’t have to worry about constraints or latent parameters.

Parameters:

• q – A numpy array.

• M – A numpy matrix.

• N – The number of data items that q represents.

Returns:

A lambda expression p -> loss, implemented in JAX.

Examples

The least squares loss, (q - M*p)’ * (q - M*p), is simply

>>> jnp.dot(q - jnp.dot(M, p), q - jnp.dot(M, p))

quapy.method.composable.ComposableQuantifier(loss, representation, **kwargs)

A generic quantification / unfolding method that solves a linear system of equations.

This class represents any quantifier that can be described in terms of a loss function, a feature transformation, and a regularization term. In this implementation, the loss is minimized through unconstrained second-order minimization. Valid probability estimates are ensured through a soft-max trick by Bunse (2022).

Parameters:

• loss – An instance of a loss class from quapy.methods.composable.

• representation – An instance of a representation class from quapy.methods.composable.

• solver (optional) – The method argument in scipy.optimize.minimize. Defaults to “trust-ncg”.

• solver_options (optional) – The options argument in scipy.optimize.minimize. Defaults to {“gtol”: 1e-8, “maxiter”: 1000}.

• seed (optional) – A random number generator seed from which a numpy RandomState is created. Defaults to None.

Examples

Here, we create the ordinal variant of ACC (Bunse et al., 2023). This variant consists of the original feature transformation of ACC and of the original loss of ACC, the latter of which is regularized towards smooth solutions.

>>> from quapy.method.composable import (
>>>     ComposableQuantifier,
>>>     TikhonovRegularized,
>>>     LeastSquaresLoss,
>>>     ClassRepresentation,
>>> )
>>> from sklearn.ensemble import RandomForestClassifier
>>> o_acc = ComposableQuantifier(
>>>     TikhonovRegularized(LeastSquaresLoss(), 0.01),
>>>     ClassRepresentation(RandomForestClassifier(oob_score=True))
>>> )

Here, we perform hyper-parameter optimization with the ordinal ACC.

>>> quapy.model_selection.GridSearchQ(
>>>     model = o_acc,
>>>     param_grid = { # try both splitting criteria
>>>         "representation__classifier__estimator__criterion": ["gini", "entropy"],
>>>     },
>>>     # ...
>>> )

To use a classifier that does not provide the oob_score argument, such as logistic regression, you have to configure a cross validation of this classifier. Here, we employ 10 cross validation folds. 5 folds are the default.

>>> from quapy.method.composable import CVClassifier
>>> from sklearn.linear_model import LogisticRegression
>>> acc_lr = ComposableQuantifier(
>>>     LeastSquaresLoss(),
>>>     ClassRepresentation(CVClassifier(LogisticRegression(), 10))
>>> )

class quapy.method.composable.DistanceRepresentation(metric: str = 'euclidean', preprocessor: AbstractRepresentation | None = None)

Bases: AbstractRepresentation

A distance-based data representation, as it is used in EDx and EDy.

Parameters:

• metric (optional) – The metric with which the distance between data items is measured. Can take any value that is accepted by scipy.spatial.distance.cdist. Defaults to “euclidean”.

• preprocessor (optional) – Another AbstractRepresentation that is called before this representation. Defaults to None.

fit_transform(X, y, sample_weight=None, average=True, n_classes=None)

This abstract method has to fit the representation and to return the transformed input data.

Note

Implementations of this abstract method should check the sanity of labels by calling check_y(y, n_classes) and they must set the property self.p_trn = class_prevalences(y, n_classes).

Parameters:

• X – The feature matrix to which this representation will be fitted.

• y – The labels to which this representation will be fitted.

• sample_weight (optional) – Importance weights for each (X[i], y[i]) pair to use during fitting. Defaults to None.

• average (optional) – Whether to return a transfer matrix M or a transformation f(X). Defaults to True.

• n_classes (optional) – The number of expected classes. Defaults to None.

Returns:

A transfer matrix M if average==True or a transformation f(X) if average==False. f(X) might contain NaN entries if the fitting permits the estimation, e.g., in bootstrapped fitting procedures.

metric: str = 'euclidean'

property n_output_features

The number of output features generated by this representation.

preprocessor: AbstractRepresentation | None = None

transform(X, sample_weight=None, average=True)

This abstract method has to transform the data X.

Parameters:

• X – The feature matrix that will be transformed.

• sample_weight (optional) – Importance weights for each X[i] to use during averaging if average==True. Defaults to None.

• average (optional) – Whether to return a vector q or a transformation f(X). Defaults to True.

Returns:

A vector q = f(X).average(axis=0, weights=sample_weight) if average==True or a transformation f(X) if average==False.

class quapy.method.composable.EnergyKernelRepresentation(preprocessor: AbstractRepresentation | None = None)

Bases: AbstractRepresentation

A kernel-based data representation, as it is used in KMM, that uses the energy kernel:

k(x_1, x_2) = ||x_1|| + ||x_2|| - ||x_1 - x_2||

Note

The methods of this representation do not support setting average=False.

Parameters:

preprocessor (optional) – Another AbstractRepresentation that is called before this representation. Defaults to None.

fit_transform(X, y, sample_weight=None, average=True, n_classes=None)

This abstract method has to fit the representation and to return the transformed input data.

Note

Implementations of this abstract method should check the sanity of labels by calling check_y(y, n_classes) and they must set the property self.p_trn = class_prevalences(y, n_classes).

Parameters:

• X – The feature matrix to which this representation will be fitted.

• y – The labels to which this representation will be fitted.

• sample_weight (optional) – Importance weights for each (X[i], y[i]) pair to use during fitting. Defaults to None.

• average (optional) – Whether to return a transfer matrix M or a transformation f(X). Defaults to True.

• n_classes (optional) – The number of expected classes. Defaults to None.

Returns:

A transfer matrix M if average==True or a transformation f(X) if average==False. f(X) might contain NaN entries if the fitting permits the estimation, e.g., in bootstrapped fitting procedures.

property n_output_features

The number of output features generated by this representation.

preprocessor: AbstractRepresentation | None = None

transform(X, sample_weight=None, average=True)

This abstract method has to transform the data X.

Parameters:

• X – The feature matrix that will be transformed.

• sample_weight (optional) – Importance weights for each X[i] to use during averaging if average==True. Defaults to None.

• average (optional) – Whether to return a vector q or a transformation f(X). Defaults to True.

Returns:

A vector q = f(X).average(axis=0, weights=sample_weight) if average==True or a transformation f(X) if average==False.

class quapy.method.composable.EnergyLoss

Bases: FunctionLoss

The loss function of EDx (Kawakubo et al., 2016) and EDy (Castaño et al., 2022).

This loss function represents the Energy Distance between two samples.

class quapy.method.composable.GaussianKernelRepresentation(sigma: float = 1.0, preprocessor: AbstractRepresentation | None = None)

Bases: AbstractRepresentation

A kernel-based data representation, as it is used in KMM, that uses the gaussian kernel:

k(x, y) = exp(-||x - y||^2 / (2σ^2))

Parameters:

• sigma (optional) – A smoothing parameter of the kernel function. Defaults to 1.

• preprocessor (optional) – Another AbstractRepresentation that is called before this representation. Defaults to None.

fit_transform(X, y, sample_weight=None, average=True, n_classes=None)

This abstract method has to fit the representation and to return the transformed input data.

Note

Implementations of this abstract method should check the sanity of labels by calling check_y(y, n_classes) and they must set the property self.p_trn = class_prevalences(y, n_classes).

Parameters:

• X – The feature matrix to which this representation will be fitted.

• y – The labels to which this representation will be fitted.

• sample_weight (optional) – Importance weights for each (X[i], y[i]) pair to use during fitting. Defaults to None.

• average (optional) – Whether to return a transfer matrix M or a transformation f(X). Defaults to True.

• n_classes (optional) – The number of expected classes. Defaults to None.

Returns:

A transfer matrix M if average==True or a transformation f(X) if average==False. f(X) might contain NaN entries if the fitting permits the estimation, e.g., in bootstrapped fitting procedures.

property n_output_features

The number of output features generated by this representation.

preprocessor: AbstractRepresentation | None = None

sigma: float = 1.0

transform(X, sample_weight=None, average=True)

This abstract method has to transform the data X.

Parameters:

• X – The feature matrix that will be transformed.

• sample_weight (optional) – Importance weights for each X[i] to use during averaging if average==True. Defaults to None.

• average (optional) – Whether to return a vector q or a transformation f(X). Defaults to True.

Returns:

A vector q = f(X).average(axis=0, weights=sample_weight) if average==True or a transformation f(X) if average==False.

class quapy.method.composable.GaussianRFFKernelRepresentation(sigma: float = 1.0, n_rff: int = 1000, preprocessor: AbstractRepresentation | None = None, seed: int | None = None)

Bases: AbstractRepresentation

An efficient approximation of the GaussianKernelRepresentation, as it is used in KMM, using random Fourier features.

Parameters:

• sigma (optional) – A smoothing parameter of the kernel function. Defaults to 1.

• n_rff (optional) – The number of random Fourier features. Defaults to 1000.

• preprocessor (optional) – Another AbstractRepresentation that is called before this representation. Defaults to None.

• seed (optional) – Controls the randomness of the random Fourier features. Defaults to None.

fit_transform(X, y, sample_weight=None, average=True, n_classes=None)

This abstract method has to fit the representation and to return the transformed input data.

Note

Implementations of this abstract method should check the sanity of labels by calling check_y(y, n_classes) and they must set the property self.p_trn = class_prevalences(y, n_classes).

Parameters:

• X – The feature matrix to which this representation will be fitted.

• y – The labels to which this representation will be fitted.

• sample_weight (optional) – Importance weights for each (X[i], y[i]) pair to use during fitting. Defaults to None.

• average (optional) – Whether to return a transfer matrix M or a transformation f(X). Defaults to True.

• n_classes (optional) – The number of expected classes. Defaults to None.

Returns:

A transfer matrix M if average==True or a transformation f(X) if average==False. f(X) might contain NaN entries if the fitting permits the estimation, e.g., in bootstrapped fitting procedures.

property n_output_features

The number of output features generated by this representation.

n_rff: int = 1000

preprocessor: AbstractRepresentation | None = None

seed: int | None = None

sigma: float = 1.0

transform(X, sample_weight=None, average=True)

This abstract method has to transform the data X.

Parameters:

• X – The feature matrix that will be transformed.

• sample_weight (optional) – Importance weights for each X[i] to use during averaging if average==True. Defaults to None.

• average (optional) – Whether to return a vector q or a transformation f(X). Defaults to True.

Returns:

A vector q = f(X).average(axis=0, weights=sample_weight) if average==True or a transformation f(X) if average==False.

class quapy.method.composable.HellingerSurrogateLoss

Bases: FunctionLoss

The loss function of HDx and HDy (González-Castro et al., 2013).

This loss function computes the average of the squared Hellinger distances between feature-wise (or class-wise) histograms. Note that the original HDx and HDy by González-Castro et al (2013) do not use the squared but the regular Hellinger distance. Their approach is problematic because the regular distance is not always twice differentiable and, hence, complicates numerical optimizations.

class quapy.method.composable.HistogramRepresentation(n_bins: int, preprocessor: AbstractRepresentation | None = None, unit_scale: bool = True)

Bases: AbstractRepresentation

A histogram-based data representation, as it is used in HDx and HDy.

Parameters:

• n_bins – The number of bins in each feature.

• preprocessor (optional) – Another AbstractRepresentation that is called before this representation. Defaults to None.

• unit_scale (optional) – Whether or not to scale each output to a sum of one. A value of False indicates that the sum of each output is the number of features. Defaults to True.

fit_transform(X, y, sample_weight=None, average=True, n_classes=None)

This abstract method has to fit the representation and to return the transformed input data.

Note

Implementations of this abstract method should check the sanity of labels by calling check_y(y, n_classes) and they must set the property self.p_trn = class_prevalences(y, n_classes).

Parameters:

• X – The feature matrix to which this representation will be fitted.

• y – The labels to which this representation will be fitted.

• sample_weight (optional) – Importance weights for each (X[i], y[i]) pair to use during fitting. Defaults to None.

• average (optional) – Whether to return a transfer matrix M or a transformation f(X). Defaults to True.

• n_classes (optional) – The number of expected classes. Defaults to None.

Returns:

A transfer matrix M if average==True or a transformation f(X) if average==False. f(X) might contain NaN entries if the fitting permits the estimation, e.g., in bootstrapped fitting procedures.

n_bins: int

property n_output_features

The number of output features generated by this representation.

preprocessor: AbstractRepresentation | None = None

transform(X, sample_weight=None, average=True)

This abstract method has to transform the data X.

Parameters:

• X – The feature matrix that will be transformed.

• sample_weight (optional) – Importance weights for each X[i] to use during averaging if average==True. Defaults to None.

• average (optional) – Whether to return a vector q or a transformation f(X). Defaults to True.

Returns:

A vector q = f(X).average(axis=0, weights=sample_weight) if average==True or a transformation f(X) if average==False.

unit_scale: bool = True

class quapy.method.composable.KernelRepresentation(kernel: Callable)

Bases: AbstractRepresentation

A general kernel-based data representation, as it is used in KMM. If you intend to use a Gaussian kernel or energy kernel, prefer their dedicated and more efficient implementations over this class.

Note

The methods of this representation do not support setting average=False.

Parameters:

kernel – A callable that will be used as the kernel. Must follow the signature (X[y==i], X[y==j]) -> scalar.

fit_transform(X, y, sample_weight=None, average=True, n_classes=None)

This abstract method has to fit the representation and to return the transformed input data.

Note

Implementations of this abstract method should check the sanity of labels by calling check_y(y, n_classes) and they must set the property self.p_trn = class_prevalences(y, n_classes).

Parameters:

• X – The feature matrix to which this representation will be fitted.

• y – The labels to which this representation will be fitted.

• sample_weight (optional) – Importance weights for each (X[i], y[i]) pair to use during fitting. Defaults to None.

• average (optional) – Whether to return a transfer matrix M or a transformation f(X). Defaults to True.

• n_classes (optional) – The number of expected classes. Defaults to None.

Returns:

A transfer matrix M if average==True or a transformation f(X) if average==False. f(X) might contain NaN entries if the fitting permits the estimation, e.g., in bootstrapped fitting procedures.

kernel: Callable

property n_output_features

The number of output features generated by this representation.

transform(X, sample_weight=None, average=True)

This abstract method has to transform the data X.

Parameters:

• X – The feature matrix that will be transformed.

• sample_weight (optional) – Importance weights for each X[i] to use during averaging if average==True. Defaults to None.

• average (optional) – Whether to return a vector q or a transformation f(X). Defaults to True.

Returns:

A vector q = f(X).average(axis=0, weights=sample_weight) if average==True or a transformation f(X) if average==False.

class quapy.method.composable.LaplacianKernelRepresentation(sigma=1.0)

Bases: KernelRepresentation

A kernel-based data representation, as it is used in KMM, that uses the laplacian kernel.

Parameters:

sigma (optional) – A smoothing parameter of the kernel function. Defaults to 1.

static create_kernel(sigma)

class quapy.method.composable.LeastSquaresLoss

Bases: FunctionLoss

The loss function of ACC (Forman, 2008), PACC (Bella et al., 2019), and ReadMe (Hopkins & King, 2010).

This loss function computes the sum of squares of element-wise errors between q and M*p.

class quapy.method.composable.QUnfoldWrapper(_method: AbstractMethod)

Bases: BaseQuantifier, BaseMixin

A thin wrapper for using qunfold methods in QuaPy.

Parameters:

_method – An instance of qunfold.methods.AbstractMethod to wrap.

Examples

Here, we wrap an instance of ACC to perform a grid search with QuaPy.

>>> from qunfold import ACC
>>> qunfold_method = QUnfoldWrapper(ACC(RandomForestClassifier(obb_score=True)))
>>> quapy.model_selection.GridSearchQ(
>>>     model = qunfold_method,
>>>     param_grid = { # try both splitting criteria
>>>         "representation__classifier__estimator__criterion": ["gini", "entropy"],
>>>     },
>>>     # ...
>>> )

fit(X, y)

Generates a quantifier.

Parameters:

• X – array-like, the training instances

• y – array-like, the labels

Returns:

self

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep (bool, default=True) – If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params – Parameter names mapped to their values.

Return type:

dict

predict(X)

Generate class prevalence estimates for the sample’s instances

Parameters:

X – array-like, the test instances

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params (dict) – Estimator parameters.

Returns:

self – Estimator instance.

Return type:

estimator instance

class quapy.method.composable.TikhonovRegularization

Bases: AbstractLoss

Tikhonov regularization, as proposed by Blobel (1985).

This regularization promotes smooth solutions. This behavior is often required in ordinal quantification and in unfolding problems.

instantiate(q, M, N)

This abstract method has to create a lambda expression p -> loss with JAX.

In particular, your implementation of this abstract method should return a lambda expression

>>> return lambda p: loss_value(q, M, p, N)

where loss_value has to return the result of a JAX expression. The JAX requirement ensures that the loss function can be auto-differentiated. Hence, no derivatives of the loss function have to be provided manually. JAX expressions are easy to implement. Just import the numpy wrapper

>>> import jax.numpy as jnp

and use jnp just as if you would use numpy.

Note

p is a vector of class-wise probabilities. This vector will already be the result of our soft-max trick, so that you don’t have to worry about constraints or latent parameters.

Parameters:

• q – A numpy array.

• M – A numpy matrix.

• N – The number of data items that q represents.

Returns:

A lambda expression p -> loss, implemented in JAX.

Examples

The least squares loss, (q - M*p)’ * (q - M*p), is simply

>>> jnp.dot(q - jnp.dot(M, p), q - jnp.dot(M, p))

quapy.method.composable.TikhonovRegularized(loss, tau=0.0)

Add TikhonovRegularization (Blobel, 1985) to any loss.

Calling this function is equivalent to calling

>>> CombinedLoss(loss, TikhonovRegularization(), weights=[1, tau])

Parameters:

• loss – An instance from qunfold.losses.

• tau (optional) – The regularization strength. Defaults to 0.

Returns:

An instance of CombinedLoss.

Examples

The regularized loss of RUN (Blobel, 1985) is:

>>> TikhonovRegularization(BlobelLoss(), tau)

quapy.method.confidence module

class quapy.method.confidence.AggregativeBootstrap(quantifier: AggregativeQuantifier, n_train_samples=1, n_test_samples=500, confidence_level=0.95, region='intervals', bonferroni=False, random_state=None, verbose=False)

Bases: WithConfidenceABC, AggregativeQuantifier

Aggregative Bootstrap allows any AggregativeQuantifier to get confidence regions around point-estimates of class prevalence values. This method implements some optimizations for speeding up the computations, which are only possible due to the two phases of the aggregative quantifiers.

During training, the bootstrap repetitions are only carried out over pre-classified training instances, after the classifier has been trained (only once), in order to train a series of aggregation functions (model-based approach).

During inference, the bootstrap repetitions are applied to the pre-classified test instances.

See Moreo, A., Salvati, N. An Efficient Method for Deriving Confidence Intervals in Aggregative Quantification. Learning to Quantify: Methods and Applications (LQ 2025), co-located at ECML-PKDD 2025. pp 12-33

Parameters:

• quantifier – an aggregative quantifier

• confidence_level – float, the confidence level for the confidence region (default 0.95)

• region – string, set to intervals for constructing confidence intervals (default), or to ellipse for constructing an ellipse in the probability simplex, or to ellipse-clr for constructing an ellipse in the Centered-Log Ratio (CLR) unconstrained space.

• bonferroni – bool (default False), whether to apply Bonferroni correction when region=’intervals’. This parameter has no effect for ellipse-based regions.

• random_state – int for replicating samples, None (default) for non-replicable samples

Para n_train_samples:

int, the number of training resamplings (defaults to 1, set to > 1 to activate a model-based bootstrap approach)

Para n_test_samples:

int, the number of test resamplings (defaults to 500, set to > 1 to activate a population-based bootstrap approach)

aggregate(classif_predictions: ndarray)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregate_conf(classif_predictions: ndarray, confidence_level=None)

aggregate_conf_sequential__(classif_predictions: ndarray, confidence_level=None)

aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

property classifier

Gives access to the classifier

Returns:

the classifier (typically an sklearn’s Estimator)

fit(X, y)

Trains the aggregative quantifier. This comes down to training a classifier (if requested) and an aggregation function.

Parameters:

• X – array-like of shape (n_samples, n_features), the training instances

• y – array-like of shape (n_samples,), the labels

Returns:

self

predict_conf(instances, confidence_level=None) -> (<class 'numpy.ndarray'>, <class 'quapy.method.confidence.ConfidenceRegionABC'>)

Adds the method predict_conf to the interface. This method returns not only the point-estimate, but also the confidence region around it.

Parameters:

• instances – a np.ndarray of shape (n_instances, n_features,)

• confidence_level – float in (0, 1), default is 0.95

Returns:

a tuple (point_estimate, conf_region), where point_estimate is a np.ndarray of shape (n_classes,) and conf_region is an object from ConfidenceRegionABC

class quapy.method.confidence.BayesianCC(classifier: BaseEstimator = None, fit_classifier=True, val_split: int = 5, num_warmup: int = 500, num_samples: int = 1000, mcmc_seed: int = 0, confidence_level: float = 0.95, region: str = 'intervals', bonferroni: bool = False, temperature=1.0, prior='uniform')

Bases: AggregativeCrispQuantifier, WithConfidenceABC

Bayesian quantification method (by Albert Ziegler and Paweł Czyż), which is a variant of ACC that calculates the posterior probability distribution over the prevalence vectors, rather than providing a point estimate obtained by matrix inversion.

Can be used to diagnose degeneracy in the predictions visible when the confusion matrix has high condition number or to quantify uncertainty around the point estimate.

This method relies on extra dependencies, which have to be installed via: $ pip install quapy[bayes]

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation. Set to None when the method does not require any validation data, in order to avoid that some portion of the training data be wasted.

• num_warmup – number of warmup iterations for the MCMC sampler (default 500)

• num_samples – number of samples to draw from the posterior (default 1000)

• mcmc_seed – random seed for the MCMC sampler (default 0)

• confidence_level – float in [0,1] to construct a confidence region around the point estimate (default 0.95)

• region – string, set to intervals for constructing confidence intervals (default), or to ellipse for constructing an ellipse in the probability simplex, or to ellipse-clr for constructing an ellipse in the Centered-Log Ratio (CLR) unconstrained space.

• bonferroni – bool (default False), whether to apply Bonferroni correction when region=’intervals’. This parameter has no effect for ellipse-based regions.

• prior – an array-like with the alpha parameters of a Dirichlet prior, a scalar real value to be broadcast to all classes, or the string ‘uniform’ for a uniform, uninformative prior (default)

aggregate(classif_predictions)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregation_fit(classif_predictions, labels)

Estimates the misclassification rates.

Parameters:

• classif_predictions – array-like with the label predictions returned by the classifier

• labels – array-like with the true labels associated to each classifier prediction

get_conditional_probability_samples()

get_prevalence_samples()

predict_conf(instances, confidence_level=None) -> (<class 'numpy.ndarray'>, <class 'quapy.method.confidence.ConfidenceRegionABC'>)

Adds the method predict_conf to the interface. This method returns not only the point-estimate, but also the confidence region around it.

Parameters:

• instances – a np.ndarray of shape (n_instances, n_features,)

• confidence_level – float in (0, 1), default is 0.95

Returns:

a tuple (point_estimate, conf_region), where point_estimate is a np.ndarray of shape (n_classes,) and conf_region is an object from ConfidenceRegionABC

sample_from_posterior(classif_predictions)

class quapy.method.confidence.ConfidenceEllipseCLR(samples, confidence_level=0.95)

Bases: ConfidenceEllipseTransformed

Instantiates a Confidence Ellipse in the Centered-Log Ratio (CLR) space.

Parameters:

• samples – np.ndarray of shape (n_samples, n_classes)

• confidence_level – float, the confidence level (default 0.95)

class quapy.method.confidence.ConfidenceEllipseILR(samples, confidence_level=0.95)

Bases: ConfidenceEllipseTransformed

Instantiates a Confidence Ellipse in the Isometric-Log Ratio (CLR) space.

Parameters:

• samples – np.ndarray of shape (n_samples, n_classes)

• confidence_level – float, the confidence level (default 0.95)

class quapy.method.confidence.ConfidenceEllipseSimplex(samples, confidence_level=0.95)

Bases: ConfidenceRegionABC

Instantiates a Confidence Ellipse in the probability simplex.

Parameters:

• samples – np.ndarray of shape (n_samples, n_classes)

• confidence_level – float, the confidence level (default 0.95)

closest_point_in_region(p, tol=1e-06, max_iter=30)

Finds the closes point to p that belongs to the region. Assumes the region is convex.

Parameters:

• p – array-like, the point

• tol – float, error tolerance

• max_iter – int, max number of iterations

Returns:

array-like, the closes point to p in the segment between p and the center of the region, that belongs to the region

coverage(true_value)

Checks whether a value, or a sets of values, are contained in the confidence region. The method computes the fraction of these that are contained in the region, if more than one value is passed. If only one value is passed, then it either returns 1.0 or 0.0, for indicating the value is in the region or not, respectively.

Parameters:

true_value – a np.ndarray of shape (n_classes,) or shape (n_values, n_classes,)

Returns:

float in [0,1]

point_estimate()

Returns the point estimate, the center of the ellipse.

Returns:

np.ndarray of shape (n_classes,)

property samples

Returns internal samples

class quapy.method.confidence.ConfidenceEllipseTransformed(samples, transformation: CompositionalTransformation, confidence_level=0.95)

Bases: ConfidenceRegionABC

Instantiates a Confidence Ellipse in a transformed space.

Parameters:

• samples – np.ndarray of shape (n_samples, n_classes)

• confidence_level – float, the confidence level (default 0.95)

closest_point_in_region(p, tol=1e-06, max_iter=30)

Finds the closes point to p that belongs to the region. Assumes the region is convex.

Parameters:

• p – array-like, the point

• tol – float, error tolerance

• max_iter – int, max number of iterations

Returns:

array-like, the closes point to p in the segment between p and the center of the region, that belongs to the region

coverage(true_value)

Checks whether a value, or a sets of values, are contained in the confidence region. The method computes the fraction of these that are contained in the region, if more than one value is passed. If only one value is passed, then it either returns 1.0 or 0.0, for indicating the value is in the region or not, respectively.

Parameters:

true_value – a np.ndarray of shape (n_classes,) or shape (n_values, n_classes,)

Returns:

float in [0,1]

point_estimate()

Returns the point estimate, the center of the ellipse.

Returns:

np.ndarray of shape (n_classes,)

property samples

Returns internal samples

class quapy.method.confidence.ConfidenceIntervals(samples, confidence_level=0.95, bonferroni_correction=False)

Bases: ConfidenceRegionABC

Instantiates a region based on (independent) Confidence Intervals.

Parameters:

• samples – np.ndarray of shape (n_samples, n_classes)

• confidence_level – float, the confidence level (default 0.95)

• bonferroni_correction – bool (default False), if True, a Bonferroni correction is applied to the significance level (alpha) before computing confidence intervals. The correction consists of replacing alpha with alpha/n_classes. When n_classes=2 the correction is not applied because there is only one verification test since the other class is constrained. This is not necessarily true for n_classes>2.

coverage(true_value)

Checks whether a value, or a sets of values, are contained in the confidence region. The method computes the fraction of these that are contained in the region, if more than one value is passed. If only one value is passed, then it either returns 1.0 or 0.0, for indicating the value is in the region or not, respectively.

Parameters:

true_value – a np.ndarray of shape (n_classes,) or shape (n_values, n_classes,)

Returns:

float in [0,1]

coverage_soft(true_value)

mean_winkler_score(true_prev, alpha=None, add_ae=False)

property n_dim

point_estimate()

Returns the point estimate, the class-wise average of the bootstrapped estimates

Returns:

np.ndarray of shape (n_classes,)

property samples

Returns internal samples

winkler_scores(true_prev, alpha=None, add_ae=False)

class quapy.method.confidence.ConfidenceIntervalsCLR(samples, confidence_level=0.95, bonferroni_correction=False)

Bases: ConfidenceIntervalsTransformed

Instantiates a Confidence Intervals in the Centered-Log Ratio (CLR) space.

Parameters:

• samples – np.ndarray of shape (n_samples, n_classes)

• confidence_level – float, the confidence level (default 0.95)

• bonferroni_correction – bool (default False), if True, a Bonferroni correction is applied to the significance level (alpha) before computing confidence intervals. The correction consists of replacing alpha with alpha/n_classes. When n_classes=2 the correction is not applied because there is only one verification test since the other class is constrained. This is not necessarily true for n_classes>2.

class quapy.method.confidence.ConfidenceIntervalsILR(samples, confidence_level=0.95, bonferroni_correction=False)

Bases: ConfidenceIntervalsTransformed

Instantiates a Confidence Intervals in the Isometric-Log Ratio (CLR) space.

Parameters:

• samples – np.ndarray of shape (n_samples, n_classes)

• confidence_level – float, the confidence level (default 0.95)

• bonferroni_correction – bool (default False), if True, a Bonferroni correction is applied to the significance level (alpha) before computing confidence intervals. The correction consists of replacing alpha with alpha/n_classes. When n_classes=2 the correction is not applied because there is only one verification test since the other class is constrained. This is not necessarily true for n_classes>2.

class quapy.method.confidence.ConfidenceIntervalsTransformed(samples, transformation: CompositionalTransformation, confidence_level=0.95, bonferroni_correction=False)

Bases: ConfidenceRegionABC

Instantiates a Confidence Interval region in a transformed space.

Parameters:

• samples – np.ndarray of shape (n_samples, n_classes)

• confidence_level – float, the confidence level (default 0.95)

• bonferroni_correction – bool (default False), if True, a Bonferroni correction is applied to the significance level (alpha) before computing confidence intervals. The correction consists of replacing alpha with alpha/n_classes. When n_classes=2 the correction is not applied because there is only one verification test since the other class is constrained. This is not necessarily true for n_classes>2.

coverage(true_value)

Checks whether a value, or a sets of values, are contained in the confidence region. The method computes the fraction of these that are contained in the region, if more than one value is passed. If only one value is passed, then it either returns 1.0 or 0.0, for indicating the value is in the region or not, respectively.

Parameters:

true_value – a np.ndarray of shape (n_classes,) or shape (n_values, n_classes,)

Returns:

float in [0,1]

coverage_soft(true_value)

mean_winkler_score(true_prev, alpha=None, add_ae=False)

point_estimate()

Returns the point estimate, the center of the ellipse.

Returns:

np.ndarray of shape (n_classes,)

property samples

Returns internal samples

winkler_scores(true_prev, alpha=None, add_ae=False)

class quapy.method.confidence.ConfidenceRegionABC

Bases: ABC

Abstract class of confidence regions

closest_point_in_region(p, tol=1e-06, max_iter=30)

Finds the closes point to p that belongs to the region. Assumes the region is convex.

Parameters:

• p – array-like, the point

• tol – float, error tolerance

• max_iter – int, max number of iterations

Returns:

array-like, the closes point to p in the segment between p and the center of the region, that belongs to the region

abstractmethod coverage(true_value) → float

Checks whether a value, or a sets of values, are contained in the confidence region. The method computes the fraction of these that are contained in the region, if more than one value is passed. If only one value is passed, then it either returns 1.0 or 0.0, for indicating the value is in the region or not, respectively.

Parameters:

true_value – a np.ndarray of shape (n_classes,) or shape (n_values, n_classes,)

Returns:

float in [0,1]

montecarlo_proportion(n_trials=10000)

Estimates, via a Monte Carlo approach, the fraction of the simplex covered by the region. This is carried out by returning the fraction of the n_trials points, uniformly drawn at random from the simplex, that are included in the region. The value is only computed once when multiple calls are made.

Returns:

float in [0,1]

ndim() → int

Number of dimensions of the region. This number corresponds to the total number of classes. The dimensionality of the simplex is therefore ndim-1

Returns:

int

abstractmethod point_estimate() → ndarray

Returns the point estimate corresponding to a set of bootstrap estimates.

Returns:

np.ndarray

abstract property samples

Returns internal samples

simplex_portion()

Computes the fraction of the simplex which is covered by the region. This is not the volume of the region itself (which could lie outside the boundaries of the simplex), but the actual fraction of the simplex contained in the region. A default implementation, based on Monte Carlo approximation, is provided.

Returns:

float, the fraction of the simplex covered by the region

class quapy.method.confidence.PQ(classifier: BaseEstimator = None, fit_classifier=True, val_split: int = 5, nbins: int = 4, fixed_bins: bool = False, num_warmup: int = 500, num_samples: int = 1000, stan_seed: int = 0, confidence_level: float = 0.95, region: str = 'intervals', bonferroni: bool = False)

Bases: AggregativeSoftQuantifier, BinaryAggregativeQuantifier

Precise Quantifier: Bayesian distribution matching quantifier <https://arxiv.org/abs/2507.06061>, which is a variant of :class:`HDy that calculates the posterior probability distribution over the prevalence vectors, rather than providing a point estimate.

This method relies on extra dependencies, which have to be installed via: $ pip install quapy[bayes]

Parameters:

• classifier – a scikit-learn’s BaseEstimator, or None, in which case the classifier is taken to be the one indicated in qp.environ[‘DEFAULT_CLS’]

• val_split – specifies the data used for generating classifier predictions. This specification can be made as float in (0, 1) indicating the proportion of stratified held-out validation set to be extracted from the training set; or as an integer (default 5), indicating that the predictions are to be generated in a k-fold cross-validation manner (with this integer indicating the value for k); or as a tuple (X,y) defining the specific set of data to use for validation. Set to None when the method does not require any validation data, in order to avoid that some portion of the training data be wasted.

• num_warmup – number of warmup iterations for the STAN sampler (default 500)

• num_samples – number of samples to draw from the posterior (default 1000)

• stan_seed – random seed for the STAN sampler (default 0)

• region – string, set to intervals for constructing confidence intervals (default), or to ellipse for constructing an ellipse in the probability simplex, or to ellipse-clr for constructing an ellipse in the Centered-Log Ratio (CLR) unconstrained space.

• bonferroni – bool (default False), whether to apply Bonferroni correction when region=’intervals’. This parameter has no effect for ellipse-based regions.

aggregate(classif_predictions)

Implements the aggregation of the classifier predictions.

Parameters:

classif_predictions – np.ndarray of classifier predictions

Returns:

np.ndarray of shape (n_classes,) with class prevalence estimates.

aggregate_conf(predictions, confidence_level=None)

aggregation_fit(classif_predictions, labels)

Trains the aggregation function.

Parameters:

• classif_predictions – array-like with the classification predictions (whatever the method classify() returns)

• labels – array-like with the true labels associated to each classifier prediction

predict_conf(instances, confidence_level=None) -> (<class 'numpy.ndarray'>, <class 'quapy.method.confidence.ConfidenceRegionABC'>)

class quapy.method.confidence.WithConfidenceABC

Bases: ABC

Abstract class for confidence regions.

REGION_TYPE = ['intervals', 'ellipse', 'ellipse-clr', 'ellipse-ilr']

classmethod construct_region(prev_estims, confidence_level=0.95, method='intervals', bonferroni=False) → ConfidenceRegionABC

Construct a confidence region given many prevalence estimations.

Parameters:

• prev_estims – np.ndarray of shape (n_estims, n_classes)

• confidence_level – float, the confidence level for the region (default 0.95)

• method – str, indicates the method for constructing regions. Set to intervals for constructing confidence intervals (default), or to ellipse for constructing an ellipse in the probability simplex, or to ellipse-clr for constructing an ellipse in the Centered-Log Ratio (CLR) unconstrained space.

• bonferroni – bool (default False), whether to apply Bonferroni correction when method=’intervals’. This parameter has no effect for ellipse-based regions.

abstractmethod predict_conf(instances, confidence_level=0.95) -> (<class 'numpy.ndarray'>, <class 'quapy.method.confidence.ConfidenceRegionABC'>)

Adds the method predict_conf to the interface. This method returns not only the point-estimate, but also the confidence region around it.

Parameters:

• instances – a np.ndarray of shape (n_instances, n_features,)

• confidence_level – float in (0, 1), default is 0.95

Returns:

a tuple (point_estimate, conf_region), where point_estimate is a np.ndarray of shape (n_classes,) and conf_region is an object from ConfidenceRegionABC

quantify_conf(instances, confidence_level=0.95) -> (<class 'numpy.ndarray'>, <class 'quapy.method.confidence.ConfidenceRegionABC'>)

Alias to predict_conf. This method returns not only the point-estimate, but also the confidence region around it.

Parameters:

• instances – a np.ndarray of shape (n_instances, n_features,)

• confidence_level – float in (0, 1), default is 0.95

Returns:

a tuple (point_estimate, conf_region), where point_estimate is a np.ndarray of shape (n_classes,) and conf_region is an object from ConfidenceRegionABC

quapy.method.confidence.bootstrap_once(i, classif_predictions, quantifier, n_samples)

quapy.method.confidence.closest_point_on_ellipsoid(p, mean, cov, chi2_critical, tol=1e-09, max_iter=100)

Computes the closest point on the ellipsoid defined by:

(x - mean)^T cov^{-1} (x - mean) = chi2_critical

quapy.method.confidence.simplex_volume(n)

Computes the volume of the n-dimensional simplex. For n classes, the corresponding volume is simplex_volume(n-1)() since the simplex has one degree of freedom less.

Parameters:

n – int, the dimensionality of the simplex

Returns:

float, the volume of the n-dimensional simplex

quapy.method.confidence.within_ellipse_prop(values, mean, prec_matrix, chi2_critical)

Checks the proportion of values that belong to the ellipse with center mean and precision matrix prec_matrix at a distance chi2_critical.

Parameters:

• values – a np.ndarray of shape (n_dim,) or (n_values, n_dim,)

• mean – a np.ndarray of shape (n_dim,) with the center of the ellipse

• prec_matrix – a np.ndarray with the precision matrix (inverse of the covariance matrix) of the ellipse. If this inverse cannot be computed then None must be passed

• chi2_critical – float, the chi2 critical value

Returns:

float in [0,1], the fraction of values that are contained in the ellipse defined by the mean (center), the precision matrix (shape), and the chi2_critical value (distance). If values is only one value, then either 0. (not contained) or 1. (contained) is returned.

quapy.method.confidence.within_ellipse_prop__(values, mean, prec_matrix, chi2_critical)

Checks the proportion of values that belong to the ellipse with center mean and precision matrix prec_matrix at a distance chi2_critical.

Parameters:

• values – a np.ndarray of shape (n_dim,) or (n_values, n_dim,)

• mean – a np.ndarray of shape (n_dim,) with the center of the ellipse

• prec_matrix – a np.ndarray with the precision matrix (inverse of the covariance matrix) of the ellipse. If this inverse cannot be computed then None must be passed

• chi2_critical – float, the chi2 critical value

Returns:

float in [0,1], the fraction of values that are contained in the ellipse defined by the mean (center), the precision matrix (shape), and the chi2_critical value (distance). If values is only one value, then either 0. (not contained) or 1. (contained) is returned.

Module contents