ExpectedErrorReduction#

class skactiveml.pool.ExpectedErrorReduction(enforce_mapping, cost_matrix=None, missing_label=nan, random_state=None)[source]#

Bases: SingleAnnotatorPoolQueryStrategy

Expected Error Reduction (EER)

This class implements the basic workflow of EER algorithms by:

determining each candidates-label pair and simulate its utility by retraining the classifier with it,

determining some kind of risk for the retrained classifier.

This workflow has been used by [1], [2], [3], and [4].

Parameters:

enforce_mappingbool: If True, an exception is raised when no exact mapping between samples in X and samples in candidates can be determined.
cost_matrix: array-like, shape (n_classes, n_classes), default=None: Cost matrix with cost_matrix[i,j] defining the cost of predicting class j for a sample with the actual class i. Used for misclassification loss and ignored for log loss.
missing_labelscalar or string or np.nan or None, default=np.nan: Value to represent a missing label.
random_stateint or np.random.RandomState or None, default=None: The random state to use.

References

[1]

Roy, N., & McCallum, A. (2001). Toward optimal active learning through monte carlo estimation of error reduction. ICML, (pp. 441-448).

[2]

Joshi, A. J., Porikli, F., & Papanikolopoulos, N. P. (2012). Scalable active learning for multiclass image classification. IEEE TrPAMI, 34(11), pp. 2259-2273.

[3]

Margineantu, D. D. (2005). Active cost-sensitive learning. In IJCAI (Vol. 5, pp. 1622-1623).

[4]

Kapoor, Ashish, Eric Horvitz, and Sumit Basu. “Selective Supervision: Guiding Supervised Learning with Decision-Theoretic Active Learning.” IJCAI. Vol. 7. 2007.

Methods

`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Get parameters for this estimator.
`query`(X, y, clf[, fit_clf, ...])	Determines for which candidate samples labels are to be queried.
`set_params`(**params)	Set the parameters of this estimator.

get_metadata_routing()#

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest: A MetadataRequest encapsulating routing information.

get_params(deep=True)#

Get parameters for this estimator.

Parameters:

deepbool, default=True: If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

paramsdict: Parameter names mapped to their values.

query(X, y, clf, fit_clf=True, ignore_partial_fit=True, sample_weight=None, candidates=None, sample_weight_candidates=None, X_eval=None, sample_weight_eval=None, batch_size=1, return_utilities=False)[source]#

Determines for which candidate samples labels are to be queried.

Parameters:

Xarray-like of shape (n_samples, n_features)

Training data set, usually complete, i.e., including the labeled and unlabeled samples.

yarray-like of shape (n_samples,)

Labels of the training data set (possibly including unlabeled ones indicated by self.missing_label).

clfskactiveml.base.SkactivemlClassifier

Model implementing the methods fit and predict_proba.

fit_clfbool, default=True

Defines whether the classifier should be fitted on X, y, and sample_weight.

ignore_partial_fitbool, default=True

Relevant in cases where clf implements partial_fit. If True, the partial_fit function is ignored and fit is used instead.

sample_weightarray-like of shape (n_samples,), default=None

Weights of training samples in X.

candidatesNone or array-like of shape (n_candidates), dtype=int or array-like of shape (n_candidates, n_features), default=None

If candidates is None, the unlabeled samples from (X,y) are considered as candidates.
If candidates is of shape (n_candidates,) and of type int, candidates is considered as the indices of the samples in (X,y).
If candidates is of shape (n_candidates, …), the candidate samples are directly given in candidates (not necessarily contained in X).

sample_weight_candidatesarray-like of shape (n_candidates,), default=None

Weights of candidates samples in candidates if candidates are directly given (i.e., candidates.ndim > 1). Otherwise, weights for candidates are given in sample_weight.

X_evalarray-like of shape (n_eval_samples, n_features), default=None

Unlabeled evaluation data set that is used for estimating the risk. Not applicable for all EER methods.

sample_weight_evalarray-like of shape (n_eval_samples,), default=None

Weights of evaluation samples in X_eval if given. Used to weight the importance of samples when estimating the risk.

batch_sizeint, default=1

The number of samples to be selected in one AL cycle.

return_utilitiesbool, default=False

If True, also return the utilities based on the query strategy.

Returns:

query_indicesnumpy.ndarray of shape (batch_size,)

The query indices indicate for which candidate sample a label is to be queried, e.g., query_indices[0] indicates the first selected sample.

If candidates is None or of shape (n_candidates,), the indexing refers to the samples in X.
If candidates is of shape (n_candidates, n_features), the indexing refers to the samples in candidates.

utilitiesnumpy.ndarray of shape (batch_size, n_samples) or numpy.ndarray of shape (batch_size, n_candidates)

The utilities of samples after each selected sample of the batch, e.g., utilities[0] indicates the utilities used for selecting the first sample (with index query_indices[0]) of the batch. Utilities for labeled samples will be set to np.nan.

If candidates is None or of shape (n_candidates,), the indexing refers to the samples in X.
If candidates is of shape (n_candidates, n_features), the indexing refers to the samples in candidates.

set_params(**params)#

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters: