MultiAnnotatorPoolQueryStrategy#

class skactiveml.base.MultiAnnotatorPoolQueryStrategy(missing_label=nan, random_state=None, target_type='auto')[source]#

Bases: PoolQueryStrategy

Base class for all pool-based active learning query strategies with multiple annotators in scikit-activeml.

Parameters:

missing_labelscalar or str or np.nan or None, default=np.nan: Value to represent a missing label.
random_stateint or RandomState instance, default=None: Controls the randomness of the estimator. If None, the RandomState singleton used by np.random is used.
target_type“auto” or “single-output” or “multi-label” or “multi-output”, default=”auto”: Declared target type. Multi-annotator strategies support only single-output classification in version 1.1.

Methods

`query`(X, y, *args[, candidates, annotators, ...])	Determines which candidate sample is to be annotated by which annotator.
`get_metadata_routing`()	Get metadata routing of this object.
`get_params`([deep])	Get parameters for this estimator.
`set_params`(**params)	Set the parameters of this estimator.

abstract MultiAnnotatorPoolQueryStrategy.query(X, y, *args, candidates=None, annotators=None, batch_size=1, return_utilities=False, **kwargs)[source]#

Determines which candidate sample is to be annotated by which annotator.

Parameters:

Xarray-like of shape (n_samples, n_features)

Training data set, usually complete, i.e., including the labeled and unlabeled samples.

yarray-like of shape (n_samples, n_annotators)

Labels of the training data set for each annotator (possibly including unlabeled ones indicated by self.MISSING_LABEL), meaning that y[i, j] contains the label annotated by annotator i for sample j.

candidatesNone or array-like of shape (n_candidates), dtype=int or array-like of shape (n_candidates, n_features), default=None

See parameter annotators.

annotatorsNone or array-like of shape (n_avl_annotators), dtype=int or array-like of shape (n_candidates, n_annotators), default=None

If candidate samples and annotators are not specified, i.e., candidates=None, annotators=None the unlabeled target values, y, are the candidates annotator-sample-pairs.
If candidate samples and available annotators are specified: The annotator-sample-pairs, for which the sample is a candidate sample and the annotator is an available annotator are considered as candidate annotator-sample-pairs.
If candidates is None, all samples of X are considered as candidate samples. In this case n_candidates equals len(X).
If candidates is of shape (n_candidates,) and of type int, candidates is considered as the indices of the sample candidates in (X, y).
If candidates is of shape (n_candidates, n_features), the sample candidates are directly given in candidates (not necessarily contained in X). This is not supported by all query strategies.
If annotators is None, all annotators are considered as available annotators.
If annotators is of shape (n_avl_annotators), and of type int, annotators is considered as the indices of the available annotators.
If annotators is a boolean array of shape (n_candidates, n_annotators) the annotator-sample-pairs, for which the sample is a candidate sample and the boolean matrix has entry True are considered as candidate annotator-sample pairs.

batch_sizeint or str, default=1

The number of annotators-sample pairs to be selected in one AL cycle. If adaptive=True, batch_size=’adaptive’ is allowed.

return_utilitiesbool, default=False

If True, also return the utilities based on the query strategy.

Returns:

query_indicesnp.ndarray of shape (batch_size, 2)

The query_indices indicate which candidate sample pairs are to be queried is, i.e., which candidate sample is to be annotated by which annotator, e.g., query_indices[:, 0] indicates the selected candidate samples and query_indices[:, 1] indicates the respectively selected annotators.

If candidates is None or of shape (n_candidates,), the indexing of refers to samples in X.
If candidates is of shape (n_candidates, n_features), the indexing refers to samples in candidates.

utilities: numpy.ndarray of shape (batch_size, n_samples, n_annotators) or numpy.ndarray of shape (batch_size, n_candidates, n_annotators)

The utilities of all candidate samples w.r.t. to the available annotators after each selected sample of the batch, e.g., utilities[0, :, j] indicates the utilities used for selecting the first sample-annotator-pair (with indices query_indices[0]).

If candidates is None or of shape (n_candidates,), the indexing refers to samples in X.
If candidates is of shape (n_candidates, n_features), the indexing refers to samples in candidates.

Notes

An exhausted candidate pool, i.e., no candidate annotator-sample pair left to be queried, is a valid acquisition state. It is answered with an empty batch of batch_size zero and a warning naming the exhaustion, so that a budget loop running one cycle past exhaustion needs no special case.

MultiAnnotatorPoolQueryStrategy.get_metadata_routing()#

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest: A MetadataRequest encapsulating routing information.

MultiAnnotatorPoolQueryStrategy.get_params(deep=True)#

Get parameters for this estimator.

Parameters:

deepbool, default=True: If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

paramsdict: Parameter names mapped to their values.

MultiAnnotatorPoolQueryStrategy.set_params(**params)#

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters: