DropQuery

class skactiveml.pool.DropQuery(dropout_rate=0.75, n_dropout_samples=5, cluster_algo=<class 'sklearn.cluster._kmeans.KMeans'>, cluster_algo_dict=None, n_cluster_param_name='n_clusters', clf_embedding_flag_name=None, missing_label=nan, random_state=None, multilabel_aggregation_fn=<function mean>, disagreement_threshold=0.5, target_type='auto')

Bases: SingleAnnotatorPoolQueryStrategy

Dropout Query (DropQuery)

This class implements the query strategy Dropout Query (DropQuery) [1] that incorporates both uncertainty and sample diversity into every selected batch. For this purpose, unlabeled samples are filtered according to a disagreement-based measure via dropout such that only the unlabeled samples with a disagreement above a threshold are clustered for selecting the unlabeled samples nearest to the respective clusters.

DropQuery was proposed for single-output classification. Multi-label support in this implementation is an extension and not part of the original proposal in [1]. For resolved multi-label targets, the disagreement is counted per label output, i.e., the per-label score of the label output j is the number of the n_dropout_samples dropout predictions whose label j differs from the label j predicted without dropout, and is therefore an integer in [0, n_dropout_samples]. multilabel_aggregation_fn reduces these per-label counts along the label axis, and the reduced count is divided by n_dropout_samples to obtain the disagreement rate compared with disagreement_threshold. This per-output decomposition ignores correlations between label outputs, i.e., a dropout prediction flipping several labels jointly is indistinguishable from independent flips of the same labels.

Parameters:

dropout_ratefloat, default=0.75

Dropout rate used to generate samples.

n_dropout_samplesint, default=3

Number of dropout samples.

cluster_algoClusterMixin.__class__, default=KMeans

The cluster algorithm to be used. It must implement a fit_transform method, which takes samples X as inputs, e.g., sklearn.clustering.KMeans and sklearn.clustering.MiniBatchKMeans.

cluster_algo_dictdict, default=None

The parameters passed to the clustering algorithm cluster_algo, excluding the parameter for the number of clusters.

n_cluster_param_namestring, default=”n_clusters”

The name of the parameter for the number of clusters.

clf_embedding_flag_namedict or str or None, default=None

Flag, which is passed to the predict method for getting the (learned) sample representations.

• If clf_embedding_flag_name is None and predict returns only one output, the input samples X are used.

• If clf_embedding_flag_name is None and predict returns two outputs, (y_pred, embeddings) are expected as outputs.

• If isinstance(clf_embedding_name, str), we call:

  clf.predict(X, **{clf_embedding_flag_name: True})
  

  and expect (y_pred, embeddings) as output.

• If isinstance(clf_embedding_name, dict), we call:

  clf.predict(X, **clf_embedding_flag_name)
  

  and expect (y_pred, embeddings) as output.

missing_labelscalar or string or np.nan or None, default=np.nan

Value to represent a missing label.

random_stateNone or int or np.random.RandomState, default=None

The random state to use.

multilabel_aggregation_fncallable, default=np.mean

Callable reducing the per-label disagreement counts of one sample to one count. It is only used for resolved multi-label classification targets. It is called with the per-label scores of shape (n_samples, n_outputs) and the label axis passed as the axis keyword argument, and must return one score per sample within the range of that sample’s per-label scores, e.g. np.mean, np.average, np.median, np.min, np.max, or a quantile. np.sum is not supported, because its result grows with the number of label outputs. Only the callability of the reduction is validated at runtime, so a violating reduction silently changes the acquisition scale.

disagreement_thresholdfloat, default=0.5

Threshold used to filter candidate samples based on their disagreement score. For multi-label targets, the per-label disagreement counts are first reduced by multilabel_aggregation_fn before being divided by n_dropout_samples. A scale-preserving reduction therefore keeps the resulting rate in [0, 1], but the multi-label path does not restrict the threshold to that interval.

target_type“auto” or “single-output” or “multi-label”, default=”auto”

Declared target type. The strategy supports single-output and multi-label classification. A fitted classifier’s target specification is authoritative when available.

References

[1] (1,2)

S. R. Gupte, J. Aklilu, J. J. Nirschl, and S. Yeung-Levy, “Revisiting Active Learning in the Era of Vision Foundation Models.” Trans. Mach. Learn., 2024.

Methods

query(X, y, clf[, fit_clf, sample_weight, ...])	Query the next samples to be labeled.
get_metadata_routing()	Get metadata routing of this object.
get_params([deep])	Get parameters for this estimator.
set_params(**params)	Set the parameters of this estimator.

DropQuery.query(X, y, clf, fit_clf=True, sample_weight=None, candidates=None, batch_size=1, return_utilities=False)

Query the next samples to be labeled.

Parameters:

Xarray-like of shape (n_samples, n_features)

Training data set, usually complete, i.e. including the labeled and unlabeled samples.

yarray-like of shape (n_samples,) or (n_samples, n_outputs)

Labels of the training data set (possibly including unlabeled ones indicated by self.missing_label). For multi-label targets, a row y[i] must either contain only observed labels or only missing_label values, i.e., no mixing within a row.

clfskactiveml.base.SkactivemlClassifier

Classifier implementing the methods fit and predict.

fit_clfbool, default=True

Defines whether the classifier clf should be fitted on X, y, and sample_weight.

sample_weightarray-like of shape (n_samples,) or (n_samples, n_outputs), default=None

Weights of training samples in X. For two-dimensional y, one weight per sample is supported. Per-target weights are forwarded to clf.fit without additional validation and require estimator support.

candidatesNone or array-like of shape (n_candidates,) of type int, default=None

• If candidates is None, the unlabeled samples from (X,y) are considered as candidates.

• If candidates is of shape (n_candidates,) and of type int, candidates is considered as the indices of the samples in (X,y).

batch_sizeint, default=1

The number of samples to be selected in one AL cycle.

return_utilitiesbool, default=False

If true, also return the utilities based on the query strategy.

Returns:

query_indicesnumpy.ndarray of shape (batch_size,)

The query indices indicate for which candidate sample a label is to be queried, e.g., query_indices[0] indicates the first selected sample. The indexing refers to the samples in X.

utilitiesnumpy.ndarray of shape (batch_size, n_samples)

The utilities of samples after each selected sample of the batch, e.g., utilities[0] indicates the utilities used for selecting the first sample (with index query_indices[0]) of the batch. Utilities for labeled samples will be set to np.nan. The indexing refers to the samples in X.

DropQuery.get_metadata_routing()

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest

A MetadataRequest encapsulating routing information.

DropQuery.get_params(deep=True)

Get parameters for this estimator.

Parameters:

deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

paramsdict

Parameter names mapped to their values.

DropQuery.set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**paramsdict

Estimator parameters.

Returns:

selfestimator instance

Estimator instance.

Examples using skactiveml.pool.DropQuery

Dropout Query (DropQuery)

Dropout Query (DropQuery)