AnnotatorLogisticRegression#

class skactiveml.classifier.multiannotator.AnnotatorLogisticRegression(n_annotators=None, tol=0.0001, max_iter=100, fit_intercept=True, annot_prior_full=1, annot_prior_diag=0, weights_prior=1, solver='Newton-CG', solver_dict=None, classes=None, cost_matrix=None, missing_label=nan, random_state=None)[source]#

Bases: SkactivemlClassifier

Logistic Regression for Crowds

Logistic Regression based on Raykar [1] is a classification algorithm that learns from multiple annotators. Besides, building a model for the classification task, the algorithm estimates the performance of the annotators. The performance of an annotator is assumed to only depend on the true label of a sample and not on the sample itself. Each annotator is assigned a confusion matrix, where each row is normalized. This contains the bias of the annotators decisions. These estimated biases are then used to refine the classifier itself.

The classifier also supports a bayesian view on the problem, for this a prior distribution over an annotator’s confusion matrix is assumed. It also assumes a prior distribution over the classifiers’ weight vectors corresponding to a regularization.

Parameters:
n_annotatorsint >= 1 or None, default=None

Number of annotators. If None, it is inferred from y during fitting.

tolfloat, default=0.0001

Threshold for stopping the EM-Algorithm and the optimization of the logistic regression weights. If the change of the respective value between two steps is smaller than tol, the respective algorithm stops.

max_iterint, default=100

The maximum number of iterations of the EM-algorithm to be performed.

fit_interceptbool, default=True

Specifies if a constant (a.k.a. bias or intercept) should be added to input samples.

annot_prior_fullint or float or array-like, default=1

Base concentration of the Dirichlet prior A for each annotator’s confusion matrix. A scalar applies to every annotator, while an array supplies one value per annotator. Values must be at least 1 because the MAP update uses A - 1 as prior counts.

annot_prior_diagint or float or array-like, default=0

Non-negative values added to the diagonal of each annotator’s Dirichlet prior A. A scalar applies to every annotator, while an array supplies one value per annotator.

weights_priorint or float >= 0, default=1

Determines Gamma as the inverse covariance matrix of the prior distribution for every weight vector (i.e., Gamma=weights_prior * np.eye(n_features)). As default, the identity matrix is used for each weight vector.

solverstr or callable, default=’Newton-CG’

Type of solver. Should be ‘Nelder-Mead’, ‘Powell’, ‘CG’, ‘BFGS’, ‘Newton-CG’, ‘L-BFGS-B’, ‘TNC’, ‘COBYLA’, ‘SLSQP’, ‘trust-constr’, ‘dogleg’, ‘trust-ncg’, ‘trust-exact’, ‘trust-krylov’, or custom - a callable object. See scipy.optimize.minimize for more information.

solver_dictdictionary, default=None

Additional solver options passed to scipy.optimize.minimize. If None, {‘maxiter’: 100} is passed.

classesarray-like of shape (n_classes), default=None

Holds the label for each class. If none, the classes are determined during the fit.

missing_labelscalar or string or np.nan or None, default=np.nan

Value to represent a missing label.

cost_matrixarray-like of shape (n_classes, n_classes)

Cost matrix with cost_matrix[i,j] indicating cost of predicting class classes[j] for a sample of class classes[i]. Can be only set, if classes is not None.

random_stateint or RandomState instance or None, default=None

Determines random number for predict method. Pass an int for reproducible results across multiple method calls.

Attributes:
n_annotators_int

Number of annotators.

W_numpy.ndarray of shape (n_features, n_classes)

The weight vectors of the logistic regression model.

Alpha_numpy.ndarray of shape (n_annotators, n_classes, n_classes)

This is a confusion matrix for each annotator, where each row is normalized. Alpha_[l,k,c] describes the probability that annotator l provides the class label c for a sample belonging to class k.

classes_array-like of shape (n_classes,)

Holds the label for each class after fitting.

cost_matrix_array-like of shape (classes, classes)

Cost matrix with C[i,j] indicating cost of predicting class self.classes_[j] for a sample of class classes_[i].

References

[1]

V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin, L. Bogoni, and L. Moy. Learning from Crowds. J. Mach. Learn. Res., 11(4):1297–1322, 2010.

Methods

fit(X, y[, sample_weight])

Fit the model using X as samples and y as class labels.

predict(X[, extra_outputs])

Return class predictions for the test samples X.

predict_proba(X[, extra_outputs])

Return class probability estimates for the test samples X.

score(X, y[, sample_weight])

Return the mean accuracy on the given test data and labels.

get_metadata_routing()

Get metadata routing of this object.

get_params([deep])

Get parameters for this estimator.

set_fit_request(*[, sample_weight])

Configure whether metadata should be requested to be passed to the fit method.

set_params(**params)

Set the parameters of this estimator.

set_predict_proba_request(*[, extra_outputs])

Configure whether metadata should be requested to be passed to the predict_proba method.

set_predict_request(*[, extra_outputs])

Configure whether metadata should be requested to be passed to the predict method.

set_score_request(*[, sample_weight])

Configure whether metadata should be requested to be passed to the score method.

AnnotatorLogisticRegression.fit(X, y, sample_weight=None)[source]#

Fit the model using X as samples and y as class labels.

Parameters:
Xarray-like of shape (n_samples, n_features)

Feature matrix representing the samples.

yarray-like of shape (n_samples, n_annotators)

It contains the class labels of the training samples, where missing labels are represented via missing_label. Specifically, label y[n, m] refers to the label of sample X[n] from annotator m.

sample_weightarray-like of shape (n_samples, n_annotators), default=None

Annotation weights with the same shape as y. An annotation with zero weight is treated as missing. Weights are applied to majority-vote initialization and confusion matrix updates. They do not directly weight the logistic regression or expectation updates.

Returns:
self: AnnotatorLogisticRegression,

The AnnotatorLogisticRegression is fitted on the training data.

AnnotatorLogisticRegression.predict(X, extra_outputs=None)[source]#

Return class predictions for the test samples X.

By default, this method returns only the class predictions y_pred. If extra_outputs is provided, a tuple is returned whose first element is y_pred and whose remaining elements are the requested additional forward outputs, in the order specified by extra_outputs.

Parameters:
Xarray-like of shape (n_samples, …)

Test samples.

extra_outputsNone or str or sequence of str, default=None

Names of additional outputs to return next to y_pred. The names must be a subset of the following keys:

  • “logits” : Additionally return the class-membership logits L_class for the samples in X.

  • “annotator_perf” : additionally return the estimated annotator performance probabilities P_perf for each sample–annotator pair.

  • “annotator_class” : Additionally return the annotator–class probability estimates P_annot for each sample, class, and annotator.

Returns:
y_prednumpy.ndarray of shape (n_samples,)

Class predictions of the test samples.

*extrasnumpy.ndarray, optional

Only returned if extra_outputs is not None. In that case, the method returns a tuple whose first element is y_pred and whose remaining elements correspond to the requested forward outputs in the order given by extra_outputs. Potential outputs are:

  • L_class : np.ndarray of shape (n_samples, n_classes), where L_class[n, c] is the logit for the class classes_[c] of sample X[n].

  • P_perf : np.ndarray of shape (n_samples, n_annotators), where P_perf[n, m] refers to the estimated label correctness probability (performance) of annotator m when labeling sample X[n].

  • P_annot : np.ndarray of shape (n_samples, n_annotators, n_classes), where P_annot[n, m, c] refers to the probability that annotator m provides the class label c for sample X[n].

AnnotatorLogisticRegression.predict_proba(X, extra_outputs=None)[source]#

Return class probability estimates for the test samples X.

By default, this method returns only the class probabilities P. If extra_outputs is provided, a tuple is returned whose first element is P and whose remaining elements are the requested additional forward outputs, in the order specified by extra_outputs.

Parameters:
Xarray-like of shape (n_samples, …)

Test samples.

extra_outputsNone or str or sequence of str, default=None

Names of additional outputs to return next to P. The names must be a subset of the following keys:

  • “logits” : Additionally return the class-membership logits L_class for the samples in X.

  • “annotator_perf” : additionally return the estimated annotator performance probabilities P_perf for each sample–annotator pair.

  • “annotator_class” : Additionally return the annotator–class probability estimates P_annot for each sample, class, and annotator.

Returns:
Pnumpy.ndarray of shape (n_samples, n_classes)

Class probabilities of the test samples. Classes are ordered according to self.classes_.

*extrasnumpy.ndarray, optional

Only returned if extra_outputs is not None. In that case, the method returns a tuple whose first element is P and whose remaining elements correspond to the requested forward outputs in the order given by extra_outputs. Potential outputs are:

  • L_class : np.ndarray of shape (n_samples, n_classes), where L_class[n, c] is the logit for the class classes_[c] of sample X[n].

  • P_perf : np.ndarray of shape (n_samples, n_annotators), where P_perf[n, m] refers to the estimated label correctness probability (performance) of annotator m when labeling sample X[n].

  • P_annot : np.ndarray of shape (n_samples, n_annotators, n_classes), where P_annot[n, m, c] refers to the probability that annotator m provides the class label c for sample X[n].

AnnotatorLogisticRegression.score(X, y, sample_weight=None)#

Return the mean accuracy on the given test data and labels.

Parameters:
Xarray-like of shape (n_samples, …)

Test samples.

yarray-like of shape (n_samples,) or (n_samples, n_outputs)

True class labels of the test samples X.

sample_weightarray-like of shape (n_samples,), default=None

Sample weights of the test sample X.

Returns:
scorefloat

Mean accuracy of self.predict(X) regarding y. For multi-label classification, this is the subset accuracy, i.e., a sample counts as correct only if all of its labels are predicted correctly.

AnnotatorLogisticRegression.get_metadata_routing()#

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:
routingMetadataRequest

A MetadataRequest encapsulating routing information.

AnnotatorLogisticRegression.get_params(deep=True)#

Get parameters for this estimator.

Parameters:
deepbool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:
paramsdict

Parameter names mapped to their values.

AnnotatorLogisticRegression.set_fit_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → AnnotatorLogisticRegression#

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to fit.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in fit.

Returns:
selfobject

The updated object.

AnnotatorLogisticRegression.set_params(**params)#

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:
**paramsdict

Estimator parameters.

Returns:
selfestimator instance

Estimator instance.

AnnotatorLogisticRegression.set_predict_proba_request(*, extra_outputs: bool | None | str = '$UNCHANGED$') → AnnotatorLogisticRegression#

Configure whether metadata should be requested to be passed to the predict_proba method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to predict_proba if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to predict_proba.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
extra_outputsstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for extra_outputs parameter in predict_proba.

Returns:
selfobject

The updated object.

AnnotatorLogisticRegression.set_predict_request(*, extra_outputs: bool | None | str = '$UNCHANGED$') → AnnotatorLogisticRegression#

Configure whether metadata should be requested to be passed to the predict method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to predict.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
extra_outputsstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for extra_outputs parameter in predict.

Returns:
selfobject

The updated object.

AnnotatorLogisticRegression.set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → AnnotatorLogisticRegression#

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:
sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns:
selfobject

The updated object.