################# Strategy Overview ################# This is an overview of all implemented active learning strategies, which are often divided into three main categories based on the utilities they compute for sample selection: 1. **Informativeness-based** strategies mostly select samples for which the model is most uncertain (e.g., via information-theoretic measures). 2. **Representativeness-based** strategies select samples that capture the overall data distribution (e.g., via clustering ordensity estimation). 3. **Hybrid** strategies combine both criteria to select samples that are informative and representative. Furthermore, we distinguish between **regression** and **classification** as supervised learning tasks, where labels canbe provided by a **single annotator** or **multiple annotators**. You can use the checkboxes below to filter the query strategies based on these distinctions. .. raw:: html Regression Classification Multi-Annotator Single-Annotator Pool ---- Baseline ~~~~~~~~ .. list-table:: :header-rows: 1 * - Method - Base Class - Tags - Reference * - :doc:`Random Sampling ` - :doc:`RandomSampling ` - pool regression classification single-annotator - Hybrid ~~~~~~ .. list-table:: :header-rows: 1 * - Method - Base Class - Tags - Reference * - :doc:`Batch Active Learning by Diverse Gradient Embedding (BADGE) ` - :doc:`Badge ` - pool classification single-annotator - :footcite:t:`ash2020deep` * - :doc:`Clustering Uncertainty-weighted Embeddings (CLUE) ` - :doc:`Clue ` - pool classification single-annotator - :footcite:t:`prabhu2021active` * - :doc:`Contrastive Active Learning (CAL) ` - :doc:`ContrastiveAL ` - pool classification single-annotator - :footcite:t:`margatina2021active` * - :doc:`Dropout Query (DropQuery) ` - :doc:`DropQuery ` - pool classification single-annotator - :footcite:t:`gupte2024revisiting` * - :doc:`Fast Active Learning by Contrastive UNcertainty (FALCUN) ` - :doc:`Falcun ` - pool classification single-annotator - :footcite:t:`gilhuber2024falcun` * - :doc:`Density-Diversity-Distribution-Distance Sampling (4DS) ` - :doc:`FourDs ` - pool classification single-annotator - :footcite:t:`reitmaier2013let` * - :doc:`Batch Density-Diversity-Distribution-Distance Sampling (Batch4DS) ` - :doc:`FourDs ` - pool classification single-annotator - :footcite:t:`reitmaier2013let` * - :doc:`Multi-class Probabilistic Active Learning (McPAL) ` - :doc:`ProbabilisticAL ` - pool classification single-annotator - :footcite:t:`kottke2016multi` * - :doc:`Querying Informative and Representative Examples (QUIRE) ` - :doc:`Quire ` - pool classification single-annotator - :footcite:t:`huang2010active`, :footcite:t:`huang2014active` * - :doc:`Regression Tree Based Active Learning (RT-AL) with Random Selection ` - :doc:`RegressionTreeBasedAL ` - pool regression single-annotator - :footcite:t:`jose2023regression` * - :doc:`Regression Tree Based Active Learning (RT-AL) with Diversity Selection ` - :doc:`RegressionTreeBasedAL ` - pool regression single-annotator - :footcite:t:`jose2023regression` * - :doc:`Regression Tree Based Active Learning (RT-AL) with Representativity Selection ` - :doc:`RegressionTreeBasedAL ` - pool regression single-annotator - :footcite:t:`jose2023regression` * - :doc:`Density-weighted Uncertainty Sampling ` - :doc:`UncertaintySampling ` - pool classification single-annotator - :footcite:t:`tang2002active` * - :doc:`Dual Strategy for Active Learning ` - :doc:`UncertaintySampling ` - pool classification single-annotator - :footcite:t:`donmez2007dual` Informativeness ~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - Method - Base Class - Tags - Reference * - :doc:`Batch Bayesian Active Learning by Disagreement (BatchBALD) ` - :doc:`BatchBALD ` - pool classification single-annotator - :footcite:t:`houlsby2011bayesian`, :footcite:t:`kirsch2019batchbald` * - :doc:`Active Learning with Cost Embedding (ALCE) ` - :doc:`CostEmbeddingAL ` - pool classification single-annotator - :footcite:t:`huang2016novel` * - :doc:`Epistemic Uncertainty Sampling (EpisUS) ` - :doc:`EpistemicUncertaintySampling ` - pool classification single-annotator - :footcite:t:`nguyen2019epistemic` * - :doc:`Expected Model Change ` - :doc:`ExpectedModelChangeMaximization ` - pool regression single-annotator - :footcite:t:`cai2013maximizing` * - :doc:`Expected Model Output Change ` - :doc:`ExpectedModelOutputChange ` - pool regression single-annotator - :footcite:t:`kading2018active` * - :doc:`Expected Model Variance Reduction ` - :doc:`ExpectedModelVarianceReduction ` - pool regression single-annotator - :footcite:t:`cohn1996active` * - :doc:`Bayesian Active Learning by Disagreement (BALD) ` - :doc:`GreedyBALD ` - pool classification single-annotator - :footcite:t:`houlsby2011bayesian` * - :doc:`Regression based Kullback Leibler Divergence Maximization ` - :doc:`KLDivergenceMaximization ` - pool regression single-annotator - :footcite:t:`elreedy2019novel` * - :doc:`Monte-Carlo EER with Log-Loss ` - :doc:`MonteCarloEER ` - pool classification single-annotator - :footcite:t:`roy2001toward` * - :doc:`Monte-Carlo EER with Misclassification-Loss ` - :doc:`MonteCarloEER ` - pool classification single-annotator - :footcite:t:`roy2001toward` * - :doc:`Query-by-Committee (QBC) with Kullback-Leibler Divergence ` - :doc:`QueryByCommittee ` - pool classification single-annotator - :footcite:t:`seung1992query`, :footcite:t:`mccallum1998employing` * - :doc:`Query-by-Committee (QBC) with Vote Entropy ` - :doc:`QueryByCommittee ` - pool classification single-annotator - :footcite:t:`seung1992query`, :footcite:t:`engelson1996minimizing` * - :doc:`Query-by-Committee (QBC) with Variation Ratios ` - :doc:`QueryByCommittee ` - pool classification single-annotator - :footcite:t:`seung1992query`, :footcite:t:`beluch2018power` * - :doc:`Query-by-Committee (QBC) with Empirical Variance ` - :doc:`QueryByCommittee ` - pool regression single-annotator - :footcite:t:`seung1992query`, :footcite:t:`burbidge2007active` * - :doc:`Uncertainty Sampling with Margin ` - :doc:`UncertaintySampling ` - pool classification single-annotator - :footcite:t:`settles2009active` * - :doc:`Uncertainty Sampling with Least-Confidence ` - :doc:`UncertaintySampling ` - pool classification single-annotator - :footcite:t:`settles2009active` * - :doc:`Uncertainty Sampling with Entropy ` - :doc:`UncertaintySampling ` - pool classification single-annotator - :footcite:t:`settles2009active` * - :doc:`Expected Average Precision ` - :doc:`UncertaintySampling ` - pool classification single-annotator - :footcite:t:`wang2018uncertainty` * - :doc:`Value of Information on Unlabeled Samples ` - :doc:`ValueOfInformationEER ` - pool classification single-annotator - :footcite:t:`joshi2009multi` * - :doc:`Value of Information on Labeled Samples ` - :doc:`ValueOfInformationEER ` - pool classification single-annotator - :footcite:t:`margineantu2005active` * - :doc:`Value of Information (VOI) ` - :doc:`ValueOfInformationEER ` - pool classification single-annotator - :footcite:t:`kapoor2007selective` Representativeness ~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - Method - Base Class - Tags - Reference * - :doc:`Core Set ` - :doc:`CoreSet ` - pool regression classification single-annotator - :footcite:t:`sener2018active` * - :doc:`Discriminative Active Learning (DAL) ` - :doc:`DiscriminativeAL ` - pool classification regression single-annotator - :footcite:t:`gissin2019discriminative` * - :doc:`Greedy Sampling on the Target Space (GSy) ` - :doc:`GreedySamplingTarget ` - pool regression single-annotator - :footcite:t:`wu2019active` * - :doc:`Improved Greedy Sampling (GSi) ` - :doc:`GreedySamplingTarget ` - pool regression single-annotator - :footcite:t:`wu2019active` * - :doc:`Greedy Sampling on the Feature Space (GSx) ` - :doc:`GreedySamplingX ` - pool regression classification single-annotator - :footcite:t:`wu2019active` * - :doc:`Probability Coverage (ProbCover) ` - :doc:`ProbCover ` - pool classification single-annotator - :footcite:t:`yehuda2022active` * - :doc:`Typical Clustering (TypiClust) ` - :doc:`TypiClust ` - pool regression classification single-annotator - :footcite:t:`hacohen2022active` Wrapper ~~~~~~~ .. list-table:: :header-rows: 1 * - Method - Base Class - Tags - Reference * - :doc:`Parallel Utility Estimation Wrapper ` - :doc:`ParallelUtilityEstimationWrapper ` - pool regression classification single-annotator - * - :doc:`Sub-sampling Wrapper ` - :doc:`SubSamplingWrapper ` - pool regression classification single-annotator - Stream ------ Baseline ~~~~~~~~ .. list-table:: :header-rows: 1 * - Method - Base Class - Tags - Reference * - :doc:`Periodic Sampling ` - :doc:`PeriodicSampling ` - stream classification single-annotator - * - :doc:`Stream Random Sampling ` - :doc:`StreamRandomSampling ` - stream classification single-annotator - Hybrid ~~~~~~ .. list-table:: :header-rows: 1 * - Method - Base Class - Tags - Reference * - :doc:`Cognitive Dual-Query Strategy with Fixed-Uncertainty ` - :doc:`CognitiveDualQueryStrategyFixUn ` - stream classification single-annotator - :footcite:t:`liu2023active` * - :doc:`Cognitive Dual-Query Strategy with Random Sampling ` - :doc:`CognitiveDualQueryStrategyRan ` - stream classification single-annotator - :footcite:t:`liu2023active` * - :doc:`Cognitive Dual-Query Strategy with Randomized-Variable-Uncertainty ` - :doc:`CognitiveDualQueryStrategyRanVarUn ` - stream classification single-annotator - :footcite:t:`liu2023active` * - :doc:`Cognitive Dual-Query Strategy with Variable-Uncertainty ` - :doc:`CognitiveDualQueryStrategyVarUn ` - stream classification single-annotator - :footcite:t:`liu2023active` * - :doc:`Split ` - :doc:`Split ` - stream classification single-annotator - :footcite:t:`zliobaite2014active` * - :doc:`Density Based Active Learning for Data Streams ` - :doc:`StreamDensityBasedAL ` - stream classification single-annotator - :footcite:t:`ienco2014high` * - :doc:`Probabilistic Active Learning in Datastreams ` - :doc:`StreamProbabilisticAL ` - stream classification single-annotator - :footcite:t:`kottke2015probalistic` Informativeness ~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - Method - Base Class - Tags - Reference * - :doc:`Fixed-Uncertainty ` - :doc:`FixedUncertainty ` - stream classification single-annotator - :footcite:t:`zliobaite2014active` * - :doc:`Randomized-Variable-Uncertainty ` - :doc:`RandomVariableUncertainty ` - stream classification single-annotator - :footcite:t:`zliobaite2014active` * - :doc:`Variable-Uncertainty ` - :doc:`VariableUncertainty ` - stream classification single-annotator - :footcite:t:`zliobaite2014active` References ---------- .. footbibliography::