Visualizations

On this page, we illustrate how different query strategies behave across various active learning scenarios. To this end, we generate synthetic, low-dimensional toy datasets that allow us to visualize the distribution of samples selected for labeling in feature space. Each example consists of an animation and accompanying code that plot the results across individual active learning cycles.

Pool-based AL Strategies

Core Set

Core Set

MaxHerding

MaxHerding

Random Sampling

Random Sampling

Active Learning with Cost Embedding (ALCE)

Active Learning with Cost Embedding (ALCE)

Contrastive Active Learning (CAL)

Contrastive Active Learning (CAL)

Typical Clustering (TypiClust)

Typical Clustering (TypiClust)

Uncertainty Sampling (US) with Entropy

Uncertainty Sampling (US) with Entropy

Epistemic Uncertainty Sampling (EpisUS)

Epistemic Uncertainty Sampling (EpisUS)

Querying Informative and Representative Examples (QUIRE)

Querying Informative and Representative Examples (QUIRE)

Uncertainty Sampling (US) with Margin

Uncertainty Sampling (US) with Margin

Uncertainty Sampling (US) with Least-Confidence

Uncertainty Sampling (US) with Least-Confidence

Probability Coverage (ProbCover)

Probability Coverage (ProbCover)

Uncertainty Sampling with Expected Average Precision (USAP)

Uncertainty Sampling with Expected Average Precision (USAP)

Density-Diversity-Distribution-Distance Sampling (4DS)

Density-Diversity-Distribution-Distance Sampling (4DS)

Monte-Carlo Expected Error Reduction (EER) with Log-Loss

Monte-Carlo Expected Error Reduction (EER) with Log-Loss

Monte-Carlo Expected Error Reduction (EER) with Misclassification-Loss

Monte-Carlo Expected Error Reduction (EER) with Misclassification-Loss

Sub-sampling Wrapper

Sub-sampling Wrapper

Discriminative Active Learning (DAL)

Discriminative Active Learning (DAL)

Parallel Utility Estimation Wrapper

Parallel Utility Estimation Wrapper

Multi-class Probabilistic Active Learning (McPAL)

Multi-class Probabilistic Active Learning (McPAL)

Query-by-Committee (QBC) with Vote Entropy

Query-by-Committee (QBC) with Vote Entropy

Query-by-Committee (QBC) with Variation Ratios

Query-by-Committee (QBC) with Variation Ratios

Query-by-Committee (QBC) with Kullback-Leibler Divergence

Query-by-Committee (QBC) with Kullback-Leibler Divergence

Value of Information (VOI)

Value of Information (VOI)

Value of Information (VOI) on Labeled Samples

Value of Information (VOI) on Labeled Samples

Value of Information (VOI) on Unlabeled Samples

Value of Information (VOI) on Unlabeled Samples

Bayesian Active Learning by Disagreement (BALD)

Bayesian Active Learning by Disagreement (BALD)

Density-weighted Uncertainty Sampling (DWUS)

Density-weighted Uncertainty Sampling (DWUS)

Dual Strategy for Active Learning

Dual Strategy for Active Learning

Batch Active Learning by Diverse Gradient Embedding (BADGE)

Batch Active Learning by Diverse Gradient Embedding (BADGE)

Clustering Uncertainty-weighted Embeddings (CLUE)

Clustering Uncertainty-weighted Embeddings (CLUE)

Dropout Query (DropQuery)

Dropout Query (DropQuery)

Fast Active Learning by Contrastive UNcertainty (FALCUN)

Fast Active Learning by Contrastive UNcertainty (FALCUN)

Batch Density-Diversity-Distribution-Distance Sampling (4DS)

Batch Density-Diversity-Distribution-Distance Sampling (4DS)

Batch Bayesian Active Learning by Disagreement (BatchBALD)

Batch Bayesian Active Learning by Disagreement (BatchBALD)

Pool-based AL Strategies for Regression

Greedy Sampling on the Feature Space (GSx)

Greedy Sampling on the Feature Space (GSx)

Greedy Sampling on the Target Space (GSy)

Greedy Sampling on the Target Space (GSy)

Improved Greedy Sampling (GSi)

Improved Greedy Sampling (GSi)

Expected Model Variance Reduction

Expected Model Variance Reduction

Expected Model Change Maximization

Expected Model Change Maximization

Query-by-Committee (QBC) with Empirical Variance

Query-by-Committee (QBC) with Empirical Variance

Expected Model Output Change

Expected Model Output Change

Regression based Kullback Leibler Divergence Maximization

Regression based Kullback Leibler Divergence Maximization

Regression Tree Based Active Learning (RT-AL) with Random Selection

Regression Tree Based Active Learning (RT-AL) with Random Selection

Regression Tree Based Active Learning (RT-AL) with Diversity Selection

Regression Tree Based Active Learning (RT-AL) with Diversity Selection

Regression Tree Based Active Learning (RT-AL) with Representativity Selection

Regression Tree Based Active Learning (RT-AL) with Representativity Selection

Pool-based AL Strategies for Mulitple Annotators

Interval Estimation Threshold

Interval Estimation Threshold

Random Sampling

Random Sampling

Core Set + Greedy Selection

Core Set + Greedy Selection

Stream-based AL Strategies

Split

Split

Periodic Sampling

Periodic Sampling

Stream Random Sampling

Stream Random Sampling

Variable-Uncertainty

Variable-Uncertainty

Density Based Active Learning for Data Streams

Density Based Active Learning for Data Streams

Fixed-Uncertainty

Fixed-Uncertainty

Randomized-Variable-Uncertainty

Randomized-Variable-Uncertainty

Cognitive Dual-Query Strategy with Random Sampling

Cognitive Dual-Query Strategy with Random Sampling

Cognitive Dual-Query Strategy with Variable-Uncertainty

Cognitive Dual-Query Strategy with Variable-Uncertainty

Cognitive Dual-Query Strategy with Randomized-Variable-Uncertainty

Cognitive Dual-Query Strategy with Randomized-Variable-Uncertainty

Cognitive Dual-Query Strategy with Fixed-Uncertainty

Cognitive Dual-Query Strategy with Fixed-Uncertainty

Probabilistic Active Learning in Datastreams

Probabilistic Active Learning in Datastreams