Visualizations

On this page, we illustrate how different query strategies behave across various active learning scenarios. To this end, we generate synthetic, low-dimensional toy datasets that allow us to visualize the distribution of samples selected for labeling in feature space. Each example consists of an animation and accompanying code that plot the results across individual active learning cycles.

🏊 Pool-based Active Learning: Classification

Core Set

Core Set

Random Sampling

Random Sampling

MaxHerding

MaxHerding

Active Learning with Cost Embedding (ALCE)

Active Learning with Cost Embedding (ALCE)

Density-Diversity-Distribution-Distance Sampling (4DS)

Density-Diversity-Distribution-Distance Sampling (4DS)

Uncertainty Herding (UHerding)

Uncertainty Herding (UHerding)

Contrastive Active Learning (CAL)

Contrastive Active Learning (CAL)

Typical Clustering (TypiClust)

Typical Clustering (TypiClust)

Querying Informative and Representative Examples (QUIRE)

Querying Informative and Representative Examples (QUIRE)

Uncertainty Sampling (US) with Entropy

Uncertainty Sampling (US) with Entropy

Epistemic Uncertainty Sampling (EpisUS)

Epistemic Uncertainty Sampling (EpisUS)

Uncertainty Sampling (US) with Margin

Uncertainty Sampling (US) with Margin

Uncertainty Sampling (US) with Least-Confidence

Uncertainty Sampling (US) with Least-Confidence

Probability Coverage (ProbCover)

Probability Coverage (ProbCover)

Uncertainty Sampling with Expected Average Precision (USAP)

Uncertainty Sampling with Expected Average Precision (USAP)

Monte-Carlo Expected Error Reduction (EER) with Log-Loss

Monte-Carlo Expected Error Reduction (EER) with Log-Loss

Query-by-Committee (QBC) with Vote Entropy

Query-by-Committee (QBC) with Vote Entropy

Query-by-Committee (QBC) with Variation Ratios

Query-by-Committee (QBC) with Variation Ratios

Monte-Carlo Expected Error Reduction (EER) with Misclassification-Loss

Monte-Carlo Expected Error Reduction (EER) with Misclassification-Loss

Sub-sampling Wrapper

Sub-sampling Wrapper

Discriminative Active Learning (DAL)

Discriminative Active Learning (DAL)

Parallel Utility Estimation Wrapper

Parallel Utility Estimation Wrapper

Multi-class Probabilistic Active Learning (McPAL)

Multi-class Probabilistic Active Learning (McPAL)

Query-by-Committee (QBC) with Kullback-Leibler Divergence

Query-by-Committee (QBC) with Kullback-Leibler Divergence

Value of Information (VOI)

Value of Information (VOI)

Value of Information (VOI) on Labeled Samples

Value of Information (VOI) on Labeled Samples

Value of Information (VOI) on Unlabeled Samples

Value of Information (VOI) on Unlabeled Samples

Density-weighted Uncertainty Sampling (DWUS)

Density-weighted Uncertainty Sampling (DWUS)

Bayesian Active Learning by Disagreement (BALD)

Bayesian Active Learning by Disagreement (BALD)

Dual Strategy for Active Learning

Dual Strategy for Active Learning

Batch Active Learning by Diverse Gradient Embedding (BADGE)

Batch Active Learning by Diverse Gradient Embedding (BADGE)

Clustering Uncertainty-weighted Embeddings (CLUE)

Clustering Uncertainty-weighted Embeddings (CLUE)

Dropout Query (DropQuery)

Dropout Query (DropQuery)

Batch Density-Diversity-Distribution-Distance Sampling (4DS)

Batch Density-Diversity-Distribution-Distance Sampling (4DS)

Fast Active Learning by Contrastive UNcertainty (FALCUN)

Fast Active Learning by Contrastive UNcertainty (FALCUN)

Batch Bayesian Active Learning by Disagreement (BatchBALD)

Batch Bayesian Active Learning by Disagreement (BatchBALD)

Label Cardinality Inconsistency (LCI)

Label Cardinality Inconsistency (LCI)

Maximum Loss Reduction with Maximal Confidence (MMC)

Maximum Loss Reduction with Maximal Confidence (MMC)

🏊 Pool-based Active Learning: Regression

Greedy Sampling on the Feature Space (GSx)

Greedy Sampling on the Feature Space (GSx)

Greedy Sampling on the Target Space (GSy)

Greedy Sampling on the Target Space (GSy)

Improved Greedy Sampling (GSi)

Improved Greedy Sampling (GSi)

Expected Model Variance Reduction

Expected Model Variance Reduction

Expected Model Change Maximization

Expected Model Change Maximization

Query-by-Committee (QBC) with Empirical Variance

Query-by-Committee (QBC) with Empirical Variance

Expected Model Output Change

Expected Model Output Change

Regression based Kullback Leibler Divergence Maximization

Regression based Kullback Leibler Divergence Maximization

Regression Tree Based Active Learning (RT-AL) with Random Selection

Regression Tree Based Active Learning (RT-AL) with Random Selection

Regression Tree Based Active Learning (RT-AL) with Diversity Selection

Regression Tree Based Active Learning (RT-AL) with Diversity Selection

Regression Tree Based Active Learning (RT-AL) with Representativity Selection

Regression Tree Based Active Learning (RT-AL) with Representativity Selection

🏊 Pool-based Active Learning: Multi-annotator Learning

Interval Estimation Threshold

Interval Estimation Threshold

Random Sampling

Random Sampling

Core Set + Greedy Selection

Core Set + Greedy Selection

🌊 Stream-based Active Learning: Classification

Split

Split

Periodic Sampling

Periodic Sampling

Stream Random Sampling

Stream Random Sampling

Variable-Uncertainty

Variable-Uncertainty

Density Based Active Learning for Data Streams

Density Based Active Learning for Data Streams

Fixed-Uncertainty

Fixed-Uncertainty

Randomized-Variable-Uncertainty

Randomized-Variable-Uncertainty

Cognitive Dual-Query Strategy with Random Sampling

Cognitive Dual-Query Strategy with Random Sampling

Cognitive Dual-Query Strategy with Variable-Uncertainty

Cognitive Dual-Query Strategy with Variable-Uncertainty

Cognitive Dual-Query Strategy with Randomized-Variable-Uncertainty

Cognitive Dual-Query Strategy with Randomized-Variable-Uncertainty

Cognitive Dual-Query Strategy with Fixed-Uncertainty

Cognitive Dual-Query Strategy with Fixed-Uncertainty

Probabilistic Active Learning in Datastreams

Probabilistic Active Learning in Datastreams