GitHub

🏆 Test-Time Adaptation Leaderboard

The world keeps changing, so should the model.
A Unified, Reproducible, Ready-to-Run Benchmark for Test-Time Adaptation,
covering Classification, Segmentation, and Detection with comprehensive Effectiveness and Efficiency evaluation.

🏆 Leaderboard

Configure Panel
1 · Select methods drag to add/remove · reorder within Selected · ·
selected ↓  ·  table & export order
2 · Display
3 · Export

🌐 Per-domain details (per corruption · best bold)

🗺️ About this Leaderboard

🗺️ 关于本排行榜

This is a unified Test-Time Adaptation (TTA) leaderboard spanning three tasks: Classification, Segmentation, and Detection. It reports both performance and efficiency across a range of models, settings, and datasets.

All methods are evaluated under one shared code framework. Performance is averaged over 5 random seeds, and efficiency is measured on a single RTX 4090. Source models and shift benchmarks are pinned to fixed models and datasets on the Hugging Face Hub.

Metrics by task: Classification uses Error % (lower is better), Segmentation uses mIoU % (higher is better), and Detection uses mAP (higher is better).

这是一个统一评测分类、分割、检测三类任务的测试时自适应（Test-Time Adaptation, TTA）排行榜，在不同模型、实验设置与数据集下报告性能与效率。

所有方法均在统一代码框架下评测，性能为 5 个随机种子重复运行后取平均，效率在单卡 RTX 4090 上测量。源模型与偏移基准均固定为 Hugging Face 上指定的模型与数据集。

各任务指标：分类用 Error %（越低越好），分割用 mIoU %（越高越好），检测用 mAP（越高越好）。

Tasks · Benchmarks · Source Models

任务 · 基准 · 源模型

The shift benchmarks apply the 15 ImageNet-C-style corruptions at severity 5. All assets are hosted under the WNJXYK Hugging Face org. Click any pill to open it.

偏移基准采用 15 种 ImageNet-C 风格的损坏（Severity 5）。所有资源均托管于 WNJXYK Hugging Face 组织下，点击任意标签即可打开。

TaskMetricBenchmarkSource models
🏷️ Classification Error % ↓ CIFAR-10-C WRN-28-10 AugMix-WRN-40-2 ShuffleNetV2 ViT-B/16
CIFAR-100-C ResNeXt-29 AugMix-WRN-40-2 ShuffleNetV2 ViT-B/16
ImageNet-C ResNet-50 ResNet-18 ConvNeXt-T ViT-B/16
🖼️ Segmentation mIoU % ↑ Cityscapes-C SegFormer-B5
ADE20K-C SegFormer-B5
🎯 Detection mAP ↑ COCO-C DETR-R50 Faster R-CNN R50-FPN

Protocols & Efficiency

协议与效率

Protocols define the characteristics of the test stream. Switch between them with the Leaderboard's Setting selector.

  • Online. Each shift is streamed on its own as an i.i.d. sequence, and the method is reset to the source model before every shift (episodic). The classic single-domain TTA setting; it measures how the method adapts under each individual shift.
  • Continual. The 15 shifts are streamed back-to-back with no reset in between, so adaptation error can accumulate. This tests the method under continuously changing conditions.
  • Dirichlet (non-i.i.d.). The stream's labels are sampled from a Dirichlet(α) class prior, producing temporally correlated, class-imbalanced batches, so the stream is no longer i.i.d. This tests the method in open, non-stationary environments.

Per-task defaults

  • Classification: Online · Continual · Dirichlet (α controls the skew)
  • Segmentation: Continual, one image per step (batch size = 1)
  • Detection: Online, one image per step (batch size = 1)

Efficiency is measured for every method in isolation on a single RTX 4090. Select the Runtime / Memory / Params Columns in the Leaderboard to show:

  • Throughput (samples/s): end-to-end adaptation and inference rate over the stream.
  • Adapter time (s/batch): wall-clock spent inside the adapter per batch, measured with CUDA synchronisation on both sides; source-model preprocessing is excluded.
  • Peak GPU memory (MiB): peak allocation including the source-model weights, reset per seed so it reflects this setup's true footprint.
  • Updated params: the number of parameters the method actually trains at test time, a proxy for adaptation overhead.

问题设置 / 评估协议定义了测试数据流的特性，可通过排行榜中的 Setting 选择器切换。

  • Online（在线适应）。每种偏移单独作为一个独立同分布的序列输入，且在每种偏移开始前将方法重置回源模型（episodic）。这是经典的测试时适应单域设定，考察算法在单一偏移下的适应行为。
  • Continual（持续适应）。15 种偏移首尾相接、连续测试，期间模型不重置，因此适配误差会不断累积。此设置考察算法在连续变化情形下的表现。
  • Dirichlet（非独立同分布适应）。数据流的标签按 Dirichlet(α) 类先验采样，产生时间上相关、类别不均衡的批次，数据流不再符合独立同分布特性。此设置考察算法在开放环境下的表现。

各任务默认设定

  • 分类任务：Online · Continual · Dirichlet（α 控制偏斜程度）
  • 分割任务：Continual，每步一张图（Batch Size = 1）
  • 检测任务：Online，每步一张图（Batch Size = 1）

效率（Efficiency）对每个方法统一在单块 RTX 4090 上隔离测量。在排行榜中选中 Runtime / Memory / Params Columns 可显示：

  • 吞吐量（samples/s）：数据流上端到端的适配与推理速率。
  • 适配耗时（s/batch）：每个批次在适配器内部花费的墙钟时间，两侧均做 CUDA 同步测量，不含源模型预处理时间。
  • 峰值显存（MiB）：包含源模型权重的峰值占用，每个随机种子前重置，以反映该配置的真实占用。
  • 更新参数量：方法在测试时实际训练的参数数量，作为适配开销的代理指标。

Code & Reproduce

代码与复现

Built on ttakit: the package is both a five-line library (TTA.wrap(model, adapter=...)) for quickly implementing TTA methods, and a YAML-driven framework for reproducing them. To reproduce any result:

基于 ttakit 构建：此工具包既是一个五行代码即可调用的库（TTA.wrap(model, adapter=...)），用于快速实现不同测试时适应算法；也是一个由 YAML 配置驱动的复现框架，用于复现各类方法。复现任意结果的命令如下：

uv run ttakit run --config configs/reproductions/<method>_<dataset>_<protocol>.yaml

📦 Code: TBD  ·  🤗 Models & data: huggingface.co/WNJXYK

📦 代码：TBD  ·  🤗 模型与数据：huggingface.co/WNJXYK