Efficient Probing - ICLR 2026 Poster Presentation

Created on April 23, 2026

2026

We are excited to present our latest research on Efficient Probing (EP) at the International Conference on Learning Representations (ICLR 2026) in Brasil. This work revisits a fundamental approach in representation learning and reveals important insights about how modern vision models structure their learned features.

Event Details

đź“Ť Location: Poster Session 4, Pavilion 4, P4-#3713
🕒 Time: Friday, April 24, 15:15–17:45

The Problem

Linear probing has long been the standard benchmark for evaluating learned representations. However, this approach may not fully capture the expressive power of modern deep learning models — particularly those trained with masked image modeling (MIM) strategies.

Many state-of-the-art models learn rich, semantically meaningful representations at the patch level, but traditional global evaluation protocols can significantly underestimate their actual capabilities. This disconnect between representation quality and evaluation methodology motivated our investigation.

Our Solution: Efficient Probing

We introduce Efficient Probing (EP), a lightweight, attention-based probing method that:

  • Selectively aggregates patch-level tokens using learned attention weights
  • Improves the accuracy–efficiency trade-off between probing complexity and performance gains
  • Reveals stronger performance from locally-trained representations that standard methods overlook
  • Demonstrates that many vision models possess more expressive power than previously estimated

By designing a more appropriate evaluation protocol, we show that representations trained with masked image modeling and other local-learning objectives are far more powerful than traditional benchmarks suggest.

Key Contributions

The work challenges conventional wisdom about representation learning and provides practitioners with a more accurate assessment tool for their models. This has important implications for model selection, transfer learning, and the design of future representation learning methods.

Resources

Poster Preview