Evict3R: Training-Free Token Eviction for Memory-Bounded Streaming Visual Geometry Transformers

Soroush Mahdi¹, Fardin Ayar¹, Ehsan Javanmardi², Manabu Tsukada², Mahdi Javanmardi¹

¹Autonomous and Intelligent Systems Lab, Amirkabir University of Technology (Tehran Polytechnic)²Tsukada Laboratory, The University of Tokyo

paper code bibtex

demo://evict3r · budget B = 0.2

Reconstructions from our method with B = 0.2. Despite using less than half the memory, our method shows no visible drop in quality.

18.63 → 9.39 GBpeak memory, 7-Scenes, long sequences

0.003drop in accuracy and completeness

training-freeplug-and-play at inference time

$ cat abstract.md

Streaming visual transformers like StreamVGGT achieve strong 3D perception but suffer from unbounded growth of key–value (KV) memory, which limits scalability. We propose a training-free, inference-time token eviction policy that bounds memory by discarding redundant tokens while keeping the most informative ones. Our method uses significantly less memory with little to no drop in accuracy: on 7-Scenes with long sequences it reduces peak memory from 18.63 GB to 9.39 GB while accuracy and completeness drop by only 0.003. Under strict memory budgets, eviction enables denser frame sampling, which improves reconstruction accuracy compared to the baseline. Experiments across video depth estimation (Sintel, KITTI), 3D reconstruction (7-Scenes, NRGBD), and camera pose estimation (Sintel, TUM-dynamics) show that our approach closely matches StreamVGGT at a fraction of the memory and makes long-horizon streaming inference more practical.

$ cat method.md

Our method manages the growing key–value (KV) cache of StreamVGGT by introducing a layer-wise token eviction framework. Each new frame produces tokens that are added to the cache; when the cache reaches its per-layer budget, we rank all tokens by their cumulative attention scores, normalized for exposure and sequence length. Tokens that receive little attention are evicted, while essential ones are preserved. This simple plug-and-play mechanism keeps memory bounded while maintaining the rich context needed for accurate 3D reconstruction.

Evict3R method overview: per-layer KV cache with attention-based token eviction

$ python kv_demo.py --stream

A toy version of the idea you can play with: stream frames into a cache, then lower the budget and watch which tokens survive.

demo://kv-cache-eviction

Why streaming 3D transformers run out of memory

Each new frame adds tokens to the cache. Lower the budget and the least-attended tokens get evicted (dim cells). Most of the attention mass survives, because attention is sparse.

budget 132

unbounded cache288 tok

budgeted cache132 tok

attention mass kept–

toy illustration with synthetic scores; not the paper's exact eviction rule

$ ./visualize --layer 1

Token eviction on a sample input, layer by layer. Pick a layer to see which tokens are evicted as frames stream in (visualizations for layers 1–15).

layer

$ cat results.md

Quantitative comparison of Evict3R with baseline StreamVGGT across different datasets and metrics. Table III samples 6–10 frames for 7-Scenes and 5–10 frames for NRGBD. Table IV samples 24–40 frames for 7-Scenes and 20–40 frames for NRGBD. For ultra-long sequences (Table IV), StreamVGGT requires 16.41 GB (NRGBD) and 18.63 GB (7-Scenes), but our method achieves similar accuracy with nearly half the memory.

Results tables comparing Evict3R with StreamVGGT

$ cat citation.bib

@misc{mahdi2025evict3rtrainingfreetokeneviction,
  title={Evict3R: Training-Free Token Eviction for Memory-Bounded Streaming Visual Geometry Transformers},
  author={Soroush Mahdi and Fardin Ayar and Ehsan Javanmardi and Manabu Tsukada and Mahdi Javanmardi},
  year={2025},
  eprint={2509.17650},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2509.17650}
}