Home The Perils of Naive Truncation: A Context Ablation Study for Dialogue Summarization on DialogSum

Publications

The Perils of Naive Truncation: A Context Ablation Study for Dialogue Summarization on DialogSum

Year : 2025

Publisher : Institute of Electrical and Electronics Engineers Inc.

Source Title : 2025 5th International Conference on Artificial Intelligence and Signal Processing, AISP 2025

Document Type :

Abstract

A common shortcut in dialogue summarization is to truncate a conversation to its most recent turns, assuming this reduces cost without hurting summary quality. We test this assumption through a controlled context ablation study on DialogSum using DistilBART as the main model, with T5- small and BART-large for confirmation. Truncation consistently hurts quality: BERTScore drops by -0.0115 and ROUGE-L by -0.0705 (p < 0.0001), showing that removing early context acts as lossy compression that distorts meaning. Meanwhile, efficiency barely improves – shrinking input length by about 75% yields only a ~ 10% gain in throughput on an NVIDIA T4 GPU. These findings reveal that naive truncation is counterproductive; future dialogue summarization systems should adopt adaptive, content- aware context selection rather than blunt length reduction.