Information density converges in dialogue: Towards an information-theoretic model

David Reitter - Penn State University

Information density converges in dialogue: Towards an information-theoretic model

Yang Xu and David Reitter
Pennsylvania State University

“Predictive coding” suggests that the mind generates expectations about sensory input, such as words, as it engages in conversation with others.  Unexpected input will cause surprisal and higher cognitive load.  Research on monological language production online and in writing has suggested that people follow the entropy rate constancy (ERC) principle.  It states that language users distribute information such that words tend to be equally predictable given previous contexts.  How does this principle apply to spoken dialogue, if at all?

The study takes into account the joint-activity nature of dialogue and the topic shift mechanisms that are different from monologue. It examines how the information contributions from the two dialogue partners interactively evolve as the discourse develops. The increase of local sentence-level information density (predicted by ERC) is shown to apply across rather than within speakers. When the different roles of interlocutors in introducing new topics are identified‚ their contribution in information content displays a new converging pattern. We put forward an information-theoretic view of dialogue‚ which eventually may unify theories of interactive alignment and common ground.

If correct, this new understanding of complementary information density can also be used to gauge the effectiveness of communication. We predict the success of a collaborative task in English and Danish dialogue corpora.  After translating the lexical entropy series of each interlocutor into frequency space, we extract two features, power spectrum overlap and relative phase. A model trained on these features significantly improved on previous task success prediction models. Thus, the strategic distribution of information density between interlocutors is relevant to task success.

Xu, Y., & Reitter, D. (2018). Information density converges in dialogue: Towards an information-theoretic model. Cognition, (170),
147–163. Retrieved from

Xu, Y., & Reitter, D. (2017). Spectral Analysis of Information Density in Dialogue Predicts Collaborative Task Performance. In Proceedings of the 55th Annual Mtg. of the Association for Computational Linguistics. Vancouver, Canada. Retrieved from

If you would like to meet with the speaker please contact Vera Demberg.


David Reitter’s website