Decoding embodied language representations in human communication

Human communication often draws on both verbal and non-verbal signals, yet the specific contribution of interlocutors' body language remains unclear. Here, we provide direct evidence for language representations emerging seconds before spoken words in both speakers' and listeners' body positions during natural conversations, using word embeddings from BERT, a widely used transformer-based large language model in artificial intelligence. Specifically, we found that contextual word embeddings outperform non-contextual embeddings in predicting interlocutors' body pose, especially in noisy environments, highlighting the importance of contextualised embodied language representations in complementing and enhancing real-world conversational speech. Notably, our encoding models showed robust cross-subject generalisation, capturing shared patterns of language embodiment across individuals. Furthermore, these models also revealed a marked distinction between speaking and listening, indicating that individuals switch between different, role specific embodied processes during conversation. Our findings showcase the potential of large language models to reveal the intricate dynamics of human communication and the need for integrated multi-modal verbal and non-verbal processes for effective, human-like conversational artificial agents.