Measuring Semantic Complexity with Embeddings: Static vs. Contextual Representation

Iva Ivanova

Rohdenburg's Complexity Principle (1996) predicts that syntactically more explicit constructions are preferred in cognitively more demanding processing environments. In this talk, I investigate whether embeddings can serve as a reliable means for measuring semantic complexity, and what role they might play in empirically grounding the Complexity Principle. To be more concrete, I will investigate two types of embedding-based measures: static (word2vec, GloVe), which encode meaning independently from context, and contextual embeddings (BERT, GPT-based), which encode meaning dynamically based on the surrounding linguistic material. Drawing on psycholinguistic accounts of processing complexity - for instance, surprisal theory (Levy 2008) - I will discuss the methodological trade-offs between the two approaches and their respective suitability for measuring semantic complexity.