Words Speak Louder Than Order: A Behavioral Evaluation of Gemma 4

When an AI language model receives two conflicting documents, how does it decide which one to trust? Does it rely on how the sources are framed or the physical order of the documents? We evaluated this behavior on Google's Gemma 4-e4b model using a completely counterbalanced experimental design. This setup allowed us to mathematically isolate the specific effects of source framing and reading position, while ensuring the model's natural vocabulary biases were canceled out. Across ten test conditions, we discovered that source framing heavily overpowers reading position by a roughly 4-to-1 margin.

• Machine intelligence