EmBARDiment: an Embodied AI Agent for Productivity in XR

XR devices running chatbots powered by Large Language Models (LLMs) have the potential to become always-on agents that significantly enhance productivity scenarios. Current screen-based chatbots fail to fully utilize the comprehensive suite of natural inputs available in XR, including inward-facing sensor data. Instead, they over-rely on explicit voice or text prompts, sometimes paired with multi-modal data included in the query. We propose a solution that leverages an attention framework to implicitly derive context from user actions, eye gaze, and contextual memory within the XR environment. Our approach minimizes the need for explicitly engineered prompts, fostering intuitive and grounded interactions that provide deeper user insights for the chatbot.