Alexandros Karatzoglou
Alexandros works in the areas of machine learning and recommender systems. At Google he conducts research in conversational recommendation and reinforcement learning. His full list of publications can be found in Google Scholar
Research Areas
Authored Publications
Google Publications
Other Publications
Sort By
Choosing the Best of Both Worlds: Diverse and Novel Recommendations through Multi-Objective Reinforcement Learning
Dusan Stamenkovic
Ioannis Arapakis
Kleomenis Katevas
Xin Xin
15th ACM International Conference on Web Search and Data Mining - WSDM 2022, Arizona, USA (to appear)
Preview abstract
Since the inception of Recommender Systems (RS), the accuracy of the recommendations in terms of relevance has been the golden criterion for evaluating the quality of RS algorithms. However, by focusing on item relevance, one pays a significant price in terms of other important metrics: users get stuck in a "filter bubble" and their array of options is significantly reduced, hence degrading the quality of the user experience and leading to churn. Recommendation, and in particular session-based/sequential recommendation, is a complex task with multiple - and often conflicting objectives -that existing state-of-the-art approaches fail to address.In this work, we take on the aforementioned challenge and introduce Scalarized Multi-Objective Reinforcement Learning (SMORL)for the RS setting, a novel Reinforcement Learning (RL) framework that can effectively address multi-objective recommendation tasks.The proposed SMORL agent augments standard recommendation models with additional RL layers that enforce it to simultaneously satisfy three principal objectives:accuracy, diversity, and novelty of recommendations. We integrate this framework with four state-of-the-art session-based recommendation models and compare it with a single-objective RL agent that only focuses on accuracy. Our experimental results on two real-world datasets reveal a substantial increase in aggregate diversity, a moderate increase in accuracy, reduced repetitiveness of recommendations, and demonstrate the importance of reinforcing diversity and novelty as complementary objectives.
View details
Supervised Advantage Actor-Critic for Recommender Systems
Xin Xin
Ioannis Arapakis
Joemon Jose
15th ACM International Conference on Web Search and Data Mining - WSDM 2022, Arizona, USA (to appear)
Preview abstract
Casting session-based or sequential recommendation as reinforcement learning (RL) through reward signals is a promising research direction towards recommender systems (RS) that maximize long-term user engagement. However, the direct use of RL algorithms under the RS setting is unfeasible due to challenges like off-policy training, huge action spaces and lack of sufficient reward signals. Recent RL approaches in the recommendation domain try to tackle these challenges, by for example combining RL and self-supervised learning. In this paper, we examine the approach of self-supervised reinforcement learning for recommendation and show that existing methods still have limitations. For example, the negative signals from the self-supervised component are not sufficient for the RL component to perform good ranking. Moreover, the length of the sequence could also introduce bias to the training procedure.
To address the above problems, we first propose to introduce negative sampling into the RL training procedure and then combine it with self-supervised learning, namely Self-Supervised Negative Q-learning (SNQN). Based on the sampled negative actions (items), we can further calculate the ``advantage" of a positive action, which can be further utilized as a weight for the self-supervised part. This lead to another learning framework: Self-Supervised Advantage Actor-Critic (SA2C). We integrate SNQN and SA2C with four state-of-the-art sequential recommendation models and conduct experiments on two real-world datesets. Experimental results show that the proposed approaches achieve better performance than existing self-supervised reinforcement learning methods. Code will be open-sourced.
View details
On Interpretation and Measurement of Soft Attributes for Recommendation
Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '21) (2021)
Preview abstract
We address how to robustly interpret natural language refinements (or critiques) in recommender systems. In particular, in human-human recommendations settings people frequently use soft attributes to express preferences about items, including concepts like the originality of a movie plot, the noisiness of a venue, or the complexity of a recipe. While binary tagging is extensively studied in the context of recommender systems, soft attributes often involve subjective and contextual aspects, which cannot be captured reliably in this way, nor be represented as objective binary truth in a knowledge base. This also adds important considerations when measuring soft attribute ranking. We propose a more natural representation as personalized relative statements, rather than as absolute item properties. We present novel data collection techniques and evaluation approaches, and a new public dataset. We also propose a set of scoring approaches, from unsupervised to weakly supervised to fully supervised, as a step towards interpreting and acting upon soft attribute based critiques.
View details
Graph Convolutional Embeddings for Recommender Systems
Paula Gomez Duran
Xin Xin
Ioannis Arapakis
IEEE Access, vol. 9 (2021), pp. 100173 - 100184
Preview abstract
Modern recommender systems (RS) work by processing a number of signals that can be inferred from large sets of user-item interaction data. The main signal to analyze stems from the raw matrix that represents interactions. However, we can increase the performance of RS by considering other kinds of signals like the context of interactions, which could be, for example, the time or date of the interaction, the user location, or sequential data corresponding to the historical interactions of the user with the system. These complex, context-based interaction signals are characterized by a rich relational structure that can be represented by a multi-partite graph. Graph Convolutional Networks (GCNs) have been used successfully in collaborative filtering with simple user-item interaction data. In this work, we generalize the use of GCNs for N-partite graphs by considering N multiple context dimensions and propose a simple way for their seamless integration in modern deep learning RS architectures. More specifically, we define a graph convolutional embedding layer for N-partite graphs that processes user-item-context interactions and constructs node embeddings by leveraging their relational structure. Experiments on several datasets show the benefits of the introduced GCN embedding layer by measuring the performance of different context-enriched tasks
View details
Disentangling Preference Representations for Recommendation Critiquing with β-VAE
30th ACM International Conference on Information and Knowledge Management (CIKM 2021), ACM, New York
Preview abstract
Modern recommender systems usually embed users and items into a learned vector space representation. Similarity in this space is used to generate recommendations, and recommendation methods are agnostic to the structure of the embedding space. Motivated by the need for recommendation systems to be more transparent and controllable, we postulate that it is beneficial to assign meaning to some of the dimensions of user and item representations. Disentanglement is one technique commonly used for this purpose. We present a novel supervised disentangling approach for recommendation tasks. Our model learns embeddings where attributes of interest are disentangled, while requiring only a very small number of labeled items at training time. The model can then generate interactive and critiquable recommendations for all users, without requiring any labels at recommendation time, and without sacrificing any recommendation performance. Our approach thus provides users with levers to manipulate, critique and fine-tune recommendations, and gives insight into why particular recommendations are made. Given only user-item interactions at recommendation time, we show that it identifies user tastes with respect to the attributes that have been disentangled, allowing for users to manipulate recommendations across these attributes.
View details
Self-Supervised Reinforcement Learning for Recommender Systems
Xin Xin
Ioannis Arapakis
Joemon Jose
Proceedings of the 43th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '20) (2020)
Preview abstract
In session-based or sequential recommendation, it is important to consider a number of factors like long-term user engagement, rich user interaction modes such as clicks, purchase history etc. The
current state of the art supervised approaches fail to model them appropriately affecting the performance. In this context, casting sequential recommendation task as a reinforcement learning is a
promising direction. A major component of RL approaches is to model the reward through the interaction between an agent and the environment. This is challenging in recommendation setting due
to the pure off-policy setting and lack of negative rewards (feedback).
In building models for recommender systems, it would often be problematic to train a model in an on-line fashion (as required by many modern RL methods) due to the requirement to expose the
users to irrelevant recommendations. As a result, off-line learning from logged implicit feedback is of vital importance.
In this paper, we propose a self-supervised reinforcement learning approach for sequential recommendation tasks. Our approach has two components: one for supervised learning; and another for reinforcement learning. The layer trained with reinforcement learning acts as a regularizer to drive the supervised head focusing on specific rewards (e.g., helping the user in purchases) while the supervised head with cross-entropy loss provides negative gradient signals for parameter updates. Based on such an approach, we propose two frameworks namely self-Supervised Q-learning (SQN) and self-Supervised Actor-Critic (SAC). We integrate four state-of-the-art generative recommendation models in our frameworks. Experimental results on two data sets demonstrate the effectiveness of our
approach.
View details
No Results Found