Combining Decision Trees and Neural Networks for Learning-to-Rank in Personal Search
Abstract
Decision Trees (DTs) like LambdaMART have been one of the
most effective types of learning-to-rank algorithms in the past
decade. They typically work well with hand-crafted dense features
(e.g., BM25 scores). Recently, Neural Networks (NNs) have shown
impressive results in leveraging sparse and complex features (e.g.,
query and document keywords) directly when a large amount of
training data is available. While there is a large chunk of work
on how to use NNs for semantic matching between queries and
documents, relatively less work has been conducted to compare
NNs with DTs for general learning-to-rank tasks, where dense
features are also available and DTs can achieve state-of-the-art
performance. In this paper, we study how to combine DTs and NNs
to effectively bring the benefits from both sides in the learning-to-
rank setting. Specifically, we focus our study on personal search
where clicks are used as the primary labels with unbiased learning-
to-rank algorithms and a significantly large amount of training data
is easily available. Our combination methods are based on ensemble
learning. We design 12 variants and compare them based on two
aspects, ranking effectiveness and ease-of-deployment, using two
of the largest personal search services: Gmail search and Google
Drive search. We show that direct application of existing ensemble
methods can not achieve both aspects. We thus design a novel
method that uses NNs to compensate DTs via boosting. We show
that such a method is not only easier to deploy, but also gives
comparable or better ranking accuracy.
most effective types of learning-to-rank algorithms in the past
decade. They typically work well with hand-crafted dense features
(e.g., BM25 scores). Recently, Neural Networks (NNs) have shown
impressive results in leveraging sparse and complex features (e.g.,
query and document keywords) directly when a large amount of
training data is available. While there is a large chunk of work
on how to use NNs for semantic matching between queries and
documents, relatively less work has been conducted to compare
NNs with DTs for general learning-to-rank tasks, where dense
features are also available and DTs can achieve state-of-the-art
performance. In this paper, we study how to combine DTs and NNs
to effectively bring the benefits from both sides in the learning-to-
rank setting. Specifically, we focus our study on personal search
where clicks are used as the primary labels with unbiased learning-
to-rank algorithms and a significantly large amount of training data
is easily available. Our combination methods are based on ensemble
learning. We design 12 variants and compare them based on two
aspects, ranking effectiveness and ease-of-deployment, using two
of the largest personal search services: Gmail search and Google
Drive search. We show that direct application of existing ensemble
methods can not achieve both aspects. We thus design a novel
method that uses NNs to compensate DTs via boosting. We show
that such a method is not only easier to deploy, but also gives
comparable or better ranking accuracy.
Research Areas
