Cloud AI
Our mission is to spread useful AI effectively around the world.
Our mission is to spread useful AI effectively around the world.
About the team
The Google Cloud AI Research team tackles AI research challenges motivated by Google Cloud’s mission of bringing AI to tech, healthcare, finance, retail and many other industries. We work on a range of unique high-impact problems with the goal of maximizing both scientific and real-world impact – both pushing the state-of-the-art in AI (>60 papers published at top research venues over the past four years) and collaborating across teams to bring innovations to production (e.g., 1, 2, 3).
Some recent directions for Cloud AI Research include:
- Developing improved foundation models to solve challenges like hallucinations, data efficiency and generalization.
- Improved adaptation methods, including distillation, task customization, grounding and multimodality.
- Developing large language models (LLMs) for data types that are a high priority to enterprises, such as structured data.
- Building LLMs for tool use.
- Retrieval-augmented LLMs and LLM-assisted search.
- Improved LLM usability through automated prompting, explainability and reliability.
Team focus summaries
Featured publications
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
Cheng-Yu Hsieh
Chun-Liang Li
Chih-Kuan Yeh
Alexander Ratner
Ranjay Krishna
ACL 2023
SPADE: Semi-supervised Anomaly Detection under Distribution Mismatch
Chun-Liang Li
Kihyuk Sohn
Transactions on Machine Learning Research (TMLR) (2023)
Better Zero-Shot Reasoning with Self-Adaptive Prompting
Hanjun Dai
Findings of the Association for Computational Linguistics: ACL 2023 (2023)
Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models
Cheng-Yu Hsieh
Si-An Chen
Chun-Liang Li
Alexander Ratner
Ranjay Krishna
arXiv preprint arXiv:2308.00675 (2023)
FormNetV2: Inductive Multimodal Graph Contrastive Learning for Form Document Information Extraction
Chun-Liang Li
Hao Zhang
Xiang Zhang
Kihyuk Sohn
Nikolai Glushnev
Joshua Ainslie
Nan Hua
ACL (2023)
Highlighted work
-
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model SizesA novel method of distillation that proposes a resource-efficient training-to-deployment paradigm compared to existing methods. The method reduces the size of the training dataset required to curate task-specific smaller models; it also reduces the model size required to achieve, and even surpass, the original LLM’s performance. -
LANISTR: Multimodal Learning from Structured and Unstructured DataA novel method of distillation that proposes a resource-efficient training-to-deployment paradigm compared to existing methods. The method reduces the size of the training dataset required to curate task-specific smaller models; it also reduces the model size required to achieve, and even surpass, the original LLM’s performance. -
Consistency-based Self-adaptive PromptingConsistency-based Self-adaptive Prompting (COSP), a novel prompt design method for LLMs. Requiring neither handcrafted responses nor ground-truth labels, COSP selects and builds the set of examples from the LLM zero-shot outputs via carefully designed criteria that combine consistency, diversity and repetition. -
Tool Documentation Enables Zero-Shot Tool-Usage with Large Language ModelsTool documentation—individual tool usage descriptions—is an alternative to LLM demonstrations. We demonstrate that zero-shot prompts on a tool-use dataset with hundreds of available tool APIs or with unseen state-of-the-art models as tools achieve better performance compared to few-shot prompts. -
TabNetA new deep learning method for tabular data that improves over other DNN and ensemble decision tree models on many datasets and provides interpretable insights. -
TSMixer: An all-MLP architecture for time series forecastingA new multivariate model that leverages linear model characteristics and performs well on long-term forecasting benchmarks. -
Unsupervised and semi-supervised anomaly detection with data-centric MLAn anomaly detection method that utilizes an ensemble of OCCs to estimate the pseudo-labels of unlabeled data independent of the given positive labeled data, thus reducing the dependency on the labels.
Some of our locations
Some of our people
-
Tomas Pfister
- Algorithms and Theory
- Machine Intelligence
- Machine Perception
-
Alex Muzio
- Algorithms and Theory
- Distributed Systems and Parallel Computing
- Machine Intelligence
-
Chen-Yu Lee
- Machine Intelligence
- Machine Perception
- Natural Language Processing
-
Chun-Liang Li
- Algorithms and Theory
- Machine Intelligence
- Machine Perception
-
Hootan Nakhost
- Machine Intelligence
- Machine Perception
- Natural Language Processing
-
Jinsung Yoon
- Algorithms and Theory
- Machine Intelligence
- Machine Perception
-
Lesly Miculicich
- Machine Intelligence
- Machine Translation
- Natural Language Processing
-
Long T. Le
- Data Mining and Modeling
- Machine Intelligence
- Machine Learning
-
Rajarishi Sinha
- Algorithms and Theory
- Machine Intelligence
- Machine Translation
-
Ruoxi Sun
- Data Mining and Modeling
- General Science
- Machine Intelligence
-
Sayna Ebrahimi
- Algorithms and Theory
- Machine Intelligence
- Machine Perception
-
Sercan O. Arik
- Algorithms and Theory
- Machine Intelligence
- Machine Perception
-
Yanfei Chen
- Machine Intelligence
- Natural Language Processing
-
Yihe Dong
- Algorithms and Theory
- Natural Language Processing
-
Zifeng Wang
- Algorithms and Theory
- Data Mining and Modeling
- Machine Intelligence
