- Goetz Graefe
- Ian James Rae
- Jeff Naughton
- Jeff Shute
- Jeremy David Wood
- Jiacheng Yang
- John Michael Cieslewicz
- Jun Ma
- Jun Xu
- Junxiong Zhou
- Kelvin Lau
- Qiang Zeng
- Qilin (Kylin) Dong
- Xi Zhao
- Yuan Gao
- Zhan Yuan
- Ziyang Chen
Abstract
The ongoing and increasing interest in HTAP (Hybrid Transactional and Analytical Processing) systems documents the intense interest from data owners in simultaneously running transactional and analytical workloads over the same data set. Much of the reported work on HTAP has arisen in the context of “green field” systems, answering the question “if we could design a system for HTAP from scratch, what would it look like?” While there is great merit in such an approach, and a lot of valuable technology has been developed with it, we found ourselves facing a different challenge: one in which there is a great deal of transactional data already existing in several transactional systems, heavily queried by an existing federated engine that does not “own” the transactional systems, supporting both new and legacy applications that demand transparent fast queries and transactions from this combination. This paper reports on our design and experiences with F1 Lightning, a system we built and deployed to meet this challenge. We describe our design decisions, some details of our implementation, and our experience with the system in production for some of Google's most demanding applications.
Research Areas
Learn more about how we do research
We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work