Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026), Association for Computational Linguistics
Preview abstract
This work introduces a fine-grained table retrieval framework for grounding large
language models in heterogeneous, open-domain relational data. Instead of encoding a query as a single vector, the approach decomposes natural
language queries into semantic components and embeds each independently, enabling
more precise matching of compositional query intent. These representations are used in a staged retrieval pipeline with component-level
search, connectivity-aware grouping, and reranking. Experiments on three TARGET benchmark corpora show consistent improvements in capped recall@k and stronger alignment between query intent and tabular structure over dense retrieval baselines, particularly for longer and more complex queries and when using lightweight embedding models.View details
Proceedings of the 8th International Workshop on Applied AI for Database Systems and Applications (AIDB 2026) at VLDB (to appear)
Preview abstract
In this vision paper, we propose a novel architectural paradigm for accelerated AI query execution via a unified compiled execution strategy. By compiling the hybrid \textit{AI Query} as a whole—integrating both standard SQL relational constructs and LLM inference layers into a single, unified tensor compute graph—we completely eliminate PCIe data movement bottlenecks across execution boundaries and enable global compiler optimizations. We demonstrate the viability of this unified execution paradigm on synthetic benchmarks and select AI queries from \textit{SemBench}, achieving up to 5.3x latency speedup and 9.8x throughput speedup on TPUs, and outline a research roadmap of open technical challenges to realize this vision.View details
Proceedings of the VLDB Endowment, VLDB Endowment (2026)
Preview abstract
LLMs are being increasingly integrated into systems that support queries
over unstructured data~\cite{shankar2025docetl, patel2025semantic, liu2024declarative}.
In these LLM-powered data processing systems, a query is expressed
as a sequence of familiar data processing operators
such as map, reduce, and filter, each described in natural language for an LLM to carry out,
with the system's optimizer determining the plan to execute.
In this paper, we design a new optimizer to extend the rewrite directive approach to multi-objective optimization.
We introduce the Multi-Objective Agentic Rewrites (MOAR) optimizer, which surfaces a Pareto frontier of pipelines that improve accuracy over the user's original while offering a range of costs---all under a limited evaluation budget (dozens of pipelines).
We model the search space of pipelines (i.e., query plans) as a graph where each node is a complete pipeline and each edge applies a rewrite to produce a new one.
Our insight is to apply a multi-armed bandit approach to searching the graph of rewrite sequences: at each iteration, MOAR must decide which pipeline to expand and which rewrite to apply, balancing exploration of new rewrite sequences against exploitation of promising ones.
We evaluate MOAR on six real-world workloads spanning legal, medical, and enterprise domains. Compared to state-of-the-art systems and naive agentic baselines,
MOAR achieves up to 1136.4\% higher output quality and 84\% lower inference cost at equivalent quality levels, while surfacing a richer set of trade-off plans for users. View details
Proceedings of the ACM on Management of Data, ACM (2026), pp. 1-27
Preview abstract
Large language models (LLMs) have achieved strong performance on natural language to SQL (NL2SQL) tasks, but their effectiveness depends heavily on system configuration. Real-world deployments involve tuning a complex pipeline of components, including LLM choice, prompting strategy, schema linking, and example selection, while navigating trade-offs between accuracy and monetary cost. Critically, the optimal configuration varies across databases, yet prior work lacks a principled framework for such database-aware optimization.
We present PRISM, a two-phase system that systematically identifies high-accuracy, cost-efficient NL2SQL configurations tailored to each database. Phase 1 applies Bayesian Optimization with a cost-aware acquisition function to efficiently explore the configuration space and select a diverse set of high-performing single-candidate pipelines. Phase 2 ensembles these configurations to further improve execution accuracy. Experiments on the BIRD benchmark demonstrate that PRISM achieves 69.48\% execution accuracy in the single candidate setting, improving accuracy by 2.34\% and reducing cost by 92\% compared to baselines. In the ensemble setting, PRISM boosts accuracy further to 75.3\%.View details
2026 IEEE 42nd International Conference on Data Engineering (ICDE)
Preview abstract
The demand for high-fidelity test data is paramount in industrial settings where access to production data is largely restricted. Traditional data generation methods often fall short, struggling with low-fidelity and the ability to model complex data structures and semantic relationships that are critical for testing complex SQL code generation services like Natural Language to SQL (NL2SQL). In this paper, we address the critical need for generating syntactically correct and semantically ``meaningful'' mock data for complex nested schema structures that are commonly expected in real-world SQL code generation services. We highlight the limitations of existing approaches used in production, particularly their inability to handle large and complex schema, as well as the lack of semantically coherent test data that lead to limited test coverage.
We demonstrate that by leveraging Large Language Models (LLMs) and incorporating strategic pre- and post-processing steps, we can generate realistic high-fidelity test data that adheres to complex structural constraints and maintains semantic integrity to the test targets (SQL queries/functions).
This approach supports comprehensive testing of complex SQL queries involving joins, aggregations, and even deeply nested subqueries, ensuring robust evaluation of SQL code generation services, like NL2SQL and SQL Code Assistant services.
Our results demonstrate the practical utility of an out-of-the-box LLM (\textit{gemini}) based test data generation for industrial SQL code generation services where generating realistic test data is essential due to the frequent unavailability of production datasets.View details
Proceedings of the ACM on Management of Data, ACM New York, NY, USA (2026), pp. 1-23
Preview abstract
Several data warehouse and database providers have recently introduced extensions to SQL called AI Queries, enabling users to specify conditions in SQL that are evaluated by LLMs, thereby broadening significantly the kinds of queries one can express over the combination of structured and unstructured data. LLMs offer remarkable semantic reasoning capabilities, making them an essential tool for complex and nuanced queries that blend structured and unstructured data. While extremely powerful, these AI queries can become prohibitively costly when invoked on tables with millions of rows.
This paper provides an extensive evaluation of an AI query approximation approach that enables low cost analytics and database applications to benefit from AI queries.
The approach deliver >100x cost and latency reduction for the semantic filter (AI.IF) operator and also important gains for semantic ranking (ORDER BY AI.SCORE). The cost and performance gains come from utilizing cheap and accurate proxy inference ML models over embedding vectors.
We show that despite the massive gains in latency and cost, these proxy models preserve accuracy across various benchmark datasets, including the extended Amazon reviews benchmark that has 10M rows.
We present an OLAP-friendly architecture of this approach for purely online queries, a database app-friendly architecture that further increases accuracy and the online gains by moving the proxy model training offline. We present techniques that accelerate the proxy model training. View details
Proceedings of the 1st Workshop on Systems for Agentic AI (SAA 2025) at SOSP
Preview abstract
We introduce Cortex, a prototype workflow-aware serving platform designed for agentic workloads.
The core principle of Cortex is stage isolation: it provisions dedicated resource pools for each distinct stage of an agentic workflow.
This simple yet powerful strategy mitigates inter-stage interference in compute and memory, leading to better KV cache utilization, higher throughput, and more predictable performance.
By customizing resource allocation and scheduling within each distinct stage of agentic workflows, Cortex lays the groundwork for more advanced, agent-native serving paradigms, including malleable resource management, speculative execution of workflow branches, and a shared, multi-tiered cache for ``agentic state.''View details
Preview abstract
Large Language Models (LLMs) have demonstrated impressive capabilities across a range of natural language processing tasks. In particular, improvements in reasoning abilities and the expansion of context windows have opened new avenues for leveraging these powerful models.
NL2SQL is challenging in that the natural language question is inherently ambiguous, while the SQL generation requires a precise understanding of complex data schema and semantics. One approach to this semantic ambiguous problem is to provide more and sufficient contextual information.
In this work, we explore the performance and the latency trade-offs of the extended context window (a.k.a., long context) offered by Google's state-of-the-art LLM (\textit{gemini-1.5-pro}).
We study the impact of various contextual information, including column example values, question and SQL query pairs, user-provided hints, SQL documentation, and schema. To the best of our knowledge, this is the first work to study how the extended context window and extra contextual information can help NL2SQL generation with respect to both accuracy and latency cost.
We show that long context LLMs are robust and do not get lost in the extended contextual information. Additionally, our long-context NL2SQL pipeline based on Google's \textit{gemini-pro-1.5} achieve a strong performance with 67.41\% on BIRD benchmark (dev) without finetuning and expensive self-consistency based techniques.View details
Proceedings of the VLDB Endowment, VLDB Endowment (2025)
Preview abstract
Large Language Models (LLMs) have demonstrated impressive capabilities across a range of natural language processing tasks. In
particular, improvements in reasoning abilities and the expansion
of context windows have opened new avenues for leveraging these
powerful models. NL2SQL is challenging in that the natural language question is inherently ambiguous, while the SQL generation
requires a precise understanding of complex data schema and semantics. One approach to this semantic ambiguous problem is to
provide more and sufficient contextual information.
In this work, we explore the performance and the latency tradeoffs of the extended context window (a.k.a., long context) offered
by Google’s state-of-the-art LLM (gemini-1.5-pro). We study the
impact of various contextual information, including column example values, question and SQL query pairs, user-provided hints, SQL
documentation, and schema. To the best of our knowledge, this
is the first work to study how the extended context window and
extra contextual information can help NL2SQL generation with
respect to both accuracy and latency cost. We show that long context LLMs are robust and do not get lost in the extended contextual
information. Additionally, our long-context NL2SQL pipeline based
on Google’s Gemini-pro-1.5 achieves strong performance across
multiple benchmark datasets without fine-tuning or expensive
self-consistency based techniques.View details
IEEE Transactions on Knowledge and Data Engineering (2019)
Preview abstract
As machine learning systems become democratized, it becomes increasingly important to help users easily debug their
models. However, current data tools are still primitive when it comes to helping users trace model performance problems all the way to
the data. We focus on the particular problem of slicing data to identify subsets of the validation data where the model performs poorly.
This is an important problem in model validation because the overall model performance can fail to reflect that of the smaller subsets,
and slicing allows users to analyze the model performance on a more granular-level. Unlike general techniques (e.g., clustering) that
can find arbitrary slices, our goal is to find interpretable slices (which are easier to take action compared to arbitrary subsets) that are
problematic and large. We propose Slice Finder, which is an interactive framework for identifying such slices using statistical
techniques. Applications include diagnosing model fairness and fraud detection, where identifying slices that are interpretable to
humans is crucial. This research is part of a larger trend of Big data and Artificial Intelligence (AI) integration and opens many
opportunities for new researchView details