Optimizing Small Language Models for NL2SQL via Chain-of-Thought Fine-Tuning
Abstract
Translating Natural Language to SQL (NL2SQL) remains a critical bottleneck for enterprise democratization of data. While Large Language Models (LLMs) like Gemini 2.5 have demonstrated impressive zero-shot capabilities, their high inference costs limit deployment at scale. This paper explores the efficacy of fine-tuning both large-scale and small-scale language models on NL2SQL tasks.
Our research reveals a counter-intuitive scaling phenomenon. Fine-tuning large models (Gemini 2.5 Flash/Lite) on standard datasets yields negligible or even negative returns, often leading to overfitting on complex queries. Conversely, small models (Qwen) show significant gains. While standard fine-tuning improved the small model baseline from 36% to 45%, enriching the dataset with explicit Chain-of-Thought (CoT) reasoning surged accuracy to 54.5%. This paper demonstrates that transferring reasoning patterns enables compute-efficient smaller models to approach production-grade performance.
Our research reveals a counter-intuitive scaling phenomenon. Fine-tuning large models (Gemini 2.5 Flash/Lite) on standard datasets yields negligible or even negative returns, often leading to overfitting on complex queries. Conversely, small models (Qwen) show significant gains. While standard fine-tuning improved the small model baseline from 36% to 45%, enriching the dataset with explicit Chain-of-Thought (CoT) reasoning surged accuracy to 54.5%. This paper demonstrates that transferring reasoning patterns enables compute-efficient smaller models to approach production-grade performance.
