Universal Self-adaptive Prompting

With the recent advancements in few-shot prompting of Language Models (LLMs), the importance of automatic prompt design for in-context learning has become increasingly crucial and challenging. However, automatic prompt design becomes even more challenging in Zero-shot setup, where there are no ground-truth labels available as demonstrations. In this study, we present an automatic prompt design approach specifically tailored for zero-shot setups. Our strategy involves converting zero-shot settings into "few-shots" scenarios by constructing pseudo-demonstrations from model-generated outputs. The generation and selection of these pseudo-demonstrations pose challenges, particularly when addressing a diverse range of tasks. To address this, we propose a universal automatic prompt design framework, denoted as Unique Selling Points (USP): by incorporating different task-selectors, our framework enables automatic prompt designs that effectively cater to various tasks such as question answering, natural language understanding, and text summarization. We demonstrate impressive results compared to alternative zero-shot prompt design approaches.