From Prompt to Probe: Left-Shifting SoC Debug with Generative AI
Abstract
Modern SoCs rely heavily on high-performance, handshake-based communication protocols like ready-valid and proprietary credit-based flow control mechanisms. In complex System-on-Chip (SoC) verification, systemic bugs such as protocol deadlocks and unknown logic state ('X') propagation are primary causes of costly delay. These issues are notoriously difficult to debug, often manifesting as vague simulation timeouts that could require days of runtime and painstaking, iterative manual analysis to root-cause. The traditional approach of manually writing assertions or monitors for every potential failure point is tedious, error-prone, and fundamentally unscalable for modern designs with thousands of interfaces and signals. This critical gap in verification automation creates a significant bottleneck, preventing the early detection, or "left shift," of showstopper bugs.
This framework introduces a project-agnostic, automated methodology leveraging Large Language Models (LLMs) to architect robust debug monitors and sophisticated Python APIs. By autonomously binding these monitors across diverse SoC hierarchical interfaces, the system drastically accelerates the detection and diagnosis of systemic failures, facilitating a critical left-shift in the verification lifecycle. Moreover, the proposed solution further optimizes design bring-up by automating force-injection on unconnected signals discovered as a workaround via advanced X-propagation analysis infrastructure.
Solution
The foundational step of this process involves the engineer authoring a precise, technical "architectural prompt" to the Large Language Model (LLM). This prompt acts as the high-level specification, detailing complex requirements such as specific handshake protocols, desired deadlock scenarios, and 'X'-propagation checking methodologies. Acting as an expert developer, the LLM synthesized the complete Python API, an orchestration engine capable of generating SystemVerilog checker modules. This proposed methodology was engineered to handle complex operations, including the analysis of hierarchical Register-Transfer Level (RTL) structures using an elaboration database, and implementing protocol-specific rules for ready-valid and credit-based timeouts, as well as automated X-propagation assertions. This methodology fundamentally redefines the engineer's function from low-level code implementer to strategic architect, focusing on defining the verification problem, specifying requirements, and critically reviewing and iteratively refining the AI-generated output. The resultant workflow is highly optimized for automation and reusability, with the LLM leveraging project context history and feedback to continuously evolve and minimize ramp-up time for subsequent verification tasks.
Figure-1: Core Automation workflow
Prompt Structure and Engineering: Effective prompting is critical. Our prompts are structured like a technical specification document, typically including the following sections :
Goal: A high-level summary of the desired output.
Requirements: A detailed, bulleted list of functionalities and features. It contains following subsections:
Input to Script: A description of precise definition of input formats, such as command-line arguments and CSV column headers.
Output of Script: A description of the expected output files and their contents.
The full technical paper details our structured prompt engineering methodology, providing a proven sample prompt alongside their synthesized monitors. By showcasing the exact steps and iterative refinement process, we provide a transparent solution that can be seamlessly adapted for diverse SoC architectures with minimal project-specific tuning
The framework addresses the below problems:
Deadlock and Hang Detection: By authoring precise technical specifications for handshake protocols, including Valid/Ready and credit-based flow control, for the LLM, the framework autonomously synthesized robust monitors bound across multiple hierarchical interfaces. The LLM independently navigated the Register-Transfer Level (RTL) elaboration database to identify intricate valid-ready patterns and figure out the interface level details to create the interface bind files. This methodology proactively isolates the exact coordinates and root causes of systemic deadlocks within both configuration and data paths.
Automated X-Propagation Detection: This tool was engineered to perform automated RTL elaboration database parsing, identifying module definitions and extracting comprehensive port lists across diverse hierarchical interfaces. To further optimize the verification lifecycle, the system integrates advanced diagnostic capabilities, including the analysis of simulation logs for 'X'-propagation errors and the autonomous generation of force-injection files as a workaround which facilitates left-shift in the verification cycle. Additionally, the system provides granular control, allowing for the disabling of specific signal or module checks and the definition of conditional dependencies via CSV files to eliminate false failures.
Salient Features
Reusability and Scalability - Infrastructure is project agnostic and can be deployed for all the designs which employ ready-valid and credit based handshake mechanisms. X-prop infrastructure can be deployed to IP and SoC environments with little to no-effort. The list of waivers can be seamlessly ported from IP to SoC.
Continuous Context Evolution - The framework utilizes a persistent context history where LLM feedback and project-specific architectural requirements are archived. By evolving the methodology via iterative feedback loops, we observed that subsequent tool generations required significantly lesser corrections. This continuous learning enables the LLM to produce production-quality SystemVerilog logic and Python scripts that are increasingly aligned with specific SoC design styles, drastically minimizing the verification cycle.
Results & Impact
The deployment of this framework had a transformative impact on our verification efforts, proving its ability to successfully "left shift" the discovery of critical bugs.
Drastic Reduction in Debug Time: The deadlock monitor identified hangs and reported the exact failure hierarchy in a matter of hours, resolving test timeouts that previously ran for over multiple days. This significantly reduced compute resource requirements and accelerated regression closure.
Improved Bug Hunting: The X-prop monitor identified numerous RTL bugs related to unconnected or improperly driven signals early in the bring-up phase. Bugs were reported and fixed in a single, parallel pass, eliminating weeks of iterative debug effort.
Quantifiable Regression Improvement: The deadlock infrastructure provided superior failure analysis, leading to a 80% reduction in the generic "hang" bucket for our SoC-level regressions.
Accelerated Development: Reduced the development time for the core Python API and monitor logic from an estimated six weeks (three for each tool) to just a few days, a productivity gain of over 95%.
Conclusion
This paper demonstrates a significant leap forward in verification automation. By treating the LLM as an expert developer guided by high-level architectural prompts, we have successfully automated the creation of robust deadlock and X-propagation monitors. The results—a drastic reduction in both development effort and debug time—confirm the viability of this new methodology. More importantly, this framework proves that the role of the verification engineer can evolve from tedious implementation to strategic problem-solving and AI-driven verification architect. This scalable, project-agnostic solution establishes a blueprint for future AI-assisted engineering, promising to resolve other complex verification challenges with similar efficiency.
This framework introduces a project-agnostic, automated methodology leveraging Large Language Models (LLMs) to architect robust debug monitors and sophisticated Python APIs. By autonomously binding these monitors across diverse SoC hierarchical interfaces, the system drastically accelerates the detection and diagnosis of systemic failures, facilitating a critical left-shift in the verification lifecycle. Moreover, the proposed solution further optimizes design bring-up by automating force-injection on unconnected signals discovered as a workaround via advanced X-propagation analysis infrastructure.
Solution
The foundational step of this process involves the engineer authoring a precise, technical "architectural prompt" to the Large Language Model (LLM). This prompt acts as the high-level specification, detailing complex requirements such as specific handshake protocols, desired deadlock scenarios, and 'X'-propagation checking methodologies. Acting as an expert developer, the LLM synthesized the complete Python API, an orchestration engine capable of generating SystemVerilog checker modules. This proposed methodology was engineered to handle complex operations, including the analysis of hierarchical Register-Transfer Level (RTL) structures using an elaboration database, and implementing protocol-specific rules for ready-valid and credit-based timeouts, as well as automated X-propagation assertions. This methodology fundamentally redefines the engineer's function from low-level code implementer to strategic architect, focusing on defining the verification problem, specifying requirements, and critically reviewing and iteratively refining the AI-generated output. The resultant workflow is highly optimized for automation and reusability, with the LLM leveraging project context history and feedback to continuously evolve and minimize ramp-up time for subsequent verification tasks.
Figure-1: Core Automation workflow
Prompt Structure and Engineering: Effective prompting is critical. Our prompts are structured like a technical specification document, typically including the following sections :
Goal: A high-level summary of the desired output.
Requirements: A detailed, bulleted list of functionalities and features. It contains following subsections:
Input to Script: A description of precise definition of input formats, such as command-line arguments and CSV column headers.
Output of Script: A description of the expected output files and their contents.
The full technical paper details our structured prompt engineering methodology, providing a proven sample prompt alongside their synthesized monitors. By showcasing the exact steps and iterative refinement process, we provide a transparent solution that can be seamlessly adapted for diverse SoC architectures with minimal project-specific tuning
The framework addresses the below problems:
Deadlock and Hang Detection: By authoring precise technical specifications for handshake protocols, including Valid/Ready and credit-based flow control, for the LLM, the framework autonomously synthesized robust monitors bound across multiple hierarchical interfaces. The LLM independently navigated the Register-Transfer Level (RTL) elaboration database to identify intricate valid-ready patterns and figure out the interface level details to create the interface bind files. This methodology proactively isolates the exact coordinates and root causes of systemic deadlocks within both configuration and data paths.
Automated X-Propagation Detection: This tool was engineered to perform automated RTL elaboration database parsing, identifying module definitions and extracting comprehensive port lists across diverse hierarchical interfaces. To further optimize the verification lifecycle, the system integrates advanced diagnostic capabilities, including the analysis of simulation logs for 'X'-propagation errors and the autonomous generation of force-injection files as a workaround which facilitates left-shift in the verification cycle. Additionally, the system provides granular control, allowing for the disabling of specific signal or module checks and the definition of conditional dependencies via CSV files to eliminate false failures.
Salient Features
Reusability and Scalability - Infrastructure is project agnostic and can be deployed for all the designs which employ ready-valid and credit based handshake mechanisms. X-prop infrastructure can be deployed to IP and SoC environments with little to no-effort. The list of waivers can be seamlessly ported from IP to SoC.
Continuous Context Evolution - The framework utilizes a persistent context history where LLM feedback and project-specific architectural requirements are archived. By evolving the methodology via iterative feedback loops, we observed that subsequent tool generations required significantly lesser corrections. This continuous learning enables the LLM to produce production-quality SystemVerilog logic and Python scripts that are increasingly aligned with specific SoC design styles, drastically minimizing the verification cycle.
Results & Impact
The deployment of this framework had a transformative impact on our verification efforts, proving its ability to successfully "left shift" the discovery of critical bugs.
Drastic Reduction in Debug Time: The deadlock monitor identified hangs and reported the exact failure hierarchy in a matter of hours, resolving test timeouts that previously ran for over multiple days. This significantly reduced compute resource requirements and accelerated regression closure.
Improved Bug Hunting: The X-prop monitor identified numerous RTL bugs related to unconnected or improperly driven signals early in the bring-up phase. Bugs were reported and fixed in a single, parallel pass, eliminating weeks of iterative debug effort.
Quantifiable Regression Improvement: The deadlock infrastructure provided superior failure analysis, leading to a 80% reduction in the generic "hang" bucket for our SoC-level regressions.
Accelerated Development: Reduced the development time for the core Python API and monitor logic from an estimated six weeks (three for each tool) to just a few days, a productivity gain of over 95%.
Conclusion
This paper demonstrates a significant leap forward in verification automation. By treating the LLM as an expert developer guided by high-level architectural prompts, we have successfully automated the creation of robust deadlock and X-propagation monitors. The results—a drastic reduction in both development effort and debug time—confirm the viability of this new methodology. More importantly, this framework proves that the role of the verification engineer can evolve from tedious implementation to strategic problem-solving and AI-driven verification architect. This scalable, project-agnostic solution establishes a blueprint for future AI-assisted engineering, promising to resolve other complex verification challenges with similar efficiency.
