Validating ISP Sensors in Pre-Silicon Platforms
Abstract
In the modern era, traditional DSLR cameras have seen a decline in popularity as smartphones have become the primary tool for capturing high-quality photography. This shift is driven by the integration of advanced image-capturing hardware and sophisticated image processing systems within mobile devices.
Modern smartphones now feature multiple cameras with ever-increasing specifications for resolution, zoom, and frame rate across various lenses like wide and telephoto. Consequently, the Image Signal Processors (ISPs) and Neural Processing Units (NPUs) responsible for tasks such as Bayer filtering, HDR construction, pixel binning, and depth mapping have become significantly more complex.
Developing drivers for new image sensors and testing these complex ISPs/NPUs with authentic image data presents a major challenge in the pre-silicon phase. Conventional simulations are often too slow to process multiple high-resolution frames within a practical timeframe. To address this, verification and validation teams are turning to FPGA and emulation platforms. These environments support high-speed testing with real-world sensors, allowing for the processing of multi-frame high-resolution images and offering a clear advantage over traditional simulation methods.
This paper evaluates various methodologies for validating Sensors and ISPs in pre-silicon, highlighting the benefits of using FPGA and emulation platforms alongside physical sensors. Key areas of focus include:
Extensive pre-silicon interoperability testing utilizing POR camera sensors
Broad ISP test coverage across diverse image formats, resolutions, and frame rates
End-to-end path testing from the Sensor to the ISP, DPU, and finally the Display
Establishment of driver development platforms specifically for camera sensors
Prioritizing the integration of actual hardware through these sophisticated platforms enhances verification coverage and facilitates the premature identification of defects, thereby bolstering confidence in the final System-on-Chip (SoC). By attaining near-silicon validation coverage six to eight months in advance of the projected timeline through pre-silicon platforms, a substantive "shift left" within the validation lifecycle is effectively achieved.
Modern smartphones now feature multiple cameras with ever-increasing specifications for resolution, zoom, and frame rate across various lenses like wide and telephoto. Consequently, the Image Signal Processors (ISPs) and Neural Processing Units (NPUs) responsible for tasks such as Bayer filtering, HDR construction, pixel binning, and depth mapping have become significantly more complex.
Developing drivers for new image sensors and testing these complex ISPs/NPUs with authentic image data presents a major challenge in the pre-silicon phase. Conventional simulations are often too slow to process multiple high-resolution frames within a practical timeframe. To address this, verification and validation teams are turning to FPGA and emulation platforms. These environments support high-speed testing with real-world sensors, allowing for the processing of multi-frame high-resolution images and offering a clear advantage over traditional simulation methods.
This paper evaluates various methodologies for validating Sensors and ISPs in pre-silicon, highlighting the benefits of using FPGA and emulation platforms alongside physical sensors. Key areas of focus include:
Extensive pre-silicon interoperability testing utilizing POR camera sensors
Broad ISP test coverage across diverse image formats, resolutions, and frame rates
End-to-end path testing from the Sensor to the ISP, DPU, and finally the Display
Establishment of driver development platforms specifically for camera sensors
Prioritizing the integration of actual hardware through these sophisticated platforms enhances verification coverage and facilitates the premature identification of defects, thereby bolstering confidence in the final System-on-Chip (SoC). By attaining near-silicon validation coverage six to eight months in advance of the projected timeline through pre-silicon platforms, a substantive "shift left" within the validation lifecycle is effectively achieved.
