A Three-Wafer-Stacked Hybrid 15-MPixel CIS + 1-MPixel EVS With 4.6-GEvent/s Readout, In-Pixel TDC, and On-Chip ISP and ESP Function

Abstract

We introduce a hybrid 4096 × 3680 CMOS image sensor (CIS) with an embedded 1032 × 928 event-based vision sensor (EVS). Within a four-by-four cluster of CIS pixels, one color channel is substituted to provide a photocurrent for an EVS pixel. The EVS readout circuit is allocated on a second wafer and wrapped behind the four-by-four pixel cluster. A third wafer is used to integrate all peripheral readout circuitry for the EVS and CIS functions while achieving a compact sensor footprint. The EVS pixel achieves sub-1 Hz noise rates and 3% contrast non-uniformity (CTNU) at a linear nominal contrast threshold (NCT) of 15%. The row and column readout circuitry bypasses elements without events in a fraction of a clock cycle. Furthermore, a skip function is realized that reads only bordering pixels of a connected set of events. This achieves an event rate of up to 4.6 GEvents/s (GEps). The sensor captures in-pixel time stamps through sampling of a gray code timer realizing a time- to-digital conversion (TDC). Image and event signal processors (ISP & ESP) are integrated on-chip. The ESP provides global ambient and global activity monitoring functions. The sensor sup- ports various flicker mitigation approaches such as adjustment of contrast threshold, in-pixel bandpass filter function and region- of-interest (ROI), and deterministic or random subsampling. The hybrid sensor mitigates challenges in synchronization and parallax error compared with a two-sensor solution using a dedicated CI