Thorough investigation of coherent synchrotron radiation (CSR) is a research focus of the Helmholtz Program Matter and Technologies at KIT. To detect and study THz CSR emission over multiple revolutions, THz detector systems are required to convert the photon signal into electrical pulses with picosecond time resolution. Continuous measurements of each bunch in CSR have turned out to be an indispensable tool for storage ring diagnostics and machine physics studies.

Technology

Sensors

The key feature of modern beam diagnostics is the possibility to optimize the spectrum sensitivity, the spatial resolution and the field-of-view of the camera, in according to the specific applications. To reach such modular approaches, several novel silicon line-array sensors from 512 up to 2048 pixels with pixel pitch of 25 and 45 µm were designed, produced and currently available to synchrotron accelerator communities. The photons transmittance into the semiconductor has been optimized by an Anti-Reflective Coating (ARC) back-end process. The ARC has been optimized for a wide-spectrum range from 320 nm to 1050 nm, figure 1.

In the new version of the Kalypso system there is an attractive possibility to extend the spectrum sensitivity range up to 5 µm by novel PbS and PbSe line arrays. In parallel, we have developed and submitted a novel ultra-fast line-array based on Trench-Isolated Low-Gain Avalanche Detector (TI-LGAD) technology. The new sensor has been designed and optimized in order to operate at several hundred million of frames per second and with an additional multiplication layer built-in in the bulk of the sensor, somewhat similar to the fast-gated intensifying camera. A cross-section of the TI-LGAD is shown in figure 2.

The new sensor technology will be employed in the transverse beam diagnostic where a fast charge collection time down to 3 ns is necessary to enable detection in a multi-bunch operation mode.

Front-end readout ASICs

Three ASICs have been developed in UMC- 110 nm CMOS technology, Gotthard version 1.6, Gotthard-HR (Proto) and the final version of the Gotthard-HR vers. 2. the readout chip is compatible with different sensors technologies (Si and InGaAs, InP) and operates over 10 MHz framerate at full detector occupancy. The block diagram of the ASICs is shown in figure 3, it features in 128 parallel readout channels, where each channel includes: 1) a full-differential folded-cascode charge-sensitive amplifier with three programmable gains by FPGA, 2) a differential correlated-double-sampling, which is indispensable to reduce the noise contribution, 3) a sampling-and-hold circuit followed by an analog channel buffer. 8 channels are then grouped, multiplexed and converted into digital bits by on-board multichannel fast ADCs. The new readout chip makes KALYPSO suitable detector system for a wide range of applications that requires very low-noise and high-framerate. The final chip is shown in figure 4.

 

Ultra-wide band RF analog front-end electronics for THz sensors

The KArlsruhe Pulse Taking and Ultrafast Readout Electronics system (KAPTURE), developed at KIT, implements a memory-efficient approach to acquire the detector signal on a bunch-by-bunch basis. Kapture is a wide-band (DC - 60 GHz) and ultrafast front-end readout system designed for direct sampling of ultrashort pulses generated from both room temperature or cryogenic THz sensors. The signal of the fast THz sensors is fed via high bandwidth connectors into a track-and-hold unit and a 12-bit, up to 2 GS/s, analog-to-digital converter. KAPTURE offers up to 8 sampling channels with individual delay units, whose sampling trigger points can be adjusted in 3 ps steps each. This enables a "local sampling" of a THz signal in minimum steps of 3 ps. This local sampling depends on the chosen delays between the channels, with a maximum rate of 330 GSa/s. The very low-noise condition allows to measure the arrived time of each pulse with picosecond time resolution and the pulse amplitude with mV resolution. All pulses are acquired in real-time, reconstructed and processed by an optimized data processing framework based on GPUs. Capture system is shown in figure 5.

 

High-density interconnects and mounting

The Kalypso detector system is based on Multichip modules (MCM) technology. The MCM is basically extensions of hybrid microcircuits, the differences being in their higher degree of density and improved electrical performance. The picture shown the Kalypso line-camera with 512 pixels, pixel pitch of 25µm. The silicon sensor has been optimized for a spectrum sensitivity peak @ 800 nm. The line sensor is connected to four readout chip Gotthard - KIT by a wedge-to-wedge wire-bonding process with 25 µm aluminium wire.

 

Heterogenous data acquisition and data processing

Both Kapture and Kalypso generate very large data volumes. To cope with this so called "big data" challenge, a high-performance heterogeneous FPGA-GPU-based computing infrastructure has been developed [Caselle] To meet real-time constraint, the data is directly transferred from the FPGA into the GPU memory bypassing CPU and CPU memory by high-performance ad-hoc direct memory access (DMA). To satisfy both the high data throughput and the fast data processing, a novel readout system based on PCIe generation 4 has been developed. The main processor is based on the ZYNQ UltraScale+ from Xilinx device which includes a 64-bit quadcore ARM processor with up to 1.5 GHz and a dual-core ARM with up to 600 MHz for Real-Time Operating System (RTOS). The ZYNQ is equipped with a large FPGA device. The readout card, shown in figure 7, is also equipped by two different system memories: a large DDR4 SODIMM memory attached to the processor systems (PS) and a 4 Gb DDR4 memory connected to the programmable logic (PL). A full-duplex FireFly electrical/optical data link operating up to 330 Gb/s in full-duplex mode, which allows to stream out the data generated by KAPTURE or KALYPSO directly to the microTCA devices.

The platform is an ideal system for the implementation of fast machine learning inference on FPGA based on the novel deep learning processor unit (DPU) technology. The integration of KAPTURE and KALYPSO with a real-time data processing based on artificial intelligence will open up a novel solution towards the autonomous accelerators. The systems could be directly integrated within the control system network of the accelerator / beam line by an embedded EPICs/TANGO server on the FPGA by standard TCP/IP port present on the readout card.

Projects

NeoDyn

Modern beam diagnostics instrumentation relies on several technologies: custom ASIC design, high-frequency technologies, custom superconducting or semiconductor design and fabrication, high-density interconnect technologies, high-throughput DAQ and advanced AI data processing. Within NeoDyn project, we develop a new generation of innovative detector systems for accelerator diagnostics which opens new windows of opportunities in photon science and user experiments. In particular, we have produced and installed several detector systems which goes well beyond the state-of-the-art and therefore they lead to new or improved applications in the THz range and for short electron and photon pulses at large-scale facilities. To study THz behaviour with an unprecedented temporal resolution and to resolve bunch-to-bunch interactions the novel front-end readout electronics so called "KAPTURE-2", has been developed. KALYPSO has originally been developed for the upgrade of the Electro-Optical Spectral Decoding experiments at the KIT research synchrotron light source KARA. Later KALYPSO was also used at the European XFEL and FLASH, ELBE and DELTA. The Key-strength of the both systems are: 1) modularity, both systems could be employed as a stand-alone detector or integrated within a MicroTCA crate, 2) high resolution combined to the high repetition rate, 3) capability to provide a base detector technology to facilitate advanced diagnostics, monitor and control of synchrotron, linear and plasma accelerators.

For students

The proposed research will develop novel advance and cutting-edge detector technologies and open new windows of opportunities in photon science and user experiments. It provides an ideal ground for the education and training of students and young researchers.