Hyperspectral In-Memory Computing with Optical Frequency Combs and Programmable Optical Memories

Hyperspectral In-Memory Computing and Applications of Optical Frequency Comb and Programmable Optical Memory

Introduction

In recent years, breakthroughs in machine learning have driven revolutionary developments in various industries, including healthcare, finance, retail, automotive, and manufacturing. These transformations have led to a surge in demand for extensive matrix-vector multiplication (MVM) operations, which are crucial for large-scale optimization and deep learning algorithms. However, this increasing computational demand challenges the traditional Von Neumann digital electronic computer architecture, which separates memory from processing units, resulting in the “Von Neumann bottleneck.” This bottleneck occurs when data transfer speed between memory and the processor limits overall system performance. To address this performance bottleneck, in-memory computing has emerged as a transformative solution, integrating non-volatile memory elements directly into the processor. This integration promotes more efficient data movement, reduces power consumption, and enables highly parallel computing.

At the same time, optical computing systems have garnered renewed attention due to their inherent suitability for parallel mathematical operations. These systems have made significant strides since their inception decades ago, showcasing the immense potential of optical computing in terms of high computational throughput and energy efficiency. However, developing high-throughput optical computing systems that can compete with these advanced electronic hardware remains a challenge. Against this backdrop, this study proposes and demonstrates a hyperspectral in-memory computing architecture that leverages both spatial and frequency multiplexing using optical frequency comb (OFC) and programmable optical memory. This highly designed three-dimensional optoelectronic computing system exhibits exceptional performance in terms of parallelism, programmability, and scalability, overcoming typical limitations of optical computing.

Paper Overview

This study was authored by Mostafa Honari Latifpour, Byoung Jun Park, Yoshihisa Yamamoto, and Myoung-Gyun Suh, and was published in the July 2024 issue, Volume 11, Issue 7, of the journal Optica. The research was primarily led by the Physics and Informatics Laboratory of NTT Research, Inc., and supported by institutions such as City University of New York and Korea University. The study draws inspiration from parallel information processing solutions in fields such as optical communications, spectroscopy, imaging, and display technology, aiming to significantly enhance computational throughput.

Experimental Processes and Results

Single Matrix-Vector Multiplication

In the experiment, researchers first demonstrated an optical MVM system utilizing hybrid space-frequency multiplexing, laying the foundation for the hyperspectral in-memory computing system. By interconnecting elements across frequency and spatial dimensions, this multiplexing approach facilitated parallel data transmission between these domains. The input source used an optical fiber OFC in the optical C-band with a pulse repetition rate of 250 MHz. It was then coarsely filtered to obtain a 36 GHz frequency interval. This effectively generated an OFC with 36 GHz frequency spacings and 250 MHz intensity modulation. Input vector elements were encoded line-by-line into the intensity of each 36 GHz comb line.

Through optical setup, the comb lines were spatially separated, laid out vertically, and concentrated onto a spatial light modulator (SLM). The encoded matrix elements multiplied with the input vector served as attenuation weights. The matrix-vector multiplication (MAC) operation was completed by capturing the output matrix of the SLM using a two-dimensional shortwave infrared camera. The system combined the output light matrix horizontally through a line-scan camera and detected its light intensity, verifying the accuracy of various MAC values.

Matrix-Matrix Multiplication via Hyperspectral MAC

To further expand the system, researchers conducted a hyperspectral setup experiment for matrix-matrix multiplication (MMM). The ultimately horizontally spread optical signal was concentrated on SLM 2 to encode the second matrix. The aggregated light signal captured by the two-dimensional camera was frequency-sorted and processed for MMM operations. Multiple MMM test results aligned with theoretical predictions, demonstrating the system’s robustness and accuracy with noise levels maintained below 5%.

Discussion

In verification experiments, the system operated in an open-loop mode, with input encoding and MAC result reading independently performed using commercial digital electronic devices. Achieving high-throughput computations necessitates fast external modulation and reading. By introducing the “two-dimensional optoelectronic neuron” device combining electronic control circuits and memory, programmable nonlinear operations and diverse algorithm executions can be realized. Moreover, the hybrid architecture showcased in this study combines the speed and energy efficiency advantages of optical computing, and plans are underway to enhance practical application through a closed-loop mode.

Conclusion

The proposed hyperspectral in-memory computing system fully exploits frequency, spatial, and temporal dimensions to maximize computational throughput and energy efficiency. The system design prioritizes scalability by uniting spatial and frequency multiplexing with scalable SLM and OFC technologies. This modular approach not only simplifies the manufacturing process but also directly benefits from the advancements in SLM and OFC technologies, thereby improving overall system performance. In the future, this architecture is expected to lead a new era of energy-efficient optical information processing, potentially surpassing Petaops in future cloud computing environments.