The benefits of joint design of computation and optics within a hybrid computational-sensing technique was lucidly illustrated over half-a-century ago by the emergence of the now-venerable technique of Fourier-transform spectroscopy. It was known from the fundamental work by Jacquinot [1] and Fellget [2] that the Fourier-transform spectrometer could achieve a three-orders-of-magnitude enhancement in optical throughput, but it was not until Cooley and Tukey reported the fast-Fourier-transform algorithm [3], and the development of digital computation enabled its application, that the technique became practical. Fourier-transform spectroscopy subsequently became established as the workhorse laboratory tool it is today. After many Moore’s-law doublings of computer power, computer processing has now become routinely integrated with physical sensing and processing techniques, enabling increasingly complex interplay between optical sensing, imaging and computation. The first OSA Conference on Computational Optical Sensing and Imaging was held in Charlotte, North Carolina, USA in 2005 and the COSI conference is now established as the key annual conference in this field. This COSI Feature Issue is associated with the 2019 conference in Munich, Germany. Due to the life-changing events of the covid-19 pandemic, the 2020 COSI conference will instead be held by video conference, facilitated by the same concepts in data compression and miniaturised computational imaging cameras that have been a feature of this conference series.

This Feature Issue includes nineteen papers ranging from new emerging techniques, such as imaging round corners or microscopy beyond the diffraction limit, to enhancements of established techniques in 3D imaging and chemical sensing. A particularly striking achievement of computational imaging is the development of techniques that provide capabilities that have not previously been possible. High-resolution, wide-field microscopy can be achieved by traditional means employing high-cost, high-numerical aperture microscope objectives and precision translation stages. Based on the principles of ptychography, Fourier ptychography is now able to generate gigapixel high-resolution phase and amplitude microscopy that is beyond the diffraction limit of the simple low-cost objective lenses used. Konda et al. [4] provide the first comprehensive review of this burgeoning field, highlighting its relationship to other computational-imaging techniques, the most important achievements in the field and the challenges for the future. On a macroscopic scale, computational imaging has demonstrated the counter-intuitive ability to image around corners using computational inversion of time-gated imaging of scattered light. Early demonstrations have tended to employ simplified scenarios involving scattering of light from planar stationary surfaces as scatterers, but La Manna et al. [5] describe how adding an additional time-of-flight detector enables non-line-of-sight imaging using light scattered off dynamic surfaces.

For long-range, time-gated 3D imaging, Li et al. [6] report the use of digital superresolution to resolve single-photon LIDAR spatial features that are approximately 2x smaller than the diffraction limit. This is achieved by performing a 3D deconvolution of LIDAR data with Total Variation Regularization, which enforces gradient-domain sparsity in reconstructed depth maps. Image resolution can also be limited by the pixel size of the focal-plane array. Digital super-resolution techniques reconstruct imagery by developing reconstruction algorithms that incorporate prior information about signals to help regularize what is otherwise an ill-posed image reconstruction problem. Kocsis et al. [7] report the incorporation of phase masks together with the popular Block-Matched 3D (BM3D) filtering algorithm to produce high resolution wavefront reconstructions directly from a lensless coherent imaging setup (e.g. Gabor hologram) with large pixels. They experimentally demonstrating super-resolution factors as high as 3.45.

Compressive imaging is based in the recognition that sparse sampling of typical scenes can enable more efficient computational reconstruction of images. Many compressive-imaging approaches have focused on detection using a single pixel to generate images, but we include two articles reporting interesting variations on this approach. Compressive sensing has been reported for super-resolution of image pixelation produced by low-resolution detector arrays, which is of particular pertinence for thermal-infrared imaging. Wu and Wang [8] describe algorithms that eliminate the degradation associated with non uniformity in medium-wave infrared detector arrays. In the field of cryptography Jiao et al. report a novel application of single-pixel imaging that combines ideas from compressive sensing and optical cryptography, to authenticate encrypted content printed or displayed on opaque surfaces [9]. Related techniques have been employed in spectrally programmable cameras, employing spatio-spectral filtering to optically compare object spectra with library spectra prior to detection. It is difficult however, to simultaneously achieve both high spatial and high spectral resolution. Saragrdam and Sankaranarayanan [10] discuss the concept of space-spectrum uncertainty and formalize the tradeoff between the two resolutions and the relationship to conventional and compressive hyperspectral cameras.

Computational imaging and microscopy have a rich shared history, in particular in addressing the profound challenges of imaging three-dimensional and transparent biological samples and for metrology of industrial and scientific samples. We include three articles describing improvements to established computational-microscopy techniques of through-focal microscopy and phase microscopy. Peng et al. describe an algorithm that corrects for nanoscale lateral translations of an object that occur during through-focus microscopy enabling lower-cost mechanics to be employed [11]. Jayakumar et al. exploit the Moiré fringes that occur when sampling high-frequency fringe patterns arising from field curvature. This enables simultaneous measurement of wide-field biological samples at high speed [12]. Wavefront sensing is an integral component of adaptive-optics systems and for metrology. Wang et al. present a unified model for image formation in Shack-Hartmann (slope tracking) and curvature-based (Transport-of-Intensity Equation) wavefront sensing [13]. The model expands the domain of validity of TIE-based approaches to larger propagation distances beyond the finite-difference approximation. The model is additionally used to identify the theoretical resolution limit in propagation-based approaches to deterministic wavefront sensing.

A particularly vivid impact of computational imaging on everyday life is the myriad uses of consumer mobile-phone based cameras for applications ranging from image-stitching of panoramas to metrology and 3D immersive imaging. These consumer devices can now be used to transfer optical metrology from the laboratory to the field such as is described by Willomitzer et al. [14]. They employ the screen of the mobile phone for structured-light illumination and multi-view image stitching to measure 3D profiles of extended specular surfaces.

In recent years, the availability of big data and powerful computing hardware have led to tremendous interest in machine learning and deep learning, and it is not surprising that they are also increasingly important as a powerful tool to complement physics-based approaches in computational-imaging. In this Feature Issue, four papers are particularly noteworthy in this regard. The first, by Zeng et al. [15], features a deep-learning methodology called residual encoder-decoder capsule network (RedCap), and applies it to digital holographic reconstruction. The approach has better efficiency in processing data than a convolutional neural network, making it more suitable for applications with more constraints on computational resources. Deng et al. [16] also consider possible improvements to traditional deep-learning methods, particularly for low-light scenarios. Specifically, they look at phase retrieval, and investigate alternatives of the loss functions to see their effects on high and low frequencies. As for Jiao et al. [17], they caution that powerful though learning-based methods are, they should not be the only tool for all optical imaging problems. In particular, they show that linear regression can still be a viable method under certain circumstances. Deep neural networks (DNN) continue to attract wide attention. In optical diffraction tomography, a conventional object reconstruction method is used to iteratively solve an optimization problem using gradient descent to update the object estimation and then projecting the result onto a convex set. Instead of replacing the whole object reconstruction process with a DNN, Yang et al. use a DNN for the projection process [18]. Then the algorithm can still get feedback from system measurements and take advantage from the powerful capability of the DNN.

Speckle phenomena can severely degrade coherent imaging, but for computational sensing of physical phenomena it offers a simple and convenient source of modulation. Fluorescence lifetime imaging is widely used for sensing of chemicals, but relies on short-pulse lasers. Junek et al. report the exploitation of the rapid temporal fluctuations of speckles produced by a rotating speckle plate to provide a low-cost alternative excitation source for fluorescence-lifetime imaging [19]. Carles et al. exploit variations in the tissue-point spread function and associated spatial-frequency spectrum of objective speckle to enable calculation of chromophore absorption in turbid media (such as partially oxygenated blood), without the degradations and practicalities of lens-based imaging [20]. Temporal correlations in speckle can also be exploited to extract polarimetric phase of an object obscured behind scattering media as described by Chen et al. [21]. Their approach exploits correlations in scattered light and combines ideas from intensity interferometry and phase-shifting interferometry to recover the unknown phase. A strength of the approach is its robustness to vibrations and environmental fluctuations. Unprocessed speckle patterns are of course, very sensitive to mechanical vibration and Wu at al. demonstrate how calculation of the optic flow of speckle can be used to produce a laser microphone without the complexity of conventional interferometers [22].

As exemplified by the 19 papers in this Feature Issue, the field of computational optical sensing and imaging is diverse and rapidly evolving. Underpinned by increasing computational power, the breadth of applications and complexity of solutions have advanced considerably since the development of the Fourier-transform spectrometer, but the fundamental principles and benefits of integration of computational processing and optical sensing and imaging remain.