Non-line-of-sight optical sensors applications

NLOS sensing will change the way how we think about optical sensing, and will bring new sensing capabilities that might increase the situational awareness and observation capabilities of the soldier dramatically, and therefore increase survivability and the superiority of own troops. In this perspective, NLOS sensing is expecting to strongly impact military operations.

The technology will enable new optical imaging and ranging within a hidden scene without direct line-of-sight, including being able to “look around corners”. This technology might be used in many operations where information about a hidden area is needed but beyond direct access. Possible operation scenarios include enhanced situational awareness, mission planning for hostage rescue (localization of persons in building) and threat analysis like detection of ambush.

Disruptive progress in quantum sensing devices is expected to bring game-changing capabilities in light sensing with high-dynamic range from daylight to very low light conditions. In the area of reconstruction algorithms, breakthrough in analysis is expected with a three dimensional resolution in real time, at performance levels close to current line-of-sight sensing. These reconstruction algorithms could also advance and push developments in current side technologies such as noise removal or viewing through turbid media (e.g. brownout, submarine).

Optical technologies are facing a paradigm shift by an evolutionary revolution from digital imaging to computational and quantum imaging. These disruptive novel optical sensing approaches could bring game-changing sensing capabilities to many military operations. As a lighthouse technology in computational and quantum imaging indeed, non-line-of-sight imaging (NLOS) could overcome limitations of classical optical sensing which are tied to the direct line-of-sight, as it can extend the perception range of an optical sensor to areas hidden from direct view, while insuring high spatial optical resolution. In the future, this emerging technology might therefore enhance soldier’s observation and detection capabilities dramatically by bringing imaging and ranging capabilities, in many operational scenarios where current technologies such as line-of-sight optical sensing or RADAR fail to deliver relevant data with appropriate resolution. Possible operation scenarios include enhanced situational awareness, mission planning for hostage rescue (localization of persons in building) and threat analysis like detection of ambush.

Specific challenge

This topic aims to push the development of novel quantum sensing devices, laser technologies and computational algorithms such as geometrical reconstruction and artificial intelligence, such as to enable a breakthrough in optical sensing and situational awareness.

Classically, the perception area of optical sensors is limited to the line-of-sight. This area can be extended by computational imaging to areas outside the direct line-of-sight: using highly sensitive devices, multiple diffuse reflected photons can be recorded and their signatures analysed by sophisticated algorithms, such as physically based back-projection or artificial intelligence. Due to multiple diffuse reflections, the expected signals are very low and require quantum sensing devices with single photon counting capabilities. Further, the sensors have to measure the photon roundtrip path length with high precision.

Expertise from different fields is to be combined to build a demonstrator to be validated in a relevant environment. In this context, the topic calls for research in fields of computer science, for the development of novel reconstruction algorithms, semiconductor electronics, for the development of highly sensitive and precise single photon counting devices, and photonics for the development of laser illumination and optical receiver. All research activities may be led to a laboratory scale demonstration system which may be tested in relevant scenarios.

Targeted activities

The proposals must cover the following activities as referred in article 10.3 of the EDF Regulation:

- Activities that aim to create, underpin and improve knowledge, products and technologies, including disruptive technologies for defence, which can achieve significant effects in the area of defence;

- Activities that aim to increase interoperability and resilience, including secured production and exchange of data, to master critical defence technologies, to strengthen the security of supply or to enable the effective exploitation of results for defence products and technologies;

- Studies, such as feasibility studies to explore the feasibility of new or upgraded products, technologies, processes, services and solutions.

The proposals should support the development of novel sensor devices dedicated of NLOS sensing, the development of new reconstruction algorithms with the aim of fast reconstruction of the hidden scene, the development of a compact laser source with performance adapted to the specific needs of NLOS sensing and the development of a laboratory system to investigate and demonstrate NLOS sensing in relevant scenarios.

An integrated experimental setup is expected with first approaches for self-calibration and self-adoption to environmental conditions within the first 3 years. Then the maturity must be increased by further integration of the system components (laser, sensor, optics, software) and investigate first relevant scenarios by testing in representative environment.

The proposals must finally give prediction on how the technology investigated could be integrated into military sensing platforms and industrial products.

Functional requirements

The proposals must fulfil the following requirements:

- Combine appropriate photon-counting sensing devices and computational imaging analysis to sense and reconstruct the NLOS scenario.

- Be able to self-calibrate the 3D LOS scenario, which is a prerequisite knowledge to reconstruct the NLOS scenario.

- Define a modular approach in order to explore different sensing layouts, system designs and to evaluate their performances (single-pixel or array sensor, single laser point illumination or scanning illumination).

- Increase the maturity level of an NLOS system from generating and integrating knowledge to study in a 5-year time scale. The aim is to test the system in representative environment in a static or dynamic scenario closer to later use in military operation.

- Develop a thorough model for NLOS scenarios, able to reliably simulate performances (resolution, SNR ^[1] …) of the system as a function of the experimental parameters such as distance, chosen operating wavelength, background noise level, system efficiency, hidden object dimensions, atmospheric effects, dark counts and gating effects.

- Improve the development of sensor and laser technology in order to increase NLOS performances (sensor sensitivity and time-resolution, laser power and pulse duration)

- Develop fast data analysis algorithms to tackle real-time capabilities or, at least, reconstruct a distant NLOS scenario (several meters round trip) on an appropriate time scale (less-than-a-minute time scale).

- Establish a laboratory testing scenario of a hidden-scene to evaluate the performance of NLOS sensing systems and reconstruction algorithms.

- Increase the stealth and eye safety of NLOS systems by shifting the illumination wavelength from the VIS ^[2] spectrum to SWIR ^[3] spectrum.

- Define and establish a military relevant testing scenario to demonstrate the potential use of NLOS technology.

[1] Signal-to-noise ratio

[2] Visible

[3] Short-wave infrared