May 7, 2024
Defence Research and Development Canada (DRDC) experimented with two artificial intelligence (AI) enabled technologies to improve sensing and surveillance, in the multinational experiment Project Convergence CAPSTONE 4, held from February 18 to March 20, 2024 in Fort Irwin, California.
“We were able to showcase Canadian solutions to operational problems as well as conduct research and development alongside our partners right here - down in the dirt,” says defence scientist Dr. Mélanie Breton.
DRDC worked closely with Canadian Army end-users to experiment with an advanced sensing technology known as JAWS, during CAPSTONE 4.
During the event, DRDC assessed the use of two sensing technologies: the ground-based Joint Algorithmic Warfighter Sensor (JAWS), developed by DRDC, and the airborne Multimodal Input Surveillance and Tracking (MIST), developed by TerraSense Analytics.
JAWS
JAWS is a suite of distributed multi-modal sensors, AI models, and data management pipelines. It is currently being developed by the DRDC Valcartier Research Centre, in close collaboration with Canadian Armed Forces (CAF) end-users and offers advanced sensing and data processing capabilities. It uses sophisticated technology to detect, identify, and track objects of interest and make it easier for CAF operators to make sense of a high volume of data, and thus reduce their cognitive burden.
A Canadian Armed Forces member uses the JAWS system and improves the use of artificial intelligence in the field with a long-range binocular system known as the JIM Compact sensor.
Over several years of development, DRDC has created pipelines allowing JAWS’ computer vision AI models to be trained using a combination of real and synthetic imagery directly in the field. The CAPSTONE 4 event added to this AI training and tested its ability to identify in real-time objects of interest within the fields of view of its sensor systems. Operators in the field could then confirm the automated detections using the Android Tactical Assault Kit (ATAK) and the vetted data was then fed into the larger common operating picture. A common operating picture combines information from multiple sources to provide a continuously updated overview of a situation for command-and-control.
MIST
MIST was developed by TerraSense Analytics with funding and support from the Innovation for Defence Excellence and Security (IDEaS) program. MIST provides automated analysis of full-motion video to generate detection and localization of objects of interest. It can work from crewed and uncrewed aircraft systems or a ground station. MIST uses AI and deep learning to help operators make sense of data from multiple sensors in real time.
The Canadian Army worked with DRDC to test an advanced surveillance system known as MIST, which was funded by the Innovation for Defence Excellence and Security (IDEaS) program, during CAPSTONE4 in Fort Irwin, California.
MIST has the potential to detect, identify and track objects in the air, land, and maritime domains. MIST’s goal to fuse data from multiple sensors into one output for use by intelligence, surveillance and reconnaissance analysts is one of the reasons the IDEaS program has invested in a longer-term evaluation of the technology, known as a Test Drive.
One of the main benefits of CAPSTONE 4 for DRDC was the opportunity to experiment collaboratively alongside international partners.
“CAPSTONE 4 allowed us to experiment in an environment that mimics an operational theatre at a scale unlike anything that could be possible nationally,” says DRDC project manager Catherine Daigle.
The DRDC team of scientists, engineers, and technicians, as well as contractors, participated in CAPSTONE4 alongside approximately 60 members of the Canadian Army. Overall, approximately 4,000 contributors from France, Japan and from our Five Eyes partner nations: Australia, New Zealand, the United Kingdom, and the United States participated in this event.