May 5, 2020: A team of American (Microsoft) and Chinese (Huazhong University) researchers recently released FairMOT (Fair Multi-Object Tracking), an open-source (free for anyone to use) AI Object Detector system. What this software system does is speed up (30 frames a second) the identification of objects on video that more quickly and precisely follows objects in the video. Civilian uses include security and monitoring hospitals or elder-care facilities to detect potential problems. This also has military uses, the major one being able to make better use of the huge American databases of UAV and ground-based security videos. Currently, all this video overwhelms the ability of troops to extract useful information. Since the 1990s there have been numerous projects undertaken to make better sense of all this data. The proliferation of video cameras on the battlefield, especially UAVs (as well as ground and underwater vehicles) for surveillance or base security has created a huge library of images that show bad guys doing what bad guys do and what they look like while doing it. This can range from moving around carrying weapons, to using those weapons, to the particular driving patterns of people up to no good. These images are a unique resource, and the U.S. is putting together a library of these images. This is similar to older still pictures libraries, which were eventually used by pattern recognition software to let machines examine the millions of images digital photo satellites began producing decades ago. The basic problem was that very soon there were too many pictures for human analysts to examine. Computers had to do much of the work, or else most of the images would go unexamined. This technology was quickly adapted to the kind of combat encountered in Iraq and Afghanistan, and terrorist operations in general.

There are other benefits of FairMOT. Research has shown that people staring at live video feeds start losing their ability to concentrate on the images after about twenty minutes. This problem has been known for some time, and the military (not to mention civilian security firms) have been seeking a technological solution. It's actually not as bad with UAVs, because the picture constantly changes, but cameras that are fixed can wear operators out real quick.

The basic tech solution is pattern analysis. Since the most common video is digital, it's possible to translate the video into numbers, and then analyze those numbers. Government security organizations have been doing this for some time but after the fact. It's one thing to have a bunch of computers analyzes satellite photos for a week, to see if there was anything useful there. It's quite another matter to do it in real-time. But computers have gotten faster, cheaper and smaller in the last few years, and programmers have kept coming up with more efficient routines for analyzing the digital images. Commercial firms already have software on the market that will analyze, in real-time, video, and alert a human operator if someone or something (you are looking for) appears to be there. The AI object detector takes advantage of faster computers and more powerful video cards to do what it does.

While some military analysis does not have to be real-time, like the system used in Iraq and Afghanistan to compare today's and yesterday’s photos of a road to see if a bomb may have been planted, the most common need is for real-time analysis. Several times a year now, a new software package shows up that does that or tries to. These systems are getting better. Many can definitely beat your average human observer over time (several hours of viewing video). The real-time analysis software is rapidly evolving. You don't hear much about it, because if the enemy knows the details of how it works, they can develop moves that will deceive it or, to be more accurate, make the pattern analysis less accurate. That is changing as the need for commercial AI object detector appears. For a decade this software has been used as an adjunct to human observers, and gradually taking over. There will always be a human in the loop, to confirm what the software believes it has found.

But the big breakthrough, which may already have been achieved, is a predictive analytics system that can quickly examine thousands of hours of video from a specific area and calculate the probability that certain vehicles, or individuals, down there, are up to no good, or will simply be traveling down a certain road. This works if you have lots of examples of people you know, and are trying to find. The predictive analysis looks for enough indicators to make it likely that something specific is going to happen. When done in real-time, the analysis software can instantly alert users that something specific is about to happen at a specific location. If nothing does happen, that is saved and added to the library of experience the analysis software uses to make predictions. In effect, the predictive analysis software gets smarter the more often it is used. And the library of combat zone video images grows larger as well, making it possible for the analysis software to sniff more behavior patterns that predict bad actions.