October 25,2008:  The proliferation of video cameras on the battlefield (in UAVs, ground robots and for base security) has provided a huge library of images that show bad guys doing what bad guys do. This can range from moving around carrying weapons, to using those weapons, to the particular driving patterns of people up to no good. This is a unique resource, and the U.S. Department of Defense is putting together a library of these images. This is similar to older still pictures libraries, which were eventually used by pattern recognition software to let machines examine the millions of images digital photo satellites began producing decades ago. The basic problem was that there were quickly too many pictures for human analysts to examine. Computers had to do much of the work, or else most of the images would go unexamined. This technology was quickly adapted to the kind of combat encountered in Iraq and Afghanistan, and terrorist operations in general.

Now we have all those cameras producing millions of hours of video a year, which has to be examined by humans, to find whatever useful may be there. While this video is usually examined, on-the-spot, by UAV sensor operators, or security troops guarding bases, a lot of it has more to reveal. The people first viewing don't always detect everything useful.

Research has shown that people staring at live video feeds start losing their ability to concentrate on the images after about twenty minutes. This problem has been known for some time, and the military (not to mention civilian security firms) have been seeking a technological solution. It's actually not as bad with UAVs, because the picture constantly changes, but cameras that are fixed can wear operators out real quick.

The basic tech solution is pattern analysis. Since the most common video is digital, it's possible to translate the video into numbers, and then analyze those numbers. Government security organizations have been doing this for some time, but after the fact. It's one thing to have a bunch of computers analyze satellite photos for a week, to see if there was anything useful there. It's quite another matter to do it in real time. But computers have gotten faster, cheaper and smaller in the last few years, and programmers have kept coming up with more efficient routines for analyzing the digital images. Commercial firms already have software on the market that will analyze, in real time, video, and alert a human operator if someone, or something (you are looking for) appears to be there.

While some military analysis does not have to be real time (like the system used in Iraq to compare today's and yesterdays photos of a road to see if a bomb may have been planted), the most common need is for real time analysis. Several times a year now, a new software package shows up that does that, or tries to. These systems are getting better. Many can definitely beat your average human observer over time (several hours of viewing video). The real time analysis software is rapidly evolving. You don't hear much about it, because if the enemy knows the details of how it works, they can develop moves that will deceive it (or, to be more accurate, make the pattern analysis less accurate.) Already, this software is being used as an adjunct to human observers, and gradually taking over. There will always be a human in the loop, to confirm what the software believes it has found.

Another area of active research is night vision goggles that think. This will be achieved by sufficiently miniaturizing the computer power needed. This will first be seen with heat imaging devices, commonly used by armored vehicles, combat robots and weapons sights, with this kind of software. Troops equipped with this equipment won't be immune to night attack, but the attackers will have to move as if it was daylight, slowly and from behind whatever cover is available. Moving swiftly under cover of night will no longer be possible if the other side has night vision devices equipped with pattern recognition software.

But the big breakthrough, which may already have been achieved, is a predictive analysis system that can quickly examine thousands of hours of video from a specific area, and calculate the probability that certain vehicles, or individuals, down there, are up to no good. This works if you have lots of examples of people you know are up to no good. The predictive analysis looks for enough indicators to make it likely that something bad is going down. When done in real time, the analysis software can instantly alert that something bad is about to happen at a specific location. If nothing does happen, that is saved and added to the library of experience the analysis software uses to make predictions. In effect, the predictive analysis software gets smarter the more often it is used. And the library of combat zone video images grows larger as well, making it possible for the analysis software to sniff more behavior patterns that predict bad actions.