November 29, 2010:
The U.S. Department of Defense is developing two new software systems that can take over the tedious job of watching video feeds from UAVs or security cameras. One is VIRAT (Video and Image Retrieval and Analysis Tool). The other is PERSEAS (Persistent Stare Exploitation and Analysis System). VIRAT is for video stakeouts of small areas (a building, or single doorway or window), while PERSEAS collects activities over a wider area and uses statistical analysis, and databases of enemy activity, to look for useful patterns. This is the continuation of a long term trend. For over a decade, the U.S. has been using software to help scour satellite and aerial recon pictures for useful information. There were simply not enough trained photo analysts to examine the growing number of photos generated in the course of intelligence work.
The boredom of watching video for hours is increasingly being alleviated by the use of pattern matching software, that can detect movement that is in need of human attention. Research has shown that people staring at live video feeds start losing their ability to concentrate on the images after about twenty minutes. This problem has been known for some time, and the military (not to mention civilian security firms) have been seeking a technological solution. It's actually not as bad with UAVs, because the picture constantly changes, but cameras that are staring at the same spot, can wear operators out real quick.
The basic tech solution is pattern analysis. Since the most common video is digital, it's possible to translate the video into numbers, and then analyze those numbers. Government security organizations have been doing this for some time, but after the fact. It's one thing to have a bunch of computers analyze satellite photos for a week, to see if there was anything useful there. It's quite another matter to do it in real time. But computers have gotten faster, cheaper and smaller in the last few years, and programmers have kept coming up with more efficient algorithms for analyzing the digital images. Commercial firms already have software on the market that will analyze, in real time, video, and alert a human operator if someone, or something (you are looking for) appears to be there.
The real time analysis software is rapidly evolving. You don't hear much about it, because if the enemy knows the details of how it works, they can develop moves that will deceive it (or, to be more accurate, make the pattern analysis less accurate.) Already, this software is being used as an adjunct to human observers, and gradually taking over. There will always be a human in the loop, to confirm what the software believes it has found.
The proliferation of video cameras on the battlefield (in UAVs, ground robots and for base security) has provided a huge library of images that show bad guys doing what bad guys do. This can range from moving around carrying weapons, to using those weapons, to the particular driving patterns of people up to no good. This is a unique resource, and the U.S. Department of Defense has put together a library of these images. This is similar to older still pictures libraries, which were eventually used by pattern recognition software to let machines examine the millions of images digital photo satellites began producing decades ago. The basic problem was that there were soon too many pictures for human analysts to examine. Computers had to do much of the work, or else most of the images would go unexamined. This technology was quickly adapted to the kind of combat encountered in Iraq and Afghanistan, and terrorist operations in general.