October 28,2008:  The war in Iraq and Afghanistan has led military intelligence and psychological warfare forces to look more towards civilian marketing techniques for new tools to deal with populations that produce terrorists, and pro-terrorist civilians. The basic idea is that the work done on analyzing and manipulating opinions in consumer markets over the last century provides a body of knowledge that would make it easier to understand, and work with, populations in combat zones. A lot of this interest came from officers who had studied the subject in college, and become interested in military applications.

Naturally, there is a difference between working with foreign, often hostile, populations, and well studied, very familiar populations at peace. But by working through this problem, the military has found ways to apply marketing analysis and manipulation techniques to battleground areas. The details of this are largely classified, not just because this sort of thing can attract a lot of hostile media attention, but also because the techniques developed become less effective is they are widely known.

Islamic terrorists are well aware of the need to understand and communicate effectively with the Moslem populations they try to survive in. They have the advantage of being from that culture. But they also lack the technical tools (the math based models, analytical tools, databases and general knowledge of marketing and communications). While the terrorists have quickly, and effectively, adopted the Internet as a communications tool, so have counter-terrorism organizations.

The American military first got into the use of databases and analysis when they, early on in the war on terror, adopted many practices that major police departments have long taken for granted. One of the more useful techniques is biometrics. That is, every time the troops encounter a "person of interest", they don't just take their name and address, they also use portable electronic tools to take fingerprints, a retinal scan and photos. All this is stored in a database, which eventually contained hundreds of thousands of records for Iraqis, Afghans, and other "persons of interest". The fingerprints were particularly useful, because when they are stored electronically, you can search and find out immediately if the print you have just lifted from somewhere else, like the fragment of a car bomb, is in there or not. The digital photos, from several angles, are also useful, because these pictures are run through software that creates a numeric ID that can be used by security cameras to look for some one specific, or for finding someone from a witness description. Other nations are digitizing their mug shots, and this enables these people to be quickly checked against those in the American database.

Expanding this data to the entire population, or large segments, of it, happened early on. Saddam Hussein, for example, was captured by collecting lots of information on people he knew, or were related by family or tribal tries. The military database on biometric data became the basis for other data collections that included family members and their connections. The troops quickly discovered that family and tribal politics often played a big role in who became a terrorists, and who was against them. This played a role in the "tribal strategy" of the Surge Offensive (which convinced most of the Sunni tribes to turn against the terrorists, as it was mainly a matter of knowing who to make the pitch to.)

The psychological warfare people used the "marketing" data from the military databases to come up with pitches that could be made to various factions of the population. But often these marketing insights just provided more information for commanders to consider as they planned where to make the next series of raids, or which tribal leaders to sit down and talk with.

This affection for collecting data is nothing new for the U.S. Army. For decades, U.S. troops have regularly collected huge amounts of information from accidents, training and combat incidents. So now, it's no surprise that forensics teams examine each bombing (car or roadside), to see if they can get fingerprints. Often bomb makers are found this way, because raids frequently encounter suspicious characters, but no evidence to justify arresting them. It only takes about two minutes per subject to take the biometric data, so any suspicious characters are added to the database. Now, after several years of this, raiding parties know to grab any guy who seems to panic at the sight of the biometrics equipment coming out. The terrorists know that biometrics is bad news for them, and they fear it.

Naturally, the FBI and U.S. intelligence agencies were data mining this database, and running cross checks with other databases of people with known, or suspected, criminal backgrounds. As was long suspected, many Islamic terrorists had spent time in Western nations, where they participated in criminal scams, and were often arrested. Same pattern in Moslem nations, where there was a strong linkage between criminal behavior in general, and association with Islamic terrorist organizations. Some of this is because the same kind of personality is attracted to both lifestyles, but it's also long been a common practice for Islamic terrorists to raise money, and gain access to weapons and explosives, via criminal activities and connections with other criminals.