Drones (also known as pilotless aircraft, UAVs, RPVs and Bugs) first showed up before aircraft with pilots in them. In 1887, Douglas Archibald rigged a camera to a large kite in order to take aerial photos. An American, William Eddy, adopted the same technique during the Spanish-American war, successfully taking hundreds of reconnaissance photos. Once piloted aircraft came along, research on drones declined, except to adapt older aircraft as pilotless aerial targets. Why? Well, for one thing, the instrumentation needed for a drone that could last as long as a piloted aircraft took a long time to arrive. Even during the Vietnam war, where the Firefly drone (originally a target drone) was widely used, 16 percent of their sorties ended with a botched landing, and a destroyed drone. Still, the Firefly flew 3,435 sorties, mainly over areas where it would be difficult to recover a downed pilot. Even so, by 1972, various types of drones flying in Southeast Asia were coming back 90 percent of the time. But that's more than ten times the loss rate for piloted aircraft.
With no war to spur development of drones, the U.S. lost interest once more. But in Israel, work proceeded. And drones figured prominently in a spectacular Israeli aerial victory over the Bekaa Valley in 1982. Using drones in cooperation with their warplanes, Israel was able to shut down the Syrian air force (and destroy 86 aircraft) in a few days. Israel pioneered the use of drones for real time surveillance, electronic warfare and decoys. But in the U.S., there was either no interest, or some inexplicably botched UAV development projects. Americans wondered how the Israelis did it while the Department of Defense continued to screw up attempts to create useful drones.
Finally, with some urging (and ridicule) from Congress, the Department of Defense began to buy drones from Israel. The Navy bought the Israeli Pioneer UAV, which is still in use. Many of these Israeli drones (plus some newly developed U.S. ones) were used in the 1991 Gulf War. There weren't that many of them, but the army and marines noted that the air force and navy were stingy with answering requests for recon missions. This made the ground troops aware of how they could create their own air force of drones. All of a sudden, the army and marines were back in the drone development business. This time they were serious, and a number of successful UAVs were developed. The Predator entered service in 1995.
When the 1999 Kosovo war came along, various NATO nations had 97 UAVs operating in the area. While 27 crashed (mostly due to accident, although the UAVs were frequently shot at), UAVs were considered a great success. The military began to get UAV fever, except for the air forces. The U.S. Air Force had actually developed several highly successful UAVs (cruise missiles) in the 1980s, but these were not considered UAVs. To the air force, these weapons were enhanced missiles, and all went on one way missions. The air force had a point, as recovering UAVs had always been a major problem. As any pilot can tell you, taking off and flying is easy, but landing is a bitch.
The CIA also got into the act, obtaining some Predator UAVs for use in Central Asia (and elsewhere) before September 11, 2001. The CIA had no hang-ups (like the U.S. Air Force) about arming their drones. They figured out how to equip a Predator with two (hundred pound) Hellfire missiles on a Predator. If the UAV spotted a likely target, it could launch missiles right away. Now this was no different from the old tactic of turning warplanes loose against "targets of opportunity." But UAVs could circle over a piece of land for hours, something most modern aircraft could not do. Most enemy troops expect jet fighter bombers to come by, make a few passes, and depart. But the CIA Predators could snoop around for a while, checking out suspected targets, and then fire on likely targets. Since the Hellfire was a highly accurate missile, they would usually hit what they aimed at.
Finally, in the late 1990s, the U.S. Air Force got enthusiastic about UAVs. The long range Global Hawk UAV was a great success, and was first used in Afghanistan (even before it was finished with all it's development tests). The air force is also developing air-to-air combat UAVs. This should not be too difficult, as tests with prototypes in the 1970s showed that a drone could outfly the best pilots. The U.S. armed forces now has nearly a thousand UAVs in service or on order and over a hundred UAV designs are in various stages of development.
Why the change of mind? Mostly it's about technology. The instrumentation that enables a pilot on the ground to control a UAV, or software that allows a UAV to fly missions successfully on its own, is now a reality. This microcomputer based stuff is a lot cheaper and more reliable than it was just ten years ago. For many decades, the concept of UAVs was good, but the means were not. Now there is affordable technology to make the UAVs work. This is a typical pattern with many items. But for UAVs, it meant an obviously good idea stayed just out of reach for many decades. There was also a problem air forces, which are run by pilots, not being eager to replace pilots with UAVs. But with UAV technology so cheap and widespread, the air forces have to develop UAVs, or see their piloted aircraft blown from the sky by someone else's UAVs.