It's a standing joke that "military intelligence" is an oxymoron, but in fact the intelligence service is a very important arm of any country's defenses. They only seem like clowns to the public because most of us only hear about their failures. If you read up on a few of their successes, they seem a lot less amateurish.
In wartime, of course, the job of an intelligence service is to give its side as much information about what the other side is doing as possible. In peacetime, the intelligence service's job is similar: to make sure its side knows what the other side is doing so the other side doesn't do something unexpected. I have a lot of books about intelligence operations in the Second World War; they're collected on my WW2 Covert Operations page. I also have a few books about peacetime intelligence.
There are a lot of ways for a good intelligence-gathering service to do its job. These days, computers provide some of the easiest ways of getting information. But like everything in the intelligence game, computers are a double-edged sword: while they provide unparalleled opportunities for spies, they also offer unprecedented ways of tracking and catching spies, as illustrated in this book:
THE CUCKOO'S EGG: Tracking a Spy Through the Maze of Computer
Espionage
Stoll, Clifford
c.1990, Pocket Books
ISBN: 0-671-72688-9
In 1989, Berkeley astronomer Clifford Stoll was helping out in the Berkeley computer center. One day he was assigned to figure out a minor glitch in some computer-time accounts. He soon traced the glitch to a hacker who had found a way into Berkeley's computer system. Stoll set up a way to follow the hacker's actitivies, and watched in disbelief as the hacker penetrated numerous allegedly-secure computers -- even military computers! Stoll made catching this hacker a private crusade. The hunt led him into some truly rarefied heights: visits to the CIA and FBI, and cooperation with numerous agencies in the US and Europe. Finally the hacker was found and arrested -- and turned out to be a paid agent of the Soviet Union, using his hacking skills to steal valuable computer technology and data.
One case in which the American intelligence community really did come off second best was the Walker family spy ring. This was revealed in 1985, when John Walker, his brother Arthur, John's son Michael, and another man named Jerry Whitworth were arrested for espionage. It was found that they'd been operating and passing cryptographic data to the KGB, the Soviet Union's main intelligence agency, for at least seventeen years. It was also found that American law enforcement agencies had had several chances to break Walker's ring, and each time had overlooked or ignored the critical information. I have a couple of books about the Walker spy ring:
BREAKING THE RING
Barron, John
c.1987, Avon
ISBN: 0-380-70520-6
This is the story of how the Walker spy ring was broken. It starts by telling how one of Walker's agents, Jerry Whitworth, attempted to sell Walker to the FBI, but got cold feet and backed out. It then reviews the evidence that should have told the American counter-intelligence community that something was wrong, that the Soviet Union had access to American naval communications. Next it tells how the FBI finally followed up on a tip from Walker's wife, and eventually trapped and arrested him in the process of making a delivery. Finally, it tells how Walker and his agents were prosecuted, and describes the damage they did to American national security.
FAMILY OF SPIES: Inside the Walker Family Spy Ring
Earley, Pete
c.1988, Bantam Books
ISBN: 0-553-28222-0
A different, more "insider" look at the Walker spy ring.