Sun Exclusive

NSA rejected system that sifted phone data legally

Dropping of privacy safeguards after 9/11, turf battles blamed

By Siobhan Gorman | Sun reporter

May 18, 2006

Article tools

E-mail

Share
- Digg
- Del.icio.us
- Facebook
- Furl
- Google
- Newsvine
- Reddit
- Spurl
- Yahoo

WASHINGTON - The National Security Agency developed a pilot program in the late 1990s that would have enabled it to gather and analyze huge amounts of communications data without running afoul of privacy laws. But after the Sept. 11 attacks, it shelved the project - not because it failed to work but because of bureaucratic infighting and a sudden White House expansion of the agency's surveillance powers, according to several intelligence officials.

The agency opted instead to adopt only one component of the program, which produced a far less capable and rigorous program. It remains the backbone of the NSA's warrantless surveillance efforts, tracking domestic and overseas communications from a vast databank of information, and monitoring selected calls.

Four intelligence officials knowledgeable about the program agreed to discuss it with The Sun only if granted anonymity because of the sensitive nature of the subject.

The program the NSA rejected, called ThinThread, was developed to handle greater volumes of information, partly in expectation of threats surrounding the millennium celebrations. Sources say it bundled four cutting-edge surveillance tools. ThinThread would have:

• Used more-sophisticated methods of sorting through massive phone and e-mail data to identify suspect communications.

• Identified U.S. phone numbers and other communications data and encrypted them to ensure caller privacy.

• Employed an automated auditing system to monitor how analysts handled the information, in order to prevent misuse and improve efficiency.

• Analyzed the data to identify relationships between callers and chronicle their contacts. Only when evidence of a potential threat had been developed would analysts be able to request decryption of the records.

An agency spokesman declined to discuss NSA operations.

"Given the nature of the work we do, it would be irresponsible to discuss actual or alleged operational issues as it would give those wishing to do harm to the U.S. insight and potentially place Americans in danger," said NSA spokesman Don Weber in a statement to The Sun. "However, it is important to note that NSA takes its legal responsibilities very seriously and operates within the law."

In what intelligence experts describe as rigorous testing of ThinThread in 1998, the project succeeded at each task with high marks. For example, its ability to sort through huge amounts of data to find threat-related communications far surpassed the existing system, sources said. It also was able to rapidly separate and encrypt U.S.-related communications to ensure privacy.

But the NSA, then headed by Air Force Gen. Michael V. Hayden, rejected both of those tools, as well as the feature that monitored potential abuse of the records. Only the data analysis facet of the program survived and became the basis for the warrantless surveillance program.

The decision, which one official attributed to "turf protection and empire building," has undermined the agency's ability to zero in on potential threats, sources say. In the aftermath of revelations about the agency's wide gathering of U.S. phone records, they add, ThinThread could have provided a simple solution to privacy concerns.

A better system

A number of independent studies, including a classified 2004 report from the Pentagon's inspector-general, in addition to the successful pilot tests, found that the program provided "superior processing, filtering and protection of U.S. citizens, and discovery of important and previously unknown targets," said an intelligence official familiar with the program who described the reports to The Sun. The Pentagon report concluded that ThinThread's ability to sort through data in 2001 was far superior to that of another NSA system in place in 2004, and that the program should be launched and enhanced.

Hayden, the president's nominee to lead the CIA, is to appear today before the Senate Select Committee on Intelligence and is expected to face tough questioning about the warrantless surveillance program, the collection of domestic phone records and other NSA programs.

While the furor over warrantless surveillance, particularly the collection of domestic phone records, has raised questions about the legality of the program, there has been little or no discussion about how it might be altered to eliminate such concerns.

ThinThread was designed to address two key challenges: The NSA had more information than it could digest, and, increasingly, its targets were in contact with people in the United States whose calls the agency was prohibited from monitoring.

With the explosion of digital communications, especially phone calls over the Internet and the use of devices such as BlackBerries, the NSA was struggling to sort key nuggets of information from the huge volume of data it took in.

By 1999, as some NSA officials grew increasingly concerned about millennium-related security, ThinThread seemed in position to become an important tool with which the NSA could prevent terrorist attacks. But it was never launched. Neither was it put into effect after the attacks in 2001. Despite its success in tests, ThinThread's information-sorting system was viewed by some in the agency as a competitor to Trailblazer, a $1.2 billion program that was being developed with similar goals. The NSA was committed to Trailblazer, which later ran into trouble and has been essentially abandoned.

Would you recommend this?

Rate it: