From the Air Force to the Treasury Department, government agencies are using either "Web bugs" or permanent cookies to monitor their visitors' behavior, even though federal law restricts the practice.

Some departments changed their practices this week after being contacted by CNET News.com. The Pentagon said it wasn't aware that its popular Defenselink.mil portal tracked visitors--in violation of a privacy notice--and said it would fix the problem. So did the Defense Threat Reduction Agency and the U.S. Chemical Safety and Hazard Investigation Board.

"We were not aware of the cookies set to expire in 2016," a Pentagon representative said Wednesday. "All of the cookies we had set with WebTrends were to be strictly (temporary) cookies, and we are taking immediate action." WebTrends is a commercial Web-monitoring service.

The practice of tracking Web visitors came under fire last week when the National Security Agency was found to use permanent cookies to monitor visitors, a practice it halted after inquiries from the Associated Press. The White House also was criticized last week for employing WebTrends' tracking mechanism that used a tiny GIF image.

A 2003 government directive says that, in general, "agencies are prohibited from using" Web bugs or cookies to track Web visitors. Both techniques are ways to identify repeat visitors and, depending on the configuration, can be used to track browsing behavior across nongovernment Web sites too.

"It's evidence that privacy is not being taken seriously," said Peter Swire, a law professor at Ohio State University, referring to the dozens of agencies tracking visitors. "The guidance is very clear." While working in the Clinton administration in 2000, Swire helped to craft an earlier Web tracking policy.

To detect which agencies engage in electronic tracking, CNET News.com wrote a computer program that connected to every agency listed in the official U.S. Government Manual, and then evaluated what monitoring techniques were used. The expiration dates of the cookies detected ranged from 2006 to 2038, with most of them marked as valid for at least a decade or two.

Many agencies appeared to have no inkling that their Web sites were configured to record the activities of users. "When the agency set up ColdFusion on our Web server, we set the software to its default value," said William Alberque, a spokesman for the Defense Threat Reduction Agency. "The default value, as you saw, creates individual session cookies that can last on your computer for either 30 years or until you delete them." (ColdFusion is Adobe Systems' Web development software.)

While the practice of setting permanent cookies is generally prohibited, it's usually not clear how they're being used. In the worst case, they could be used to invade privacy by correlating one person's visits to thousands of Web sites. They also can be as innocuous as permitting someone to set a Web site's default language.

Not all monitoring of Web visitors is prohibited. The 2003 directive provides an exception for federal agencies that have a "compelling need," clearly disclose the tracking and have approval from the agency head. In addition, the directive does not apply to state government Web sites, court Web sites or sites created by members of Congress.

The perils of third-party cookies

Probably the most intrusive type of tracking comes from third-party cookies set by commercial vendors. Such cookies permit correlation of visits to thousands of Web sites. A visitor to the Pentagon's Web site could be identified as the same person who stopped by Hilton.com and HRBlock.com--because both of those companies are WebTrends customers.

For its part, WebTrends says it does not correlate that information. "There are companies that tried to do that in the past and got a lot of bad public exposure," said Brent Hieggelke, WebTrends' vice president of corporate marketing.

"We do not track cross-site traffic," Hieggelke said. "We do not offer any services that let you understand cross-domain traffic at unrelated sites at all."

Privacy advocates tend to be leery of such third-party cookies, however, warning that a change in company management or ownership could result in a policy shift, or that a security breach would expose Web browsing habits.

"If WebTrends has the ability to link the White House visit to the commercial site visit, then that does look like persistent tracking," said Swire, the Ohio law professor. "It would be useful to have a third-party audit of that."

Statcounter.com is another Web-statistics program, used by the Commerce Department and the Energy Department, which also sets third-party cookies.

The Dublin, Ireland-based company says it does not correlate information from multiple Internet sites. "We do not sell any information to third parties," said its U.S. representative. "All we're interested in gathering is information that can tell (a Webmaster) what area the visitor comes from, what they looked at, what they went back to, data that shows how their sites are used."

During the Clinton administration, the White House's Office of Management and Budget published initial guidelines (click here for PDF) for federal Web sites in June 1999. That 10-page document gave federal agencies three months to post "clearly labeled and easily accessed" privacy policies on their sites and suggested model language.

Then came a public flap over the tracking technologies employed by the White House's antidrug site Freevibe.com. Shortly afterward, the White House published a directive restricting agencies from using any sort of "cookies" or other "automatic means of collecting information" at their sites except in narrow circumstances. The latest, 2003, directive continued the restriction on permanent (sometimes called persistent) cookies but permitted temporary ones that last only as long as the browser window is open.

Failure to follow the rules has plagued government agencies before. In 2001, the Defense Department's Inspector General reviewed the agency's 400 sites and found "persistent" cookies on 128 of them. The Central Intelligence Agency admitted in 2002 that it had also been using the proscribed cookies without proper clearance, and it stripped them from its sites.

The level of compliance with the rules appears to have changed little since a 2000 General Accounting Office survey (click here for PDF), which revealed that at least a dozen agencies were still using cookies in apparent violation of the rules.

Persistent by default

Many of the cookies appearing on the errant Web sites were generated by ColdFusion, the popular Web authoring tool. When the software creates creates certain types of cookies, it automatically assigns them a default "persistent" setting, which sets them to expire about 30 years in the future, said senior project manager Tim Buntel.

ColdFusion's software architects encourage Web developers to use an application that allows them to manage and make changes to the cookie settings as they see fit, Buntel said, adding that "any ColdFusion application can be built completely without any cookie use."

Representatives at several agencies said they were astonished to see cookies on their Web sites, and they blamed their Web designer's lack of understanding of ColdFusion's default settings.

The Defense Threat Reduction Agency immediately altered the settings on discovering that its ColdFusion developers had neglected to tweak the defaults. "We never have kept a database of any such information," said spokesman William Alberque.

"Frankly, I don't think anybody here even realized they existed, but now they do, and we'll follow up on it," said Daniel Horowitz, a spokesman for the U.S. Chemical Safety and Hazard Investigation Board.

One Smithsonian Institution Web staffer, who initially denied the existence of persistent cookies detected by CNET News.com on the National Air and Space Museum's site, said that ColdFusion settings were probably to blame. "Regardless, I can assure you that we are not currently using or distributing cookie information," the representative said in a statement sent to CNET News.com.

A few others, including the Federal Reserve Bank System and the U.S. Institute of Peace, said they're independent agencies that are not bound by the 2003 directive from the Office of Management and Budget (OMB). "We are not a government agency," said Calvin Mitchell, senior vice president at the Federal Reserve Bank of New York. "We try to fulfill the spirit of certain government regulations as we can, but we're not obliged to follow those."

A White House official suggested a different interpretation. "When it comes to federal government Web sites, the policy is clear, and so anything that ends in a .mil or a .gov would fall underneath the federal policy as outlined in the OMB guidance," said David Almacy, the White House's Internet director.

Only one federal agency contacted this week appeared to comply fully with the directive. The National Institute of Dental and Craniofacial research says it received the necessary permission in January 2005 to enable cookies on its Web site for a survey. The cookies, which expire in one month, are used to avoid asking the same people to complete the survey.

The White House says that because it only uses a 1 pixel-by-1 pixel image that loads from WebTrends' site, it complies with the 2003 directive from the Office of Management and Budget. "There are no cookies being placed either on the Web site, from the White House or from WebTrends," Almacy said. "No personal information was gleaned, no cookies were being used, but OMB guidance is pretty clear. The White House Web site is and always has been in compliance with OMB guidance."