See No Evil

The senior lawyer for the National Security Agency stated on last week that US technology companies were fully aware of the surveillance agency’s widespread collection of data.

Rajesh De, the NSA general counsel, said all communications content and associated metadata harvested by the NSA under a 2008 surveillance law occurred with the knowledge of the companies – both for the internet collection program known as Prism and for the so-called “upstream” collection of communications moving across the internet.

Asked during a hearing of the US government’s institutional privacy watchdog if collection under the law, known as Section 702 or the Fisa Amendments Act, occurred with the “full knowledge and assistance of any company from which information is obtained,” De replied: “Yes.”

When the Guardian and the Washington Post broke the Prism story in June, thanks to documents leaked by whistleblower Edward Snowden, nearly all the companies listed as participating in the program – Yahoo, Apple, Google, Microsoft, Facebook and AOL – claimed they did not know about a surveillance practice described as giving NSA vast access to their customers’ data. Some, like Apple, said they had “never heard” the term Prism.

De explained: “Prism was an internal government term that as the result of leaks became the public term,” De said. “Collection under this program was a compulsory legal process, that any recipient company would receive.”

After the hearing, De added that service providers also know and receive legal compulsions surrounding NSA’s harvesting of communications data not from companies but directly in transit across the internet under 702 authority.

The disclosure of Prism resulted in a cataclysm in technology circles with tech giants launching extensive PR campaigns to assure their customers of data security.

These tech giants also wanted to demonstrate to the public that they were successfully pressing the Obama administration to allow them greater leeway to disclose the volume and type of data requests served to them by the government.

Last week, it is rumored that Facebook founder Mark Zuckerberg said he had called US president Barack Obama to voice concern about “the damage the government is creating for all our future.” 

We All Spy

WASHINGTON (NYTimes)— American officials have long considered Huawei, the Chinese telecommunications giant, a security threat, blocking it from business deals in the United States for fear that the company would create “back doors” in its equipment that could allow the Chinese military or Beijing-backed hackers to steal corporate and government secrets.

But even as the United States made a public case about the dangers of buying from Huawei, classified documents show that the National Security Agency was creating its own back doors — directly into Huawei’s networks.

The agency pried its way into the servers in Huawei’s sealed headquarters in Shenzhen, China’s industrial heart, according to N.S.A. documents provided by the former contractor Edward J. Snowden. It obtained information about the workings of the giant routers and complex digital switches that Huawei boasts connect a third of the world’s population, and monitored communications of the company’s top executives.

One of the goals of the operation, code-named “Shotgiant,” was to find any links between Huawai and the People’s Liberation Army, one 2010 document made clear. But the plans went further: to exploit Huawai’s technology so that when the company sold equipment to other countries — including both allies and nations that avoid buying American products — the N.S.A. could roam through their computer and telephone networks to conduct surveillance and, if ordered by the president, offensive cyberoperations.

“Many of our targets communicate over Huawei-produced products,” the N.S.A. document said. “We want to make sure that we know how to exploit these products,” it added, to “gain access to networks of interest” around the world.

The documents were disclosed by The New York Times and Der Spiegel, and are also part of a book by Der Spiegel, “The N.S.A. Complex.” The documents, as well as interviews with intelligence officials, offer new insights into the United States’ escalating digital cold war with Beijing. While President Obama and China’s president, Xi Jinping, have begun talks about limiting the cyber conflict, it appears to be intensifying.  To read more click here.

The N.S.A., for example, is tracking more than 20 Chinese hacking groups — more than half of them Chinese Army and Navy units — as they break into the networks of the United States government, companies including Google, and drone and nuclear-weapon part makers, according to a half-dozen current and former American officials.

Brazilian Wisdom

Brazil is considering ways to make local use of the internet less dependent on US-based services, following leaks about Washington's cyberspy operations.

The South American nation has suggested forcing internet firms to open data centers in Brazil, which would be used to store locally generated material.

It is also pursuing a plan to build a new internet cable.  The project would offer a way for data to bypass the US.

Brazil's President, Dilma Rousseff, has postponed a state visit to Washington after allegations that the US National Security Agency (NSA) had targeted her emails and phone calls.

It has also been alleged that the NSA hacked state-run oil company Petrobras and intercepted billions of emails and calls to Brazilians.

US Secretary of State John Kerry has previously defended the NSA's actions, saying they were necessary to combat terrorism.

"Brazil and other countries will understand exactly what we are doing, why and how - and we will work together to make sure that whatever is done is done in a way that respects our friends and our partners," he said last month on a visit to the country.

Brazil's IT policy secretary Virgilio Almeida has suggested that internet firms would have to operate data centers in the country, which would make them subject to local privacy laws.

Her decision to cancel (or officially, to postpone) the Washington visit will be seized upon by some as an act of petty nationalism.

Some Brazilian business leaders, worried by the precarious economic climate, will question the wisdom of antagonizing such an important business.

But the political pressure was greater still. There was fury in Brazil, not only at the revelation that the president's own conversations and communications may have been spied upon by the NSA but that US interests were allegedly involved in blatant economic espionage against major Brazilian interests, including Petrobras.

Dilma Rousseff will have been aware of the feelings of ordinary Brazilians had her Washington trip gone ahead.

The perception here in Brazil is that the Obama administration has yet to give an adequate response or an apology.

In addition, he said, the government might move to ensure that its own data about tax information and other sensitive subjects would be stored locally rather than in the cloud.

Last week a Brazilian official specifically named Facebook, Google and Microsoft as examples of companies that would have to change their practices, according to a report by the Reuters news agency.

The three companies are among those that have acknowledged handing over data about "national security matters" after legally binding requests from the US authorities.  Read more:

Confusion Opens the Door to Enlightenment

The U.S. government's efforts to determine which highly classified materials leaker Edward Snowden took from the National Security Agency have been frustrated by Snowden's sophisticated efforts to cover his digital trail by deleting or bypassing electronic logs, government officials told The Associated Press. Such logs would have showed what information Snowden viewed or downloaded.

The government's forensic investigation is wrestling with Snowden's apparent ability to defeat safeguards established to monitor and deter people looking at information without proper permission, said the officials, who spoke on condition of anonymity because they weren't authorized to discuss the sensitive developments publicly.

The disclosure undermines the Obama administration's assurances to Congress and the public that the NSA surveillance programs can't be abused because its spying systems are so aggressively monitored and audited for oversight purposes: If Snowden could defeat the NSA's own tripwires and internal burglar alarms, how many other employees or contractors could do the same?

In July, nearly two months after Snowden's earliest disclosures, NSA Director Keith Alexander declined to say whether he had a good idea of what Snowden had downloaded or how many NSA files Snowden had taken with him, noting an ongoing criminal investigation.

NSA spokeswoman Vanee Vines told the AP that Alexander "had a sense of what documents and information had been taken," but "he did not say the comprehensive investigation had been completed." Vines would not say whether Snowden had found a way to view and download the documents he took without the NSA knowing.

In defending the NSA surveillance programs that Snowden revealed, Deputy Attorney General James Cole told Congress last month that the administration effectively monitors the activities of employees using them.

"This program goes under careful audit," Cole said. "Everything that is done under it is documented and reviewed before the decision is made and reviewed again after these decisions are made to make sure that nobody has done the things that you're concerned about happening."

The disclosure of Snowden's hacking prowess inside the NSA also could dramatically increase the perceived value of his knowledge to foreign governments, which would presumably be eager to learn any counter-detection techniques that could be exploited against U.S. government networks.  Read more:

NSA Data Lab

The National Security Agency is funding a “top secret” $60 million dollar data analysis lab at North Carolina State University which will scrutinize information collected from private emails, phone calls and Google searches.

“The Laboratory for Analytic Sciences will be launched in a Centennial Campus building that will be renovated with money from the federal agency, but details about the facility are top secret. Those who work in the lab will be required to have security clearance from the U.S. government,” reports the News & Observer.

The project was initially supposed to be revealed in June, but the scandal surrounding the NSA’s PRISM surveillance program prompted the university to delay the announcement, with faculty staff citing, “that bit out of The Guardian (newspaper) on NSA collecting phone records of Verizon customers.”

According to NCSU Chancellor Randy Woodson, the program will revolve around “making sense out of the deluge of data that we’re all swimming in every day,” although the university denies that it will be involved in “mass surveillance”.

However, according to an Associated Press report, the data lab will analyze information collected by the NSA’s new $2 billion dollar data center in Bluffdale, Utah, which is set to collect ”complete contents of private emails, cell phone calls, and Google searches, as well as all sorts of personal data trails—parking receipts, travel itineraries, bookstore purchases, and other digital “pocket litter.”

According to the AP report, the new data lab will help perfect technology that will “analyze that data for patterns identifying terrorists and other security threats.”

The announcement of the new data center coincides with a Washington Post report which reveals that the NSA “has broken privacy rules or overstepped its legal authority thousands of times each year since Congress granted the agency broad new powers in 2008.”

During a press conference last week, President Obama claimed that the agency was not “actually abusing these programs and, you know, listening in on people’s phone calls or inappropriately reading people’s e-mails.”

Data mining (the analysis step of the "Knowledge Discovery in Databases" process, or KDD), an interdisciplinary subfield of computer science, is the computational process of discovering patterns in large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics, and database systems. 

The overall goal of the data mining process is to extract information from a data set and transform it into an understandable structure for further use. Aside from the raw analysis step, it involves database and data management aspects, data pre-processing, model and inference considerations, interestingness metrics, complexity considerations, post-processing of discovered structures, visualization, and online updating.

The term is a buzzword, and is frequently misused to mean any form of large-scale data or information processing (collection, extraction, warehousing, analysis, and statistics) but is also generalized to any kind of computer decision support system, including artificial intelligence, machine learning, and business intelligence. In the proper use of the word, the key term is discovery[citation needed], commonly defined as "detecting something new". 

Even the popular book "Data mining: Practical machine learning tools and techniques with Java" (which covers mostly machine learning material) was originally to be named just "Practical machine learning", and the term "data mining" was only added for marketing reasons. Often the more general terms "(large scale) data analysis", or "analytics" – or when referring to actual methods, artificial intelligence and machine learning.

Big Brother

is

watching us...

Spying on the Spys

Edward Snowden

A week after Edward Snowden's leaks about National Security Agency surveillance and data-gathering were first reported, and four days after he revealed himself as the leaker, the news media is figuring out how the 29-year-old IT systems administrator managed his potentially huge data heist.

If you're concerned about national security, the new revelations will probably dismay you; if you appreciate leaking of government secrets, Snowden's technique is likely encouraging: Theft by thumb drive.

The NSA and other spy and military agencies have long known the dangers of the innocent-seeming portable USB flash drive. In October 2008, the NSA discovered that a thumb drive loaded with malware had infected the military's secure internal network. The Pentagon then (at least temporarily) banned the use of thumb drives — NSA commanders even reportedly ordered USB ports filled in with liquid cement.

But "of course, there are always exceptions," especially for system administrators, a former NSA official tells the Los Angeles Times. "There are people who need to use a thumb drive and they have special permission. But when you use one, people always look at you funny."

That doesn't appear to have fazed Snowden. Not only do investigators know he pilfered the top secret files on a thumb drive, they "know how many documents he downloaded and what server he took them from," a U.S. official tells the Los Angeles Times. They don't know how he accessed those files, but as a system administrator, Snowden had broad access to key parts of the NSA network — and, says Ken Dilanian at the Los Angeles Times, "presumably a keen understanding of how those networks are monitored for unauthorized downloads."

In any case, Dilanian says, "confirmation of a thumb drive solved one of the central mysteries in the case: How Snowden, who worked for contracting giant Booz Allen Hamilton, physically removed classified material from a spy agency famous for strict security and ultra-secrecy."

Didn't Snowden's behavior, or his decision to take unpaid leave just a month after starting his job in Hawaii, arouse any suspicions? Sort of, says Mark Hosenball at Reuters. According to Hosenball's sources, Snowden's prolonged absence "prompted a hunt for the contractor, first by his employer Booz Allen Hamilton and then by the U.S. government."      Read more: