{"id":1688,"date":"2023-03-13T21:06:27","date_gmt":"2023-03-14T02:06:27","guid":{"rendered":"https:\/\/dda.ndus.edu\/ddreview\/?p=1688"},"modified":"2023-04-18T16:22:16","modified_gmt":"2023-04-18T21:22:16","slug":"ai-security-the-national-security-commission-on-artificial-intelligence-and-adversarial-machine-learning","status":"publish","type":"post","link":"https:\/\/dda.ndus.edu\/ddreview\/ai-security-the-national-security-commission-on-artificial-intelligence-and-adversarial-machine-learning\/","title":{"rendered":"AI SECURITY: The National Security Commission on Artificial Intelligence and Adversarial Machine Learning"},"content":{"rendered":"\n

The National Security Commission on Artificial Intelligence (NSCAI) was established by Congress in 2018 to \u201cconsider the methods and means necessary to advance the development of artificial intelligence \u2026 to comprehensively address the national security and defense needs of the United States.\u201di<\/sup>\u00a0And this it did. More than 19 of the recommendations from NSCAI\u2019s \u201cThe Final Report\u201d were included in the FY2021 National Defense Authorization Act, with dozens of other recommendations influencing Acts of Congress and executive orders related to national defense, intelligence, innovation and competition.<\/p>\n\n\n\n

Although the final report was released in March 2021\u2014and the commission sunset in October that year\u2014one element of the report is still widely underappreciated, especially outside of government circles. While the Department of Defense (DOD), as well as other government agencies ranging from the Department of Veteran Affairsii<\/sup>\u00a0to the Internal Revenue Service,iii<\/sup>\u00a0have responded to the charge to adopt <\/em>AI, very few government and commercial entities have invested in the calls to secure <\/em>AI.<\/p>\n\n\n\n

Organizations that adopt AI also adopt AI\u2019s risks and vulnerabilities. Intentional attacks against AI are a nascent style of cyberattack by which an attacker can manipulate an AI system. So, while the U.S. does need more investment in AI, it also needs to decisively secure industry security. The purpose of the book is to provide decision-makers with context that will sharpen their critique as they embrace the power of AI in government and industry. We also offer recommendations on how the intersection of technology, policy and law can provide us with a secure future.<\/p>\n\n\n\n

Below is an excerpt from the book\u2019s first chapter, \u201cDo You Want to Be a Part of the Future?\u201d<\/p>\n\n\n\n

NSCAI Genesis Story<\/h1>\n\n\n\n

Ylli Bajraktari is not a household name, but in national security circles, he has a reputation for getting things done. Andrew Exum, a former Deputy it from its adversaries.<\/p>\n\n\n\n

\"\"
Not with a Bug, But with a Sticker: Attacks on Machine Learning Systems and What To Do About Them, by Ram Shankar Siva Kumar and Hyrum Anderson, PhD,
John Wiley & Sons, Inc., 2023.<\/figcaption><\/figure><\/div>\n\n\n\n

In response, we wrote a book, Not With a Bug, But With a Sticker: Attacks on Machine Learning Systems and What To Do About Them<\/em>, which will be published by John Wiley & Sons, Inc., on May 2, 2023. The book explains how AI systems are significantly at risk from attacks in both simple and sophisticated ways, which can jeopardize national security, as well as corporate and Secretary of Defense for the Middle East, described Bajraktari and his brother Ylberiv<\/sup>\u00a0as \u201ctwo of the most important and best people in the federal government you\u2019ve likely never heard of.\u201dv<\/sup>\u00a0Ylli Bajraktari escaped war-torn Kosovo and moved to the U.S. in his 20s. Burnished with bonafide <\/em>credentials from Harvard\u2019s prestigious Kennedy School, he steadily rose through the ranks at the Pentagon, eventually becoming an advisor to the Deputy Secretary of Defense.vi<\/sup><\/p>\n\n\n\n

\"\"
Researchers asked an AI computer vision model, which typically performs well on image classification tasks, \u201cWhat is pictured in this image?\u201d To confuse the AI vision model, the researchers created images to maximize \u201cpenguin-ness,\u201d \u201csnakeness,\u201d and \u201cschoolbus-ness.\u201d As a result, the AI vision model was 99 percent confident that the top image represents a penguin, the middle image represents a green snake, and the bottom image represents a school bus. Courtesy of Ann Nguyen<\/figcaption><\/figure><\/div>\n\n\n\n

Whether it was fatigue that came from years of traveling the world to shape international policy or the pressure of working at the White House,vii<\/sup>\u00a0Bajraktari left the executive branch to join the National Defense University\u2019s (NDU) Institute for National Strategic Studies as a visiting research fellow. A year-long assignment from the Office of the Secretary of Defense, this break from the intensity in the White House gave him time to study and recalibrate at the NDU\u2019s libraries. He used some rare downtime for self-education on what he thought would be an instrumental cornerstone of U.S. competitiveness: artificial intelligence (AI). Bajraktari quickly absorbed the information he gleaned from pouring over books and watching YouTube videos on machine learning (ML). While at NDU, he organized the university\u2019s first ever AI symposium in November 2018. Expecting a meager 10 people to attend, the response was overwhelming. Bajraktari had to turn away hundreds of would-be attenders because of the room\u2019s fire code. While Bajraktari didn\u2019t know it yet, his time at DOD, the White House and now at NDU had been preparing him for a leadership role to shape the nation\u2019s strategy for investing in AI.<\/p>\n\n\n\n

That came in 2018, when the NSCAI was born out of the House Armed Services Committee. The leadership and guidance for the commission was to come from 15 appointed commissioners, a mix of tech glitterati that included current and former leadership at Google, Microsoft, Amazon, Oracle, and directors of laboratories at universities and research institutes that support national security.viii<\/sup>\u00a0It was an independent, temporary and bipartisan commission set up to study AI\u2019s national security implications.<\/p>\n\n\n\n

When NSCAI commissioner and former Google CEO, Eric Schmidt, called Bajraktari to ask him to lead the commission, Bajraktari didn\u2019t answer the phone. It was Christmas break. Plus, he had been primed by his White House days to ignore calls from unknown numbers. So eventually, the former CEO of Google resorted to the plebian tactic: He sent Bajraktari an email. In it, Schmidt described that he had just been voted by the other commissioners to chair NSCAI and needed somebody to run the commission\u2019s day-to-day operations. Bajraktari\u2019s email response was a simple one-liner: \u201cI\u2019ll do it.\u201dix<\/sup><\/p>\n\n\n\n

With an ambitious goal, a tight deadline and a budget smaller than what it takes to air a 60-second Superbowl commercial, Bajraktari assembled a team of over 130 staff members to deliver a series of reports that would culminate in NSCAI\u2019s final report.<\/p>\n\n\n\n

Even the initial findings packed a punch. Bajraktari and the two chairs of the commission headed to the White House to brief President Trump about the findings. Scheduled for 15 minutes, the meeting at the Oval Office lasted for nearly an hour. In December 2020, at the twilight of his presidency, President Trump signed an executive order entitled \u201cPromoting the Use of Trustworthy Artificial Intelligence in Government.\u201dx<\/sup><\/p>\n\n\n\n

Even with a change in the executive office, NSCAI\u2019s recommendations continued to make an impact. On July 13, 2021\u2014well into the Biden presidency\u2014the commission held a summit to discuss the final report at the Mayflower Hotel Ballroom in Washington, the Secretaries of Defense, Commerce and State, the National Security Advisor, and the Director of the Office of Science and Technology Policy all made in-person appearances and spoke to the masked and socially distanced audience members. It was a powerful signal from the American government to both its allies and adversaries that the U.S., from its highest levels of trade, diplomacy, defense, security, and science and technology, was ready to invest in AI and take the NSCAI\u2019s recommendation seriously.<\/p>\n\n\n\n

\"\"
AI can be fooled by carefully constructed patterns. What looks like a sweater designed around psychedelic, multicolor swirls is actually a carefully constructed pattern to fool object-recognition systems into mispredicting that the wearer does not exist. However, the deceptive traits of this sweater work only in a controlled setting and are generally ineffective to avoid surveillance in real-world settings. Courtesy of Zuxuan Wu and Tom Goldstein.<\/figcaption><\/figure>\n\n\n\n

Adversarial ML: Attacks Against AI<\/h1>\n\n\n\n

Delivered on March 2021 to the president and Congress, the first page of NSCAI\u2019s final report includes a realistic assessment of where the country stands, which by the commission\u2019s own admission was uncomfortable to deliver. The NSCAI report was 756 pages long, but its opening lines summarize our unpreparedness. \u201cAmerica is not prepared to defend or compete in the AI era,\u201d it reads. \u201cThis is the tough reality we must face. And it is this reality that demands comprehensive, whole-of-nation action.\u201d To \u201cdefend \u2026 in the AI era\u201d certainly refers to holistic national security but also entails defending vulnerabilities in AI defenses that are spelled out in numerous recommendations to ensure that \u201cmodels are resilient to \u2026 attempts to undermine AI-enabled systems.\u201dxi<\/sup><\/p>\n\n\n\n

But defend from whom? <\/em>Defend from what?<\/em><\/p>\n\n\n\n

Adversarial ML is sometimes called \u201ccounter AI\u201d in military circles. Distinct from the case where AI is used to empower an attacker (often called offensive <\/em>AI<\/em>), in adversarial ML, the adversary is targeting <\/em>an AI system as part of an attack. In this kind of cyberattack against AI, attackers actively subvert vulnerabilities in the ML system to accomplish their goal.<\/p>\n\n\n\n

What are the vulnerabilities? What kinds of attacks are possible? Consider a few scenarios:<\/p>\n\n\n\n

\"\"
To demonstrate the limitation his custom-built autonomous vehicle vision system, artist James Bridle drew two concentric circles with table salt sprinkled on the pavement: The inner circle was a solid line, and the outer circle was a dashed line. The car quite readily drove into the circle but was unable to drive out, mistaking the implausible lane markings as valid traffic indicators. Source: Autonomous Trap 001 (James Bridle, 2017), courtesy of the artist.<\/figcaption><\/figure><\/div>\n\n\n\n