National Security Commission on Artificial Intelligence. Final Report - page 1
LETTER FROM THE CHAIR AND VICE CHAIR
Letter from the Chair
and Vice Chair
Americans have not yet grappled with just how profoundly the artificial
intelligence (AI) revolution will impact our economy, national security,
and welfare. Much remains to be learned about the power and limits
of AI technologies. Nevertheless, big decisions need to be made now
to accelerate AI innovation to benefit the United States and to defend
against the malign uses of AI.
When considering these decisions, our leaders confront the classic dilemma of statecraft
identified by Henry Kissinger: “When your scope for action is greatest, the knowledge on
which you can base this action is always at a minimum. When your knowledge is greatest,
the scope for action has often disappeared.” The scope for action remains, but America’s
room for maneuver is shrinking.
As a bipartisan commission of 15 technologists, national security professionals, business
executives, and academic leaders, the National Security Commission on Artificial
Intelligence (NSCAI) is delivering an uncomfortable message: America is not prepared to
defend or compete in the AI era. This is the tough reality we must face. And it is this reality
that demands comprehensive, whole-of-nation action. Our final report presents a strategy
to defend against AI threats, responsibly employ AI for national security, and win the
broader technology competition for the sake of our prosperity, security, and welfare. The
U.S. government cannot do this alone. It needs committed partners in industry, academia,
and civil society. And America needs to enlist its oldest allies and new partners to build a
safer and freer world for the AI era.
AI is an inspiring technology. It will be the most powerful tool in generations for benefiting
humanity. Scientists have already made astonishing progress in fields ranging from
biology and medicine to astrophysics by leveraging AI. These advances are not science
fair experiments; they are improving life and unlocking mysteries of the natural world. They
are the kind of discoveries for which the label “game changing” is not a cliché.
AI systems will also be used in the pursuit of power. We fear AI tools will be weapons of
first resort in future conflicts. AI will not stay in the domain of superpowers or the realm of
science fiction. AI is dual-use, often open-source, and diffusing rapidly. State adversaries
p
1
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
are already using AI-enabled disinformation attacks to sow division in democracies and jar
our sense of reality. States, criminals, and terrorists will conduct AI-powered cyber attacks
and pair AI software with commercially available drones to create “smart weapons.” It
is no secret that America’s military rivals are integrating AI concepts and platforms to
challenge the United States’ decades-long technology advantage. We will not be able to
defend against AI-enabled threats without ubiquitous AI capabilities and new warfighting
paradigms. We want the men and women in national security departments and agencies
to have access to the best technology in the world to defend themselves and us, and to
protect our interests and those of our allies and partners.
Despite exciting experimentation and a few small AI programs, the U.S. government is a
long way from being “AI-ready.” The Commission’s business leaders are most frustrated by
slow government progress because they know it’s possible for large institutions to adopt
AI. AI integration is hard in any sector—and the national security arena poses some unique
challenges. Nevertheless, committed leaders can drive change. We need those leaders
in the Pentagon and across the Federal Government to build the technical infrastructure
and connect ideas and experimentation to new concepts and operations. By 2025, the
Department of Defense and the Intelligence Community must be AI-ready.
We should embrace the AI competition. Competition already infuses the quests for data,
computing power, and the holy grail: the rare talent to make AI breakthroughs. The fact
that AI courses through so many adjacent technologies and is leveraged across so many
fields explains its power and leads inexorably to another critical point: AI is part of a
broader global technology competition. Competition will speed up innovation. We should
race together with partners when AI competition is directed at the moonshots that benefit
humanity like discovering vaccines. But we must win the AI competition that is intensifying
strategic competition with China. China’s plans, resources, and progress should concern
all Americans. It is an AI peer in many areas and an AI leader in some applications. We
take seriously China’s ambition to surpass the United States as the world’s AI leader within
a decade.
The AI competition is also a values competition. China’s domestic use of AI is a chilling
precedent for anyone around the world who cherishes individual liberty. Its employment
of AI as a tool of repression and surveillance—at home and, increasingly, abroad—is
a powerful counterpoint to how we believe AI should be used. The AI future can be
democratic, but we have learned enough about the power of technology to strengthen
authoritarianism abroad and fuel extremism at home to know that we must not take for
granted that future technology trends will reinforce rather than erode democracy. We must
work with fellow democracies and the private sector to build privacy-protecting standards
into AI technologies and advance democratic norms to guide AI uses so that democracies
can responsibly use AI tools for national security purposes.
p
2
LETTER FROM THE CHAIR AND VICE CHAIR
We would like to emphasize a few areas where action is necessary
because the stakes of the competition are so high:
Leadership.
Ultimately, we have a duty to convince the leaders in the U.S. Government to make the
hard decision and the down payment to win the AI era. In America, the buck stops with
the President, and AI strategy starts in the White House. We built a National Security
Council to confront the challenges of the post-World War II era. Now we need to create
a Technology Competitiveness Council to build a strategy that accounts for the complex
security, economic, and scientific challenges of AI and its associated technologies. That
leadership imperative extends into all critical national security departments and agencies.
Talent.
The human talent deficit is the government’s most conspicuous AI deficit and the single
greatest inhibitor to buying, building, and fielding AI-enabled technologies for national
security purposes. This is not a time to add a few new positions in national security
departments and agencies for Silicon Valley technologists and call it a day. We need to
build entirely new talent pipelines from scratch. We should establish a new Digital Service
Academy and civilian National Reserve to grow tech talent with the same seriousness of
purpose that we grow military officers. The digital age demands a digital corps. Just as
important, the United States needs to win the international talent competition by improving
both STEM education and our system for admitting and retaining highly skilled immigrants.
Hardware.
Microelectronics power all AI, and the United States no longer manufactures the world’s
most sophisticated chips. We do not want to overstate the precariousness of our position,
but given that the vast majority of cutting-edge chips are produced at a single plant
separated by just 110 miles of water from our principal strategic competitor, we must
reevaluate the meaning of supply chain resilience and security. A recent chip shortage for
auto manufacturing cost an American car company an estimated $2.5 billion. A strategic
blockage would cost far more and put our security at risk. The federal investment and
incentives needed to revitalize domestic microchip fabrication—perhaps $35 billion—
should be an easy decision when the alternative is relying on another country to produce
the engines that power the machines that will shape the future.
p
3
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
Innovation Investment.
We worry that only a few big companies and powerful states will have the resources
to make the biggest AI breakthroughs. Despite the diffusion of open-source tools, the
needs for computing power and troves of data to improve algorithms are soaring at the
cutting edge of innovation. The federal government must partner with U.S. companies to
preserve American leadership and to support development of diverse AI applications that
advance the national interest in the broadest sense. If anything, this report underplays the
investments America will need to make. The $40 billion we recommend to expand and
democratize federal AI research and development (R&D) is a modest down payment on
future breakthroughs. We will also need to build secure digital infrastructure across the
nation, shared cloud computing access, and smart cities to truly leverage AI for the benefit
of all Americans. We envision hundreds of billions in federal spending in the coming years.
This is not a time for abstract criticism of industrial policy or fears of deficit spending to
stand in the way of progress. In 1956, President Dwight Eisenhower, a fiscally conservative
Republican, worked with a Democratic Congress to commit $10 billion to build the
Interstate Highway System. That is $96 billion in today’s world. Surely we can make a
similar investment in the nation’s future.
We are proud of the NSCAI’s bipartisan work. We have debated together, learned together,
and achieved consensus on critical points. It is our privilege to submit our recommendations
to Congress and the President. To paraphrase Winston Churchill, we are at the beginning
of the beginning of the competition that will shape our prosperity, national security, and the
well-being of our citizens. Our report presents the first steps the United States should take
to defend, compete, and win in the AI era.
Eric Schmidt,
Bob Work,
Chair
Vice Chair
p
4
LETTER FROM THE EXECUTIVE DIRECTOR: THE BEGINNING OF THE BEGINNING
LETTER FROM THE EXECUTIVE DIRECTOR:
The Beginning
of the Beginning
When we started our journey two years ago, little did we know what
was in front of us. What we encountered was willingness and hope
among many friends and allies to get our mission from Congress right
to maintain the United States’ advantage in artificial intelligence (AI).
We enjoyed support from U.S. Departments and Agencies. Many of them loaned us
resources, including detailing both civilian and military personnel, and dedicated countless
hours to help us understand their missions and priorities. Members of Congress and
congressional staff worked closely with us to accelerate our government’s adoption of AI
for national security purposes.
Over the course of the Commission’s work, we engaged with hundreds of representatives
from the private sector, academia, civil society, and across the government. We received
countless briefings—classified and unclassified. We met with anyone who thinks about AI,
works with AI, and develops AI who was willing to make time for us.
We found consensus among nearly all of our partners on three points: the conviction that
AI is an enormously powerful technology, acknowledgement of the urgency to invest more
in AI innovation, and responsibility to develop and use AI guided by democratic principles.
p
5
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
We also talked to our allies—old and new. From New Delhi to Tel Aviv to London, there
was a willingness and desire to work with the United States to deepen cooperation on AI.
I am indebted to the many individuals who volunteered with us, interned with us, provided
expertise, and were friends of the Commission. I am particularly grateful to the dedicated
full-time staff of the Commission, who in many cases stepped away from important jobs to
join this essential mission.
In the last two years, we encountered widespread hope that AI could generate incredible
benefits for our nation’s economy, welfare, and security. We also heard concern that AI—
like any technology—could create new challenges and exacerbate existing problems. We
listened and took those concerns seriously.
We ultimately came away with a recognition that if America embraces and invests in AI
based on our values, it will transform our country and ensure that the United States and its
allies continue to shape the world for the good of all humankind.
Thank you!
Yll Bajraktari
p
6
EXECUTIVE SUMMARY
Executive Summary
No comfortable historical reference captures the impact of artificial
intelligence (AI) on national security. AI is not a single technology
breakthrough, like a bat-wing stealth bomber. The race for AI
supremacy is not like the space race to the moon. AI is not even
comparable to a general-purpose technology like electricity. However,
what Thomas Edison said of electricity encapsulates the AI future: “It
is a field of fields … it holds the secrets which will reorganize the life of
the world.” Edison’s astounding assessment came from humility. All
that he discovered was “very little in comparison with the possibilities
that appear.”
The National Security Commission on Artificial Intelligence (NSCAI) humbly acknowledges
how much remains to be discovered about AI and its future applications. Nevertheless, we
know enough about AI today to begin with two convictions.
First, the rapidly improving ability of computer systems to solve problems and to perform
tasks that would otherwise require human intelligence—and in some instances exceed
human performance—is world altering. AI technologies are the most powerful tools in
generations for expanding knowledge, increasing prosperity, and enriching the human
experience. AI is also the quintessential “dual-use” technology. The ability of a machine
to perceive, evaluate, and act more quickly and accurately than a human represents a
competitive advantage in any field—civilian or military. AI technologies will be a source of
enormous power for the companies and countries that harness them.
Second, AI is expanding the window of vulnerability the United States has already entered.
For the first time since World War II, America’s technological predominance—the backbone
of its economic and military power—is under threat. China possesses the might, talent,
and ambition to surpass the United States as the world’s leader in AI in the next decade if
current trends do not change. Simultaneously, AI is deepening the threat posed by cyber
attacks and disinformation campaigns that Russia, China, and others are using to infiltrate
our society, steal our data, and interfere in our democracy. The limited uses of AI-enabled
attacks to date represent the tip of the iceberg. Meanwhile, global crises exemplified by
the COVID-19 pandemic and climate change highlight the need to expand our conception
of national security and find innovative AI-enabled solutions.
p
7
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
“The NSCAI Final Report
presents an integrated national
strategy to reorganize the
government, reorient the nation,
and rally our closest allies and
partners to defend and compete
in the coming era of
AI-accelerated competition
and conflict.”
Given these convictions, the Commission concludes that the United States must act now
to field AI systems and invest substantially more resources in AI innovation to protect
its security, promote its prosperity, and safeguard the future of democracy. Today, the
government is not organizing or investing to win the technology competition against a
committed competitor, nor is it prepared to defend against AI-enabled threats and rapidly
adopt AI applications for national security purposes. This is not a time for incremental
toggles to federal research budgets or adding a few new positions in the Pentagon for
Silicon Valley technologists. This will be expensive and require a significant change in
mindset. America needs White House leadership, Cabinet-member action, and bipartisan
Congressional support to win the AI era.
The NSCAI Final Report presents an integrated national strategy to reorganize the
government, reorient the nation, and rally our closest allies and partners to defend and
compete in the coming era of AI-accelerated competition and conflict. It is a two-pronged
approach. Part I, “Defending America in the AI Era,” outlines the stakes, explains what
the United States must do to defend against the spectrum of AI-related threats, and
recommends how the U.S. government can responsibly use AI technologies to protect
the American people and our interests. Part II, “Winning the Technology Competition,”
addresses the critical elements of the AI competition and recommends actions the
government must take to promote AI innovation to improve national competitiveness and
protect critical U.S. advantages. The recommendations are designed as interlocking and
mutually reinforcing actions that must be taken together.
p
8
EXECUTIVE SUMMARY
Part I: Defending America in the AI Era.
AI-enhanced capabilities will be the tools of first resort in a new era of conflict as strategic
competitors develop AI concepts and technologies for military and other malign uses
and cheap and commercially available AI applications ranging from “deepfakes” to lethal
drones become available to rogue states, terrorists, and criminals. The United States must
prepare to defend against these threats by quickly and responsibly adopting AI for national
security and defense purposes. Defending against AI-capable adversaries operating
at machine speeds without employing AI is an invitation to disaster. Human operators
will not be able to keep up with or defend against AI-enabled cyber or disinformation
attacks, drone swarms, or missile attacks without the assistance of AI-enabled machines.
National security professionals must have access to the world’s best technology to protect
themselves, perform their missions, and defend us. The Commission recommends that the
government take the following actions:
Defend against emerging AI-enabled threats to America’s free and open society. Digital
dependence in all walks of life is transforming personal and commercial vulnerabilities
into potential national security weaknesses. Adversaries are using AI systems to enhance
disinformation campaigns and cyber attacks. They are harvesting data on Americans
to build profiles of their beliefs, behavior, and biological makeup for tailored attempts to
manipulate or coerce individuals. This gathering storm of foreign influence and interference
requires organizational and policy reforms to bolster our resilience. The government needs
to stand up a task force and 24/7 operations center to confront digital disinformation. It
needs to better secure its own databases and prioritize data security in foreign investment
screening, supply chain risk management, and national data protection legislation. The
government should leverage AI-enabled cyber defenses to protect against AI-enabled
cyber attacks. And biosecurity must become a top-tier priority in national security policy.
Prepare for future warfare. Our armed forces’ competitive military-technical advantage
could be lost within the next decade if they do not accelerate the adoption of AI across
their missions. This will require marrying top-down leadership with bottom-up innovation
to put operationally relevant AI applications into place. The Department of Defense (DoD)
should:
First, establish the foundations for widespread integration of AI by 2025. This includes
building a common digital infrastructure, developing a digitally-literate workforce, and
instituting more agile acquisition, budget, and oversight processes. It also requires
strategically divesting from military systems that are ill-equipped for AI-enabled warfare
and instead investing in next-generation capabilities.
Second, achieve a state of military AI readiness by 2025. Pentagon leadership must act
now to drive organizational reforms, design innovative warfighting concepts, establish
AI and digital readiness performance goals, and define a joint warfighting network
p
9
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
architecture. DoD must also augment and focus its AI R&D portfolio. Readiness will also
require promoting AI interoperability with allies and partners.
Manage risks associated with AI-enabled and autonomous weapons. AI will enable new
levels of performance and autonomy for weapon systems. But it also raises important
legal, ethical, and strategic questions surrounding the use of lethal force. Provided their
use is authorized by a human commander or operator, properly designed and tested AI-
enabled and autonomous weapon systems can be used in ways that are consistent with
international humanitarian law. DoD’s rigorous, existing weapons review and targeting
procedures, including its dedicated protocols for autonomous weapon systems and
commitment to strong AI ethical principles, are capable of ensuring that the United States
will field safe and reliable AI-enabled and autonomous weapon systems and use them in a
lawful manner. While it is neither feasible nor currently in the interests of the United States
to pursue a global prohibition of AI-enabled and autonomous weapon systems, the global,
unchecked use of such systems could increase risks of unintended conflict escalation and
crisis instability. To reduce the risks, the United States should (1) clearly and publicly affirm
existing U.S. policy that only human beings can authorize employment of nuclear weapons
and seek similar commitments from Russia and China; (2) establish venues to discuss
AI’s impact on crisis stability with competitors; and (3) develop international standards of
practice for the development, testing, and use of AI-enabled and autonomous weapon
systems.
Transform national intelligence. The Intelligence Community
(IC) should adopt and
integrate AI-enabled capabilities across all aspects of its work, from collection to analysis.
Intelligence will benefit from AI more than any other national security mission. To capitalize
on AI, the Office of the Director of National Intelligence needs to empower and resource its
science and technology leaders. The entire IC should leverage open-source and publicly
available information in its analysis and prioritize collection of scientific and technical
intelligence. For better insights, intelligence agencies will need to develop innovative
approaches to human-machine teaming that use AI to augment human judgment.
Scale up digital talent in government. National security agencies need more digital experts
now or they will remain unprepared to buy, build, and use AI and associated technologies.
The talent deficit in DoD and the IC represents the greatest impediment to being AI-ready
by 2025. The government needs new talent pipelines, including a U.S. Digital Service
Academy to train current and future employees. It needs a civilian National Digital Reserve
Corps to recruit people with the right skills—including industry experts, academics, and
recent college graduates. And it needs a Digital Corps, modeled on the Army Medical
Corps, to organize technologists already serving in government.
Establish justified confidence in AI systems. If AI systems routinely do not work as designed
or are unpredictable in ways that can have significant negative consequences, then leaders
will not adopt them, operators will not use them, Congress will not fund them, and the
p
10
EXECUTIVE SUMMARY
American people will not support them. To establish justified confidence, the government
should focus on ensuring that its AI systems are robust and reliable, including through
research and development (R&D) investments in AI security and advancing human-AI
teaming through a sustained initiative led by the national research labs. It should also
enhance DoD’s testing and evaluation capabilities as AI-enabled systems grow in number,
scope, and complexity. Senior-level responsible AI leads should be appointed across the
government to improve executive leadership and policy oversight.
Present a democratic model of AI use for national security. AI tools are critical for U.S.
intelligence, homeland security, and law enforcement agencies. Public trust will hinge on
justified assurance that government use of AI will respect privacy, civil liberties, and civil
rights. The government must earn that trust and ensure that its use of AI tools is effective,
legitimate, and lawful. This imperative calls for developing AI tools to enhance oversight
and auditing, increasing public transparency about AI use, and building AI systems that
advance the goals of privacy preservation and fairness. It also requires ensuring that those
impacted by government actions involving AI can seek redress and have due process.
The government should strengthen oversight and governance mechanisms and establish
a task force to assess evolving concerns about AI and privacy, civil liberties, and civil
rights.
Part II: Winning the Technology Competition.
The race to research, develop, and deploy AI and associated technologies is intensifying the
technology competition that underpins a wider strategic competition. China is organized,
resourced, and determined to win this contest. The United States retains advantages
in critical areas, but current trends are concerning. While a competitive response is
complicated by deep academic and commercial interconnections, the United States must
do what it takes to retain its innovation leadership and position in the world. The U.S.
government must embrace the AI competition and organize to win it by orchestrating and
aligning U.S. strengths.
Organize with a White House-led strategy for technology competition. The United
States must elevate AI considerations from the technical to the strategic level. Emerging
technologies led by AI now underpin our economic prosperity, security, and welfare. The
White House should establish a new Technology Competitiveness Council led by the
Vice President to integrate security, economic, and scientific considerations; develop a
comprehensive technology strategy; and oversee its implementation.
Win the global talent competition. The United States risks losing the global competition for
scarce AI expertise if it does not cultivate more potential talent at home and recruit and
retain more existing talent from abroad. The United States must move aggressively on both
fronts. Congress should pass a National Defense Education Act II to address deficiencies
across the American educational system—from K-12 and job reskilling to investing in
p
11
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
thousands of undergraduate- and graduate-level fellowships in fields critical to the AI
future. At the same time, Congress should pursue a comprehensive immigration strategy
for highly skilled immigrants to encourage more AI talent to study, work, and remain in the
United States through new incentives and visa, green card, and job-portability reforms.
Accelerate AI innovation at home. The government must make major new investments
in AI R&D and establish a national AI research infrastructure that democratizes access
to the resources that fuel AI development across the nation. The government should: (1)
double non-defense funding for AI R&D annually to reach $32 billion per year by 2026,
establish a National Technology Foundation, and triple the number of National AI Research
Institutes; (2) establish a National AI Research Infrastructure composed of cloud computing
resources, test beds, large-scale open training data, and an open knowledge network
that will broaden access to AI and support experimentation in new fields of science and
engineering; and (3) strengthen commercial competitiveness by creating markets for AI
and by forming a network of regional innovation clusters.
Implement comprehensive intellectual property (IP) policies and regimes. The United States
must recognize IP policy as a national security priority critical for preserving America’s
leadership in AI and emerging technologies. This is especially important in light of China’s
efforts to leverage and exploit IP policies. The United States lacks the comprehensive
IP policies it needs for the AI era and is hindered by legal uncertainties in current U.S.
patent eligibility and patentability doctrine. The U.S. government needs a plan to reform IP
policies and regimes in ways that are designed to further national security priorities.
Build a resilient domestic base for designing and fabricating microelectronics. After
decades leading the microelectronics industry, the United States is now almost entirely
reliant on foreign sources for production of the cutting-edge semiconductors that power
all the AI algorithms critical for defense systems and everything else. Put simply: the
U.S. supply chain for advanced chips is at risk without concerted government action.
Rebuilding domestic chip manufacturing will be expensive, but the time to act is now. The
United States should commit to a strategy to stay at least two generations ahead of China
in state-of-the-art microelectronics and commit the funding and incentives to maintain
multiple sources of cutting-edge microelectronics fabrication in the United States.
Protect America’s technology advantages. As the margin of U.S. technological advantage
narrows and foreign efforts to acquire American know-how and dual-use technologies
increase, the United States must reexamine how to best protect ideas, technology, and
companies without unduly hindering innovation. The United States must:
First, modernize export controls and foreign investment screening to better protect
critical dual-use technologies—including by building regulatory capacity and fully
implementing recent legislative reforms, implementing coordinated export controls on
advanced semiconductor manufacturing equipment with allies, and expanding disclosure
requirements for investors from competitor nations.
p
12
EXECUTIVE SUMMARY
Second, protect the U.S. research enterprise as a national asset—by providing government
agencies, law enforcement, and research institutions with tools and resources to conduct
nuanced risk assessments and share information on specific threats and tactics,
coordinating research protection efforts with allies and partners, bolstering cybersecurity
support for research institutions, and strengthening visa vetting to limit problematic
research collaborations.
Build a favorable international technology order. The United States must work hand-in-
hand with allies and partners to promote the use of emerging technologies to strengthen
democratic norms and values, coordinate policies and investments to advance global
adoption of digital infrastructure and technologies, defend the integrity of international
technical standards, cooperate to advance AI innovation, and share practices and resources
to defend against malign uses of technology and the influence of authoritarian states in
democratic societies. The United States should lead an Emerging Technology Coalition
to achieve these goals and establish a Multilateral AI Research Institute to enhance the
United States’ position as a global research hub for emerging technology. The Department
of State should be reoriented, reorganized, and resourced to lead diplomacy in emerging
technologies.
Win the associated technologies competitions. Leadership in AI is necessary but not
sufficient for overall U.S. technological leadership. AI sits at the center of the constellation
of emerging technologies, enabling some and enabled by others. The United States must
therefore develop a single, authoritative list of the technologies that will underpin national
competitiveness in the 21st century and take bold action to catalyze U.S. leadership in
AI, microelectronics, biotechnology, quantum computing, 5G, robotics and autonomous
systems, additive manufacturing, and energy storage technology. U.S. leadership across
these technologies requires investing in specific platforms that will enable transformational
breakthroughs and building vibrant domestic manufacturing ecosystems in each. At the
same time, the government will need to continuously identify and prioritize emerging
technologies farther over the horizon.
p
13
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
Conclusion
This new era of competition promises to change the world we live in
and how we live within it. We can either shape the change to come or
be swept along by it. We now know that the uses of AI in all aspects
of life will grow and the pace of innovation will continue to accelerate.
We know adversaries are determined to turn AI capabilities against
us. We know China is determined to surpass us in AI leadership. We
know advances in AI build on themselves and confer significant first-
mover advantages. Now we must act. The principles we establish,
the federal investments we make, the national security applications
we field, the organizations we redesign, the partnerships we forge,
the coalitions we build, and the talent we cultivate will set America’s
strategic course. The United States should invest what it takes to
maintain its innovation leadership, to responsibly use AI to defend
free people and free societies, and to advance the frontiers of science
for the benefit of all humanity. AI is going to reorganize the world.
America must lead the charge.
p
14
PREFACE
Preface
The National Security Commission on Artificial Intelligence’s (NSCAI)
task is to make recommendations to the President and Congress
to “advance the development of artificial intelligence [AI], machine
learning, and associated technologies to comprehensively address
the national security and defense needs of the United States.” In
establishing the Commission, Section 1051 of the John S. McCain
National Defense Authorization Act for Fiscal Year 2019 instructs
NSCAI to examine AI through the lenses of national competitiveness,
the means to sustain technological advantage, trends in international
cooperation and competitiveness, ways to foster investment in basic
and advanced research, workforce and training, potential risks of
military use, ethical concerns, establishment of data standards and
incentivization of data sharing, and the future evolution of AI.1
The 15 commissioners were nominated by Congress and the Executive Branch. They
represent a diverse group of technologists, business executives, academic leaders, and
national security professionals. They have approached all inquiries in bipartisan fashion
and reached consensus on the Final Report. The Commission’s operations have been
guided by two principles: the need for action and the importance of transparency.
Action.
The Commission’s work includes an initial report in July 2019, interim reports in November
2019 and October 2020, two additional quarterly memorandums, a series of special
papers in response to the COVID-19 pandemic, and now a final report. Waiting to deliver
recommendations in a final report was not an option when we began our work in the spring
of 2019. Assessing the broad national security implications of a dynamic technology like
AI at a single point in time is like trying to catch lightning in a bottle. Scientists continue
to deliver AI breakthroughs and the commercial sector is finding new ways to apply AI
at an accelerating pace. Competitors around the world are developing AI strategies and
investing resources. The Commission delivered recommendations on a continuous basis,
aiming to match the speed of AI developments and the desires from the Executive Branch
and Congress for help in deciding what to do. Congress has already adopted a number of
our recommendations in the William M. (Mac) Thornberry National Defense Authorization
Act for Fiscal Year 2021,2 and the Executive Branch has incorporated recommendations
p
15
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
as well. And we have continuously sought to learn from and educate a wide range of
stakeholders to build a shared understanding about how AI will impact national security.
Transparency.
The NSCAI has been committed to transparency. As a Federal Advisory Committee, it has
held five public plenary sessions totaling approximately 15 hours of deliberations, streamed
live online, and archived meeting recordings on the NSCAI website. It has responded to
more than two dozen Freedom of Information Act requests and released more than 2,500
pages of material. NSCAI has posted more than 700 pages of draft materials for public
review and comment. With the exception of materials and issues classified for national
security reasons, the Commission has endeavored to offer full transparency. We have
proactively engaged with the media after every plenary session, quarterly report, and
submission to Congress. In dozens of separate engagements, we have partnered with
non-governmental organizations, federal government organizations, and international
organizations to communicate our recommendations to the media and the public.
Most important, we have taken on the hardest issues with AI in public settings and made
recommendations only after consulting with a wide range of civil society, private sector,
and government groups. We have tried to listen and understand views across the spectrum
on deeply complicated aspects of AI. We have engaged ethicists, technologists, and
national security strategists. We have spoken with warriors and diplomats. We have talked
to academics and entrepreneurs. All told, commissioners and staff have participated in
hundreds of discussions. As the commissioners built consensus on recommendations, we
approached issues with care and humility.
The Final Report.
The Final Report presents the NSCAI’s recommendations as a strategy for winning
the AI era. The 16 chapters in the Main Report provide topline recommendations. The
accompanying Blueprints for Action outline concrete steps that departments and agencies
can take to implement NSCAI recommendations. The Commission has provided as much
specificity as possible—including by providing draft legislative text and executive orders—
to help the President and Congress move rapidly from understanding AI to acting for the
benefit of the American people.
The Final Report represents an important step, but it is not the NSCAI’s final act. For
the remaining life of the Commission, our work will focus on implementation to help the
President and Congress make the investments and take the actions recommended to win
the AI era.
1 For full text, see Pub. L. 115-232, 132 Stat. 1636 (2018), https://www.congress.gov/115/bills/hr5515/
BILLS-115hr5515enr.pdf.
2 For full text, see Pub. L. 116-283, 134 Stat. 3388 (2021), https://www.congress.gov/bill/116th-congress/
house-bill/6395/text.
p
16
THE NATIONAL S ECU RIT Y COMMIS S ION ON AR TIFICIAL INTELLIG ENCE
Introduction
19
Artificial Intelligence in
31
Context
PART I: DEFENDING AMERICA IN THE AI ERA
41
Chapter 1: Emerging Threats in the AI Era
43
Chapter 2: Foundations of Future Defense
59
Chapter 3: AI and Warfare
75
Chapter 4: Autonomous Weapon Systems and Risks
89
Associated with AI-Enabled Warfare
Chapter 5: AI and the Future of National Intelligence
107
Chapter 6: Technical Talent in Government
119
Chapter 7: Establishing Justified Confidence in AI Systems
131
Chapter 8: Upholding Democratic Values: Privacy, Civil Liberties,
141
and Civil Rights in Uses of AI for National Security
PART II: WINNING THE TECHNOLOGY COMPETITION
155
Chapter 9: A Strategy for Competition and Cooperation
157
Chapter 10: The Talent Competition
171
Chapter 11: Accelerating AI Innovation
183
Chapter 12: Intellectual Property
199
Chapter 13: Microelectronics
11
Chapter 14: Technology Protection
23
Chapter 15: A Favorable International Technology
241
Order Chapter 16: Associated Technologies
253
Blueprints for Action
271
Appendices
599
p
18
INTRODUCTION
Introduction
Artificial Intelligence (AI) technologies promise to be the most powerful
tools in generations for expanding knowledge, increasing prosperity,
and enriching the human experience. The technologies will be the
foundation of the innovation economy and a source of enormous
power for countries that harness them. AI will fuel competition
between governments and companies racing to field it. And it will be
employed by nation-states to pursue their strategic ambitions.
Americans have not yet seriously grappled with how profoundly the AI revolution will
impact society, the economy, and national security. Recent AI breakthroughs, such as a
computer defeating a human in the popular strategy game of Go1, shocked other nations
into action, but it did not inspire the same response in the United States. Despite our
private-sector and university leadership in AI, the United States remains unprepared for
the coming era. Americans must recognize the assertive role that the government will have
to play in ensuring the United States wins this innovation competition. Congress and the
President will have to support the scale of public resources required to achieve it.
The magnitude of the technological opportunity coincides with a moment of strategic
vulnerability. China is a competitor possessing the might, talent, and ambition to challenge
America’s technological leadership, military superiority, and its broader position in the
world. AI is deepening the threat posed by cyber attacks and disinformation campaigns
that Russia, China, and other state and non-state actors are using to infiltrate our society,
steal our data, and interfere in our democracy. The limited uses of AI-enabled attacks to
date are the tip of the iceberg. Meanwhile, global crises exemplified in the global pandemic
and climate change are expanding the definition of national security and crying out for
innovative technological solutions. AI can help us navigate many of these new challenges.
We are fortunate. The AI revolution is not a strategic surprise. We are experiencing its
impact in our daily lives and can anticipate how research progress will translate into real-
world applications before we have to confront the full national security ramifications. This
commission can warn of national security challenges and articulate the benefits, rather
than explain why previous warnings were ignored and opportunities were missed. We
still have a window to make the changes to build a safer and better future. The pace of AI
innovation is not flat; it is accelerating. If the United States does not act, it will likely lose
its leadership position in AI to China in the next decade and become more vulnerable to a
spectrum of AI-enabled threats from a host of state and non-state actors.
p
19
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
The Commission concludes that the United States needs to implement a strategy to
defend and compete in the AI era. The White House must lead the effort to reorganize the
government and reorient the nation. This report presents the core elements of the strategy.
• Part I, “Defending America in the AI Era” (Chapters 1-8), outlines what the United States
must do to defend against the spectrum of AI-related threats from state and non-state
actors and recommends how the U.S. government can responsibly use AI technologies
to protect the American people and our interests.
• Part II, “Winning the Technology Competition” (Chapters 9-16), outlines AI’s role
in a broader technology competition. Each chapter addresses a critical element of
the competition and recommends actions the government must take to promote AI
innovation to improve national competitiveness and protect critical U.S. advantages.
Why Does AI Matter?
In
1901, Thomas Edison was asked to predict electricity’s impact on humanity. Two
decades after the development of the light bulb, he foresaw a general-purpose technology
of unlimited possibilities. “[Electricity] is the field of fields,” he said. “It holds the secrets
which will reorganize the life of the world.”2 AI is a very different kind of general-purpose
technology, but we are standing at a similar juncture and see a similarly wide-ranging
impact.3 The rapidly improving ability of computer systems to solve problems and to
perform tasks that would otherwise require human intelligence is transforming many
aspects of human life and every field of science. It will be incorporated into virtually all
future technology. The entire innovation base supporting our economy and security will
leverage AI. How this “field of fields” is used—for good and for ill—will reorganize the
world.
The Commission’s assessment is rooted in a realistic understanding of AI’s current state of
development and a projection of how the technology will evolve.
AI is already ubiquitous in everyday life and the pace of innovation is accelerating. We take
for granted that AI already shapes our lives in ways small and big. A “smartphone” has multiple
AI-enabled features including voice assistants, photo tagging, facial recognition security,
search apps, recommendation and advertising engines, and less obvious AI enhancements
in its operating system. AI is helping predict the spread and escalation of a pandemic
outbreak, planning and optimizing the distribution of goods and services, monitoring traffic
flow and safety, speeding up drug and therapeutic discovery, and automating routine office
functions. Recognizing the pace of change is critical to understanding the power of AI. The
application of AI techniques to solve problems is compressing innovation timescales and
turning once-fantastical ideas into realities across a range of disciplines.
p
20
INTRODUCTION
Deploying and adopting AI remains a hard problem. AI cannot magically solve problems. As
AI moves from an elite niche science to a mainstream tool, engineering will be as important
as scientific breakthroughs. Early adopters across sectors have learned similar lessons:
Trying to employ AI is a slog even after the science is settled. Many of the most important
real-world impacts will come from figuring out how to employ existing AI algorithms and
systems, some more than a decade old. The integration challenge is immense. Harnessing
data, hardening and packaging laboratory algorithms so they are ready for use in the field,
and adapting AI software to legacy equipment and rigid organizations all require time, effort,
and patience. Integrating AI often necessitates overcoming substantial organizational and
cultural barriers, and it demands top-down leadership.
AI tools are diffusing broadly and rapidly. Cutting-edge deep learning techniques are often
prohibitively expensive, requiring vast amounts of data, computing power, and specialized
knowledge. However, AI will not be the provenance of only big states and big tech. Many
machine learning tools that fuel AI applications are publicly available and usable even for
non-experts. Open-source applications and development tools combined with inexpensive
cloud computing and less data-intensive approaches are expanding AI opportunities
across the world to state and non-state actors.
AI is changing relationships between humans and machines. In modern society, we already
rely much more on machines and automation than we may be aware. The U.S. military, for
instance, has used autonomous systems for decades. However, as AI capabilities improve,
the dynamics within human-machine “teams” will change. In the past, computers could
only perform tasks that fell within a clearly defined set of parameters or rules programmed
by a human. As AI becomes more capable, computers will be able to learn and perform
tasks based on parameters that humans do not explicitly program, creating choices and
taking actions at a volume and speed never before possible. Across many fields of human
activity, AI innovations are raising important questions about what choices to delegate to
intelligent machines, in what circumstances, and for what reasons. In the national security
sphere, these questions will take on greater significance as AI is integrated into defense
and intelligence systems. Across our entire society, we will need to address these new
complexities with nuanced approaches, intellectual curiosity, and care that recognizes the
increasing ubiquity of AI.
Part I: Defending America in the AI Era.
Technology so ubiquitous in other facets of society will have an equivalent impact on
international competition and conflict.4 We must adopt AI to change the way we defend
America, deter adversaries, use intelligence to make sense of the world, and fight and win
wars. The men and women who protect the United States must be able to leverage the AI
and associated technologies that can help them accomplish their missions as quickly and
safely as possible.
p
21
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
AI is the quintessential “dual use” technology—it can be used for civilian and military
purposes. The AI promise—that a machine can perceive, decide, and act more quickly, in
a more complex environment, with more accuracy than a human—represents a competitive
advantage in any field. It will be employed for military ends, by governments and non-state
groups.
We can expect the large-scale proliferation of AI-enabled capabilities. Many national
security applications of AI will require only modest resources and good, but not great,
expertise to use. AI algorithms are often accessible. The hardware is “off-the-shelf” and
in most cases generally available to consumers (as with graphics processing units, for
example). “Deepfake” capabilities can be easily downloaded and used by anyone.5 AI-
enabled tools and mutating malware are in the hands of hackers.6 Cheap, lethal drones will
be common. Azerbaijan’s use of Turkish drones and Israeli loitering munitions in combat
against Armenia in October 2020 confirmed that autonomous military capabilities are
spreading.7 Many states are watching and learning from these experiences. The likelihood
of reckless or unethical uses of AI-enabled technologies by rogue states, criminals, or
terrorists is increasing.
AI-enabled capabilities will be tools of first resort in a new era of conflict. State and non-state
actors determined to challenge the United States, but avoid direct military confrontation,
will use AI to amplify existing tools and develop new ones. Adversaries are exploiting our
digital openness through AI-accelerated information operations and cyber attacks. Ad-
tech will become natsec-tech as adversaries recognize what advertising and technology
firms have recognized for years: that machine learning is a powerful tool for harvesting
and analyzing data and targeting activities. Using espionage and publicly available data,
adversaries will gather information and use AI to identify vulnerabilities in individuals,
society, and critical infrastructure. They will model how best to manipulate behavior, and
then act.
AI will transform all aspects of military affairs. AI applications will help militaries prepare,
sense and understand, decide, and execute faster and more efficiently. Numerous weapon
systems will leverage one or more AI technologies. AI systems will generate options for
commanders and create battle networks connecting systems across all domains. It will
transform logistics, procurement, training, and the design and development of new hardware.
Adopting AI will demand the development of new tactics and operational concepts. In the
future, warfare will pit algorithm against algorithm. The sources of battlefield advantage will
shift from traditional factors like force size and levels of armaments to factors like superior
data collection and assimilation, connectivity, computing power, algorithms, and system
security.
Competitors are actively developing AI concepts and technologies for military use. Russia
has plans to automate a substantial portion of its military systems.8 It has irresponsibly
deployed autonomous systems in Syria for testing on the battlefield.9 China sees AI as the
p
22
INTRODUCTION
path to offset U.S. conventional military superiority by “leapfrogging” to a new generation
of technology. Its military has embraced “intelligentized war”--investing, for example, in
swarming drones to contest U.S. naval supremacy.10 China’s military leaders talk openly
about using AI systems for “reconnaissance, electromagnetic countermeasures and
coordinated firepower strikes.”11 China is testing and training AI algorithms in military
games designed around real-world scenarios. As these authoritarian states field new AI-
enabled military systems, we are concerned that they will not be constrained by the same
rigorous testing and ethical code that guide the U.S. military.
AI will revolutionize the practice of intelligence. There may be no national security function
better suited for AI adoption than intelligence tradecraft and analysis. Machines will sift
troves of data amassed from all sources, locate critical information, translate languages,
fuse data sets from different domains, identify correlations and connections, redirect assets,
and inform analysts and decision-makers. To protect the American people, perhaps the
most urgent and compelling reason to accelerate the use of AI for national security is the
possibility that more advanced machine analysis could find and connect the dots before
the next attack, when human analysis alone may not see the full picture as clearly.
Defending against AI-capable adversaries without employing AI is an invitation to disaster.
AI will compress decision time frames from minutes to seconds, expand the scale of attacks,
and demand responses that will tax the limits of human cognition. Human operators will not
be able to defend against AI-enabled cyber or disinformation attacks, drone swarms, or
missile attacks without the assistance of AI-enabled machines. The best human operator
cannot defend against multiple machines making thousands of maneuvers per second
potentially moving at hypersonic speeds and orchestrated by AI across domains. Humans
cannot be everywhere at once, but software can.
Compelling logic dictates quick, but careful and responsible, AI adoption. The government
should adopt AI following the principle of legendary basketball coach John Wooden: “Be
quick, but don’t hurry.”12 Like other “safety critical” applications of AI, military and intelligence
functions require deliberation and caution before they are developed and fielded. Some
current AI systems are narrow and brittle. All require rigorous testing, safeguards, and an
understanding of how they might operate differently in the real world than in a testbed.
AI-enabled autonomous weapon systems could be more precise, and as a result, reduce
inadvertent civilian casualties. But they also raise important ethical questions about the
role of human judgment in employing lethal force. If improperly designed or used, they
could also increase the risk of military escalation.
There is an emerging consensus on principles for using AI responsibly in the defense and
intelligence communities.13 If an AI-powered machine does not work as designed with
predictability and guided by clear principles, then operators will not use it, organizations
will not embrace it, and the American people will not support it. Hurrying would be
counterproductive and dangerous if it caused Americans to lose confidence in the
p
23
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
“The best human operator
cannot defend against multiple
machines making thousands
of maneuvers per second
potentially moving at hypersonic
speeds and orchestrated by AI
across domains. Humans cannot
be everywhere at once, but
sofware can.”
benefits AI could confer. Risk, however, is inescapable. Failing to use AI to solve real
national security challenges risks putting the United States at a disadvantage, leaving
American service members more vulnerable, and spending taxpayer money unwisely on
antiquated and inefficient equipment. Delaying AI adoption will push all of the risk onto the
next generation of Americans—who will have to defend against, and perhaps fight, a 21st
century adversary with 20th century tools.
The U.S. government still operates at human speed, not machine speed. Adopting AI
requires profound adjustments in national security business practices, organizational
cultures, and mindsets from the tactical to the strategic levels—from the battlefield to
the Pentagon. The government lags behind the commercial state of the art in most AI
categories, including basic business automation. It suffers from technical deficits that range
from digital workforce shortages to inadequate acquisition policies, insufficient network
architecture, and weak data practices. Bureaucracy is thwarting better partnerships with
the AI leaders in the private sector that could help. The government must become a better
customer and a better partner. National security innovation, in the absence of an impetus
like a major war or terrorist attack, will require strong leadership.
Part II: Winning the Technology Competition.
In addition to AI’s narrow national security and defense applications, AI is the fulcrum of a
broader technology competition in the world. AI will be leveraged to advance all dimensions
p
24
INTRODUCTION
of national power, from healthcare to food production to environmental sustainability. The
successful adoption of AI in adjacent fields and technologies will drive economies, shape
societies, and determine which states exert influence and exercise power in the world.
Many countries have national AI strategies, but only the United States and China have the
resources, commercial might, talent pool, and innovation ecosystem to lead the world in
AI. In some areas of research and applications, China is already an AI peer, and it is more
technically advanced in some applications.14 Within the next decade, China could surpass
the United States as the world’s AI superpower.15
On a level playing field, the United States is capable of out-innovating any competitor.
However, today, there is a fundamental difference in the U.S. and China’s approaches to
AI innovation that puts American AI leadership in peril. For decades, the U.S. innovation
model has been the envy of the world. The open exchange of ideas, free markets, and
limited government involvement to support basic research are pillars of the American way
of innovation and reflect American values. In America, tech firms compete for market share.
They are not instruments of state power. Researchers collaborate in an open research
environment in competition with their peers to make AI breakthroughs without regard for
borders. The international flow of venture capital and AI-related commerce is encouraged
as firms compete for profits and the next big idea.
Most AI progress in the United States should remain with the private sector and universities.
We must not lose an innovation culture that is bottom-up and infused with a garage-startup
mentality. However, a fully distributed approach is not a winning strategy in this strategic
competition. Even large tech firms cannot be expected to compete with the resources
of China or make the big investments the U.S. will need to stay ahead. We will need a
hybrid approach meshing government and private-sector efforts to win the technology
competition.
China is organized, resourced, and determined to win the technology competition. AI is
central to China’s global expansion, economic and military power, and domestic stability.
It has a head start on executing a national AI plan as part of larger plans to lead the world
in several critical and emerging technology fields. Beginning in 2017, China established
AI goals, objectives, and strategies tied to specific timelines with resources backed by
committed leadership to lead the world in AI by 2030.16 China is executing a centrally
directed systematic plan to extract AI knowledge from abroad through espionage, talent
recruitment, technology transfer, and investments. It has ambitious plans to build and train
a new generation of AI engineers in new AI hubs. It supports “national champion” firms
(including Huawei, Baidu, Alibaba, Tencent, iFlytek, and SenseTime) to lead development
of AI technologies at home, advance state-directed priorities that feed military and security
programs under the rubric of military-civil fusion, and capture markets abroad.17 It funds
massive digital infrastructure projects across several continents. China developed an
intellectual property (IP) strategy and is trying to set global technical standards for AI
development.18 And its laws make it all but impossible for a company in China to shield its
data from the authorities.19
p
25
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
Advancements in AI are contributing to a broad platform technology competition in
e-commerce, search engines, social media, and much else. The countries, companies,
and researchers that win the AI competition—in computing, data, talent, and
commercialization—will be positioned to win a much larger game. In essence, more
and better data, fed by a larger consumer/participant base, produce better algorithms,
which produce better results, which in turn produces more users, more data, and better
performance—until, ultimately, fewer companies will become entrenched as the dominant
platforms. If China’s firms win these competitions, it will not only disadvantage U.S.
commercial firms, it will also create the digital foundation for a geopolitical challenge to the
United States and its allies. Platform domination abroad allows China to harvest the data of
its users and permits China to extend aspects of its domestic system of control. Wherever
China controls the digital infrastructure, social media platforms, and e-commerce, it would
possess greater leverage and power to coerce, propagandize, and shape the world to
conform to its goals.
The AI competition is complicated by deep interconnections. The United States and China
are not operating in parallel lanes like the Soviets and Americans did in the space race,
with disconnected research and development (R&D) enterprises and minimal commercial
contacts. The research ecosystems in China and the United States are deeply connected
through shared research projects, talent circulation (particularly from China to the United
States), and commercial linkages that include supply chains, markets, and joint research
ventures. It would be counterproductive to sever the technology ties to China that benefit
basic research and U.S. companies. However, the United States must protect the integrity
of open research, prevent the theft of American IP, and employ targeted tools like export
controls and investment screening to protect technology industries critical to national
security.
The United States retains advantages in critical areas, but trends are concerning. The
world’s best scientific talent is more likely to stay home or migrate elsewhere today
than in our recent past.20 The U.S. lead in microelectronics—the hardware on which all
AI runs—has diminished, and for cutting-edge chips it is dependent on foreign supply
chains and manufacturers in Asia that are vulnerable to coercion or disruption.21 While
many machine learning tools are widely available and per-unit computing costs have
declined, the computing power and data access needed for cutting-edge deep learning
research breakthroughs are making it harder for university-based researchers and smaller
companies to compete.22 The geography of innovation remains concentrated in only some
parts of the country.23
The U.S. government must take a hands-on approach to national technology
competitiveness. Promoting a diverse and resilient R&D ecosystem and commercial sector
is a government responsibility. Expanding talent pipelines to attract the world’s best and
redoubling efforts to educate AI-ready Americans are public policy choices. Judiciously,
but aggressively, protecting critical AI intellectual property and thwarting the systemic
p
26
INTRODUCTION
campaign of illicit knowledge transfer being conducted by competitors is a government
obligation. Protecting hardware advantages and building resiliency into supply chains
necessitate legislation and federal incentives. Bringing together like-minded allies and
partners to build an international coalition that ensures a democratic vision for AI that will
shape the digital future requires U.S.-led diplomacy.
The AI competition will require White House leadership. The critical elements of the strategy
are too complicated for any one department or agency to lead because they cut across
national security, economic, and technology policy. Only strong executive leadership from
the White House can drive policy, force tradeoffs, and mobilize the country to make the
necessary investments.
AI for What Ends? Technology and Values.
The widespread adoption of AI by governments around the world is impacting not only the
international order among states, but also the political order within them. The stakes of the AI
future are intimately connected to the enduring contest between authoritarian and democratic
political systems and ideologies.
Technology itself does not possess an ideology, but how it is designed, where it is employed,
and which laws govern its use reflect the priorities and values of those who design and employ
it. More AI-enabled surveillance and analysis capabilities will soon be in the hands of most or all
governments. As the technology diffuses, the main difference between states will have less to
do with the quality or sophistication of the technology and more to do with the way it is used—
for what purpose, and under what rules.
Authoritarian regimes will continue to use AI-powered face recognition, biometrics, predictive
analytics, and data fusion as instruments of surveillance, influence, and political control. China’s
use of AI-powered surveillance technologies to repress its Uyghur minority and monitor all of its
citizens foreshadows how authoritarian regimes will use AI systems to facilitate censorship, track
the physical movements and digital activities of their citizens, and stifle dissent.24 The global
circulation of these digital systems creates the prospect of a wider adoption of authoritarian
governance. But liberal democracies also employ AI for internal security and public safety
purposes. More than half of the world’s advanced democracies use AI-enabled surveillance
systems.25 Such technologies have legitimate public purposes and are compatible with the rule
of law. Yet in states edging toward illiberal practices, utilizing digital tools in ways that undermine
the rule of law could tip the scales toward further democratic backsliding. The preservation of
individual liberties calls for continued vigilance. A responsible democracy must ensure that the
use of AI by the government is limited by wise restraints to comport with the rights and liberties
that define a free and open society.
The U.S. government should develop and field AI-enabled technologies with adequate
transparency, strong oversight, and accountability to protect against misuse. Merely stating U.S.
p
27
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
opposition to the authoritarian use of AI is not enough. The United States must also demonstrate
how a democracy should use AI to protect the security of its citizens in ways that uphold liberal
democratic values. There is an urgent need to field AI for national security purposes against, for
instance, foreign and domestic terrorists operating within our borders. There is also an enduring
need to ensure that security applications of AI conform to core values of individual liberty and
equal protection under law.
The United States must lead a coalition of democracies. As we ensure that AI is developed
and used in ways that are safe for democracy at home, we must also promote global norms
to make its use safe for democracy abroad. While the U.S. government’s ability to influence
the governance practices of other states is limited, a strong plank of the U.S. foreign policy
agenda with respect to AI must be to promote human rights and counter techno-authoritarian
trends. The United States can use diplomacy and leverage its global partnerships to advocate
for establishing privacy-protecting technical standards and norms in international bodies,
and it can work with like-minded nations to ensure that other nations have an alternative to
embracing China’s technology and methods of social control and access to technologies
that protect democratic values like privacy. We do not seek a fragmented digital world. We
want the United States and its allies to exist in a world with a diverse set of choices in digital
infrastructure, e-commerce, and social media that will not be vulnerable to authoritarian coercion
and that support free speech, individual rights, privacy, and tolerance for differing views.
Conclusion
We are at the beginning of the beginning of this new era of competition.
We now know the uses of AI in all aspects of life will grow and the pace
of innovation will accelerate. We know adversaries are determined to
turn AI capabilities against us. We know a competitor is determined to
surpass us in AI leadership. We know AI is accelerating breakthroughs
in a wide array of fields. We know that whoever translates AI
developments into applications first will have the advantage. Now
we must act. The principles we establish, the federal investments we
make, the national security applications we field, the organizations
we redesign, the partnerships we forge, the coalitions we build, and
the talent we cultivate will set America’s strategic course. The United
States should invest what it takes to maintain its innovation leadership,
to responsibly use AI to defend free people and free societies, and to
advance the frontiers of science for the benefit of all humanity. AI is
going to reorganize the world. America must lead the charge.
p
28
INTRODUCTION
Introduction - Endnotes
com/alphago-korea.
2 Quoted in Orison Swett Marden, How They Succeeded: Life Stories of Successful Men Told by
Themselves, Lothrop Publishing Co. at 238 (1901).
3 Andrew Ng is widely credited with making this comparison. See e.g., Shana Lynch, Andrew Ng: Why
AI Is the New Electricity, Insights by Stanford Business (March 11, 2017), https://www.gsb.stanford.
edu/insights/andrew-ng-why-ai-new-electricity.
4 For an overview of AI and international relations see Michael Horowitz, Artificial Intelligence,
International Competition, and the Balance of Power, Texas National Security Review (May 2018),
https://doi.org/10.15781/T2639KP49.
5 Karen Hao & Will Douglas Heaven, The Year Deepfakes Went Mainstream, MIT Technology Review
(Dec. 24, 2020), https://www.technologyreview.com/2020/12/24/1015380/best-ai-deepfakes-of-2020/.
com/9315/2354/6488/2018-Webroot-Threat-Report_US-ONLINE.pdf; Implications of Artificial
Intelligence for Cybersecurity: Proceedings of a Workshop, National Academies of Sciences,
Engineering, & Medicine (2019), https://www.nap.edu/catalog/25488/implications-of-artificial-
intelligence-for-cybersecurity-proceedings-of-a-workshop; Ben Buchanan, et al., Automating Cyber
georgetown.edu/research/automating-cyber-attacks/; Deep Exploit: Fully Automatic Penetration
com/13o-bbr-bbq/machine_learning_security/tree/master/DeepExploit.
7 Robyn Dixon, Azerbaijan’s Drones Owned the Battlefield in Nagorno-Karabakh—and Showed Future
karabkah-drones-azerbaijan-aremenia/2020/11/11/441bcbd2-193d-11eb-8bda-814ca56e138b_story.
html.
stanleycenter.org/wp-content/uploads/2020/05/MilitarizationofAI-Russia.pdf.
9 Dylan Malyasov, Combat Tests in Syria Brought to Light Deficiencies of Russian Unmanned Mini-
light-deficiencies-russian-unmanned-mini-tank.html.
10 Testimony of Elsa Kania before the U.S.-China Economic and Security Review Commission, Hearing
files/June%207%20Hearing_Panel%201_Elsa%20Kania_Chinese%20Military%20Innovation%20in%20
Artificial%20Intelligence_0.pdf; Elsa Kania, “AI Weapons” in China’s Military Innovation, Brookings
at 1 (April 20, 2020), https://www.brookings.edu/wp-content/uploads/2020/04/FP_20200427_ai_
weapons_kania_v2.pdf.
com/2020/11/the-plas-ai-competitions/.
12 Andrew Hill & John Wooden, Be Quick—But Don’t Hurry: Finding Success in the Teachings of a
Lifetime, Simon & Schuster at 69 (2001).
13 Press Release, U.S. Department of Defense, DoD Adopts Ethical Principles for Artificial Intelligence
adopts-ethical-principles-for-artificial-intelligence/; Principles of Artificial Intelligence Ethics for
features/2763-principles-of-artificial-intelligence-ethics-for-the-intelligence-community.
14 Graham Allison & Eric Schmidt, Is China Beating the U.S. to AI Supremacy?, Belfer Center for
Science and International Affairs (Aug. 2020), https://www.belfercenter.org/publication/china-beating-
us-ai-supremacy.
15 See e.g., Alexandra Mousavizadeh, et al., The Global AI Index, Tortoise Media (Dec. 3, 2019),
https://www.tortoisemedia.com/2019/12/03/global-ai-index/.
p
29
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
16 See Graham Webster, et al., Full Translation: China’s ‘New Generation Artificial Intelligence
digichina/blog/full-translation-chinas-new-generation-artificial-intelligence-development-plan-2017/
(translating China’s State Council Notice on the Issuance of the New Generation Artificial Intelligence
Development Plan, dated July 20, 2017).
17 Benjamin Larsen, Drafting China’s National AI Team for Governance, New America (Nov. 18, 2019),
https://www.newamerica.org/cybersecurity-initiative/digichina/blog/drafting-chinas-national-ai-team-
governance/ (originally published in Graham Webster, ed., AI Policy and China: Realities of State-
cloudfront.net/documents/DigiChina-AI-report-20191029.pdf); Meng Jing, China to Boost Its ‘National
Team’ to Meet Goal of Global AI Leadership by 2030, South China Morning Post (Nov. 15, 2018),
global-ai-leadership-2030; Gregory C. Allen, Understanding China’s AI Strategy, Center for a New
American Security (Feb. 6, 2019), https://www.cnas.org/publications/reports/understanding-chinas-
ai-strategy (“The price of SenseTime and the other AI Champions being allowed to dominate these
technologies is the Champions’ extensive cooperation with China’s national security community. Even
beyond direct cooperation, China’s success in commercial AI and semiconductor markets brings
funding, talent, and economies of scale that both reduce China’s vulnerability from losing access to
international markets and offer useful technology for the development of weaponry and espionage
capabilities.”).
18 Emily de La Bruyère & Nathan Picarsic, China Standards 2035, Horizon Advisory (April 2020),
https://www.horizonadvisory.org/china-standards-2035-first-report.
19 Murray Scot Tanner, Beijing’s New National Intelligence Law: From Defense to Offense, Lawfare
(July 20, 2017), https://www.lawfareblog.com/beijings-new-national-intelligence-law-defense-offense.
20 Remco Zwetsloot, China’s Approach to Tech Talent Competition: Policies, Results, and the
content/uploads/2020/04/FP_20200427_china_talent_policy_zwetsloot.pdf.
21 “Ninety percent of all high-volume, leading-edge [semiconductor] production will soon be based in
Taiwan, China, and South Korea.” The Department of Defense estimates that by 2022 only 8% of all
semiconductor fabrication will occur in the United States, “down from 40% in the 1990s.” Michaela D.
Platzer, et al., Semiconductors: U.S. Industry, Global Competition, and Federal Policy, Congressional
Switzer, U.S. National Security Implications of Microelectronics Supply Chain Concentration in Taiwan,
South Korea, and the People’s Republic of China, U.S. Air Force [Sept. 2019]).
22 For example, non-elite universities and AI startups have difficulty affording the cost of compute
resources and data for training sophisticated machine learning (ML) models. Nur Ahmed & Muntasir
Wahed, The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence
Research, arXiv (Oct. 22, 2020), https://arxiv.org/abs/2010.15581. The need to bolster the nationwide
AI infrastructure is the first recommendation in the 20-Year Roadmap issued by the Association for the
Advancement of Artificial Intelligence. See A 20-Year Community Roadmap for Artificial Intelligence
Research in the US, Computing Community Consortium and AAAI, at 3 (August 2019), https://cra.org/
ccc/wp-content/uploads/sites/2/2019/08/Community-Roadmap-for-AI-Research.pdf.
23 More than 90% of U.S. innovation-sector job creation occurred in just five major coastal cities
between 2005 and 2017. Robert D. Atkinson, et al., The Case for Growth Centers: How to Spread Tech
centers-how-to-spread-tech-innovation-across-america/.
24 See, e.g., Patrice Taddonio, How China’s Government Is Using AI on Its Uighur Muslim Population,
PBS Frontline (Nov. 21, 2019), https://www.pbs.org/wgbh/frontline/article/how-chinas-government-
is-using-ai-on-its-uighur-muslim-population/; Bethany Allen-Ebrahimian, Exposed: China’s Operating
Manuals for Mass Internment and Arrest by Algorithm, International Consortium of Investigative
Journalists (Nov. 24, 2019), https://www.icij.org/investigations/china-cables/exposed-chinas-
operating-manuals-for-mass-internment-and-arrest-by-algorithm/.
25 See Steven Feldstein, The Global Expansion of AI Surveillance, Carnegie Endowment for
International Peace (Sept. 2019), https://carnegieendowment.org/2019/09/17/global-expansion-of-ai-
surveillance-pub-79847.
p
30
Artificial Intelligence
in Context
Artificial intelligence
(AI) is not a single piece of hardware or software, but rather a
constellation of technologies that depend on interrelated elements that can be envisioned
as a stack.
Integration
Applications
Algorithms
Planning &
Computer
Optimization
Vision
Hardware
Uses for
Modeling &
Data
Deployed AI
Simulation
Prediction
Today
Talent
Robotic
Natural Language
Process Automation
Understanding
Edge
Computing
Novel
Learning
Research
Advanced
Frontiers
Reasoning
Education
Human-AI
Teaming
Science
More
General AI
AI-Enabled
Future
Space
Smart Cities
Healthcare
Artificial Intelligence (AI) is not a single piece of hardware or software,
but rather a constellation of technologies. To address such a broad
topic, the Commission’s legislative mandate provided guidance
on how to scope its work to include technologies that solve tasks
requiring human-like perception, cognition, planning, learning,
communication, or physical action; and technologies that may learn
and act autonomously, whether in the form of software agents or
embodied robots.1
Successful development and fielding of AI technologies depends on a number of
interrelated elements that can be envisioned as a stack.2 AI requires talent, data,
hardware, algorithms, applications, and integration. We regard talent as the most essential
requirement because it drives the creation and management of all the other elements.
Data is critical for most AI systems.3 Labeled and curated data enables much of current
machine learning (ML) used to create new applications and improve the performance
of existing AI applications. The underlying hardware provides the computing power to
analyze ever-growing data pools and run applications. This hardware layer includes cloud-
based compute and storage, supported by a networking and communications backbone,
instrumental for connecting smart sensors and devices at the network edge. Algorithms
are the mathematical operations that tell the system how to navigate the data to provide
answers in response to specific questions. An application makes the answers useful for
specific tasks. Integration of these elements is critical to fielding a successful end-to-
end AI system. This requires significant engineering talent and investment to integrate
existing data flows, decision pipelines, legacy equipment, testing designs, etc. This task
of integration can be daunting and historically has been underestimated.4
AI technologies and applications such as pattern recognition, ML, computer vision, natural
language understanding, and speech recognition have evolved for many decades. In the
early years of AI, the period the Defense Advanced Research Projects Agency (DARPA)
describes as the “first wave,” researchers explored many approaches, including symbolic
logic, expert systems, and planning. Some of the most effective results were based on
“handcrafted knowledge” defined by humans and then used by the machine for reasoning
and interacting.5
Within the past 10 years, we have witnessed a “second wave” of AI, propelled by large-
scale statistical ML that enables engineers to create models that can be trained to
specific problem domains if given exemplar data or simulated interactions. Learning from
data, these systems are designed to solve specific tasks and achieve particular goals
with competencies that, in some respects, parallel the cognitive processes of humans:
perceiving, reasoning, learning, communicating, deciding, and acting. Today most fielded
large-scale AI systems employ elements of both first- and second-wave AI approaches.
Age of Deployed AI.
Today, we have reached an inflection point. Global digital transformation has led to an
overwhelming supply of data. Statistical ML algorithms, particularly deep neural networks,
have matured as problem solvers—albeit with limitations.6 The powerful and networked
computing that fuels ML capabilities has become widely available. The convergence of
these factors now places this capable technology in the hands of the technical and non-
technical alike. The fundamental “question is no longer how this technology works, but
what it can do for you.”7
While the current technology still has significant limitations, it is well-suited for certain use
cases. We have entered the age of deployed AI. AI is now ubiquitous, embedded in devices
we use and interact with on a daily basis—for example, in our smartphones, wireless
routers, and cars. We routinely rely on AI-enriched applications, whether searching for a
new restaurant, navigating traffic, selecting a movie, or getting customer service over the
phone or online.
Current Application
Prediction
Natural Language Understanding (NLU)
of AI in Key Areas.
Examples:
Machines are able to
Event forecasting and pattern
Example:
Preventative
process, analyze,
analysis have impacted nearly all
Vehicle
understand, and mimic
Richer
industries (e.g., finance, farming,
Maintenance
human language, either
Human-Computer
and transportation).
Interaction
Precision
spoken or written.
Agriculture
Current
Planning and Optimization
Computer Vision
Application of
Determining
Perceiving and
AI in Key Areas
Example:
necessary steps to
Example:
learning visual tasks from
complete a series of
the world through
Livestock
Transportation
Monitoring
tasks can save time
Planning
cameras and sensors.
and money, and
in Cities
improve safety.
Robotic Process Automation (RPA)
Modeling and Simulation
Software robotics to help
Example:
Modeling our physical,
organizations automate tedious
Examples:
RPA Platforms
economical, and social world to
and repetitive tasks.
COVID-19
support the study, optimization,
Research
and testing of operations through
simulation without interfering or
Tracking
Space Debris
interrupting ongoing processes.
Forecasting the future of AI is difficult. Five years ago, few would have predicted the recent
breakthroughs in natural language understanding that have resulted in systems that can
generate full text almost indistinguishable from human prose.8 With a remarkable increase
of investments in the global AI industry over the past five years9 and an unprecedented
amount of general R&D dollars being invested worldwide,10 there is no AI slowdown in
sight—only new horizons for deployed AI.
“With a remarkable increase
of investments in the global AI
industry over the past five years
and an unprecedented amount
of general R&D dollars being
invested worldwide, there is no
AI slowdown in sight—only new
horizons for deployed AI.”
Frontiers of AI Technology.
The next decade of AI research will likely be defined by efforts to incorporate existing
knowledge, push forward novel ways of learning, and make systems more robust,
generalizable, and trustworthy.11 Research on advancing human-machine teaming will be
at the forefront, as will improvements in hybrid AI techniques, enhanced training methods,
and explainable AI.
Human-AI Teaming. Mastering human-AI collaboration and teaming is a foundational
element for future application of AI. Synergy between humans and AI holds the promise
of a whole greater than the sum of its parts. Researchers are addressing this challenge by
studying issues of delegated authority, observability, predictability, directability, and trust.12
Gaining greater understanding of how humans will learn to work with AI will provide insights
for creating effective training programs for humans. Advances in language understanding
are being pursued to create systems that can summarize complex inputs and engage
through human-like conversation, a critical component of next-generation teaming. The
frontier of teaming includes the need for collaborative intelligence among cohorts of agents,
whether mixed groups of humans and machines or teams of coordinating machines.
Novel Ways of Learning. New learning methods are allowing for greater efficiency in both
training and inference from data.13 This decreases dependence on vast data sets and
widens the aperture of systems to handle tasks beyond their original scope, building
pathways toward contextual learning and commonsense reasoning. Hybrid AI techniques
combine different AI approaches to capitalize on their complementary strengths.14 For
example, neuro-symbolic research is combining symbolic manipulation with neural
networks.15 Model-based and data-based approaches may also be combined; for example,
leveraging physics knowledge within statistical ML frameworks.16 Researchers are also
advancing supervised learning techniques with low supplies of labeled data,17 while others
have devised more efficient methods of labeling data.18 Synthetic data generation through
simulation is one such promising approach.19 It allows a model to see conditions and
scenarios it may not have encountered with a real data set, while preserving relationships
between important variables in the original data and privacy of sensitive data.20
Edge Computing. Breaking size, weight, and power barriers also increases the ubiquity
of AI and aids privacy protection. Companies are working to pack more computational
power into tighter, specialized chips that use less energy to train and run the same models.
Such chips allow consumer devices to run complex models locally, rather than transmit
data externally and wait for models to run remotely. Retaining data entirely on the device
where a model is being trained or run is an advancement that could potentially enhance
individual privacy in AI-powered systems.21
Advances in Reasoning. In comparison to humans, even our most capable current AI
systems lack what one might think of as “commonsense reasoning.” Efforts are underway
to create systems that can generalize knowledge and translate learning across domains.
An AI system endowed with commonsense reasoning could effectively model the
human ability to make and exploit presumptions about the physical properties, purpose,
intentions, and behavior of people and objects and thereby characterize the probable
consequences of an action or interaction. Advancements in categorization, in creating
generalized structured ontologies, and in language understanding will drive the ability for
machines to learn while understanding context and content and allow people to discover
rapid solutions to problems that would historically take years to examine.22 This research
promises to pave the way for more explainable AI along with greater ability to detect and
mitigate bias, which will be essential to improving trustworthiness of these more general
AI technologies.23
Toward More General Artificial Intelligence. AI solutions to date have demonstrated
narrow and deep competencies, but with fundamental distinction from capabilities
demonstrated by humans. Humans perform tasks by learning without explicit supervised
signals; they generalize skills required for one task and apply them to other tasks; and they
accrue, manipulate, and reason with large amounts of commonsense knowledge. Some
researchers have used the phrase “artificial general intelligence” (AGI) to refer to a goal
of extending AI beyond narrow, vertical wedges of expertise. Debates have focused on
whether there might be specific breakthroughs that would lead to more general, human-like
capabilities or whether the field will more likely continue to push more general AI along one
or more dimensions of skills. No matter what the perspective, significant progress across
the research areas mentioned in this section will be required to create more general AI
systems.24 If achieved, more general AI methods could have enormous benefits, but could
also introduce new risks if safety challenges are not addressed. While breakthroughs are
in no way guaranteed, the United States should continue to research systems with more
human-like capabilities, accompanied by commensurate investments to ensure that those
systems are safe and controllable.
Advances in AI, including the mastery of more general AI capabilities along one or more
dimensions, will likely provide new capabilities and applications. Some of these advances
could lead to inflection points or leaps in capabilities. Such advances may also introduce
new concerns and risks and the need for new policies, recommendations, and technical
advances to assure that systems are aligned with goals and values,25 including safety,
robustness and trustworthiness.26 The US should monitor advances in AI and make
necessary investments in technology and give attention to policy so as to ensure that AI
systems and their uses align with our goals and values.27
Looking to an AI-Enabled Future.
Following the trajectories of the research threads outlined above sketches a future in which
AI empowers humanity in unprecedented ways, unlocking capabilities across science,
education, space technology, healthcare, infrastructure, manufacturing, agriculture,
entertainment, and countless other sectors. For example, advances in natural language
understanding could enable real-time, ubiquitous translation for more obscure languages
for which written and spoken training data is limited.28 This would transform the way we
communicate across geographic and cultural barriers, enabling business, diplomacy, and
free exchange of ideas.
Breakthroughs in integration of multi-modal, multi-source data could enable real-time
AI-driven modeling and simulation for federal responses to crises including pandemics
and natural disasters.29 Drone feeds augmented with maps, building layouts, and other
visual data layers could empower first responders with lifesaving emergency-scene
understanding,30 and AI could help build response plans, expedite command and control,
and optimize logistics for a range of disaster-response scenarios.31
AI as the Engine
Healthcare
of Invention.
Examples:
Advancing AI-assisted disease prevention
Averting cardiovascular disease and stroke
Aiding in early disease detection and monitoring
Biological sensors
Mitigating the effects of disabilities
Assisting the visually impaired
Supporting health care providers in patient care
Telehealth robots
Science
Education
Examples:
Examples:
Accelerating
AI as the
Dynamic
Expediting the
materials
Optimizing learning
curriculum
discovery and
development
augmentation
effective utilization
Engine of
for individuals
and resilience
of materials
Increasing safety
Remote digital
Expediting drug and
Invention
in training
tutors that use
vaccine discovery
virtual and
augmented reality
Space
Smart Cities
Example:
Examples:
Enabling extended and flexible
Autonomous
Improving safety
Connected smart
space exploration
spacecraft and
sensors
Enabling intelligent
smart habitats
infrastructure
Smart roads and
bridges
Optimizing complex
transportation hubs
Coordinating
movement of people
Sustaining the
and goods
environment
Supporting low-carbon
energy systems
Artificial Intelligence in Context - Endnotes
1 The John S. McCain National Defense Authorization Act for Fiscal Year 2019 includes the following
definition to guide the Commission’s work: 1. Any artificial system that performs tasks under varying
and unpredictable circumstances without significant human oversight, or that can learn from
experience and improve performance when exposed to data sets. 2. An artificial system developed
in computer software, physical hardware, or other context that solves tasks requiring human-like
perception, cognition, planning, learning, communication, or physical action. 3. An artificial system
designed to think or act like a human, including cognitive architectures and neural networks. 4. A
set of techniques, including machine learning that is designed to approximate a cognitive task. 5.
An artificial system designed to act rationally, including an intelligent software agent or embodied
robot that achieves goals using perception, planning, reasoning, learning, communicating, decision-
making, and acting. See Pub. L. 115-232, 132 Stat. 1636, 1965 (2018).
2 Andrew W. Moore, et al., The AI Stack: A Blueprint for Developing and Deploying Artificial
Intelligence, Proc. SPIE 10635, Ground/Air Multisensor Interoperability, Integration, and Networking
for Persistent ISR IX, 106350C (May 4, 2018), https://doi.org/10.1117/12.2309483; see also Dave
Martinez, et al., Artificial Intelligence: Short History, Present Developments, and Future Outlook, MIT
Lincoln Laboratory at 27 (Jan. 2019), https://www.ll.mit.edu/media/9526.
3 Note that model-based AI requires data for the manual construction of the model(s). Typically, this
involves less data than statistical machine learning, but more human effort.
4 Saleema Amershi, et al., Software Engineering for Machine Learning: A Case Study, ICSE-SEIP
’19 Proceedings of the 41st International Conference on Software Engineering at 291-300 (2019),
Engineering-for-Machine-Learning-A-Case-Study; D. Sculley, et al., Machine Learning: The High
Interest Credit Card of Technical Debt, SE4ML: Software Engineering for Machine Learning (NIPS
2014 Workshop), https://ai.google/research/pubs/pub43146.
www.darpa.mil/attachments/AIFull.pdf.
6 The limitations of today’s statistical machine learning, as an example, include the vulnerability of
unknowingly learning and amplifying biases in the training data; the fact that they are often complex
models composed of a very large number of learned parameters, making them opaque and difficult
to interpret; the fact that they are trained to solve narrow tasks and lack generalization to other related
problems (such as when operationally encountered data fundamentally changes characteristic from
the training data); and the fact that they require large amounts of labeled training data.
7 Andrew Moore, When AI Becomes an Everyday Technology, Harvard Business Review (June 7,
2019), https://hbr.org/2019/06/when-ai-becomes-an-everyday-technology.
8 Tom B. Brown, et al., Language Models are Few-Shot Learners, arXiv (July 22, 2020), https://arxiv.
org/abs/2005.14165.
9 Zachary Arnold, et al., Tracking AI Investment: Initial Findings from the Private Markets, Center for
Security and Emerging Technology (Sept. 2020), https://cset.georgetown.edu/wp-content/uploads/
CSET-Tracking-AI-Investment.pdf.
10 According to UNESCO, global spending on R&D has reached a record high of almost US$1.7
trillion. See How Much Does Your Country Invest in R&D?, UNESCO Institute for Statistics (last
accessed Jan. 7, 2021), http://uis.unesco.org/apps/visualisations/research-and-development-
spending/.
11 For a recent debate from AI experts see AI DEBATE 2: Moving AI Forward: An Interdisciplinary
Approach, Montreal Artificial Intelligence (Dec. 23, 2020), https://montrealartificialintelligence.com/
aidebate2.html.
Artificial Intelligence in Context - Endnotes
12 See e.g., Bryan Wilder, et al., Learning to Complement Humans, International Joint Conferences
on Artificial Intelligence Organization (2020), https://doi.org/10.24963/ijcai.2020/212; Ece Kamar, et
al., Combining Human and Machine Intelligence in Large-scale Crowdsourcing, Proceedings of the
11th International Conference on Autonomous Agents and Multiagent Systems (A AMAS 2012) (June
4-8, 2012), https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/galaxyZoo.pdf;
Ramya Ramakrishnan, et al., Overcoming Blind Spots in the Real World: Leveraging Complementary
Abilities for Joint Execution, Proceedings of the AAAI Conference on Artificial Intelligence (July
17, 2019), https://doi.org/10.1609/aaai.v33i01.33016137; Saleema Amershi, et al., Guidelines for
Human-AI Interaction, CHI ’19: Proceedings of the 2019 CHI Conference on Human Factors in
Computing Systems (May 2019), https://doi.org/10.1145/3290605.3300233; Eric Horvitz, Reflections
on Challenges and Promises of Mixed-Initiative Interaction, AI Magazine (June 15, 2007), https://doi.
org/10.1609/aimag.v28i2.2036.
13 A goal of these new methods is to eliminate the need for many complex calculations that make
traditional training very slow. Vincent Dutordoir, Sparse Gaussian Processes with Spherical Harmonic
Features, arXiv (June 30, 2020), https://arxiv.org/abs/2006.16649.
14 For a discussion on hybrid intelligence architectures that combine symbolic manipulation with deep
learning see Gary Marcus, The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence,
arXiv at 14-19 (Feb. 19, 2020), https://arxiv.org/abs/2002.06177.
mitibmwatsonailab.mit.edu/category/neuro-symbolic-ai/.
16 Anuj Karpatne, et al., Physics-guided Neural Networks (PGNN): An Application in Lake Temperature
Modeling, Association for Computing Machinery’s Special Interest Group on Knowledge Discovery
and Data Mining (ACM SIGKDD) 2018 (Feb. 20, 2018), https://arxiv.org/pdf/1710.11431.pdf.
17 Rajat Raina, et al., Self-Taught Learning: Transfer Learning from Unlabeled Data, Proceedings
of the 24th International Conference of Machine Learning (June 2007), https://dl.acm.org/doi/
abs/10.1145/1273496.1273592; Dr. Bruce Draper, Learning with Less Labeling, DARPA (last accessed
Dec. 19, 2020), https://www.darpa.mil/program/learning-with-less-labeling.
18 Rahul Dixit, et al., Artificial Intelligence and Machine Learning in Sparse/Inaccurate Data Situations,
IEEE (Aug. 21, 2020), https://ieeexplore.ieee.org/document/9172612.
19 See Cem Dilmegani, The Ultimate Guide to Synthetic Data in 2021, AI Multiple (Jan. 12, 2021),
https://research.aimultiple.com/synthetic-data/.
promise-synthetic-data-1016.
technologyreview.com/10-breakthrough-technologies/2020/#tiny-ai.
22 The World’s Largest and Most Complete Common-Sense Knowledge Base, Cycorp (last accessed
Dec. 19, 2020), https://www.cyc.com/the-cyc-platform/the-knowledge-base; John Pavlus, Common
org/common-sense-comes-to-computers-20200430/; Antoine Bosselut, et al., COMET: Commonsense
Transformers for Automatic Knowledge Graph Construction, Proceedings of the 57th Annual Meeting
of the Association for Computational Linguistics (2019), https://homes.cs.washington.edu/~msap/
pdfs/bosselut2019comet.pdf.
sciencedaily.com/releases/2020/08/200827105937.htm.
24 See Benedict Neo, Top 4 AI companies leading in the race towards Artificial General Intelligence,
Towards Data Science (April 13, 2020), https://towardsdatascience.com/four-ai-companies-on-
the-bleeding-edge-of-artificial-general-intelligence-b17227a0b64a; Srishti Deoras, 9 Companies
analyticsindiamag.com/9-companies-doing-exceptional-work-in-agi-just-like-openai/.
25 In the Key Considerations, the Commission describes practices, technologies and operational policies
to develop and field systems that align with key values. Importantly, agencies must consider values
as (1) embodied in choices about engineering trade-offs and (2) explicitly represented in the goals
and utility functions of an AI system. See Key Considerations for Responsible Development & Fielding
of Artificial Intelligence, NSCAI
(July
Key-Considerations-for-Responsible-Development-Fielding-of-AI.pdf. In addition to actions and
investments needed now (see the Blueprint for Action associated with Chapter 7 of this report), the Key
Considerations include policies and practices that should be updated to reflect new AI considerations
as the technology evolves.
26 This will require R&D as noted in the Key Considerations and Chapter 7 of this report. It will also
require continued investment in system architectures to limit the consequences of system failure, to
monitor AI performance as systems run to assess if they are performing as intended, and to overcome
S&T gaps for audit and oversight. For more information, see Chapters 7 and 8 of this report.
27 As noted in Chapter 8 and its Blueprint for Action, this will require, for instance, sustained assessment
and recurring guidance on privacy and civil liberties implications of AI applications and emerging
technologies. Disallowed outcomes and policy guidance will need to be updated over time as community
norms and technical capabilities change. Further, the Key Considerations note that engineering
practices will need to assess general feasibility and compliance with disallowed outcomes expressed
in policy, as well as the demonstrated technical maturity of specific candidate AI technologies. See
Key Considerations for Responsible Development & Fielding of Artificial Intelligence, NSCAI (July
2020),
Development-Fielding-of-AI.pdf.
28 Envisioned translation systems will leverage feedback to the system by actively correcting
translation and recognition errors the software makes, improving performance as the interaction
between translating parties goes on. A major goal is rapid deployment to new languages that have
not been seen before. For example, Carnegie Mellon’s DIPLOMAT Project makes interactive speech
translation possible through a new architecture called Multi Engine Machine Translation (MEMT).
DIPLOMAT gives users the ability to provide translation corrections to support rapid development to
new languages that have not been seen before. Robert Frederking, Interactive Speech Translation in
the DIPLOMAT Project, Carnegie Mellon University Language Technologies Institute (last accessed
Dec. 19, 2020), http://www.cs.cmu.edu/~air/papers/acl97-workshop.pdf.
29 Modeling and simulation can also help prepare for effective pandemic supply chain
responses orchestrated by the government. Madhav Marathe, High Performance Simulations to
pdf/10.1145/3384441.3395993.
30 Edgybees (last accessed Dec. 19, 2020), https://edgybees.com/.
31 Department of Energy Announces the First Five Consortium, U.S. Department of Energy (Aug. 18,
2020), https://www.energy.gov/articles/department-energy-announces-first-five-consortium.
PART ONE
PART ONE
p
41
THE NATIONAL SECURITY COMMISSION ON ARTIFICIAL INTELLIGENCE
PART I: DEFENDING AMERICA IN THE AI ERA
41
Chapter 1: Emerging Threats in the AI Era
43
Chapter 2: Foundations of Future Defense
59
Chapter 3: AI and Warfare
75
Chapter 4: Autonomous Weapon Systems and Risks
89
Associated with AI-Enabled Warfare
Chapter 5: AI and the Future of National Intelligence
107
Chapter 6: Technical Talent in Government
119
Chapter 7: Establishing Justified Confidence in AI Systems
131
Chapter 8: Upholding Democratic Values: Privacy, Civil Liberties,
141
and Civil Rights in Uses of AI for National Security
p
42
CHAPTER 1
Chapter 1: Emerging
Threats in the AI Era
p
43
EMERGING THREATS IN THE AI ERA
Societal Level of Conflict
Data Harvesting
AI-Enabled
and Targeting of
Information
Individuals
Operations
Adversarial AI
Accelerated
Cyber Attacks
AI-Enabled
Biotechnology
p
44
CHAPTER 1
The U.S. government is not prepared to defend the United States in the
coming artificial intelligence (AI) era. AI applications are transforming
existing threats, creating new classes of threats, and further emboldening
state and non-state adversaries to exploit vulnerabilities in our open
society.1 AI systems will extend the range and reach of adversaries into
the United States just as the missile age and terrorism brought threats
closer to home. Because of AI, adversaries will be able to act with
micro-precision, but at macro-scale and with greater speed. They will
use AI to enhance cyber attacks and digital disinformation campaigns
and to target individuals in new ways. AI will also help create precisely
engineered biological agents. And adversaries will manipulate the AI
systems we will rely upon.
Current Threats
New Threats
Threats TO AI Stacks
Future Threats
How AI is
Advanced BY AI Systems
FROM AI Systems
Themselves
VIA AI Systems
Transforming the
Threat Landscape
AI transforms existing
AI creates new
AI itself is also a new
Examples of potential
range and reach of threats
threat phenomena
attack surface
threats to keep in view
Self-replicating
Deepfakes and
AI attack involves the
Rapid machine-to-
AI-generated
computational
whole “AI stack”.
machine escalation
malware
propaganda
Examples include:
via automated C2
Improved
Micro-targeting:
Model inversion
AI-enabled human
and autonomous
AI-fused data for
Training data
augmentation by
disinformation
targeting or blackmail
manipulation
peer competitors
campaigns
“Data lake”
AI swarms and
poisoning
Proliferation of
AI-engineered and
nano-swarms
simple lethal
targeted pathogens
autonomous weapons
to terrorists
AI technologies exacerbate two existing national security challenges:
• First, digital dependence in all walks of life increases vulnerabilities to cyber intrusion
across every segment of our society: corporations, universities, government, private
organizations, and the homes of individual citizens. In parallel, new sensors have
flooded the modern world. The internet of things (IoT), cars, phones, homes, and social
media platforms collect streams of data, which can then be fed into AI systems that can
identify, target, and manipulate or coerce our citizens.2
• Second, state and non-state adversaries are challenging the United States below
the threshold of direct military confrontation by using cyber attacks, espionage,
psychological and political warfare, and financial instruments. Adversaries do not
need AI to conduct widespread cyber attacks, exfiltrate troves of sensitive data about
American citizens, interfere in our elections, or bombard us with malign information on
p
45
EMERGING THREATS IN THE AI ERA
digital platforms. However, AI is starting to change these attacks in kind and in degree,
creating new threats to the U.S. economy, critical infrastructure, and societal cohesion.3
Moreover, these AI-enabled capabilities will be used across the spectrum of conflict.
They will be used as tools of first resort in non-military conflicts, as a prelude to military
actions, or in concert with military actions in war.
Americans are waking to some of the privacy implications of their digital dependence
and the potential threats from AI-powered malign information, like deep fakes. However,
debate in the United States has not yet accounted for the full scope and danger of the AI-
enabled threats and the overall security risks to the AI systems all around us. The prospect
of adversaries using machine learning (ML), planning, and optimization to create systems
to manipulate citizens’ beliefs and behavior in undetectable ways is a gathering storm.4
Most concerning is the prospect that adversaries will use AI to create weapons of mass
influence to use as leverage during future wars, in which every citizen and organization
becomes a potential target.
“The prospect of adversaries
using machine learning,
planning, and optimization to
create systems to manipulate
citizens’ beliefs and behavior in
undetectable ways is a gathering
storm. Most concerning is the
prospect that adversaries will
use AI to create weapons of mass
influence to use as leverage
during future wars, in which
every citizen and organization
becomes a potential target.”
p
46
CHAPTER 1
Societal
Level Impact.
The rest of this chapter discusses five AI-related threats that already have
been, or soon will be, developed and used against the United States.
1. AI-Enabled Information Operations.
AI and associated technologies will increase the magnitude, precision, and persistence
of adversarial information operations. AI exacerbates the problem of malign information in
three ways:
• Message. AI can produce original text-based content and manipulate images, audio,
and video, including through generative adversarial network (GAN)-enabled and
reinforcement learning (RL) deep fakes that will be very difficult to distinguish from
authentic messages.
p
47
EMERGING THREATS IN THE AI ERA
• Audience. AI can construct profiles of individuals’ preferences, behaviors, and beliefs to
target specific audiences with specific messages.
• Medium. AI can be embedded within platforms, such as through ranking algorithms, to
proliferate malign information.
AI-enabled malign information campaigns will not just send one powerful message to 1
million people, like 20th century propaganda. They also will send a million individualized
messages—configured on the basis of a detailed understanding of the targets’ digital lives,
emotional states, and social networks.5 Rival states are already using AI-powered malign
information. For example, according to Taiwan authorities, China’s government tested its
AI-powered malign information capacities during the 2020 Taiwan elections.6 A National
Basketball Association general manager was harassed on social media for supporting
protesters in Hong Kong, in an effort that may have involved autonomous bots.7 Other
techniques rely on AI-generated fake personas.8 The control and manipulation of digital
information has become central to the Kremlin’s strategy, including in efforts to undermine
the integrity of the democratic process in the United States and elsewhere.9
In the United States, the private sector has taken the leading role in combating foreign
malign information. Social media companies in particular have extensive operations to track
and manage information on their platforms. But coordination between the government and
the social media firms remains ad hoc. We need a more integrated public-private response
to the problem of foreign-generated disinformation. Moreover, the government needs to
devote greater attention and resources to the technical challenges of detection, attribution,
and media authentication. The government should:
Create a Joint Interagency Task Force and Operations Center. Congress has authorized
Recommendation
a Foreign Malign Influence Response Center to be established within the Office of the
Director of National Intelligence (ODNI).10 The government should use this authority to
create a technologically advanced, 24-hour task force and operations center to lead and
integrate government efforts to counter foreign-sourced malign information. It would survey
the landscape of relevant public and private actors, coordinate among them, and act in
real time to counter foreign information campaigns. To expose, attribute, and respond
effectively, the center must be equipped with modern AI-enabled digital tools and staff
with specialized expertise.
Fund the Defense Advanced Research Projects Agency (DARPA) to coordinate multiple
Recommendation
research programs to detect, attribute, and disrupt AI-enabled malign information
campaigns and to authenticate the provenance of digital media. Additional funding would
amplify ongoing DARPA research programs to detect synthetic media and expand its
efforts into attributing and disrupting malign information campaigns.11 However promising
some of these detection technologies may prove to be individually, funding to develop
alternative technologies to authenticate the provenance of the digital media will provide
a more technologically robust means to prevent the impersonation of trusted sources
of information.12 DARPA should pursue these programs and help transition all of these
p
48
CHAPTER 1
technologies and applications to government departments and agencies, in order to assist
with detecting, attributing, and disrupting malign information campaigns in real time.
Create a task force to study the use of AI and complementary technologies, including
Recommendation
the development and deployment of standards and technologies, for certifying content
authenticity and provenance. The White House Office of Science and Technology
Policy should take the lead in creating this task force. In response to the challenges
of misinformation, efforts are underway to develop standards and pipelines aimed at
certifying the authenticity and provenance of audiovisual content.13 These efforts make
use of technologies, including encryption and fragile watermarking, to secure and track
the expected transformations of content via production and transmission pipelines.
These efforts offer the opportunity to mitigate malign information campaigns that seek to
corrupt or spoof highly trusted sources of information across our digital ecosystem. This
technology area is ripe for public-private partnership. Several private organizations are
already forming to fight disinformation efforts in this realm.14
2. Data Harvesting and Targeting of Individuals.
“Potential adversaries will
recognize what every advertiser
and social media company
knows: AI is a powerful targeting
tool.”
Data security is a national security problem. “Ad-tech” has become “natsec-tech.” Potential
adversaries will recognize what every advertiser and social media company knows: AI
is a powerful targeting tool. Just as AI-powered analytics transformed the relationship
between companies and consumers, now it is transforming the relationship between
governments and individuals. The broad circulation of personal data drives commercial
innovation but also creates vulnerabilities.15 We fear that adversaries’ systematic efforts
to harvest data on U.S. companies, individuals, and the government is about more than
traditional espionage.16 Adversaries will combine widely available commercial data with
data acquired illicitly—as in the 2015 Office of Personnel Management hack—to track,
manipulate, and coerce individuals.17 The reach of tools that China, for instance, uses
p
49
EMERGING THREATS IN THE AI ERA
to monitor, control, and coerce its own citizens—big data analytics, surveillance, and
propaganda—can be extended beyond its borders and directed at foreigners.18 Without
adequate data protection, AI makes it harder for anyone to hide his or her financial situation,
patterns of daily life, relationships, health, and even emotions. Personal and commercial
vulnerabilities become national security weaknesses as adversaries map individuals,
networks, and social fissures in society; predict responses to different stimuli; and model
how best to manipulate behavior or cause harm. The rise and spread of these techniques
represent a major counterintelligence challenge.19
For the government to treat the data of its citizens and businesses as a national security asset,
substantial changes are required in the way we think about data security and in our policies
and laws to strengthen it. We need to identify categories and combinations of personal
and commercial data that are most sensitive. Early efforts to limit foreign adversaries’ data
harvesting—such as the government’s decision to force a Chinese company to relinquish
ownership of a popular dating application for fear of what a hostile adversary could do with
sensitive private data20—represent important initial steps. However, the government lacks
a broad approach with clear policies, criteria, or authorities to confront this multifaceted
problem. The government should:
Develop policies that treat data security as national security, including in these areas:
Recommendation
• First, from a technical standpoint, the government must ensure that a security
development lifecycle approach is in place for its own AI systems (including
commercial systems it acquires), which should include a focus on potential privacy
attacks.21 Red teaming must include privacy expertise. Government databases should
be federated and anonymized whenever possible, and personal data retained no longer
than is necessary, in order to make it more difficult for adversaries to utilize information
for malicious purposes.
• Second, the government should ensure that data privacy and security are priority
considerations as part of larger efforts to strengthen foreign investment screening and
supply chain intelligence and risk management.22
• Third, national efforts to legislate and regulate data protection and privacy must
integrate national security considerations, such as limiting the ability of hostile
foreign actors to acquire sensitive data on Americans on the commercial market.23
3. Accelerated Cyber Attacks.
Malware in the AI era will be able to mutate into thousands of different forms once it is lodged
on a computer system. Such mutating polymorphic malware already accounts for more than
90% of malicious executable files.24 Deep RL tools can already find vulnerabilities, conceal
malware, and attack selectively.25 While it is uncertain which methods will dominate, there
is a clear path for U.S. adversaries to transform the effectiveness of cyber attack and
espionage campaigns with an ensemble of new and old algorithmic means to automate,
optimize, and inform attacks.26 This goes beyond AI-enhanced malware. Machine learning
has current and potential applications across all the phases of cyber attack campaigns
p
50
CHAPTER 1
“Machine learning has current
and potential applications across
all the phases of cyber attack
campaigns ...”
and will change the nature of cyber warfare and cyber crime.27 The expanding application
of existing AI cyber capabilities will make cyber attacks more precise and tailored,
further accelerate and automate cyber warfare, enable stealthier and more persistent
cyberweapons, and make cyber campaigns more effective on a larger scale.
U.S. defenses have proven incapable of handling even more elementary cyber challenges.
Vulnerabilities remain open in outdated infrastructure and medical devices, while new
vulnerabilities are proliferating in 5G networks, billions of IoT devices, and in software supply
chains.28 The multibillion-dollar global damage caused by Russia’s 2017 NotPetya attack
concretely demonstrates the power of even basic automated malware, the risk tolerance
of capable state actors, and the consequences of such capabilities proliferating.29 Though
defensive applications of AI bring the promise to improve our national cyber defenses,
AI can’t defend inherently vulnerable digital infrastructure. To address the present
threat, Congress must continue implementing the Cyberspace Solarium Commission’s
recommendations.30 With this foundation for cyber defense, the U.S. can prepare for
expanding threats via testing and building the instrumented infrastructure required for AI-
enabled cyber defenses, establishing better incentives for security, properly organizing to
meet the challenge, and keeping attackers off balance. Pervasive cyber-enabled espionage
and attacks on U.S. computer networks and critical infrastructure will continue—and will
become more damaging with AI—unless urgent federal action is taken. The government
should:
Develop and deploy AI-enabled defenses against cyber attacks. National security
Recommendation
agencies need to acquire the sensors and instrumentation needed to train AI systems
to detect and respond to threats on their networks. AI-enabled cyber defenses will also
need large-scale, instrumented, and realistic testing, and they must be robust enough
to withstand adversarial attacks. The defenses should be employed to expand machine
speed information sharing, behavior-based anomaly detection, and malware mitigation
across government networks. To capitalize on these capabilities, the government should
accelerate the establishment of a Joint Cyber Planning and Operations Center, modeled
after the National Counterterrorism Center.31 The Center would serve as a centralized cyber
intelligence sharing and collaboration unit with multi-agency jurisdiction and authorities to
investigate threats, proactively support defensive mitigations, and coordinate responses.
p
51
EMERGING THREATS IN THE AI ERA
4. Adversarial AI.
AI systems represent a new target for attack. While we are on the front edge of this
phenomenon, commercial firms and researchers have documented attacks that involve
evasion, data poisoning, model replication, and exploiting traditional software flaws to
deceive, manipulate, compromise, and render AI systems ineffective.32 This threat is related
to, but distinct from, traditional cyber activities, because AI systems will be vulnerable to
adversarial attacks from any domain where AI augments action—civilian or military.33 Given
the reliance of AI systems on large data sets and algorithms, even small manipulations of
these data sets or algorithms can lead to consequential changes for how AI systems operate.
The threat is not hypothetical: adversarial attacks are happening and already impacting
commercial ML systems.34 With rare exceptions, the idea of protecting AI systems has
been an afterthought in engineering and fielding AI systems, with inadequate investment
in research and development.35 Only three of 28 organizations recently surveyed have “the
right tools in place to secure their ML systems.”36 There has not yet been a uniform effort
to integrate AI assurance across the entire U.S. national security enterprise. To improve AI
“assurance,” the government should:
Create a National AI Assurance Framework. All government agencies will need to develop
Recommendation
and apply an adversarial ML threat framework to address how key AI systems could be
attacked and should be defended. An analytical framework can help to categorize threats to
government AI systems and assist analysts with detecting, responding to, and remediating
threats and vulnerabilities.37
Create dedicated red teams for adversarial testing. Red teams should assume an offensive
Recommendation
posture, trying to break systems and make them violate rules for appropriate behavior.
Because of the scarcity of required expertise and experience for AI red teams, DoD and
ODNI should consider establishing government-wide communities of AI red-teaming
capabilities that could be applied to multiple AI developments.38
5. AI-Enabled Biotechnology.
Biology is now programmable. New technologies such as the gene editing tool CRISPR
ushered in an era where humans are able to edit DNA. Combined with massive computing
power and AI, innovations in biotechnology may provide novel solutions for mankind’s most
vexing challenges, including in health, food production, and environmental sustainability.
Like other powerful technologies, however, applications of biotechnology can have a dark
side. The COVID-19 pandemic reminded the world of the dangers of a highly contagious
pathogen. AI may enable a pathogen to be specifically engineered for lethality or to target
a genetic profile—the ultimate range and reach weapon. Also, AI, when applied to biology,
could optimize for the physiological enhancement of human beings, including intelligence
and physical attributes. To the extent that brain waves can be represented as a machine
vision challenge for AI, the mysteries of the brain may be unlocked and programmed.
p
52
CHAPTER 1
Individuals, societies, and states will have different moral and ethical views and accept
different degrees of risk in the name of progress, and U.S. competitors are comparatively
likely to take more risk-tolerant actions and conform less rigidly to bioethical norms and
standards. China understands the tremendous upside associated with leading the bio
revolution. Massive genomic data sets at places like BGI Group (formerly known as the
Beijing Genomics Institute), coupled with China’s now-global genetic data collection
platform and “all-of-nation” approach to AI, will make them a formidable competitor in the
bio realm.39 BGI may be serving, wittingly or unwittingly, as a global collection mechanism
for Chinese government genetic databases, providing China with greater raw numbers and
diversity of human genome samples as well as access to sensitive personal information
about key individuals around the world.40 The United States cannot afford to look back in 10
years and be “surprised” by the biotechnology equivalent of Huawei. Additionally, Russia’s
long-standing disregard for scientific norms and bioethical principles, demonstrated by its
development and employment of novel nerve agents such as Novichok for assassination
attempts and U.S. government concerns over Russia’s compliance with the Biological
Weapons Convention, could presage a willingness to utilize advanced biotechnology
abilities for nefarious purposes.41 The government should:
Increase the profile of biosecurity and biotechnology issues within U.S. national security
Recommendation
agencies. Given how AI will substantially increase the rate of technical advancement
in biotechnology, the government should update the National Biodefense Strategy to
include a wider vision of biological threats, such as human enhancement, exploitation of
genetic data for malicious ends, and ways U.S. competitors could utilize biotechnology
or biodata advantages for novel purposes. Additionally, U.S. officials should warn of the
dangers associated with foreign actors obtaining personal genetic information, specifically
highlighting concerns about the links between BGI and the Chinese government.42
p
53
EMERGING THREATS IN THE AI ERA
Chapter 1 - Endnotes
1 A threat can be understood as an adversary capability paired with a vulnerability that can create a
harmful consequence. See Terry L. Deibel, Foreign Affairs Strategy: Logic for American Statecraft,
Cambridge University Press at 142-150 (2007). Threats can be graded further by their seriousness,
likelihood, imminence, and tractability.
2 Stuart A. Thompson & Charlie Warzel, Twelve Million Phones, One Dataset, Zero Privacy, New York
Times (Dec. 19, 2019), https://www.nytimes.com/interactive/2019/12/19/opinion/location-tracking-
cell-phone.html. For example, the internet of things (IoT) and AI-powered applications can turn your
new robotic vacuum into a listening device. See Sriram Sami, et al., Spying with Your Robot Vacuum
Cleaner: Eavesdropping via Lidar Sensors, Proceedings of the 18th Conference on Embedded
Networked Sensor Systems (Nov. 2020), https://dl.acm.org/doi/10.1145/3384419.3430781.
3 This is in some ways analogous to what Cold War strategists called “counter-value targeting.” See
Lawrence Freedman, The Evolution of Nuclear Strategy, Palgrave Macmillan Vol. 20 at 119-122 (1989).
In the realm of nuclear strategy, this was also known as counter-city or counter-economy targeting.
4 Some observers have used the concept of “sharp power” to describe such efforts to wield influence
in open societies. These uses of power are sharp “in the sense that [authoritarian states aim to]
pierce, penetrate, or perforate the information environments in the targeted countries.” Sharp Power:
www.ned.org/sharp-power-rising-authoritarian-influence-forum-report/. See also Testimony of Dr.
Eric Horvitz, Microsoft, before the U.S. Senate Committee on Commerce, Science, & Transportation,
Subcommittee on Space, Science, & Competitiveness, Hearing on the Dawn of Artificial Intelligence at
13 (Nov. 30, 2016), http://erichorvitz.com/Senate_Testimony_Eric_Horvitz.pdf.
5 Some have characterized AI-driven information operations as “computational propaganda.” See Matt
Chessen, The MADCOM Future: How Artificial Intelligence Will Enhance Computational Propaganda,
Reprogram Human Culture, and Threaten Democracy… and What Can Be Done About It, Atlantic
Council (Sept. 2017), https://www.atlanticcouncil.org/wp-content/uploads/2017/09/The_MADCOM_
Future_RW_0926.pdf.
6 Philip Sherwell, China Uses Taiwan for AI Target Practice to Influence Elections, The Australian (Jan.
5, 2020), https://www.theaustralian.com.au/world/the-times/china-uses-taiwan-for-ai-target-practice-
to-influence-elections/news-story/57499d2650d4d359a3857688d416d1e5.
7 Ben Cohen, et al., How One Tweet Turned Pro-China Trolls Against the NBA, Wall Street Journal
nba-11571238943. On automated bots, see, e.g., Sarah Kreps & Miles McCain, Not Your Father’s
fathers-bots.
8 James Vincent, An Online Propaganda Campaign Used AI-Generated Headshots to Create Fake
headshots-profile-pictures-fake-journalists-daily-beast-investigation.
9 For recent studies on technical aspects of Russia’s interference in the 2016 election, see Alexander
Spangher, et al., Characterizing Search-Engine Traffic to Internet Research Agency Web Properties,
Web Conference (2020), https://www.microsoft.com/en-us/research/publication/characterizing-
search-engine-traffic-to-internet-research-agency-web-properties/; Ryan Boyd, et al., Characterizing
the Internet Research Agency’s Social Media Operations During the 2016 U.S. Presidential Election
Using Linguistic Analyses, PsyArXiv Preprints (2018), https://psyarxiv.com/ajh2q/. See also Alina
Polyakova, Weapons of the Weak: Russian and AI-driven Asymmetric Warfare, Brookings Institution
asymmetric-warfare/.
10 See Pub. L. 116-92, National Defense Authorization Act for Fiscal Year 2020, 133 Stat. 1198, 2129
(2019).
11 These include the Media Forensics (MediFor) and Semantic Forensics (SemaFor) programs. See Dr.
www.darpa.mil/program/semantic-forensics.
p
54
CHAPTER 1
Chapter 1 - Endnotes
12 See, e.g., Paul England, et al., AMP: Authentication of Media via Provenance, arXiv (June 20, 2020),
https://arxiv.org/abs/2001.07886.
13 See, e.g., Paul England, et al., AMP: Authentication of Media via Provenance, arXiv.
14 See Creating the Standard for Digital Content Attribution, Content Authenticity Initiative, https://
contentauthenticity.org/; and Project Origin: Protecting Trusted Media, Project Origin, https://www.
originproject.info/about.
15 Robert Williams has described how policy makers face an “innovation-security conundrum,” one
aspect of which is “the worry that data privacy and national security are increasingly interconnected.
Data (and data networks) can be exploited in ways that threaten security, but they also form the
lifeblood of technological innovation on which both economic growth and national security depend.”
Robert D. Williams, Crafting a Multilateral Technology and Cybersecurity Policy, Brookings at 1 (Nov.
2020), https://www.brookings.edu/wp-content/uploads/2020/11/Robert-D-Williams.pdf.
16 Ellen Nakashima, With a Series of Major Hacks, China Builds a Database on Americans, Washington
Post (June 5, 2015), https://www.washingtonpost.com/world/national-security/in-a-series-of-hacks-
china-appears-to-building-a-database-on-americans/2015/06/05/d2af51fa-0ba3-11e5-95fd-
d580f1c5d44e_story.html.
17 Another example of an adversary acquiring significant data on U.S. individuals is the hack of the
credit reporting agency Equifax. Press Release, Department of Justice, Chinese Military Personnel
Charged with Computer Fraud, Economic Espionage and Wire Fraud for Hacking into Credit Reporting
computer-fraud-economic-espionage-and-wire-fraud-hacking; Aruna Viswanatha, et al., Four
Members of China’s Military Indicted Over Massive Equifax Breach, Wall Street Journal (Feb. 11,
breach-11581346824.
18 See, e.g., Drew Harwell & Eva Dou, Huawei Tested AI Software That Could Recognize
washingtonpost.com/technology/2020/12/08/huawei-tested-ai-software-that-could-recognize-uighur-
minorities-alert-police-report-says/; Hugh Harsono, China’s Surveillance Technology Is Keeping Tabs
chinas-surveillance-technology-is-keeping-tabs-on-populations-around-the-world/.
19 See James Baker, Counterintelligence Implications of Artificial Intelligence--Part III, Lawfare (Oct.
10, 2018), https://www.lawfareblog.com/counterintelligence-implications-artificial-intelligence-part-iii.
20 Yuan Yang & James Fontanella-Khan, Grindr Sold by Chinese Owner After US National Security
fa40a0d65a98.
21 On privacy attacks, see Maria Rigaki & Sebastian Garcia, A Survey of Privacy Attacks in Machine
Learning, arXiv (July 15, 2020), https://arxiv.org/abs/2007.07646.
22 The Committee on Foreign Investment in the United States (CFIUS) has the authority to review
transactions that include “sensitive personal data of United States citizens that may be exploited in
a manner that threatens national security.” For background, see Laura Jehl, Spotlight on Sensitive
Personal Data As Foreign Investment Rules Take Force, The National Law Review (Feb. 18, 2020),
force. The National Counterintelligence and Security Center (NCSC) includes “sensitive government
data, and personally-identifiable information” in its conception of key supply chain risks. See Supply
tri-fold.pdf.
23 See, e.g., Graham Webster, App Bans Won’t Make U.S. Security Risks Disappear, MIT Technology
Review (Sept. 21, 2020), https://www.technologyreview.com/2020/09/21/1008620/wechat-tiktok-ban-
china-us-security-policy-opinion/.
p
55
EMERGING THREATS IN THE AI ERA
24 Nicholas Duran, et al., 2018 Webroot Threat Report, Webroot at 6 (2018), https://www-cdn.webroot.
com/9315/2354/6488/2018-Webroot-Threat-Report_US-ONLINE.pdf.
25 Gary J. Saavedra, et al., A Review of Machine Learning Applications in Fuzzing, arXiv (Oct. 9, 2019),
https://arxiv.org/pdf/1906.11133.pdf; Isao Takaesu, Machine Learning Security: DeepExploit, GitHub
(Aug. 29, 2019), https://github.com/13o-bbr-bbq/machine_learning_security/tree/master/DeepExploit;
Marc Ph. Stoecklin, et al., DeepLocker: How AI Can Power a Stealthy New Breed of Malware, Security
Intelligence (Aug. 8, 2018), https://securityintelligence.com/deeplocker-how-ai-can-power-a-stealthy-
new-breed-of-malware/.
26 Implications of Artificial Intelligence for Cybersecurity: Proceedings of a Workshop, National
Academies of Sciences, Engineering, and Medicine (2019), https://doi.org/10.17226/25488; Nektaria
Kaloudi & Jingyue Li, The AI-Based Cyber Threat Landscape, ACM Computing Surveys at 1-34 (Feb.
2020), https://dl.acm.org/doi/abs/10.1145/3372823; Ben Buchanan, et al., Automating Cyber Attacks,
Center for Security and Emerging Technology (Nov. 2020), https://cset.georgetown.edu/research/
automating-cyber-attacks/; Dakota Cary & Daniel Cebul, Destructive Cyber Operations and Machine
edu/research/destructive-cyber-operations-and-machine-learning/.
27 Implications of Artificial Intelligence for Cybersecurity: Proceedings of a Workshop, National
Academies of Sciences, Engineering, and Medicine (2019), https://doi.org/10.17226/25488.
28 The recent SolarWinds attack demonstrates deep vulnerabilities in our software supply chains.
See Joint Statement by the Federal Bureau of Investigation (FBI), the Cybersecurity and Infrastructure
Security Agency (CISA), and the Office of the Director of National Intelligence (ODNI), Office of the
Director of National Intelligence (Dec. 16, 2020), https://www.dni.gov/index.php/newsroom/press-
releases/item/2175-joint-statement-by-the-federal-bureau-of-investigation-fbi-the-cybersecurity-and-
infrastructure-security-agency-cisa-and-the-office-of-the-director-of-national-intelligence-odni.
29 Implications of Artificial Intelligence for Cybersecurity: Proceedings of a Workshop, National
Academies of Sciences, Engineering, and Medicine (2019), https://doi.org/10.17226/25488; Ben
Buchanan, et al., Automating Cyber Attacks, Center for Security and Emerging Technology at 3 (Nov.
2020), https://cset.georgetown.edu/research/automating-cyber-attacks/.
30 Cyberspace Solarium Commission Report, U.S. Cyberspace Solarium Commission (March 2020),
https://www.solarium.gov/report.
31 See recommendation 5.4 in Cyberspace Solarium Commission Report, U.S. Cyberspace Solarium
Commission at 87 (March 2020), https://www.solarium.gov/report.
32 Adversarial AI Threat Matrix: Case Studies, GitHub (last accessed Jan. 10, 2021), https://github.
com/mitre/advmlthreatmatrix/blob/master/pages/case-studies-page.md. For more on applications
of adversarial AI, see Naveed Akhtar & Ajmal Mian, Threat of Adversarial Attacks on Deep Learning
jsp?arnumber=8294186.
33 Adversarial AI is about what can be done to AI systems. The science of protecting and defending
AI applications against attacks is called “AI Assurance.” The science of attacking each technological
component of AI is called “Counter-AI.”
34 Ram Shankar Siva Kumar & Ann Johnson, Cyberattacks Against Machine Learning Systems Are
security/blog/2020/10/22/cyberattacks-against-machine-learning-systems-are-more-common-than-
you-think/.
35 It has been estimated that less than 1% of AI R&D funding is directed toward the security of AI
systems. See Nathan Strout, The Three Major Security Threats to AI, Center for Security and Emerging
Technology (Sept. 10, 2019), https://cset.georgetown.edu/article/the-three-major-security-threats-to-
ai/.
36 Ram Shankar Siva Kumar, et al., Adversarial Machine Learning—Industry Perspectives, arXiv at 2
(May 21, 2020), https://arxiv.org/pdf/2002.05646.pdf.
p
56
CHAPTER 1
Chapter 1 - Endnotes
37 There are various ongoing public and private efforts including, for instance, the MITRE-Microsoft
adversarial ML framework. See Ram Shankar Siva Kumar & Ann Johnson, Cyberattacks Against
Machine Learning Systems Are More Common Than You Think, Microsoft Security (Oct. 22, 2020),
are-more-common-than-you-think/; Adversarial AI Threat Matrix: Case Studies, MITRE (last accessed
Jan. 10, 2021), https://github.com/mitre/advmlthreatmatrix/blob/master/pages/case-studies-page.md.
38 For a similar recommendation, see Michèle Flournoy, et al., Building Trust Through Testing,
WestExec Advisors at 27 (Oct. 2020), https://cset.georgetown.edu/wp-content/uploads/Building-Trust-
Through-Testing.pdf. (Flournoy, et al., argue for “a national AI and ML red team as a central hub to
test against adversarial attacks, pulling together DoD operators and analysts, AI researchers, T&E
[Central Intelligence Agency (CIA), Defense Intelligence Agency (DIA), National Security Agency
(NSA)], and other IC components, as appropriate. This would be an independent red-teaming
organization that would have both the technical and intelligence expertise to mimic realistic adversary
attacks in a simulated operational environment.”)
39 BGI built and operates China National GeneBank, the Chinese government’s national genetic
database. It also is a major global supplier of COVID-19 testing, which potentially provides access
to large international genetic data sets; by June 30, 2020, it had supplied more than 35 million test
kits to 180 countries, including the United States, and built 58 testing labs in 18 countries. See Kirsty
Needham, Special Report: COVID Opens New Doors for China’s Gene Giant, Reuters (Aug. 5, 2020),
new-doors-for-chinas-gene-giant-idUSKCN2511CE.
p
57
EMERGING THREATS IN THE AI ERA
40 John Wertheim, China’s Push to Control Americans’ Health Care Future, 60 Minutes (Jan. 31,
10aab7d&linkId=110169507; Kirsty Needham, Special Report: COVID Opens New Doors for China’s
specialreport/special-report-covid-opens-new-doors-for-chinas-gene-giant-idUSKCN2511CE.
41 See Richard Pérez-Peña, What Is Novichok, the Russian Nerve Agent Tied to Navalny Poisoning?,
New York Times (Sept. 2, 2020), https://www.nytimes.com/2020/09/02/world/europe/novichok-skripal.
html; 2020 Adherence to and Compliance with Arms Control, Nonproliferation, and Disarmament
Agreements and Commitments (Compliance Report), U.S. Department of State at Pt. V (2020),
and-disarmament-agreements-and-commitments-compliance-report-2//index.html#_Toc43298166.
42 See Chapter 16 of this report for additional recommendations pertaining to the nexus of AI and
biotechnology.
p
58
CHAPTER 2
Chapter 2: Foundations
of Future Defense
p
59
FOUNDATIONS OF FUTURE DEFENSE
AI-Enabled Future Defense
Build the
Technical
Backbone
Accelerate
Train and Educate
Adoption of
Warfighters
Existing Digital
Technologies
Democratize AI
Invest in Next-
Development
Generation
Capabilities
p
60
//////////////////////