National Security Commission on Artificial Intelligence. Final Report

CHAPTER 2

The U.S. military has enjoyed military-technical superiority over all

potential adversaries since the end of the Cold War. Now, its technical

prowess is being challenged, especially by China and Russia. Senior

military leaders have warned that if current trend lines are not altered,

the U.S. military will lose its military-technical superiority in the coming

years.¹ Artificial intelligence (AI) is a key aspect of this challenge, as

both of our great power competitors believe they will be able to offset

our military advantage using AI-enabled systems and AI-enabled

autonomy. In the coming decades, the United States will win against

technically sophisticated adversaries only if it accelerates adoption of

AI-enabled sensors and systems for command and control, weapons,

and logistics.

The Department of Defense (DoD) must set an ambitious goal. By 2025, the foundations

for widespread integration of AI across DoD must be in place. Those foundations include a

common digital infrastructure that is accessible to internal AI development teams and critical

industry partners alike, a digitally literate workforce, and modern AI-enabled business practices

that improve efficiency. All are prerequisites to achieving a state of military AI readiness, which

is discussed in Chapter 3 of this report.

“By 2025, the foundations for

widespread integration of AI

across DoD must be in place.”

DoD lags far behind the commercial sector in integrating new and disruptive technologies such

as AI into its operations. Pockets of excellence started to emerge in 2017 when Project Maven

was launched with the aim to simplify work for intelligence analysts by recognizing objects

in video footage captured by drones and other platforms.² Other promising initiatives are

occurring in defense labs and agencies, and proof-of-concept demonstrations are ongoing in

service-level tests.³ However, visionary technologists and warfighters largely remain stymied

by antiquated technology, cumbersome processes, and incentive structures that are designed

for outdated or competing aims.⁴ Successes are usually based on workarounds--in spite of

the system.

FOUNDATIONS OF FUTURE DEFENSE

“... visionary technologists

and warfighters largely

remain stymied by antiquated

technology, cumbersome

processes, and incentive

structures that are designed for

outdated or competing aims.”

The obstacles to integrating AI are many. DoD has long been hardware-oriented toward ships,

planes, and tanks. It is now trying to make the leap to a software-intensive enterprise. Spending

remains concentrated on legacy systems designed for the industrial age and Cold War.⁵ Many

Departmental processes still rely too much on PowerPoint and manually driven work streams.

The data that is needed to fuel machine learning (ML) is currently stovepiped, messy, or often

discarded. Platforms are disconnected. Acquisition, development, and fielding practices

largely follow rigid, sequential processes, inhibiting early and continuous experimentation and

testing critical for AI. Even promising AI programs have not yet delivered as hoped and often

remain bound to proprietary software and data storage of commercial vendors. Steps such

as building the cloud infrastructure necessary to scale AI applications proceed slowly. Data-

sharing agreements and software updates that take hours or days in industry turn into months-

long delays. Service members at every level lack the technical education and experience to

employ AI.

Meanwhile, bureaucracy hinders partnerships with technology firms and critical efforts to

expand the National Security Innovation Base.⁶ The prospect of bureaucratic snarls deters

companies from working with DoD; it is economically irrational for many startups to even try.

Traditional defense companies will continue to play a central role in building and integrating large

systems for AI-enabled warfare.⁷ However, even these contractors, who have the resources and

expertise to navigate the system, face process and technical roadblocks that slow efforts to

build and integrate AI systems.

As a result, change will not be easy. It will require a Secretary of Defense who focuses the

Department on speeding the adoption of new technologies, and a dedicated Steering

Committee on Emerging Technology to drive implementation and align priorities between the

DoD and the Intelligence Community. The Secretary should direct action in five areas:

CHAPTER 2

Recommendation

1. Build the technical backbone. DoD should make foundational investments to support a

Department-wide technical infrastructure for ubiquitous development and fielding of AI. It took a

promising first step in 2020 with the issuance of a DoD Data Strategy.⁸ However, the Department

lacks the modern digital ecosystem, collaborative tools and environments, and broad on-

demand access to shared AI resources that it needs to integrate AI across the organization.⁹

The Department should avoid reinventing core infrastructure for each new AI-driven program

or capability, and it should look to leverage and interoperate with proven solutions from the

Intelligence Community (IC) wherever possible. A broader platform that could be used across

the Department would enable more dynamic development and employment of AI and would

more efficiently utilize scarce technical expertise.¹⁰

Common

The Secretary of Defense should direct

Interfaces

the establishment of a DoD-wide digital

AI Digital

Ecosystem.

ecosystem. The Secretary should require

USERS

that all new joint and service programs

adhere to the design of this ecosystem and

APPLICATIONS

that, wherever possible, existing programs

become interoperable with it by 2025.¹¹ Key

PLATFORM ENVIRONMENTS

elements should include:

• Data architecture composed of a secure,

SOFTWARE

federated system of distributed repositories

linked by a data catalog and appropriate

access controls¹² that facilitates finding,

DATA

accessing, and moving desired data across

the DoD.¹³

• Packaged AI environments¹⁴ that

HARDWARE

enable agile and iterative AI capabilities

development,¹⁵ testing, fielding, and updating

in support of a diverse set of stakeholders.¹⁶

•

A marketplace of shared AI resources¹⁷ that builds upon federated repositories of data,

sofware, and trained models,¹⁸ along with pre-negotiated computing and storage

services from a pool of vetted cloud providers.

•

A bolstered network and communications backbone to provide bandwidth to support

transport and data fusion, secure processing, continuous development and fielding of AI

applications, and sofware system integration at all levels.

•

Common interfaces that allow swif integration of mission-oriented investments.

FOUNDATIONS OF FUTURE DEFENSE

“Warfighters cannot change

the way they fight without also

changing the way they think.”

Train and Educate

Pillars

Warfighters.

Data-informed

Computational

Human-machine

Organizational

Maker

decision making

thinking

teaming

transformation

culture

Recommendations

Integrate digital skill sets and

Integrate emerging and disruptive

Create emerging and disruptive technology

computational thinking into junior military

technologies into service-level professional

coded billets in the DoD

leader education

military education

2. Train and educate warfighters. Warfighters cannot change the way they fight without also

Recommendation

changing the way they think. Most service members only use the powerful computers they

have to create PowerPoint presentations, build spreadsheets, or send emails. Our service

members need to develop core competencies in building, using, and responsibly teaming

with machine systems to recognize AI’s potential for building a faster and more effective

force. In particular, they need to know:

• How to use data in decision-making in ways that complement intuition and experience.

• How to use information processing agents and how to get a computer to perform

calculations and analytics that could not be done efficiently by a human.

• How to develop and thrive in a “maker” culture that encourages continuous contact and

regular experimentation with and development of new tools.

• How to move toward a “teammate model” for interacting with autonomous systems and

navigate issues of delegated authority, observability, predictability, directability, and

trust.

• How to bring organizations into the AI era—including when and how to integrate AI-

related tasks into priority missions, allocate resources to build and maintain the AI stack,

oversee new systems, and support the careers of technical experts.

CHAPTER 2

To improve training and education along these lines, DoD should:

• Identify service members who excel at computational thinking during the accession

process;

• Invest in upskilling its workforce through self-guided education courses and coding

language incentives;

• Teach junior leaders about problem curation, the AI lifecycle, data collection and

management, probabilistic reasoning and data visualization, and data-informed

decision-making as part of their pre-commissioning requirements and initial training;

• Integrate emerging and disruptive technology training into professional military

education courses; and

• Create emerging technology coded billets and an emerging technology certification

program comparable to the joint billet and qualification system.

3. Accelerate the adoption of existing digital technologies. DoD has largely relied on

Recommendation

workarounds to adopt new technologies, while the core acquisition processes remain sclerotic.

There are some bright spots, including the release of the Department’s tailorable acquisition

framework, contracting resources,¹⁹ and approaches taken by certain programs within the Air

Force.²⁰ The Department must scale these innovative practices and take further steps to align

acquisition workforce training, program incentives, budget, and organizational structures to

better support the delivery of digitally enabled capabilities.

A number of the Department’s digital innovation initiatives have delivered results,²¹ but they

are uncoordinated and under-resourced. DoD signaling of technology priorities is ad hoc and

is not supported by a track record of significant DoD investments in digital technology with

non-traditional vendors. As a result, national security AI applications attract less private-market

investment. The Department should focus on four actions:

• Integrate commercial AI to optimize core business processes. DoD should embrace

proven commercial AI applications and incentivize their use to generate labor and

cost savings, speed administrative actions, and inform decision-making.²² As a critical

first step, DoD should prioritize construction of enterprise data sets across core

administration areas.

• Network digital innovation initiatives to scale impact. Pockets of bottom-up

innovation need to be married with top-down leadership. The Department should

harmonize its innovation initiatives to carry out a coordinated go-to-market strategy for

commercial technology solutions. The Under Secretary of Defense for Research and

Engineering, working closely with the Under Secretary of Defense for Acquisition and

Sustainment, the military services and other headquarters counterparts, should provide

strategic direction for this effort.

• Expand use of specialized acquisition pathways and contracting approaches. DoD

should accelerate efforts to train acquisition professionals on the full range of available

options for acquisition and contracting and incentivize their use for AI and digital

technologies.²³

CHAPTER 2

“At every level, technologists,

operators, and domain

experts should function as

integrated teams.”

4. Democratize AI development. The Department must promote bottom-up AI

Recommendation

development.²⁴ At every level, technologists, operators, and domain experts should

function as integrated teams.²⁵ This would facilitate user feedback and improve trust and

confidence in AI systems. DoD should:

• Designate the Joint Artificial Intelligence Center (JAIC) as the Department’s

AI Accelerator. The JAIC cannot identify every potential use for AI in the

Department, but it can and should serve as a central hub of AI expertise. In this

“accelerator” model, JAIC would coordinate with relevant acquisition, technology,

and governance offices to inform strategy; develop AI applications that address

shared challenges at the Combatant Commands; and provide resources that enable

distributed AI development across the Department and the military services.²⁶

Enhanced AI R&D Investment, FY 2015-2030

Source: Govini and NSCAI

$80

Enhanced AI R&D

$70

Investment, FY

2015-2030.

Projection of spending on AI adoption with

recommended core AI spending of $8B/yr.

$60

$50

Projection of spending on AI adoption with

current core AI spending of $1.5B/yr.

$40

$30

$20

NSCAI recommendation: increase Core AI

spending from $1.5B to $8B per year by 2025.

$10

Projection of Core AI spending

under the status quo.

2015

2018

2021

2024

2027

This figure illustrates the correlation between R&D investment in Core AI technologies and AI adoption

projected to the year 2030. Two scenarios are represented in this figure. In the first, the DoD maintains

its current level of investment in core AI (~1.5B/year). In the second scenario, the DoD increases its

FOUNDATIONS OF FUTURE DEFENSE

investments in core AI to $8B/year. A significant increase in core AI spending is required to drive the

rate of AI adoption higher.

NSCAI staff teamed with two external partners to analyze historical and planned DoD investments in

AI RDT&E. The source data for the analysis is DoD’s annual RDT&E budget expenditures (for FY2015

- FY 2020) and annual RDT&E budget requests (for FY2021-FY2025). For the methodology employed

and lessons learned from this work, see Analysis of DoD RDT&E Investments in AI, NSCAI (on file with

the Commission). Disclaimer: We believe this analysis yields important insights into general trends in

AI spending and solutions for better future analyses, but caution that quality issues in the source data

detailed in our on file report mean that the spending level estimates presented contain significant,

difficult to estimate margins of error.

AI-enabled programs develop (in the case of RDT&E programs) and field (in the case of procurement

programs) the gamut of DoD warfighting and business systems, incorporating Core AI applications

for analyzing, automating, communicating, maneuvering, monitoring, sensing, and many other tasks.

While AI spending is usually a small percentage of these programs, their system’s performance may be

critically dependent upon the incorporation of core AI.

AI-enabling programs include technologies such as cloud computing and advanced microelectronics

required to support the deployment of effective AI capabilities at scale.

• Establish integrated AI delivery teams at each Combatant Command. These

commands have specific operational needs that routinely outpace centralized

development. AI delivery teams should be embedded at each Combatant Command

and capable of supporting the full lifecycle of AI development and fielding, including

data science, engineering, testing, and production—leveraging common resources

through the digital ecosystem.²⁷ Teams should include forward-deployable

components to act as the local interface with operational units.²⁸

5. Invest in next-generation capabilities. DoD leaders anticipate flat or declining defense

Recommendation

budgets for the coming years.²⁹ Despite potential budgetary pressures, DoD must continue

accelerating its modernization programs by prioritizing emerging and disruptive technologies

such as AI.³⁰

• Fund AI research and development. The Department should commit to spending

at least 3.4% of its budget on science and technology and allocate at least $8 billion

toward AI R&D annually.³¹ Additional resources should be focused on organizations

with significant AI expertise, such as the Defense Advanced Research Projects Agency

(DARPA), the Office of Naval Research (ONR), the Air Force Office of Scientific Research

(AFOSR), the Army Research Office (ARO), and the service laboratories.

CHAPTER 2

“To make AI ubiquitous

throughout its business processes

and military systems, DoD must

make tough budget tradeoffs and

prioritize modernization.”

• Retire legacy systems ill-equipped to compete in AI-enabled warfare. To make AI

ubiquitous throughout its business processes and military systems, DoD must make

tough budget tradeoffs and prioritize modernization.³² DoD should pursue a balanced

approach to update existing systems with leading-edge technologies to buy time for

investments in longer-term bets. Further, to guard against bias in favor of defending the

status quo, DoD should require an evaluation of AI alternatives prior to funding Major

Defense Acquisition Programs (MDAP).³³

• Produce a technology annex to the National Defense Strategy. To link DoD’s

technology investment strategy to future operational needs, the annex should include

roadmaps for designing, developing, fielding, and sustaining critical technologies that

are needed to address the operational challenges identified in the strategy.

FOUNDATIONS OF FUTURE DEFENSE

Chapter 2 - Endnotes

¹ General Joseph Dunford, then Chairman of the Joint Chiefs of Staff, testified in 2017 that “The U.S.

military’s competitive advantage against potential adversaries is eroding […] I assess that within five

years we will lose our ability to project power; the basis of how we defend the homeland, advance

U.S. interests, and meet our alliance commitments.” Posture Statement of General Joseph Dunford,

Chairman of the Joint Chiefs of Staff before the Senate Armed Services Committee, Senate Armed

Services Budget Hearing at 2 (June 13, 2017), https://www.armed-services.senate.gov/imo/media/

doc/Dunford_06-13-17.pdf.

² Big Data at War: Special Operations Forces, Project Maven, and Twenty-First-Century Warfare,

Modern War Institute (Aug. 25, 2020), https://mwi.usma.edu/big-data-at-war-special-operations-

forces-project-maven-and-twenty-first-century-warfare/; Cheryl Pellerin, Project Maven to Deploy

Computer Algorithms to War Zone by Year’s End, DoD (July 21, 2017), https://www.defense.gov/

Explore/News/Article/Article/1254719/project-maven-to-deploy-computer-algorithms-to-war-zone-by-

years-end/. Project Maven now includes detecting, classifying, and tracking objects within full motion

video images (e.g., person, vehicle, and weapon) and other AI algorithms for text-based projects. PE

0305245D8Z: Intelligence Capabilities and Innovation, Office of the Secretary of Defense (Feb. 2019),

https://www.dacis.com/budget/budget_pdf/FY20/RDTE/D/0305245D8Z_187.pdf.

³ For example, the Army’s Project Convergence exercise in September 2020 demonstrated use of

AI at multiple stages of the targeting process. Jen Judson & Nathan Strout, At Project Convergence,

the US Army Experienced Success and Failure—and It’s Happy About Both, Defense News (Oct. 12,

2020), https://www.defensenews.com/digital-show-dailies/ausa/2020/10/12/at-project-convergence-

the-us-army-experienced-success-and-failure-and-its-happy-about-both/. The Air Force has held

similar exercises, most notably as part of its efforts associated with the Advanced Battle Management

System—the technical infrastructure which will support the DoD’s Joint All-Domain Command and

Control concept. Theresa Hitchens, ABMS Demo Proves AI Chops For C2, Breaking Defense (Sept. 3,

2020), https://breakingdefense.com/2020/09/abms-demo-proves-ai-chops-for-c2/.

⁴ This includes the traditional process by which concepts of operation interact with technology

development. Chapter 3 of this report offers recommendations to adapt this approach and ensure that

technological advancements inform concepts as much as concepts drive technology development.

⁵ As one observer has noted: “While DoD’s investment accounts have grown substantially in

the last three years, this growth has been highly concentrated in buying systems from existing

production lines and doing prototypes of military systems.” Testimony of Andrew Hunter, Director,

Defense-Industrial Initiatives Group, CSIS, before the U.S. House of Representatives Armed

Services Committee, Hearing on DoD’s Role in Competing with China at 6 (Jan. 15, 2020), https://

armedservices.house.gov/_cache/files/5/8/5818cc1f-b86f-4dca-8aee-10ca788e6f43/9F4A03ABF1DE

AB747AF2D1302087A426.20200115-hasc-andrew-hunter-statement-vfinal.pdf.

⁶ The National Defense Strategy highlights the importance of the National Security Innovation Base

in maintaining the Department’s technological advantage. Summary of the 2018 National Defense

Strategy, U.S. Department of Defense at 3 (2018), https://dod.defense.gov/Portals/1/Documents/

pubs/2018-National-Defense-Strategy-Summary.pdf. The Center for Strategic and International

Studies offers a useful definition of the term, noting that the “[National Security Innovation Base] is a

significant expansion in scope […] compared to the traditional concept of the defense industrial base”

and includes tech firms out of innovation hubs such as Silicon Valley, Boston, and Austin. See Andrew

Hunter, A Strategic Approach to Defense Investment, CSIS (March 26, 2018), https://www.csis.org/

analysis/strategic-approach-defense-investment.

⁷ “The largest six prime defense suppliers (Lockheed Martin, Boeing, Northrop Grumman,

Raytheon, General Dynamics, and BAE Systems) […] represented 32 percent of all DoD prime

obligations in 2019.” Fiscal Year 2020: Industrial Capabilities, U.S. Department of Defense at 40

(Dec. 23, 2020), https://www.businessdefense.gov/Portals/51/USA002573-20%20ICR_2020_Web.

pdf?ver=o3D76uGwxcg0n0Yxvd5k-Q%3d%3d.

⁸ The strategy lays the foundation for the Department to treat data as a strategic asset and details the

goals to make DoD data visible, accessible, understandable, linked, trustworthy, interoperable, and

secure. Executive Summary: DoD Data Strategy, U.S. Department of Defense (Sept. 30, 2020), https://

media.defense.gov/2020/Oct/08/2002514180/-1/-1/0/DOD-DATA-STRATEGY.PDF.

CHAPTER 2

Chapter 2 - Endnotes

⁹ In recent years the Department has made promising initial steps to establish managed services

constructs for platforms, cloud infrastructure, and software development. For example, the Air Force’s

CloudOne and PlatformOne offerings (https://software.af.mil/dsop/services/); the Navy’s Black Pearl

(https://blackpearl.us/ ); and the Army’s Coding Repository and Transformation Environment (CReATE).

Further, the Office of the Secretary of Defense has built a data management platform, ADVANA, with

the goal to establish it as the single authoritative source for audit and business data analytics. See

Written Statement for the Record of David L. Norquist, Deputy Secretary of Defense before the U.S.

Senate Armed Services Committee Subcommittee on Readiness at 6 (Nov. 20, 2019), https://www.

armed-services.senate.gov/imo/media/doc/Norquist_11-20-19.pdf.

¹⁰ Components of this platform are underway as a result of the Joint Artificial Intelligence Center

(JAIC)’s Joint Common Foundation initiative—particularly the marketplace of shared AI resources

including data, algorithms, and trained AI models.

¹¹ Use of a common technical infrastructure will vastly improve DoD’s ability to ensure interoperability

and increase the effectiveness of the joint force. However, it is important to note that even without

such critical technical infrastructure, the Department is taking important policy steps to drive

interoperability and AI readiness for programs designed to meet joint capability needs. See Aaron

Mehta, Hyten to Issue New Joint Requirements on Handling Data, Defense News (Sept. 23, 2020),

https://www.defensenews.com/pentagon/2020/09/23/hyten-to-issue-new-joint-requirements-on-

handling-data/. Chapter 3 of this report outlines additional recommendations for achieving a state of

military AI readiness by 2025.

¹² Secured access to data sets as well as other shared resources should be managed by user- and

role-based authentication facilitated by an end-to-end identity, credential, and access management

infrastructure.

¹³ This hinges on implementation of the DoD’s new data strategy. Executive Summary: DoD

Data Strategy, U.S. Department of Defense (Sept. 30, 2020), https://media.defense.gov/2020/

Oct/08/2002514180/-1/-1/0/DOD-DATA-STRATEGY.PDF.

¹⁴ These are platform environments with ready-made workflows that can be tailored and launched

depending on user type (e.g., researcher, industry partner, operator) and use case (e.g.,

development, TEVV [test, evaluation, validation, and verification], fielding).

¹⁵ In other words, the DevSecOps application lifecycle. “DevSecOps improves the lead time and

frequency of delivery outcomes through enhanced engineering practices, promoting a more cohesive

collaboration between Development, Security, and Operations teams as they work towards continuous

integration and delivery.” Understanding the Differences Between Agile & DevSecOps—From a

Business Perspective, GSA (last accessed Jan. 1, 2021), https://tech.gsa.gov/guides/understanding_

differences_agile_devsecops/.

¹⁶ Stakeholders could include embedded development teams working at the tactical edge; private-

sector partners contributing pre-trained models; academic researchers working on open, relevant

challenge problems; government science and technology (S&T) researchers working within a service

lab; or international partners co-developing interoperable AI capabilities.

¹⁷ Shared AI resources should be managed with continuous Authorization to Operate (ATO)

frameworks and with mandated default ATO reciprocity across the Department.

¹⁸ Similar to or relying upon the platform delivery and features of Git (https://git-scm.com ), GitHub

(https://github.com), and GitLab (https://about.gitlab.com ).

¹⁹ The Pentagon acquisition office’s Adaptive Acquisition Framework and Contracting Cone mark

important steps by the Department to promote the use of alternate authorities for acquisitions and

contracting. These include, for example, other transaction authorities, middle-tier acquisitions, rapid

prototyping and rapid fielding, and specialized pathways for software acquisition.

²⁰ For example, the Air Force’s Advanced Battle Management System (ABMS), which is managing

systems intended to support the new Joint All-Domain Command and Control concept as a portfolio

and based heavily on experimentation to drive innovation and an iterative approach to requirements.

Notably, the Department of Defense Appropriations Bill for Fiscal Year 2021 expresses concern with

various aspects of the Air Force’s approach, including the “absence of firm requirements, acquisition

FOUNDATIONS OF FUTURE DEFENSE

strategy, or cost estimate” and system of systems integration. See H. Rept. 116-453, at 294-295 (July

16, 2020), https://www.congress.gov/116/crpt/hrpt453/CRPT-116hrpt453.pdf.

²¹ The term “digital innovation initiatives” is used here to describe the various entities across the

Office of the Secretary of Defense and the military services--such as the Defense Innovation Unit

(DIU), AFWERX, NavalX, and Army Applications Laboratory (AAL)--that are focused on bridging

the gap with the commercial technology sector, especially startups and non-traditional vendors, and

accelerating the delivery of best-of-breed technology solutions.

²² The Defense Innovation Unit (DIU) is currently pursuing a number of AI projects to optimize

business processes in the DoD, ranging from using AI-driven Robotic Process Automation to reduce

labor costs for the Army Comptroller to improving Air Force readiness with AI-driven predictive

maintenance and leveraging AI-constructed knowledge graphs to rapidly identify supply chain

risks for the Defense Intelligence Agency. See JAIC Partners with DIU on AI/ML Models to Resolve

Complex Financial Errors, JAIC (Oct. 1, 2020), https://www.ai.mil/blog_10_01_20-jaic_partners_with_

diu_on_aiml_models_to_resolve_complex_finanical_errors.html; U.S. Defense Department Awards

C3.ai $95M Contract Vehicle to Improve Aircraft Readiness Using AI, Business Wire (Jan. 15, 2020),

https://www.businesswire.com/news/home/20200115005413/en/US-Defense-Department-Awards-C3.

ai-95M-Contract-Vehicle-to-Improve-Aircraft-Readiness-Using-AI; Accelerate.AI Accelerates Growth

and Product Adoption with Defense Innovation Unit Contract, Accrete.ai (April 23, 2020), https://blog.

accrete.ai/newsroom/accrete.ai-wins-million-dollar-contract-with-the-defense-innovation-unit.

²³ As an example, DIU uses several acquisition pathways and contracting strategies that could help

improve both the adoption and operational relevance of AI solutions and also expand the National

Security Innovation Base. DIU pioneered the Commercial Solutions Opening with Army Contracting

Command-New Jersey, which leverages section 2371b of title 10 U.S.C. Other Transaction authority

to create a “fast, flexible, and collaborative” contract vehicle to prototype capabilities for the

Department. DIU has also used Section 2374a of title 10 U.S.C. Prize Challenge authority to advance

various AI-related priorities for DoD and the broader AI research community.

²⁴ The Department-wide digital infrastructure described above is critical to enabling this approach,

but structural changes are also required to maximize its utility.

²⁵ There are notable examples of warfighter-technologist pairings within DoD, such as the Air Force’s

software factories and the forward-deployed tactical data teams used by Special Operations and

Army Futures Command. They found that partnering technologists (such as data scientists) with

operators or analysts at the tactical edge: 1) significantly reduces the time it typically takes a

contractor to understand the problem set and deploy a solution; 2) incentivizes iterative development

techniques and fast-fielding of minimum viable products that yield higher-impact solutions on an

accelerated timeline; and 3) generates increased buy-in to data and AI technologies as critical

mission enablers. NSCAI Engagements (Nov. 2020). To ensure U.S. forces maintain overmatch in the

long-term, DoD must scale this user-centered development.

²⁶ Important offices for coordination with the JAIC include but are not limited to USD(R&E), USD

Acquisition & Sustainment (USD(A&S)), Director Operational Test & Evaluation (DOT&E), and the

DoD Chief Information Officer (CIO) and Chief Data Officer (CDO). Within USD(R&E), DIU is a key

enabler of the JAIC that pursues a project-based approach by transitioning commercial prototypes for

specific applications. The JAIC currently serves the Combatant Commands through its Component

Mission Initiatives (CMIs), including a Mission Initiative for Joint Warfighting Operations. See Mission

Initiatives, JAIC (last accessed Dec. 28, 2020), https://www.ai.mil/mi_joint_warfighting_operations.

html.

²⁷ Such applications could be developed by other Combatant Commands, Service software factories,

or the JAIC and discoverable via the recommended digital ecosystem. Each Combatant Command

should ensure that the AI delivery teams are staffed with the appropriate talent to manage the full

lifecycle of AI solutions, including in disciplines such as data science, AI testing and model training,

software engineering, product management, and full stack development.

²⁸ As an example, both Army Futures Command (AFC) and Army Special Operations Command

(USASOC) use a model known as “tactical data teams.” This model brings AI/ML expertise forward

to the field in the form of three- to six-person teams to build AI solutions for real-time operational

problems. Executed by a small business, Striveworks, under contract with AFC and USASOC, they are

currently supporting efforts in Central Command and Indo-Pacific Command Areas of Responsibility.

CHAPTER 3

Even with the right artificial intelligence

(AI)-ready technology

foundations in place, the U.S. military will still be at a battlefield

disadvantage if it fails to adopt the right concepts and operations

to integrate AI technologies. Throughout history, the best adopters

and integrators, rather than the best technologists, have reaped the

military rewards of new technology.¹ The Department of Defense

(DoD) should not be a witness to the AI revolution in military affairs,

but should deliver it with leadership from the top, new operating

concepts, relentless experimentation, and a system that rewards

agility and risk.

A new warfighting paradigm is emerging because of AI. Our competitors are making

substantial investments to take advantage of it. This idea has been called “algorithmic”

or “mosaic” warfare²; China’s theorists have called it “intelligentized” war.³ All of these

terms capture, in various ways, how a new era of conflict will be dominated by AI and pit

algorithms against algorithms. Advantage will be determined by the amount and quality of

a military’s data, the algorithms it develops, the AI-enabled networks it connects, the AI-

enabled weapons it fields, and the AI-enabled operating concepts it embraces to create

new ways of war.

Today’s DoD is trying to execute an AI pivot, but without urgency. Despite pockets of

imaginative reform and a few farsighted leaders, DoD remains locked in an Industrial

Age mentality in which great-power conflict is seen as a contest of massed forces and

monolithic platforms and systems. The emerging ubiquity of AI in the commercial realm

and the speed of digital transformation punctuate the risk of not pivoting fast enough. The

Department must act now to integrate AI into critical functions, existing systems, exercises,

and wargames to become an AI-ready force by 2025. Simultaneously, DoD must develop

more creative warfighting concepts that are paired with investments in future AI-enabled

technologies to continuously out-innovate potential adversaries. If our forces are not

equipped with AI-enabled systems guided by new concepts that exceed those of their

adversaries, they will be outmatched and paralyzed by the complexity of battle.

An AI-Ready DoD by 2025:

Warfighters enabled with baseline digital literacy and access

to the digital infrastructure and software required for ubiquitous

AI integration in training, exercises, and operations.

CHAPTER 3

How AI Will Change Warfare.

AI-enabled warfare will not hinge on a single new weapon, technology, or operational concept;

rather, it will center on the application and integration of AI-enabled technologies into every

facet of warfighting. AI will transform the way war is conducted in every domain from undersea

to outer space, as well as in cyberspace and along the electromagnetic spectrum. It will

impact strategic decision-making, operational concepts and planning, tactical maneuvers

in the field, and back-office support. In this new kind of warfare, traditional confines of the

battlefield will be expanded through AI-enabled micro-targeting, disinformation, and cyber

operations, as described in Chapter 1 of this report. AI will reshape many attributes of war,

such as its speed, tempo, and scale; the relationships service members have with machines;

the persistence with which the battlefield can be monitored; and the discrimination and

precision with which targets can be attacked. There will be a premium on speed and accuracy

in developing knowledge, acting, and reacting as the conflict unfolds.

“AI-enabled warfare will not

hinge on a single new weapon,

technology, or operational

concept; rather, it will center on the

application and integration of AI-

enabled technologies into every

facet of warfighting.”

AI AND WARFARE

DoD AI RDT&E Investments by Warfighting Domains, FY 2015-2025

Source: Govini

DoD AI RDT&E

Investments.

DoD’s AI investments are well distributed across the various warfighting domains (land, naval, air,

space, cyber, electromagnetic spectrum, information), with over 25% of investments in multi-domain

applications of AI, signaling AI’s potential for integrating multi-domain operations. Investments in AI

applications for Space operations more than quadrupled from FY2019 to FY2025, from $500M to $2.2B,

increasing from 3% to almost 9% of AI-enabled investment.

Note the spending levels presented in figure represent estimates based on an analysis of DoD RDT&E

budget documents for FY2021-FY2025. See Analysis of DoD RDT&E Investments in AI, NSCAI (on final

with the Commission). Due to inherent quality issues in the source data, estimates presented contain

significant, difficult to estimate margins of error.

AI will make the process of finding and hitting targets of military value faster and more efficient.

It will also increase accuracy of target identification and minimize collateral damage. Currently,

this process generally involves passing data in a serial fashion from a sensor, through a

series of humans, to a platform that can shoot at the target. AI will help automate some of

the intermediate stages of the decision process. AI will also create opportunities for more

advanced processes that would operate more akin to a web, fusing multiple sensors and

platforms to manage complex data flows and transmitting actionable information to human

operators and machines across all domains.⁴

In war, many of the military uses of AI will complement, rather than supplant, the role of humans.

AI tools will improve the way service members perceive, understand, decide, adapt, and act

in the course of their missions. However, new concepts for military operations will also need to

account for the changing ways in which humans will be able to delegate increasingly complex

tasks to AI-enabled systems. In the near term, this will be managed through the military’s

principle of “mission command,” which stresses decentralized execution and disciplined

initiative by subordinates who follow a commander’s intent. This human-centric approach to

fighting should remain the standard for the foreseeable future. But as AI continues to advance

into the cognitive and neuromorphic domain, and human-machine teaming becomes more

sophisticated, the military will need to develop more imaginative concepts and organizational

constructs that take full advantage of AI technologies without relinquishing the principles that

undergird mission command.

CHAPTER 3

Ways to

Operationalize AI.

Business processes. Robotic Process Automation and AI-enabled analysis can generate significant

savings, speed administrative actions, and provide decision-makers with superior insights into core

business processes such as finance, budget, contracting, travel, and human resources.

Design. AI will support a holistic system-of-systems approach to developmental force design via digital

engineering, digital twins, and modeling and simulation to enable a more comprehensive understanding of

system vulnerabilities and adjacent capabilities, concepts, and technologies.

Readiness. AI will enhance training by relieving the cognitive burden of doing repetitive tasks that can be

performed better by machines. AI will be prevalent in all exercises and wargames and will enhance the

military’s ability to train in live, virtual, and constructive environments.

Plan and task collection. Through automation, AI-enabled systems will optimize tasking and collection

for platforms, sensors, and assets in near-real time in response to dynamic intelligence requirements or

changes in the environment.

Collect. At the tactical edge, “smart” sensors will be capable of pre-processing raw intelligence and

prioritizing what data to transmit and store, which will be especially helpful in degraded or low-bandwidth

environments.

Process. AI-enabled natural language processing, computer vision, and audiovisual analysis can vastly

reduce manual data processing. AI can also be used to automate data conversion such as translations

and decryptions, accelerating the ability to derive actionable insights.

Exploit and analyze. AI-enabled tools have the potential to augment filtering, flagging, and triage across

multiple data sets. Such tools can identify connections and correlations more efficiently and at a greater

scale than human analysts and can flag those findings and the most important content for human analysis.

AI will improve indications and warnings for military leaders.

• AI can fuse data from multiple sources, types of intelligence, and classification levels to produce

accurate predictive analysis in a way that is not currently possible.

• Advances in speech-to-text transcription and language analytics now enable reading comprehension,

question answering, and automated summarization of large quantities of text.

Disseminate. AI will be able to automatically generate machine-readable versions of intelligence products

and disseminate them at machine speed so that computer systems across the IC and the military can

ingest and use them in real time without manual intervention.

Planning. AI decision-support applications will utilize modeling and simulation algorithms and real-time

data sets to optimize planning options.

Deciding. AI will integrate command-and-control networks and compress the speed of finding and

attacking targets of military value.

Tasking, delegation, and distribution. Edge processing enhanced by delegated authorities will allow

frontline units to operate in a coordinated manner with minimal to no communications. AI techniques like

machine learning, and rule-based models will support network resiliency.

Logistics and sustainment. AI-enabled predictive analytics, optimization, and tracking will improve

efficiency and effectiveness across all facets of logistics. Intelligent systems will aid in the development

of courses of action for routine and contingency logistics and sustainment operations. Robotic process

automation will streamline human-centric maintenance and supply chain workflows.

Movement. AI will enhance the ability of commanders to maneuver, position, and protect units and forces.

AI will help network and coordinate movements of autonomous swarms via human-machine and machine-

machine teaming.

Targeting. AI-enabled systems will expand a single targeting chain into a complex targeting web that

considers numerous variables across units and domains.

Precision and accuracy. Through AI-enabled smart weapons and autonomous platforms, AI will enable

the military to be more precise and discern friendly forces, non-combatants, and adversary targets with

greater accuracy.

AI AND WARFARE

This list of how AI might transform warfighting principles and capabilities—as well as

others like it—is by no means exhaustive. Innovation will lead to future capabilities that are

unknowable at present and will only become clearer in time.

“Innovation will lead to future

capabilities that are unknowable

at present and will only become

clearer in time.”

Stronger Together.

If the United States wants to fight with AI, it will need allies and partners with AI-enabled

militaries and intelligence agencies. Uneven adoption of AI will threaten interoperability

and the political cohesion and resiliency of U.S. alliances.⁵ As it deepens and expands

conventional defense arrangements across the globe--especially in Europe and the Indo-

Pacific--the United States should incorporate AI and emerging technology into coordinated

defense and intelligence activities. Given the dual-use nature of many software-based

capabilities, DoD will need more flexibility to work with civilian agencies, companies, and

research institutions in partner nations.

Promote AI interoperability and the adoption of critical emerging technologies among allies

Recommendation

and partners, including the Five Eyes, the North Atlantic Treaty Organization (NATO), and

across the Indo-Pacific. This should include:

• Enhancing existing Five Eyes AI-related defense and intelligence efforts.

• Supporting NATO efforts to accelerate agreements on architectures and standards,

develop allied technical expertise, and pursue coalition AI use cases for exercises and

wargames.

• Fostering the Joint Artificial Intelligence Center (JAIC)’s International AI Partnership for

Defense as a critical vehicle to further AI defense and security cooperation.⁶

• Creating an Atlantic-Pacific Security Technology Partnership to improve military and

intelligence capabilities and interoperability across European and Indo-Pacific allies

and partners.

CHAPTER 3

“Uneven adoption of AI will

threaten military interoperability,

and the political cohesion and

resiliency of U.S. alliances.”

AI-enabled

Alliances and

Partnerships.

Achieve a State of Military AI Readiness by 2025.

To reach this goal, the DoD should:

Drive organizational reform through top-down leadership. Senior civilian and military officials

Recommendation

should set clear priorities and direction, empower subordinates, and accept higher

uncertainty and risk in pursuing new technologies. Specifically, DoD should:

• Establish a high-level Steering Committee on Emerging Technology, tri-chaired by the

Deputy Secretary of Defense, the Vice Chairman of the Joint Chiefs of Staff, and the

Principal Deputy Director of National Intelligence⁷;

• Ensure that the JAIC Director remains a three-star general or flag officer with significant

operational experience who reports directly to the Secretary of Defense or Deputy

Secretary of Defense;

• Appoint the Under Secretary of Defense for Research and Engineering as the co-chair

and chief science advisor to the Joint Requirements Oversight Council; and

AI AND WARFARE

• Assign an AI Operational Advocate on the staff of every Combatant Command. This

officer would perform a similar role to that played by the Staff Judge Advocate. He or she

would be an expert in AI systems, advise the commander and staff on the capabilities

and limitations of AI systems, and identify when AI-enabled systems are being used

inappropriately.

Develop innovative operational concepts that integrate new warfighting capabilities with

Recommendation

emerging technologies.8 These concepts should strive for seamless interoperability across

the military services and across operational domains. The concept developers should

work closely with technologists to articulate how the military could fight most effectively in

future scenarios, and they should assume that AI-enabled capabilities will be ubiquitous

on future battlefields. These concepts can also drive future investments.

By the end of 2021, establish AI and digital readiness performance goals.⁹ To achieve more

Recommendation

substantial integration of AI across DoD, the Deputy Secretary of Defense should:

• Direct DoD components to assess military AI and digital readiness through existing

readiness management forums and processes. The Tri-Chaired Steering Committee

on Emerging Technology should work closely with the Under Secretary of Defense for

Personnel and Readiness and the Joint Staff to ensure the identified AI readiness criteria

are incorporated into the military services’ readiness reporting and resourcing strategies.

• Direct the military services to accelerate review of specific skill gaps in AI to inform

recruitment and talent-management strategies.¹⁰

• Direct the military services—in coordination with the Under Secretary of Defense for

Acquisition and Sustainment, the Joint Staff, the Defense Logistics Agency, and the

JAIC—to prioritize integration of AI into logistics and sustainment systems wherever

possible.

• Integrate AI into major wargames and exercises to promote field-to-learn approaches

to technology adoption. Operators need persistent interaction with AI-enabled

capabilities early in the development cycle to generate critical feedback on how they

function and how they impact the mission. Widespread experimentation will advance

both concept development and the performance of the technology.¹¹

• Incentivize experimentation with AI-enabled applications through the Warfighting

Lab Incentive Fund, which could be overseen by the proposed Tri-Chaired Steering

Committee.¹²

Define a joint warfighting network architecture by the end of 2021. The key objective of this

Recommendation

joint warfighting network should be a secure, open-standards systems network that

supports the integration of AI applications at operational levels and across domains.¹³

It should be accessible by all of the military services and encompass several elements,

including command and control networks; data transport, storage, and secure processing;

and weapon system integration. The technical infrastructure for the network should be

supported by best practices in digital engineering.¹⁴ It should also be interoperable with

the digital ecosystem described in Chapter 2 of this report.¹⁵

Invest in priority AI R&D areas that could support future military capabilities, including the

Recommendation

following:

AI AND WARFARE

Chapter 3 - Endnotes

¹ On military adoption, see, e.g., Michael C. Horowitz, The Diffusion of Military Power: Causes and

Consequences for International Politics, Princeton University Press (2010).

² The Defense Advanced Research Projects Agency (DARPA) Mosaic warfare central concept is

built around the “adaptability for U.S. forces and complexity or uncertainty for the enemy through

the rapid composition and recomposition of a more disaggregated U.S. military force using human

command and machine control.” Bryan Clark, et al., Mosaic Warfare: Exploiting Artificial Intelligence

and Autonomous Systems to Implement Decision-Centric Operations, CSBA at vi (Feb. 11, 2020),

https://csbaonline.org/research/publications/mosaic-warfare-exploiting-artificial-intelligence-and-

autonomous-systems-to-implement-decision-centric-operations/publication/1.

³ The People’s Liberation Army has developed a warfighting concept for what it calls “intelligentized

operations” with AI at its core. Within this construct, China theorizes that in future conflict, the central

contest will be between adversarial battle networks rather than traditional weapons platforms, and

that information advantage and algorithmic superiority will be a determinant of victory. See Elsa

Kania, Chinese Military Innovation in Artificial Intelligence, CNAS at 1 (June 7, 2019), https://www.

cnas.org/publications/congressional-testimony/chinese-military-innovation-in-artificial-intelligence

(testimony before the U.S.-China Economic and Security Review Commission).

⁴ See Creating Cross-Domain Kill Webs in Real Time, DARPA (Sept. 18, 2020), https://www.darpa.mil/

news-events/2020-09-18a. See also AI Fusion: Enabling Distributed Artificial Intelligence to Enhance

Multi-Domain Operations & Real-Time Situational Awareness, Carnegie Mellon University (2020),

http://www.cs.cmu.edu/~ai-fusion/overview.

⁵ On military interoperability challenges related to AI, see Erik Lin-Greenberg, Allies and Artificial

Intelligence: Obstacles to Operations and Decision-Making, Texas National Security Review (Spring

2020), https://tnsr.org/2020/03/allies-and-artificial-intelligence-obstacles-to-operations-and-decision-

making/.

⁶ The AI Partnership for Defense, launched in September 2020, includes the United States and 12

partner nations: Australia, Canada, Denmark, Estonia, Finland, France, Israel, Japan, Norway, South

Korea, Sweden, and the United Kingdom. It seeks to “provide values-based global leadership” on

adoption of AI in the defense and security context and align “like-minded nations to promote the

responsible use of AI, advance shared interests and best practices on AI ethics implementation,

establish frameworks to facilitate cooperation, and coordinate strategic messaging on AI policy.” Joint

Statement, AI Partnership for Defense (Sept. 15-16, 2020), https://www.ai.mil/docs/AI_PfD_Joint_

Statement_09_16_20.pdf. The Partnership held its second formal dialogue in January 2021. DoD Joint

AI Center Facilitates Second International AI Dialogue for Defense, JAIC (Jan. 27, 2021), https://www.

ai.mil/news_01_27_21-dod_joint_ai_center_facilitates_second_international_ai_dialogue_for_defense.

html.

⁷ The Commission acknowledges Section 236 of the Fiscal Year 2021 National Defense Authorization

Act, which permits the Secretary of Defense to establish a steering committee on emerging

technology and national security threats composed of the Deputy Secretary of Defense, the Vice

Chairman of the Joint Chiefs of Staff, the Under Secretary of Defense for Intelligence and Security,

the Under Secretary of Defense for Research and Engineering, the Under Secretary of Defense for

Personnel and Readiness, the Under Secretary of Defense for Acquisition and Sustainment, the Chief

Information Officer, and such other officials of the Department of Defense as the Secretary determines

appropriate. However, the structure described in Sec. 236 does not include leadership from the

Intelligence Community and will thus not drive the intended action. See Pub. L. 116-283, William M.

(Mac) Thornberry National Defense Authorization Act for Fiscal Year 2021, 134 Stat. 3388 (2021).

⁸ Notably, the National Defense Strategy emphasizes the need to “evolve innovative operational

concepts” and “foster a culture of experimentation and calculated risk-taking.” Tighter coordination

between concept writers and technologists would create a more dynamic cycle of technology

development and integration. Summary of the 2018 National Defense Strategy, U.S. Department of

Defense at 7 (2018), https://dod.defense.gov/Portals/1/Documents/pubs/2018-National-Defense-

Strategy-Summary.pdf.

CHAPTER 3

Chapter 3 - Endnotes

⁹ “Readiness” is a key measure of military effectiveness and remains at the heart of budget,

policy, and oversight debates on defense preparedness. In this context, DoD should establish key

AI and digital readiness performance objectives to measure and drive Department and service

accountability. See G. James Herrera, The Fundamentals of Military Readiness, Congressional

Research Service at 2 (Oct. 2, 2020), https://fas.org/sgp/crs/natsec/R46559.pdf.

¹⁰ As noted in Chapter 6 of this report, there is already an identified need for the creation of digital

corps, civilian and military AI and AI-related career fields, an expansion of recruiting pathways, and

the creation of recruiting offices. The military services need to assess the number of personnel in

those fields and structures, not the need to establish them.

¹¹ Although AI will be ubiquitous across all domains, the high-data volumes associated with the space,

cyber, and information operations domains make use cases in those domains particularly well-suited

for prioritized integration of AI-enabled applications in wargames, exercises, and experimentation.

¹² The Warfighting Lab Incentive Fund is intended to spur field experiments and demonstrations to

“evaluate, analyze and provide insight into more effective ways of using current capabilities, and

to identify new ways to incorporate technologies into future operations and organizations.” See

Memorandum from the Deputy Secretary of Defense, Warfighting Lab Incentive Fund and Governance

Structure, U.S. Department of Defense (May 6, 2016), https://defenseinnovationmarketplace.dtic.mil/

wp-content/uploads/2018/02/DSD_memo.pdf.

AI AND WARFARE

¹³ The network envisioned is well-aligned with ongoing DoD efforts to embrace standards-driven

interoperability, system adaptability, and data-sharing. See Memorandum from the Secretary of the

Navy, Secretary of the Army, and Secretary of the Air Force for Service Acquisition Executives and

Program Executive Officers, U.S. Department of Defense (Jan. 7, 2019), https://www.dsp.dla.mil/

Portals/26/Documents/PolicyAndGuidance/Memo-Modular_Open_Systems_Approach.pdf.

¹⁴ Such as the goals and focus areas outlined in the DoD Digital Engineering Strategy; terms, any

knowledge, and guidelines shared as part of the Digital Engineering Body of Knowledge; and

incorporating Section 231 of the National Defense Authorization Act for Fiscal Year 2020, which

requires the creation of a digital engineering capability to automate testing and evaluation. See

Department of Defense Digital Engineering Strategy, Office of the Deputy Assistant Secretary of

Defense for Systems Engineering (June 2018); see also Pub. L. 116-92, The National Defense

Authorization Act for Fiscal Year 2020, 133 Stat. 1198 (2019). For a description of the Digital

Engineering Body of Knowledge, see Andrew Monje, Future Direction of Model-Based Engineering

Across the Department of Defense, U.S. Department of Defense (Jan. 27, 2020), https://ac.cto.mil/wp-

content/uploads/2020/05/RAMS-Monje-27Jan2020-Future.pdf.

¹⁵ See the Chapter 2 Blueprint for Action for details on how this architecture should interact with the

digital ecosystem.

CHAPTER 4

World military powers both large and small are pursuing artificial

intelligence (AI)-enabled and autonomous weapon systems. Such

systems have the potential to help commanders make faster, better,

and more relevant decisions. They will enable weapon systems to

be capable of levels of performance, speed, and discrimination that

exceed human capabilities. And they will enable hitherto impossible

complex tasks. If properly designed, tested, and used, they could

improve compliance with International Humanitarian Law (IHL)¹ by

reducing the risk of accidental engagements, decreasing civilian

casualties, minimizing collateral infrastructure damage, and allowing

for detailed auditing of the decisions and actions of operators and

their command chains. Although U.S. weapons platforms have

utilized autonomous functionalities for more than eight decades,²

AI technologies have the potential to enable novel, sophisticated

offensive and defensive autonomous capabilities.

The increasing use of AI technologies in weapon systems has generated important

questions regarding whether such systems are lawful, safe, and ethical. Those critical of

using AI technologies in weapons argue that states should negotiate limits or restrictions

on such systems and their use. There is also concern that autonomous weapon systems

may make conflict escalation more likely, and debate continues over what steps are

needed to ensure that such systems minimize the risk of unintended military engagements

or inadvertent and uncontrollable conflict escalation. Since 2014, the United Nations

Convention on Certain Conventional Weapons (CCW) has held meetings among states

parties to discuss the technological, military, legal, and ethical dimensions of “emerging

technologies in the area of lethal autonomous weapon systems (LAWS).”³ Specifically, it is

examining whether autonomous technologies will be capable of complying with IHL and

whether additional measures are necessary to ensure that humans maintain an appropriate

degree of control over the use of force.

The Commission has consulted with civil society, academic organizations, and government

agencies in studying the legal, ethical, and strategic questions that surround AI-enabled

and autonomous weapon systems, including their potential military benefits and risks,

possible ethical issues coming to the fore, international efforts to regulate them, and their

compliance with IHL. The Commission offers the following four judgments to reflect its

conclusions on these discussions.

AUTONOMOUS WEAPON SYSTEMS AND RISKS ASSOCIATED WITH AI-ENABLED WARFARE

Judgment 1: Provided their use is

authorized by a human commander or

NSCAI Judgments Regarding

operator, properly designed and tested

AI-Enabled and Autonomous

AI-enabled and autonomous weapon

Weapon Systems

systems have been and can continue

•

Provided their use is authorized by a

to be used in ways which are consistent

human commander or operator, properly

with IHL.

designed and tested AI-enabled and

autonomous weapon systems have been

This judgment is grounded in several

and can continue to be used in ways

which are consistent with IHL.

elements of IHL:

•

Existing DoD procedures are capable

•

Distinction: The principle of distinction

of ensuring that the United States will

holds that parties to an armed conflict

field safe and reliable AI-enabled and

must distinguish between civilians

autonomous weapon systems and use

and combatants.⁴ Weapons with

them in a manner that is consistent with

increasingly accurate AI-enabled

IHL.

target recognition systems have the

potential to reduce cases of target

•

There is little evidence that U.S.

misidentification, the leading cause

competitors have equivalent rigorous

of inadvertent engagements during

procedures to ensure their AI-enabled

combat operations, and thus reduce

and autonomous weapon systems will be

civilian casualties and collateral

responsibly designed and lawfully used.

infrastructure damage.⁵

•

The Commission does not support a

•

Proportionality: The principle of

global prohibition of AI-enabled and

proportionality prohibits attacks

autonomous weapon systems.

which would cause incidental loss of

civilian life excessive to the anticipated

military advantage.⁶ AI-enabled and

autonomous weapon systems can and

should also be designed to carry out operations in accordance with human judgments

and directions regarding the proportionality of an attack. The moral reasoning involved

in this calculus—weighing anticipated military advantage against potential civilian

harm—remains the responsibility of a human commander.⁷

•

Accountability: Ensuring accountability and command responsibility is essential

to compliance with IHL. A human can and should be held accountable for the

development, testing, use, and behavior of any autonomous weapon system, AI-

enabled or otherwise. Autonomous weapon systems operate within the same general

parameters as those used for human command and control systems, which are

specifically designed to ensure accountability for actions and compliance with IHL. This

is no different than for any other weapon system.⁸

The Commission endorses DoD’s body of policy that states that human judgment must

be involved in decisions to take human life in armed conflict. The kind of involvement

necessary for humans to remain accountable for the use of autonomous weapon systems

will vary depending on the time criticality of the situation as well as the operational context,

circumstance, and type of weapon systems involved.⁹ It is incumbent upon states to

establish processes which ensure that appropriate levels of human judgment are relied

CHAPTER 4

“... human judgment must be

involved in decisions to take

human life in armed conflict.”

upon in the use of AI-enabled and autonomous weapon systems and that human operators

of such systems remain accountable for the results of their employment.

Human accountability for the results of lethal engagements does not necessarily require

human oversight of every step of an engagement process. Once a human authorizes an

engagement against a target or group of targets, subsequent steps in the attack sequence

can be completed autonomously without relinquishing human accountability. The exact

number of steps in this sequence is dependent on the system’s technical capabilities and

the context and must consider factors such as the uncertainty associated with the system’s

behavior and potential outcomes, the magnitude of the threat, and the time available

for action. For instance, an autonomous weapon system located in a rapidly changing

environment, such as an urban setting, for an extended period, may require more frequent

human authorization to ensure sufficient human accountability over autonomous actions

than an equivalent system, operated for a similar amount of time, in a highly predictable

and less populated environment—such as underwater or in space. This logic can and

should be incorporated into the system’s design, testing, and operational planning. Taking

these factors into consideration, when feasible and deemed necessary operation designs

should include points of required human guidance amid a sequence of automated actions.

At such points, a human must review the system’s status and authorize its next actions

before the system’s mission can continue. A blanket decision to compel every discrete step

in an engagement involving lethal force to be subject to explicit authorization by a human

is neither realistic nor desirable. Indeed, such a policy could instead spur commanders to

use less precise, unguided weapon systems that might result in greater levels of collateral

damage.

Judgment 2: Existing DoD procedures are capable of ensuring that the United States will

field safe and reliable AI-enabled and autonomous weapon systems and use them in a

manner that is consistent with IHL.

DoD’s commitment to rigorous procedures for the development and use of autonomous

weapon systems—as well as its commitment to strong AI ethical principles¹⁰—instills

confidence that it will be able to field AI-enabled and autonomous weapon systems that are

used lawfully. DoD has comprehensive processes for ensuring that the use of any weapon

it fields is compliant with IHL and has a demonstrated commitment to operating within

IHL, minimizing civilian casualties, and learning from its mistakes.¹¹ DoD has established

AUTONOMOUS WEAPON SYSTEMS AND RISKS ASSOCIATED WITH AI-ENABLED WARFARE

a cross-department legal group, the DoD Law of War Working Group, to “develop and

coordinate law of war initiatives and issues, such as analysis regarding the legality of

new means or methods of warfare under consideration by DoD components.”¹² This

standing body is well positioned to examine implications for IHL as technology evolves

over time. The International Committee on the Red Cross (ICRC) has lauded the strength

and transparency of this system, listing the United States as one of eight countries that

have “national mechanisms to review the legality of weapons and that have made the

instruments setting up these mechanisms available to the ICRC.”¹³

In addition to baseline legal review, the Department has taken special precautions for

autonomous weapon systems to ensure these systems undergo sufficient test and evaluation,

verification and validation (TEVV). In 2012, DoD added to an extensive list of guiding directives

and instructions regarding weapons development within the Department by publishing DoD

Directive (DoDD) 3000.09, Autonomy in Weapon systems, which establishes DoD policy

for the development and use of autonomous weapon systems. It requires that all systems

be designed “to allow commanders and operators to exercise appropriate levels of human

judgment over the use of force” and requires senior DoD leaders to approve any autonomous

weapon with lethal capabilities first when development begins, and again before fielding.¹⁴

It also mandates any autonomous or semi-autonomous weapon that undergoes a revision

to its operating state to undergo additional testing and evaluation. DoDD 3000.09 provides

important definitions and baseline requirements for such systems and must be reviewed

annually as technology evolves.¹⁵ Chapter 7 of this report provides specific recommendations

on how the United States should adapt its TEVV policies and capabilities to ensure it retains

justified confidence in AI-enabled systems.¹⁶

“The U.S. commitment to

IHL is longstanding, and AI-

enabled and autonomous

weapon systems will not change

this commitment.”

In addition, DoD’s command and control procedures to authorize target selection and

employment of munitions are rigorous and designed to ensure compliance with IHL.

Operational commanders in the field are directly supported by lawyers embedded at

multiple levels to advise on decisions about the use of force. The U.S. commitment to IHL

CHAPTER 4

is long-standing, and AI-enabled and autonomous weapon systems will not change this

commitment.¹⁷ These same principles will be ingrained into the design of those weapons,

demonstrated in TEVV, and maintained by commanders overseeing their deployment. DoD’s

policy for autonomy in weapon systems and its adoption of ethical principles for AI in 2020

further highlight and reinforce this commitment.¹⁸

Judgment 3: There is little evidence that U.S. competitors have equivalent rigorous

procedures to ensure their AI-enabled and autonomous weapon systems will be

responsibly designed and lawfully used.

Battlefield success may become increasingly dependent on AI performance, and AI-

enabled weapons are likely to proliferate given the open-source and dual-use nature of AI.

This could cause pressure to mount on states to rapidly field new and untested systems

and algorithms. Such pressures could also tilt designs toward systems that react more

quickly, limiting the amount of time available for effective human oversight on engagement

decisions. U.S. competitors, particularly Russia and China, likely do not have equivalent

operational and targeting procedures to ensure the use of such systems is compliant

with IHL and to preserve human accountability over the use of lethal force. Russia and

China also have not published anything equivalent to DoDD 3000.09, outlining their

policies and processes governing the acquisition, development, testing, and deployment

of autonomous weapon systems. Unlike in the United States, in Russia and China these

processes are secret, if they exist at all.

U.S. competitors have demonstrated that they are unlikely to adhere to the same ethical

and legal standards in developing and utilizing AI-enabled weapon systems. Russia in

particular has historically demonstrated a willingness to deploy risky and under-tested

weapon systems, and it has deployed poorly performing unmanned ground vehicles

“A global treaty prohibiting

the development, deployment,

or use of AI-enabled and

autonomous weapon systems is

not currently in the interest of

U.S. or international security ...”

AUTONOMOUS WEAPON SYSTEMS AND RISKS ASSOCIATED WITH AI-ENABLED WARFARE

with limited autonomous functionalities in combat in Syria.¹⁹ China is not only actively

pursuing increased autonomous functionality across a range of military systems, but it

is also currently exporting armed drones with autonomous functionalities to other nations.

This includes systems such as the Blowfish A3, which Ziyan, the system’s manufacturer,

advertises as capable of conducting autonomous, lethal, targeted strikes.²⁰

Judgment 4: The Commission does not support a global prohibition of AI-enabled and

autonomous weapon systems.

A global treaty prohibiting the development, deployment, or use of AI-enabled and

autonomous weapon systems is not currently in the interest of U.S. or international

security and would be inadvisable to pursue for several reasons:

•

First is the basic definitional problem. With respect to autonomous weapon systems,

although the UN discussions about LAWS date back to 2014, states have yet to agree

on a definition for them. This makes any treaty negotiation problematic, as it may be

impossible to define the category of systems to be restricted in such a way that provides

adequate clarity while not overly constraining existing U.S. military capabilities.

•

Even if the definitional problem could be overcome, we judge that, at present,

implementation of such an agreement would be impractical because compliance could

not be verified. There is no feasible technical manner in which states could demonstrate

to one another that specific weapon systems are or are not autonomous, or that they

possess or lack certain capabilities. Doing so would require foreign inspectors to have

short-notice access to the underlying code in weapon systems of concern. States

are unlikely to agree to such an intrusive verification regime because revealing that

information would create unacceptable risks to the security of their systems.

•

Additionally, the effects of a prohibition agreement likely would run counter to U.S.

strategic interests. Commitments from states such as Russia or China likely would be

empty ones. Such an agreement would not serve the goal of putting political pressure

on the states that are most likely to deploy autonomous weapon systems in unsafe

and ethically concerning ways. Rather, the primary impact of an agreement would be

to increase pressure on those countries that abide by international law, including the

United States and its democratic allies and partners. Moreover, differing views on a

prohibition among U.S. allies could deepen divisions among them on the employment

of AI-enabled autonomous weapon systems. If U.S. allies joined an agreement

while the United States did not, that divergence would likely hinder allied military

interoperability.²¹

For these reasons, we believe the practical and strategic problems with a prohibition

treaty outweigh potential benefits for the United States and its allies and partners,

and therefore we support the current U.S. policy in opposition to such an agreement.

However, this does not preclude other agreements or policies to address strategic

risks associated with AI-enabled and autonomous weapon systems, or the

future possibility of regulating specific types of technologies in AI-enabled and

autonomous weapons technologies when such an agreement could be verifiable.

CHAPTER 4

Recommendations to Mitigate Strategic Risks of AI.

While the Commission believes that properly designed, tested, and utilized AI-enabled and

autonomous weapon systems will bring substantial military and even humanitarian benefit, the

unchecked global use of such systems potentially risks unintended conflict escalation and

crisis instability. The United States cannot assume that AI-enabled and autonomous weapon

systems fielded by other countries will be developed, acquired, and fielded with the appropriate

testing and verification to enable them to act as intended. Unintended escalations may occur for

numerous reasons, including when systems fail to perform as intended, because of challenging

and untested complexities of interaction between AI-enabled and autonomous weapon systems

on the battlefield, and, more generally, as the result of machines or humans misperceiving

signals or actions. AI-enabled systems will likely increase the pace and automation of warfare

across the board, reducing the time and space available for de-escalatory measures. Beyond

testing and robustness, we cannot assume that AI-enabled and autonomous weapons

developed by other nations will be designed to behave in accordance with IHL.

Therefore, countries must take actions which focus on reducing risks associated with AI-

enabled and autonomous weapon systems and encourage safety and compliance with IHL

when discussing their development, deployment, and use. Such efforts should and must

be led by the United States, which is uniquely situated to lead them given its technical

expertise, military prowess, and clear and transparent policies and ethical principles

governing the deployment and use of AI-enabled and autonomous weapon systems. The

Commission presents the following five recommendations regarding actions the United

States should take to mitigate risks associated with AI-enabled and autonomous weapon

systems.

Strategic Risks

Associated with

NSCAI Recommended Actions

Objectives

AI-Enabled Weapons

U.S. Actions

Nation states could

Clearly and publicly affirm existing

allow AI to authorize

U.S. policy that only human beings

Prevent unintended nuclear

employment of key

can authorize employment of

conflict due to AI-enabled

strategic weapon

nuclear weapons, and seek similar

launch authorization.

systems.

commitments from Russia and China.

Enable effective verification

States cannot verify

of potential future

Pursue technical means to verify

compliance with

agreements, which provides

compliance with future arms control

potential international

confidence systems are

agreements pertaining to AI-enabled

agreements pertaining

working as intended

and autonomous weapon systems.

to AI-enabled weapons.

without revealing sensitive

operational details.

AUTONOMOUS WEAPON SYSTEMS AND RISKS ASSOCIATED WITH AI-ENABLED WARFARE

Strategic Risks

Associated with

NSCAI Recommended Actions

Objectives

AI-Enabled Weapons

Design and incorporate

proliferation-resistant

Global, unregulated

Fund research on technical means

features into sophisticated

proliferation of

to prevent proliferation of AI-enabled

AI-enabled and autonomous

AI-enabled and

and autonomous weapon systems.

weapons, and potentially

autonomous weapons.

share them with Russia and

China.

U.S. Actions with Allies

Set international norms

Poorly designed or

Develop international standards of

guiding responsible

improperly utilized AI-

practice for the development and

development and use of

enabled weapons could

use of AI-enabled and autonomous

AI-enabled and autonomous

behave unpredictably.

weapon systems.

U.S. Actions with Russia and China

Improve understanding

Discuss AI’s impact on crisis stability

of doctrine and develop

AI-enabled systems

in the existing U.S.-Russia Strategic

confidence-building

could cause inadvertent

Security Dialogue and create an

measures regarding use of

conflict escalation.

equivalent meaningful dialogue with

AI-enabled and autonomous

China.

weapon systems.

Clearly and publicly affirm existing U.S. policy that only human beings can authorize employment

Recommendation

of nuclear weapons, and seek similar commitments from Russia and China. The United States

should make a clear, public statement that decisions to authorize nuclear weapons

employment must only be made by humans, not by an AI-enabled or autonomous system,

and should include such an affirmation in the DoD’s next Nuclear Posture Review.²² This

would cement and highlight existing U.S. policy, which states that “[t]he decision to employ

nuclear weapons requires the explicit authorization of the President of the United States.”²³

It would also demonstrate a practical U.S. commitment to employing AI and autonomous

functions in a responsible manner, limiting irresponsible capabilities, and preventing AI

systems from escalating conflicts in dangerous ways. It could also have a stabilizing effect,

as it would reduce competitors’ fears of an AI-enabled, bolt-from-the-blue strike from the

United States and could incentivize other countries to make equivalent pledges.

The United States should also actively press Russia and China, as well as other states

that possess nuclear weapons, to issue similar statements. Although joint political

commitments that only humans will authorize employment of nuclear weapons would not

be verifiable, they could still be stabilizing, responding to a classic prisoner’s dilemma: as

long as countries have confidence that others are not building risky command and control

structures that have the potential to inadvertently trigger massive nuclear escalation, they

would have less incentive to develop such systems themselves.²⁴ While this norm is widely

accepted in the United States, it is unclear if Russia and China share the same strategic

CHAPTER 4

“... countries must take actions

which focus on reducing risks

associated with AI-enabled and

autonomous weapon systems,

and encourage safety and

compliance with IHL when

discussing their development,

deployment, and use. Such

efforts should and must be led

by the United States ...”

concerns. Public reports indicate that Russia previously installed a “dead hand” system to

automate nuclear launch authorization,²⁵ and China’s representatives in Track II dialogues

with the United States have been hesitant to state that China would make an equivalent

commitment. If neither Russia nor China is willing to agree to such a proposal, the United

States should mount a strong international pressure campaign to condemn this decision

and highlight how Russia and China refuse to commit to responsible military uses of AI.

Discuss AI’s impact on crisis stability in the existing U.S.-Russia Strategic Security Dialogue

Recommendation

(SSD) and create an equivalent meaningful dialogue with China. The Departments of State

and Defense should discuss AI’s impact on crisis stability within the existing U.S.-Russia

SSD and create an equivalent meaningful dialogue with China. The SSD is an interagency

bilateral dialogue focused on reducing misunderstandings and misperceptions on key

strategic issues and threats, as well as reducing the likelihood of inadvertent escalation.

Although the dialogue has traditionally focused on nuclear arms control and doctrine, it

has recently been used to also discuss emerging technologies and space security.²⁶

The United States has no equivalent dialogue with China, as China has resisted U.S.

attempts to establish one for nearly a decade. However, within the last year there has

been increasing evidence that China is interested in formal talks with the United States

concerning AI-enabled military systems.²⁷ This interest should be cultivated and leveraged

into establishing a U.S.-China SSD that includes the relevant military, diplomatic, and

security officials from both sides.

AUTONOMOUS WEAPON SYSTEMS AND RISKS ASSOCIATED WITH AI-ENABLED WARFARE

Given that the United States, Russia, and China are all aggressively pursuing AI-enabled

capabilities, and that Russia and China are likely to field AI-enabled systems that have

undergone less rigorous TEVV than comparable U.S. systems and may be unsafe or unreliable,

it is crucial to improve mutual understanding of each other’s military doctrines, including with

respect to AI-enabled and autonomous systems. The United States should use this channel to

highlight how deploying unsafe systems could risk inadvertent conflict escalation, emphasize

the need to conduct rigorous TEVV, and discuss where each side sees risks of a conventional

conflict rapidly escalating in order to better anticipate future responses in a crisis.

“... it is crucial to improve mutual

understanding of each other’s

military doctrines, including

with respect to AI-enabled and

autonomous systems.”

These dialogues could also plant the seeds for a future, standing dialogue exclusively

focused on establishing practical and concrete confidence building measures surrounding

AI-enabled and autonomous weapon systems. For instance, the United States, Russia, and

China could work to develop an “international autonomous incidents agreement,” modeled

after the 1972 Incidents at Sea Agreement, which would seek to define the “rules of the

road” for behavior of autonomous military systems to create a more predictable operating

environment and avoid accidents and miscalculations.²⁸ They could also agree to integrate

“automated escalation tripwires” into systems that would prevent the automated escalation

of conflict in specific scenarios without human intervention, to include nuclear weapons

employment as noted above.

Work with allies to develop international standards of practice for the development, testing, and

Recommendation

use of AI-enabled and autonomous weapon systems. The United States must work closely

with its allies to develop standards of practice regarding how states should responsibly

develop, test, and employ AI-enabled and autonomous weapon systems. This could build

off of existing work, to include the 11 Guiding Principles agreed to by the LAWS Group of

Governmental Experts (GGE) in 2019,²⁹ DoDD 3000.09, the DoD Ethical Principles for AI,

and the NSCAI Key Considerations for Responsible Development and Fielding of AI.³⁰ As

part of this effort, the DoD Law of War Working Group should meet regularly to review any

future technical developments that pertain to autonomous weapon systems and IHL, and

the tri-chaired Steering Committee on Emerging Technology (separately recommended

100

CHAPTER 4

by the Commission in Chapter 3 of this report) should advise on how such future technical

developments impact policy and national defense.

The outputs of both groups should inform future DoD engagements with both allies and

competitors on AI-enabled and autonomous weapon systems. Obtaining allied consensus

regarding standards for the development, testing, and use of such systems will set

important norms regarding these systems, help to ensure they are developed and used

safely, and further highlight the commitment of the United States and its allies to ethical

and responsible uses of AI. The United States should also use these consultations to

highlight the ways in which AI will become a crucial part of future military operations and

develop common frameworks guiding the appropriate and responsible use of AI-enabled

and autonomous weapon systems on the battlefield. This should seek to incentivize allies

to invest in the digital modernization of their own forces while also highlighting the risks to

military interoperability should any ally agree to join a treaty prohibiting LAWS.

Pursue technical means to verify compliance with future arms control agreements pertaining to

Recommendation

AI-enabled weapon systems. The United States should actively pursue the development of

technologies and strategies that could enable effective and secure verification of future

arms control agreements involving uses of AI technologies. Although arms control of AI-

enabled weapon systems is currently technically unverifiable, effective verification will

likely be necessary to achieve future legally binding restrictions on AI capabilities. DoD

and the Department of Energy (DoE) should spearhead efforts to design and implement

technologies which could provide other countries confidence that an AI-enabled and

autonomous weapon system is working as intended without revealing sensitive operational

details. For instance, it could examine ways for AI-enabled weapons platforms to produce

authenticatable records of operation, which could be spot-checked via international

challenge inspections if noncompliant activity is suspected. Technical creativity will be

necessary to enable any future international restrictions on AI capabilities without revealing

sensitive information.

Fund research on technical means to prevent proliferation of AI-enabled and autonomous

Recommendation

weapon systems. Controlling the proliferation of AI-enabled and autonomous weapon

systems poses significant challenges given the open-source, dual-use, and inherently

transmissible nature of AI algorithms.³¹ The proliferation of makeshift autonomous weapon

systems which primarily utilize commercial components will be particularly difficult to

control via regulation and will necessitate capable intelligence sharing and domestic law

enforcement efforts to prevent their use by terrorists and other non-state actors. Regarding

more sophisticated autonomous weapon systems, the United States should double down

on efforts to design and incorporate proliferation-resistant features, such as standardized

ways to prevent unauthorized users from utilizing such weapons, or reprogramming a

system’s functionality by changing key system parameters. DoD and DoE should fund

technical research on such methods, and if appropriate, these methods could be shared

with Russia and China, or potentially other countries, to prevent the proliferation or loss of

control of certain AI-enabled autonomous weapon systems.³²

101

CHAPTER 4

Chapter 4 - Endnotes

¹ IHL is also referred to as the law of armed conflict (LOAC) and the law of war.

² Paul Scharre, Army of None: Autonomous Weapons and the Future of War, W.W. Norton & Co. at 39

(April 24, 2018).

³ Background on Lethal Autonomous Weapon systems in the CCW, United

Nations (last accessed Jan. 11, 2021), https://www.unog.ch/80256EE600585943/

(httpPages)/8FA3C2562A60FF81C1257CE600393DF6?OpenDocument.

⁴ Distinction, International Committee of the Red Cross (last accessed Jan. 15, 2021), https://

casebook.icrc.org/glossary/distinction.

⁵ There is room for improvement in reducing target misidentification in U.S. military operations. In

the Afghanistan war, for example, a study indicated that about half of all civilian casualty incidents

caused by U.S. forces resulted from target misidentification. The use of AI-enabled systems to make

more accurate targeting decisions is perhaps the principal way in which the proper employment

of AI could make warfare more humane. Larry Lewis, Redefining Human Control: Lessons from the

Battlefield for Autonomous Control, CNA at 4 (March 2018), https://www.cna.org/cna_files/pdf/DRM-

2017-U-016281-Final.pdf.

⁶ Proportionality, International Committee of the Red Cross (last accessed Jan. 15, 2021), https://

casebook.icrc.org/glossary/proportionality.

⁷ See Paul Scharre, Army of None: Autonomous Weapons and the Future of War, W.W. Norton & Co. at

255-257 (2018).

⁸ For a properly designed and tested autonomous system which correctly carries out the commander’s

intent, the commander is clearly accountable for the actions of that system. It is incumbent on states

to properly design, test, and use such systems and also put in place rigorous procedures ensuring

that any weapon use complies with IHL, including by ensuring individual accountability.

⁹ The Commission believes DoD’s existing formulation of “appropriate human judgment,” discussed

in the following Judgment, captures that necessary variation and ensures that any decision to employ

lethal force begins with and is under the control of human judgment, and that a human ultimately will

remain accountable for any decision to employ force.

¹⁰ Press Release, U.S. Department of Defense, DoD Adopts Ethical Principles for Artificial Intelligence

(Feb. 24, 2020), https://www.defense.gov/Newsroom/Releases/Release/Article/2091996/dod-adopts-

ethical-principles-for-artificial-intelligence/.

¹¹ DoDD 5000.01 requires any weapon fielded by DoD to undergo a legal review to ensure compliance

with the Law of Armed Conflict (LOAC), adhering to the requirements set out in Article 36 of the

Protocol Additional to the Geneva Conventions of 12 August 1949. DoDD 3000.09 and the DoD AI

Ethics Principles build on top of this baseline. See Department of Defense Directive 5000.01: The

Defense Acquisition System, U.S. Department of Defense at 9 (Sept. 9, 2020), https://www.esd.whs.

mil/Portals/54/Documents/DD/issuances/dodd/500001p.pdf?ver=2020-09-09-160307-310; Protocol

Additional to the Geneva Conventions of 12 August 1949, and relating to the Protection of Victims of

International Armed Conflicts (Protocol I), 8 June 1977, International Committee of the Red Cross (last

accessed Jan. 5, 2021), https://ihl-databases.icrc.org/applic/ihl/ihl.nsf/WebART/470-750045.

¹² Department of Defense Directive No. 2311.01: DoD Law of War Program, U.S. Department of

Defense at 11 (July 2, 2020), https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/

dodd/231101p.pdf?ver=2020-07-02-143157-007.

¹³ A Guide to the Legal Review of New Weapons, Means and Methods of Warfare: Measures to

Implement Article 36 of Additional Protocol I of 1977, International Committee of the Red Cross at 5, n.

8 (Jan. 2006), https://www.icrc.org/en/doc/assets/files/other/icrc_002_0902.pdf.

¹⁴ Department of Defense Directive 3000.09: Autonomy in Weapon systems, U.S. Department of

Defense at 2 (Nov. 21, 2012, incorp. change 1 May 8, 2017), https://www.esd.whs.mil/portals/54/

documents/dd/issuances/dodd/300009p.pdf. The weapons-review processes established in DoDD

3000.09 are designed specifically to ensure that any U.S. autonomous weapon system complies with

IHL principles such as discrimination and proportionality while also maintaining appropriate levels of

human judgment and ensuring accountability.

103

AUTONOMOUS WEAPON SYSTEMS AND RISKS ASSOCIATED WITH AI-ENABLED WARFARE

¹⁵ Department of Defense Instruction 5025.01: DoD Issuances Program at 22 (Aug. 1, 2016, incorp.

change 3 May 22, 2019), https://www.esd.whs.mil/Portals/54/Documents/DD/issuances/dodi/502501p.

pdf?ver=2020-05-20-081854-657.

¹⁶ See the Appendix of this report containing the abridged version of NSCAI’s Key Considerations

for Responsible Development & Fielding of AI. For additional details on the Commission’s

recommendation for future R&D needed to advance capabilities for Testing, Evaluation, Verification,

and Validation of AI systems, see the section on “System Performance” in Key Considerations for

Responsible Development & Fielding of Artificial Intelligence: Extended Version, NSCAI (2021) (on file

with the Commission).

¹⁷ The DoD Law of War manual serves as a detailed resource for all DoD personnel responsible for

implementing the law of war and executing military operations. See Department of Defense Law of

War Manual, U.S. Department of Defense (Dec. 2016), https://dod.defense.gov/Portals/1/Documents/

pubs/DoD%20Law%20of%20War%20Manual%20-%20June%202015%20Updated%20Dec%202016.

pdf?ver=2016-12-13-172036-190.

¹⁸ Press Release, U.S. Department of Defense, DoD Adopts Ethical Principles for Artificial Intelligence

(Feb. 24, 2020), https://www.defense.gov/Newsroom/Releases/Release/Article/2091996/dod-adopts-

ethical-principles-for-artificial-intelligence/.

¹⁹ David Axe, Don’t Panic, But Russia Is Training its Robot Tanks to Understand Human Speech,

Forbes (June 30, 2020), https://www.forbes.com/sites/davidaxe/2020/06/30/dont-panic-but-russia-is-

training-its-robot-tanks-to-understand-human-speech/?sh=7373377914f2.

²⁰ Patrick Tucker, SecDef: China Is Exporting Killer Robots to the Mideast, Defense One (Nov. 5,

2019), https://www.defenseone.com/technology/2019/11/secdef-china-exporting-killer-robots-

mideast/161100/.

²¹ The United States has expressed similar concerns with respect to treaties banning cluster munitions

and nuclear weapons. See Q&A: Convention on Cluster Munitions, HRW (Nov. 6, 2010), https://www.

hrw.org/news/2010/11/06/qa-convention-cluster-munitions#; Heather Williams, What the Nuclear

Ban Treaty Means for America’s Allies, War on the Rocks (Nov. 5, 2020), https://warontherocks.

com/2020/11/what-the-nuclear-ban-treaty-means-for-americas-allies/. As of March 2021, no ally with

which the United States has a mutual defense agreement has expressed support for a treaty banning

LAWS.

²² The Commission recognizes that AI should assist in some aspects of the nuclear command and

control apparatus, such as early warning, early launch detection, and multi-sensor fusion to validate

single sensor detections and potentially eliminate false detections.

²³ Nuclear Matters Handbook 2020, Office of the Deputy Assistant Secretary of Defense for Nuclear

Matters at 18 (2020), https://fas.org/man/eprint/nmhb2020.pdf.

²⁴ There could be other reasons countries may delegate nuclear weapons launch authority to

autonomous systems, particularly if leadership trusts machines to execute launch orders more than

humans. A political agreement is unlikely to be able to address these concerns, although offering it

would highlight how other nations are engaging in irresponsible and dangerous behavior.

²⁵ Michael Peck, Russia’s ‘Dead Hand’ Nuclear Doomsday Weapon is Back, The National Interest

(Dec. 12, 2018), https://nationalinterest.org/blog/buzz/russias-dead-hand-nuclear-doomsday-weapon-

back-38492.

²⁶ Press Release, U.S. Department of State, Deputy Secretary Sullivan’s Participation in Strategic

Security Dialogue with Russian Deputy Foreign Minister Sergey Ryabkov (July 17, 2019),

https://2017-2021.state.gov/deputy-secretary-sullivans-participation-in-strategic-security-

dialogue-with-russian-deputy-foreign-minister-sergey-ryabkov/index.html; Press Release, U.S.

Department of State, The United States and Russia Hold Space Security Exchange (July 28, 2020),

https://2017-2021.state.gov/the-united-states-and-russia-hold-space-security-exchange/index.html.

²⁷ Over the last year, Chinese experts have participated actively in several Track II dialogues with U.S.

experts on the safety of military AI systems, potentially signaling a desire for formal government-to-

government communication on these issues.

104

CHAPTER 4

Chapter 4 - Endnotes

²⁸ See Michael C. Horowitz and Paul Scharre, AI and International Stability: Risks and Confidence-

Building Measures, Center for a New American Security (Jan. 2021), https://s3.us-east-1.amazonaws.

com/files.cnas.org/documents/AI-and-International-Stability-Risks-and-Confidence-Building-

Measures.pdf?mtime=20210112103229&focal=none.

²⁹ Final Report of the 2019 Session of the Group of Governmental Experts on Emerging Technologies

in the Area of Lethal Autonomous Weapon systems, Group of Governmental Experts of the

High Contracting Parties to the Convention on Prohibitions or Restrictions on the Use of Certain

Conventional Weapons Which May Be Deemed to Be Excessively Injurious or to Have Indiscriminate

Effect, CCW/MSP/2019/CRP.2/Rev.1, (Nov. 13-15, 2019), https://undocs.org/CCW/MSP/2019/9.

³⁰ See the Appendix of this report containing the abridged version of NSCAI’s Key Considerations

for Responsible Development & Fielding of AI. For additional details on the Commission’s

recommendation for future action on International collaboration and cooperation, see the section on

“System Performance” in Key Considerations for Responsible Development & Fielding of Artificial

Intelligence: Extended Version, NSCAI (2021) (on file with the Commission).

105

CHAPTER 5

Intelligence will benefit from rapid adoption of artificial intelligence (AI)-

enabled technologies more than any other national security mission.

As every possible platform—both machine and human—contributes

to the global information grid, and as the number of sensors grows

exponentially, the volume, velocity, and variety of data threaten to

overwhelm intelligence analysis. Ascertaining the veracity and value

of information will be harder. Analysts will be challenged to provide

the context crucial for turning information into actionable intelligence.

AI will help intelligence professionals find needles in haystacks, connect the dots, and disrupt

dangerous plots by discerning trends and discovering previously hidden or masked indications

and warnings. AI-enabled capabilities will improve every stage of the intelligence cycle from

tasking through collection, processing, exploitation, analysis, and dissemination. AI algorithms

can sift through vast amounts of data to find patterns, detect threats, identify correlations, and

make predictions. AI tools can make satellite imagery, communications signals, economic

indicators, social media data, and other large sources of information more intelligible. AI

can find correlations between open-source data and other sources of intelligence, and help

the Intelligence Community (IC) be more precise, efficient, and effective in its targeting and

collections activities. The constellation of current and emerging AI technologies applicable to

intelligence missions includes computer vision for imagery analysis, biometric technologies

(such as face, voice, and gait recognition), natural language processing, and algorithmic search

and query functions for large databases, among others. Most important, AI enables data fusion

from dissimilar data streams to create a composite picture.¹

In military scenarios—against technologically advanced adversaries, rogue states, or terrorist

organizations—AI-enabled intelligence, surveillance, and reconnaissance platforms and AI-

enabled indication and warning (I&W) systems will be critical for the kind of advanced warfighting

capabilities discussed in Chapter 3 of this report. Through automation, AI-enabled systems will

optimize tasking and collection for platforms, sensors, and assets in near-real time in response

to dynamic intelligence requirements or changes in the environment. At the tactical edge,

“smart” sensors will be capable of pre-processing raw intelligence and prioritizing the data to

transmit and store, which will be especially helpful in degraded or low-bandwidth environments.

Once collected, intelligent processing systems can triage the information, identify trends and

patterns, summarize key implications, and prepare the highest-priority information for human

review (or flag items of particular interest, based on analyst-defined conditions). This includes

advanced I&W systems that will enable warfighters to anticipate and understand emerging

threats earlier, allowing them to proactively shape the environment, as well as systems close to

the tactical edge identifying adversarial denial and deception efforts. When paired with human

judgment, these capabilities will enhance all-domain awareness, lead to tighter and more

informed decision cycles, offer recommendations for different courses of action, and allow

rapid counter-actions to adversary actions.

109

AI AND THE FUTURE OF NATIONAL INTELLIGENCE

The need to adapt is made urgent by the quickening diffusion of these new technologies.

Once exquisite IC capabilities are now in wide use around the world.² Our adversaries’ ability

to quickly adopt AI tools means that the IC may be more vulnerable to deception, information

operations, sources and methods exposure, cyber operations, and counterintelligence

activities. The IC has been an early mover within the government in establishing some of

the underlying infrastructure to enable the adoption of AI, such as contracting an IC-wide

commercial cloud service in 2013.³ In addition, the IC’s 2019 Augmenting Intelligence

using Machines (AIM) initiative provided direction and a framework for broader adoption,

and some intelligence agencies have made great strides in AI adoption, putting them

ahead of others in government. Still, critical barriers in authorities, policies, budgets, data

sharing, and technical standards keep the IC from fully realizing its potential, and none

of these recommendations will be effective without substantial reforms of the security

clearance process.

An Ambitious Agenda: AI-Ready by 2025.

To build on the progress that individual agencies have made, the IC should set the

ambitious goal of adopting and integrating AI-enabled capabilities across every possible

aspect of the intelligence enterprise as part of a larger vision for the future of intelligence.

An AI-Ready IC by 2025:

Intelligence professionals enabled with baseline digital literacy

and access to the digital infrastructure and software required for

ubiquitous AI integration in each stage of the intelligence cycle.

Starting immediately, the IC should prioritize automating each stage of the intelligence

cycle to the greatest extent possible and processing all available data and information

through AI-enabled analytic systems before human analyst review. Products should also

be disseminated at machine speed-which means they must be in machine-readable

formats-and systems across the IC must be able to ingest and use them without manual

intervention. Optimizing AI-enabled systems in this way will require an entirely different

approach to the creation and review of finished intelligence products. The IC should

require that all intelligence products include both a human-readable version and, just

as important, an automated machine-readable version that can be ingested into other

analytic systems throughout the IC. All future intelligence systems should be optimized for

AI-oriented data collection and processing.

110

CHAPTER 5

Preparing for an AI-ready 2025 demands the following actions:

Empower the IC’s science and technology leadership. The Director of National Intelligence

Recommendation

(DNI) should designate the Director of Science and Technology (S&T) within the Office of

the Director of National Intelligence (ODNI) as the IC’s Chief Technology Officer (CTO) and

task and empower this position to drive the IC’s adoption of AI-enabled applications to

solve operational intelligence requirements. To do so, the IC CTO should oversee the AIM

strategy, establish and enforce common technical standards and policies necessary to

rapidly and responsibly scale AI-enabled applications across the IC, and lead acquisition

reform to ensure that the IC can rapidly procure and field systems to its intelligence

professionals. The IC CTO should be granted additional authorities for establishing policies

on and supervising IC research and engineering, technology development, technology

transition, appropriate prototyping activities, experimentation, and developmental testing

activities.

Change risk management practices to accelerate new technology adoption. The IC needs

Recommendation

to balance the technical risks involved in bringing new technologies online and quickly

updating them with the substantial operational risks that result from not keeping pace,

similar to DoD. Regular software upgrades should be automated to the extent possible. To

share software tools more easily among agencies, reciprocal accreditation of information

technology systems should be the standard.⁴

“The IC needs to balance

the technical risks involved

in bringing new technologies

on line and quickly updating

them with the substantial

operational risks that result

from not keeping pace ...”

113

AI AND THE FUTURE OF NATIONAL INTELLIGENCE

To coordinate these changes, the ODNI should establish a Senior Risk Management

Council focused on technology modernization.⁵ Its task should be to weigh the risks of

adopting new technologies with the opportunity costs of not doing so. Its goal should be

to ensure that analysts have access to the tools they need to do their jobs.

The IC will need support from the intelligence committees in Congress--for example, in the

flexible use of funds within a more agile software development framework. To support the

argument for greater flexibility, the IC should develop data-driven ways of communicating

operational gains, as well as credible assessments of the risk of inaction.

Improve coordination and interoperability between the IC and DoD. The IC must aggressively

Recommendation

pursue automated interoperability with the DoD for intelligence operations conducted at

machine speeds.⁶ To do this, security managers and network administrators must build

greater confidence in fast and secure data exchanges. ODNI, the Under Secretary of

Defense for Intelligence and Security, and the Joint Artificial Intelligence Center (JAIC)

should coordinate more on intelligence-related AI projects to minimize duplication of

effort while maximizing common approaches to AI capability development, testing and

evaluation, deployment, international engagement, and policies and authorities. They

should work together to create interoperable and sharable resources and tools--such

as those envisioned in the AI R&D ecosystem described in Chapter 2 of this report--and

should establish a culture of sharing all AI-enabled capabilities whenever feasible.⁷

Capitalize on AI-enabled analysis of open-source and publicly available information.⁸ The IC

Recommendation

should develop a coordinated and federated approach to applying AI-enabled applications

to open-source intelligence (OSINT) and should strive to integrate open-source analysis

into existing intelligence processes wherever possible in every intelligence domain.⁹

Prioritize and accelerate collection of scientific and technical intelligence to better understand

Recommendation

adversary capabilities and intentions. Such collection requires the IC to significantly increase

the technical sophistication, capabilities, and capacity of its analytic workforce. That must

involve aggressive efforts to train, recruit, and retain analysts who have the requisite

skills. These analysts must guide collection requirements and provide timely, accurate

assessments. To better coordinate intelligence on these topics, including collecting on

scientific and technical cooperation among our competitors, the DNI should appoint an

Emerging Technology Collection Executive within the National Intelligence Council.¹⁰

To recruit more S&T experts into the IC, aggressively pursue security clearance reform for

Recommendation

clearances at the Top Secret level and above, and enforce security clearance reciprocity

among members of the IC. ODNI should develop and implement an AI-enabled data and

science-based approach to security-clearance adjudication that significantly shortens

investigation timelines.¹¹

114

CHAPTER 5

Advance and continue to develop a purpose-built IC Information Technology Environment that

Recommendation

can fuse intelligence from different domains and sources. An AI-enabled technical architecture

of this kind could help autonomously integrate intelligence across stove-piped intelligence

domains, which currently often require manual intervention to share raw data or finished

analysis.¹² Doing so would help the IC blend insights from different streams of information to

create a composite picture. For example, signals intelligence often depends upon human

intelligence or geospatial intelligence. Likewise, the value of human intelligence can almost

always be enhanced by layering signals intelligence or open-source information on top of it.

Embrace fused, predictive analysis as the new standard. Successfully fusing all-source/all-

Recommendation

domain intelligence will enable accurate predictive analysis in a way that is not currently

possible. The government’s response to the COVID-19 virus has offered glimpses into

the potential for fused data sets to inform such analysis. For example, U.S. Northern

Command (working with the JAIC and the National Guard Bureau) built predictive models

from dozens of different data sets that helped to identify COVID-19 hotspots and reconcile

demands for critical supplies.¹³

Develop innovative human-centric approaches to human-machine teaming. The kind of data

Recommendation

fusion envisioned here through autonomous machine-to-machine integration will require

new concepts for human-machine teaming that optimize the strengths of each.¹⁴ The IC

will need new approaches that amplify and extend human cognition to effectively handle

the scale and complexity of the information generated by all-source intelligence analytic

engines. When developing these systems, the IC must understand and make deliberate

decisions on when and under what conditions the human or machine should act alone and

under what conditions human-machine teaming is desirable.

“The kind of data fusion

envisioned here through

autonomous machine-to-

machine integration will require

new concepts for human-

machine teaming that optimize

the strengths of each.”

115

AI AND THE FUTURE OF NATIONAL INTELLIGENCE

Chapter 5 - Endnotes

¹ For additional information on AI-enabled use cases throughout the intelligence cycle, see the

discussion on “Applications” in Maintaining the Intelligence Edge: Reimagining and Reinventing

Intelligence Through Innovation, CSIS Technology and Intelligence Task Force at 8-22 (Jan. 13, 2021),

https://csis-website-prod.s3.amazonaws.com/s3fs-public/publication/210113_Intelligence_Edge.pdf.

² AIM Initiative: A Strategy for Augmenting Intelligence Using Machines, Office of the Director of

National Intelligence (2019), https://www.dni.gov/files/ODNI/documents/AIM-Strategy.pdf (foreword by

the Honorable Sue Gordon, Principal Deputy Director of National Intelligence).

³ Frank Konkel, The Details about the CIA’s Deal with Amazon, The Atlantic (July 14, 2014),

https://www.theatlantic.com/technology/archive/2014/07/the-details-about-the-cias-deal-with-

amazon/374632/.

⁴ In adopting new software systems, the IC follows a risk-management framework developed by the

National Institute of Standards and Technology (NIST). While it is a useful framework overall, it can

also create delays or prevent the IC from keeping up with cutting-edge AI tools that are commercially

available. For more information, see FISMA Implementation Project, NIST (Dec. 3, 2020), https://csrc.

nist.gov/projects/risk-management/rmf-overview.

⁵ The Senior Risk Management Council would help the IC implement guidance from the proposed

Tri-Chair Committee on Emerging Technology and function similarly to the role this commission

recommended for the Under Secretary of Defense for Research and Engineering as a co-chair on the

Joint Requirements Oversight Council in DoD.

⁶ For more information, see Kent Linnebur, et al., Intelligence After Next: The Future of the IC

Workplace, MITRE Center for Technology and National Security (Nov. 1, 2020), https://www.mitre.org/

sites/default/files/publications/pr-20-1891-intelligence-after-next-the-future-of-the-ic-workplace.pdf.

⁷ These efforts should leverage the JAIC’s Joint Common Foundation (JCF).

⁸ Pub. L. 116-260, The Consolidated Appropriations Act (2021), Division W, Section 326 (“Open

source intelligence strategies and plans for the intelligence community”), Section 623 (“Independent

study on open-source intelligence”), and Section 624 (“Survey on Open Source Enterprise”) provide a

starting point for the IC to reimagine the role of open-source intelligence.

⁹ It is important to note that open-source intelligence (OSINT) is not limited to traditional media

sources (newspapers, radio broadcasts, etc.) and social media. OSINT also includes publicly

available information such as public government data sources (official reports, budget documents,

hearing testimonies, etc.), professional and academic publications, commercial data sources

(industry reports, financial statements, commercial imagery, etc.), and more.

¹⁰ For additional information, see the discussion on “Elevating Technical Intelligence” in Maintaining

the Intelligence Edge: Reimagining and Reinventing Intelligence Through Innovation, CSIS Technology

and Intelligence Task Force at 12 (Jan. 13, 2021), https://csis-website-prod.s3.amazonaws.com/s3fs-

public/publication/210113_Intelligence_Edge.pdf.

¹¹ For more information on the need for an academic and scientific review of behavioral approaches

to security clearance adjudication, see David Luckey, et al., Assessing Continuous Evaluation

Approaches for Insider Threats: How Can the Security Posture of the U.S. Departments and Agencies

Be Improved?, RAND Corporation at 28-34 (2019), https://www.rand.org/pubs/research_reports/

RR2684.html.

116

National Security Commission on Artificial Intelligence. Final Report - page 2