NIST · NIST AI
An AI system can malfunction if an adversary flags a way to confuse its decision making
Compiled by KHAO Editorial — aggregated from 1 source. See llms.txt for citation guidance.
★ Tier-1 Source
In this example, errant markings on the road mislead a driverless car, potentially making it veer into oncoming traffic.
Key facts
- Their work, titled Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST.AI.100-2), is part of NIST’s broader effort to support the development of trustworthy AI
- Most of these attacks are fairly easy to mount and require minimum knowledge of the AI system and limited adversarial capabilities,” said co-author Alina Oprea, a professor at Northeastern University
- We are providing an overview of attack techniques and methodologies that consider all types of AI systems,” said NIST computer scientist Apostol Vassilev, one of the publication’s authors
- An AI system can malfunction if an adversary finds a way to confuse its decision making
Summary
An AI system can malfunction if an adversary finds a way to confuse its decision making. Adversaries can deliberately confuse or even “poison” artificial intelligence (AI) systems to make them malfunction, and there’s no foolproof defense that their developers can employ. Their work, titled Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST.AI.100-2), is part of NIST’s broader effort to support the development of trustworthy AI, and it can help put NIST’s AI Risk Management Framework into practice. “We are providing an overview of attack techniques and methodologies that consider all types of AI systems,” said NIST computer scientist Apostol Vassilev, one of the publication’s authors. AI systems have permeated modern society, working in capacities ranging from driving vehicles to helping doctors diagnose illnesses to interacting with customers as online chatbots.