To help understand the evolving cybersecurity environment, we developed HiddenLayer’s 2024 AI Threat Landscape Report as a practical guide to understanding the security risks that can affect every industry and to provide actionable steps to implement security measures at your organization.
Understanding Advancements in Security for AI
Understanding new technologies’ vulnerabilities is crucial before implementing security measures. Offensive security research plays a significant role in planning defenses, as initial security measures are often built on the foundation of these offensive insights.
Security for AI is no exception. Early research and tools in this field focused on offensive strategies. Initially, AI attacks were mainly explored in academic papers and through exercises by security professionals. However, there has been a significant shift in the last few years.
Offensive Security Tooling for AI
Just as in traditional IT security, offensive security tools for AI have emerged to identify and mitigate vulnerabilities. While these tools are valuable for enhancing AI system security, malicious actors can also exploit them.
Automated Attack Frameworks
Pioneering tools like CleverHans (2016) and IBM’s Adversarial Robustness Toolbox (ART, 2018) have paved the way for testing AI comprehensively. Subsequent tools such as MLSploit (2019), TextAttack (2019), Armory (2020), and Counterfit (2021) have further advanced the field, offering a variety of attack techniques to evaluate AI defenses.
Anti-Malware Evasion Tooling
Specialized tools like MalwareGym (2017) and its successor MalwareRL (2021) focus on evading AI-based anti-malware systems. These tools highlight the need for continuous improvement in security for AI measures.
Model Theft Tooling
KnockOffNets (2021) demonstrates the feasibility of AI model theft, emphasizing the importance of securing AI intellectual property.
Model Deserialization Exploitation
Fickling (2021) and Charcuterie (2022) showcase vulnerabilities in AI model serialization, underscoring the need for secure model handling practices.
Defensive Frameworks for AI
Leading cybersecurity organizations have developed comprehensive defensive frameworks to address the rising threats to AI.
MITRE ATLAS
Launched in 2021, MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) provides a knowledge base of adversarial tactics and techniques. Modeled after the MITRE ATT&CK framework, ATLAS helps professionals stay updated on AI threats and defenses.
“This survey demonstrates the prominence of real-world threats on AI-enabled systems, with 77% of participating companies reporting breaches to their AI applications this year. The MITRE ATLAS community is dedicated to characterizing and mitigating these threats in a global alliance. We applaud our community collaborators who enhance our collective ability to anticipate, prevent, and mitigate risks to AI systems, including HiddenLayer and their latest threat report.”
– Dr. Christina Liaghati, MITRE ATLAS Lead
NIST AI Risk Management Framework
Released in January 2023, the NIST AI Risk Management Framework (AI RMF) offers guidance for the responsible design, deployment, and use of AI systems, promoting trust and security in AI.
Google Secure AI Framework (SAIF)
Introduced in June 2023, SAIF outlines best practices for securing AI systems, emphasizing strong security foundations, automated defenses, and contextualized risk management.
Policies and Regulations
Global policies and regulations are being established to ensure AI’s safe and ethical use. The EU’s GDPR and AI Act, OECD AI Principles, and national frameworks like Singapore’s Model AI Governance Framework and the US’s AI Bill of Rights highlight the growing emphasis on security for AI and governance.
Concluding Thoughts
As AI technology evolves, so must the security measures that secure it. By combining offensive and defensive strategies, leveraging comprehensive frameworks, and adhering to evolving regulations, the industry can better safeguard AI systems against emerging threats. Collaboration between academia, industry, and policymakers is essential to anticipate and mitigate risks effectively.
Continuous innovation and vigilance in security for AI will be crucial in maintaining trust and reliability in AI applications, ensuring they can be safely integrated into various sectors.
View the full Threat Landscape Report here.
