AI Security & Red-Teaming Micro-Credential

AI Security & Red-Teaming Micro-Credential

Duration: 3-4 Hours

Description

An LLM-powered application has an attack surface a traditional app doesn’t: the prompt itself. This session is a hands-on, developer-focused introduction to attacking and defending AI systems, built around the OWASP Top 10 for LLM Applications. Participants attempt prompt injection and jailbreak techniques against a sandboxed demo application, then implement the corresponding defenses — input and output filtering, permission boundaries for agent tool use, and basic monitoring for anomalous behavior.

Audience

This session is for developers and AI/ML practitioners who are responsible for the security of an LLM-powered application or agent — the hands-on, code-level counterpart to the organization-level AI Security course. A traditional application security background is helpful but not required.

Objectives

  • Identify the OWASP Top 10 risks specific to LLM applications
  • Attempt prompt injection and jailbreak techniques in a safe, sandboxed environment
  • Implement input/output filtering and permission boundaries for agent tool use
  • Design a basic approach for monitoring AI-specific abuse

Prerequisites

Basic programming experience and familiarity with how LLM applications are built (via an API, or a RAG or agent pipeline) is expected. General application security background is helpful but not required.

Related AI Courses

See the full AI training roadmap and course directory for how this fits into a broader learning path.

For the organization-level, non-technical treatment of AI security for leaders and risk owners, see AI Security. Pair this with AI Governance & Risk Management Micro-Credential for the policy side of the same risks. Building multi-agent systems? See Building Multi-Agent Systems Micro-Credential for where agent permission boundaries come from.

Course Outline

Module 1: The New Attack Surface

  • Why LLM Applications Break Traditional Security Assumptions
  • The OWASP Top 10 for LLM Applications
  • Prompt Injection: Direct and Indirect
  • Lab – Attempt a Prompt Injection Against a Sandboxed App

Module 2: Attacking and Defending

  • Jailbreaks and Guardrail Bypass Techniques
  • Data Leakage and Sensitive Information Disclosure
  • Insecure Tool Use in Agentic Systems
  • Lab – Implement Input/Output Filtering and Tool Permission Boundaries

Module 3: Operating Securely

  • Monitoring and Logging for AI-Specific Abuse
  • Red-Teaming Your Own System Before Launch
  • Building a Basic Incident Response Plan
  • Lab – Red-Team a Peer Team’s Agent Configuration