Safe & Secure Agentic AI: Governance, Identity, and Red-Teaming Training Course

Foundations: Threat Modeling for Agentic AI

• Categorizing agentic threats: misuse, privilege escalation, data leakage, and supply-chain vulnerabilities.
• Defining adversary profiles and attacker capabilities specific to autonomous agent interactions.
• Mapping critical assets, trust boundaries, and control points associated with agent operations.

Governance, Policy, and Risk Management

• Applying governance frameworks to agentic systems, including roles, responsibilities, and approval gates.
• Developing policies for acceptable use, escalation protocols, data handling, and auditability.
• Addressing compliance requirements and gathering evidence for audit purposes.

Non-Human Identity and Authentication for Agents

• Constructing agent identities using service accounts, JWTs, and short-lived credentials.
• Implementing least-privilege access patterns and just-in-time credential issuance.
• Managing the identity lifecycle, including rotation, delegation, and revocation strategies.

Access Controls, Secrets, and Data Protection

• Employing fine-grained access control models and capability-based patterns for agent security.
• Managing secrets, ensuring encryption in transit and at rest, and applying data minimization techniques.
• Safeguarding sensitive knowledge bases and PII against unauthorized agent access.

Observability, Auditing, and Incident Response

• Developing telemetry for agent behavior, including intent tracing, command logging, and provenance tracking.
• Integrating with SIEM systems, defining alerting thresholds, and maintaining forensic readiness.
• Creating runbooks and playbooks for managing agent-related incidents and containment.

Red-Teaming Agentic Systems

• Planning red-team exercises, defining scope, rules of engagement, and safe failover mechanisms.
• Exploring adversarial techniques such as prompt injection, tool misuse, chain-of-thought manipulation, and API abuse.
• Executing controlled attacks to measure system exposure and impact.

Hardening and Mitigations

• Implementing engineering controls like response throttling, capability gating, and sandboxing.
• Establishing policy and orchestration controls, including approval flows, human-in-the-loop interventions, and governance hooks.
• Deploying model and prompt-level defenses such as input validation, canonicalization, and output filtering.

Operationalizing Safe Agent Deployments

• Utilizing deployment patterns such as staging, canary releases, and progressive rollouts for agents.
• Managing change control, testing pipelines, and pre-deployment safety checks.
• Facilitating cross-functional governance across security, legal, product, and operations teams.

Capstone: Red-Team / Blue-Team Exercise

• Executing a simulated red-team attack within a sandboxed agent environment.
• Acting as the blue team to defend, detect, and remediate using established controls and telemetry.
• Presenting findings, outlining a remediation plan, and proposing policy updates.

Summary and Next Steps