Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Offline EXO Deployment
- Utilizing EXO_OFFLINE to prevent runtime internet access
- Pre-loading models into EXO_MODELS_READ_ONLY_DIRS from trusted internal mirrors
- Verifying model weight integrity via SHA-256 checksums and signed model cards
- Operating EXO in air-gapped networks without HuggingFace dependencies
Dashboard and API Access Control
- Installing and configuring reverse proxies (nginx, Caddy) with TLS termination
- Implementing role-based access control for the EXO dashboard and REST API
- Storing secrets for API authentication using macOS keychain or Linux pass
- Restricting administrative endpoints to specific source IP ranges
Cluster Isolation and Network Security
- Segmenting EXO clusters with EXO_LIBP2P_NAMESPACE and VLANs
- Configuring host firewalls (macOS application firewall, iptables, nftables) for EXO ports
- Preventing unauthorized device discovery and rogue node injection
- Encrypting libp2p traffic between nodes when RDMA is unavailable
Model Governance and Provenance
- Building an internal model registry featuring approved model lists and metadata
- Tagging and versioning quantized weights (4-bit, 8-bit) alongside source checkpoints
- Enforcing the loading of only specific HuggingFace repos or internal artifacts
- Documenting model lineage, license terms, and acceptable use policies
Audit Logging and Compliance
- Configuring EXO log forwarding to immutable audit trails (SIEM, WORM storage)
- Correlating API call logs with user identity and timestamp
- Capturing events for model instance creation, deletion, and inference requests
- Generating periodic compliance reports for internal and external auditors
Threat Modeling and Incident Response
- Identifying threats such as data exfiltration through model outputs, prompt injection, and side-channel leaks
- Implementing prompt monitoring and content filtering pipelines
- Creating incident response runbooks for cluster compromise scenarios
- Isolating affected nodes, preserving forensic logs, and rebuilding clean environments
Physical Security and Hardware Boundaries
- Securing Thunderbolt ports against unauthorized RDMA cable connections
- Utilizing secure enclaves and Apple Silicon hardware attestation where applicable
- Controlling physical access to clustered Macs and shared storage
- Documenting hardware lifecycle and decommissioning procedures
Regulatory Considerations
- Mapping EXO deployments to GDPR, HIPAA, and SOC 2 requirements
- Maintaining data residency by keeping inference on-premise
- Documenting vendor supply-chain risks (MLX, EXO, model weights)
- Preparing for AI governance frameworks such as EU AI Act Article 53
Requirements
- Experience with EXO or another local LLM runtime
- Understanding of Unix filesystem permissions and networking ACLs
- Familiarity with TLS/SSL certificate management and encryption basics
Audience
- Security engineers
- Compliance officers
- AI infrastructure administrators handling sensitive data
14 Hours
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
