Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Offline EXO Deployment
- Utilizing EXO_OFFLINE to block runtime internet access
- Pre-loading models from trusted internal mirrors into EXO_MODELS_READ_ONLY_DIRS
- Validating model weight integrity via SHA-256 checksums and signed model cards
- Operating EXO within air-gapped networks, independent of HuggingFace dependencies
Dashboard and API Access Control
- Setting up and configuring reverse proxies (nginx, Caddy) with TLS termination
- Establishing role-based access control for the EXO dashboard and REST API
- Storing API authentication secrets securely using macOS keychain or Linux pass
- Limiting administrative endpoint access to designated source IP ranges
Cluster Isolation and Network Security
- Segmenting EXO clusters using EXO_LIBP2P_NAMESPACE and VLANs
- Configuring host firewalls (macOS application firewall, iptables, nftables) for EXO ports
- Blocking unauthorized device discovery and rogue node injection attempts
- Encrypting libp2p traffic between nodes in environments where RDMA is unavailable
Model Governance and Provenance
- Creating an internal model registry that includes approved model lists and metadata
- Tagging and versioning quantized weights (4-bit, 8-bit) alongside their source checkpoints
- Restricting model loading to specific HuggingFace repositories or internal artifacts only
- Recording model lineage, licensing terms, and acceptable use policies
Audit Logging and Compliance
- Directing EXO log output to immutable audit trails (SIEM, WORM storage)
- Linking API call logs with user identity and precise timestamps
- Recording events related to model instance creation, deletion, and inference requests
- Generating regular compliance reports for internal and external auditors
Threat Modeling and Incident Response
- Identifying potential threats such as data exfiltration via model outputs, prompt injection, and side-channel leaks
- Deploying prompt monitoring and content filtering pipelines
- Developing incident response runbooks for cluster compromise scenarios
- Isolating affected nodes, securing forensic logs, and reconstructing clean environments
Physical Security and Hardware Boundaries
- Protecting Thunderbolt ports from unauthorized RDMA cable connections
- Leveraging secure enclaves and Apple Silicon hardware attestation where feasible
- Regulating physical access to clustered Macs and shared storage units
- Documenting hardware lifecycle management and decommissioning procedures
Regulatory Considerations
- Aligning EXO deployments with GDPR, HIPAA, and SOC 2 requirements
- Ensuring data residency by executing inference on-premise
- Documenting vendor supply-chain risks associated with MLX, EXO, and model weights
- Preparing for AI governance frameworks, including EU AI Act Article 53
Requirements
- Practical experience with EXO or other local LLM runtimes
- Knowledge of Unix filesystem permissions and network ACLs
- Familiarity with TLS/SSL certificate management and fundamental encryption concepts
Target Audience
- Security engineers
- Compliance officers
- AI infrastructure administrators responsible for sensitive data
14 Hours
Testimonials (1)
The trainer had an excellent knowledge of fortigate and delivered the content very well. Thanks a lot to Soroush.
