Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Introduction to LLMOps
- LLMOps versus MLOps: Unique challenges in operating LLMs.
- The LLM application lifecycle: Prompt, evaluate, deploy, and monitor.
- Production readiness checklist for Generative AI applications.
Prompt Management and Versioning
- Prompt templating systems and variable injection techniques.
- Semantic versioning for prompts with automated regression testing.
- Prompt registries and collaboration workflows.
LLM Evaluation at Scale
- Evaluation dimensions: Accuracy, relevance, safety, and groundedness.
- LLM-as-judge metrics and human evaluation pipelines.
- Automated evaluation frameworks: RAGAS, DeepEval, and custom evaluators.
- Quality gates in CI/CD for LLM deployments.
Safety Guardrails and Content Governance
- Input and output guardrails using NeMo Guardrails and Guardrails AI.
- PII detection, toxicity filtering, and topic boundary enforcement.
- Strategies for defending against jailbreaks and prompt injection attacks.
- Conducting red-teaming exercises for LLM applications to ensure safety.
LLM Observability and Monitoring
- Telmetry data: Token usage, latency, cost, and quality metrics.
- Detecting drift in LLM outputs and embedding spaces.
- Session-level tracing for multi-turn agent conversations.
- Setting up dashboards and alerts with LangSmith, Arize, and OpenTelemetry.
AI Gateway and Model Orchestration
- Multi-provider routing using LiteLLM and Portkey.
- Fallback strategies, retry logic, and circuit breakers.
- Cost-aware model selection and load balancing techniques.
- Rate limiting, quota management, and API key governance.
Performance Optimization
- Semantic caching using vector stores and exact-match strategies.
- Enforcing structured outputs via constrained decoding.
- Implementing batching, streaming, and concurrency patterns.
- Optimizing latency across different model providers.
Governance, Compliance, and Audit
- LLM audit trails: Prompt logs, response logs, and decision provenance.
- Data residency and privacy considerations for LLM APIs.
- Policy-as-code for regulating LLM usage within organizations.
- Developing an internal playbook for LLM operations.
Requirements
- Experience in building or integrating applications powered by large language models.
- Familiarity with Python and REST APIs.
- A fundamental understanding of prompt engineering concepts.
Audience
- ML engineers and MLOps practitioners transitioning into LLM operations.
- Platform engineers responsible for LLM infrastructure.
- Technical leads overseeing production deployments of Generative AI.
14 Hours
Testimonials (2)
use of proper and effective prompt
Marses Pacaldo
Course - Generative AI and Prompt Engineering for Corporate Professionals
The interactive style, the exercises
