MLOps & LLMOps

MLOps & LLMOps

Duration: 3 Days

Description

A model that performs well in a notebook and a model that performs well in production are two different engineering problems, and most teams underestimate the gap until they’re already in it. This course is the discipline that closes it: versioning data and models so results are reproducible, building pipelines that don’t break the third time someone retrains, deploying and serving models at real request volume, and monitoring for drift — the slow, silent way a model’s accuracy degrades as the real world stops matching its training data. A dedicated thread on LLMOps covers what’s genuinely different about operating generative AI systems: prompt versioning, cost per request, and quality monitoring when “correct” isn’t a fixed target.
Class time is divided between instructor-led explanation (40 percent) and hands-on exercises (60 percent).

Audience

This course is for ML and data engineers, developers, and platform engineers who own or are about to own an ML or LLM system in production — not people building their first model. Prior ML or AI development experience is assumed; this is the course for after a model works, not the course for building one.

Objectives

  • Explain the goals and scope of MLOps and LLMOps
  • Version and manage data, models, and pipelines
  • Build reproducible training and deployment workflows
  • Deploy and serve models reliably
  • Monitor for performance, drift, and quality
  • Address the unique operational needs of LLM systems

Prerequisites

Participants should have prior ML or AI development experience. The following will help:

  • Working knowledge of Python (see Python Foundations)
  • Machine learning foundations (see Machine Learning Foundations) or AI development experience
  • Basic familiarity with cloud or deployment concepts

Related AI Courses

See the full AI training roadmap and course directory for how this fits into a broader learning path.

Need the ML fundamentals first? Start with Machine Learning Foundations. Operating LLM-specific systems built with a retrieval layer? See RAG Deep Dive. For the data-pipeline foundation underneath production ML, see Data Engineering.

Course Outline

  • Module 1 – Course Introduction
    • Welcome and objectives
    • Why MLOps and LLMOps
    • The production ML lifecycle
  • Module 2 – Versioning and Reproducibility
    • Versioning data and models
    • Experiment tracking
    • Reproducible environments and pipelines
    • Hands-on exercise: track an experiment
  • Module 3 – Deployment and Serving
    • Packaging models for deployment
    • Serving patterns: batch, online, and streaming
    • APIs and scaling
    • Hands-on exercise: deploy a model service
  • Module 4 – Monitoring and Maintenance
    • Monitoring performance and health
    • Data and model drift
    • Retraining and rollback strategies
    • Alerting and incident response
  • Module 5 – LLMOps Specifics
    • Operating LLM systems versus traditional ML
    • Prompt and version management
    • Evaluating LLM quality in production
    • Cost, latency, and token management
  • Module 6 – Governance and Reliability
    • Pipelines, automation, and CI/CD for ML
    • Security and access control
    • Auditability and compliance
    • Hands-on exercise: an automated pipeline
  • Module 7 – Course Wrap-Up
    • Key takeaways
    • Resources for continued learning
    • Next steps