Machine Learning Lectures: Step 10

    “Data are becoming the new raw material of business.”
    -- Craig Mundie

    Step 10: Build A Production-Ready System

    Now, the prototype can be transformed into a ML/AI system that works in tandem with the production code. This might range from something as straightforward as jotting down an outline to aid you in making decisions up to developing an automated model designed for scalability and resilience against external attacks. These complex aspects were not covered under prototyping training sessions, so lots of effort is necessary!

    Table of Contents

    S10: Productionization

    S10: Repurposing Data

    S10: Latency Problems

    • Just Launch It?
      • Enable it to go live
      • Retest it on live traffic
      • Stay very cautious
    • Sounds easy, Is difficult
    • New data flowing in
    • Adjust solution
      • what is the good direction to repurpose
    • Retrain on the side


    S10: Retraining Frequency

    S10: Training-Serving Skew

    S10: Chained ML Systems

    • The world represented by your training data is the only world you can expect to succeed in 

    • Training-Serving skew
    • Fleeting changes in the data
      • eg. Retail -- Christmas sales results in February
      • selectively "blind" the system
      • exclude certain data
      • want a rich representation of the real world
    • Machine Learning models rarely exists in isolation
    • Systems are chained to many other machine learning models
      • inputs come from outputs of other models
      • they are changing



    S10: Tiny Changes

    S10: When Retesting Fails

    S10: The Long Tail

    • Changing anything in ML changes everything

    • "When", no if
    • It WILL fail in retesting

    • You will meet the long tail

    S10: Outliers

    S10: AI Safety & Policy

    • Outliers: plan to fail gracefully
    • Need a policy layer
      • like a filter
    • Building safety nets
      • sit on top of model's output