Machine Learning Lectures: Steps 0-1

Table of Contents

• Step 0: Where to start? (1:35)
• Step 0: Classification vs Regression (1:19)
• Step 0: Instances, Features, Targets (1:03)
  
  
• Step 0: Supervised Learning (1:54)
• Step 0: Unsupervised Learning (7:28)
• Step 0: Semi-supervised Learning (2:36)
  
  
• Step 0: Reinforcement Learning (2:59) 
• Step 0: What is Data Science? (2:54)
• Step 0: Data Science Flowchart (4:53)
  
  
• Step 0: Don't forget data (1:45)

• Step 1: What is "good behavior"? (1:57)
• Step 1: False Positives & True Negatives (1:23)
  
  
• Step 1: Confusion Matrix (2:54)
• Step 1: Performance Metrics (2:08)
• Step 1: Ground Truth (1:43)
  
  
• Step 1: Precision vs Recall (5:47)
• Step 1: What is Optimization (2:07)
  
  
• Step 1: Loss Function (6:32)
• Step 1: Setting Performance Criteria (3:39)

• Begin with labels
• Articulate the decisions you want

• What kind of label do you want?
• Classification (binary or multiclass)
• Prediction (numerical)

• Jargon - Instance / Label / Feature
• Instance: row; the example
• Feature: column (variable)
• Label: answer or truth or response

Top of Page

• For any example you give the system, you have the correct label handy
• Keywords: labeled data

• Splits into two groups
• Just separates them into groups
• Keywords: Data mining / Clustering

• Only some of the correct labels
• Blend of supervised and unsupervised learning
• Keywords: Partial guidance

Top of Page

• Not immediate feedback for every move - delayed feedback
• Keywords: sequence of actions, reward / punishment, delayed feedback, system influences its environment
• Lots of failure

• Map of data science: (number of decisions)
• Analytics: None - get inspired
• ML/AI: Many - make recipe
• Statistics: Few - decide wisely

• Are we making decisions?
• No - Use descriptive analytics
• Yes - Can you look up the answer?
• -- Yes - Use descriptive analytics
• -- No - Many decisions? -
• ---- Yes - Use ML
• ---- No - Few Decisions?
• ---- Do you need to control risk?
• No - Use Descriptive Analytics
• Yes - Use Statistical Inference

Top of Page

• ML does not work without data
• Need data to learn from

• Set Objectives first!
• Can be nuanced
• What are you trying to accomplish

• Write output label (Cat / No Cat)
• Consider mistakes
• Assign project score
• Create performance metric
• Think about loss function
• Compare the functions
• Set performance criteria

Top of Page

• Project scoring
  • True Positive
  • False Positive
  • False Negative
  • True Negative

• How to calculate the score?
• Business Performance Metric

• Accuracy
• Precision
• Recall
• Blends of the above

Top of Page

• Two questions ... 
• 1. Did it get it right or not? True/False
• 2. Did it think it found a (label)? Positive/Negative
  
  
• Precision is the fraction of actual (label) among the (label) the system found
• Precision -- don't waste my time -- leave things out -- return things that are correct
• Precision -- returns high quality, but leaves some items out
• Precision: TP / (TP + FP)
  
  
• Recall is the proportion of (label) the ML system found correctly among all the (label) in all the images
• Recall -- I need an exhaustive list and am willing to accept duds
• Recall -- returns lots of duds, but returns most of the good items
• Recall: TP (TP + FN)

• Optimization is about picking the best parameters
• Find values that give the best performance metric

Top of Page

• Loss function (objective function - cost function) 
• Bring "badness" down
• Compare the functions -- Accuracy vs. Loss function

• Minimum performance to accept
  
  

Top of Page