Module 5: Machine Learning Basics

Learning Content

Introduction

Machine Learning (ML) is the foundation of modern AI. This module teaches you how ML systems actually work—how they learn from data, make predictions, and improve over time. Understanding these fundamentals helps you evaluate AI solutions, set realistic expectations, and participate effectively in AI projects.

What is Machine Learning?

Traditional programming: You write explicit rules (if X then Y). Machine Learning: The system learns rules from examples.

The ML Workflow

Step 1: Problem Definition

Clearly define what you're trying to predict or optimize:

• Classification: Categorize items (spam/not spam, fraud/legitimate)
• Regression: Predict numerical values (price, revenue, temperature)
• Clustering: Group similar items (customer segmentation)
• Ranking: Order items by relevance (search results, recommendations)

Step 2: Data Collection

Gather relevant historical data:

• Quantity: Generally, more data = better models (thousands to millions of examples)
• Quality: Accurate, complete, relevant data is crucial
• Representativeness: Data should reflect real-world scenarios the model will encounter
• Labeling: For supervised learning, data needs correct answers (labels)

Step 3: Data Preparation

Clean and format data for ML:

• Cleaning: Handle missing values, remove duplicates, fix errors
• Feature Engineering: Create useful input variables from raw data
• Normalization: Scale numerical values to similar ranges
• Encoding: Convert categorical data (colors, categories) to numbers
• Splitting: Divide data into training set (learn patterns), validation set (tune model), test set (final evaluation)

Step 4: Model Selection

Choose appropriate ML algorithm:

• Decision Trees
• Random Forests
• Support Vector Machines
• Neural Networks
• Gradient Boosting (XGBoost, LightGBM)

Step 5: Training

The model learns patterns from training data:

• Model makes predictions on training data
• Compares predictions to actual answers
• Adjusts internal parameters to reduce errors
• Repeats many times until performance plateaus

Step 6: Evaluation

Test model on data it hasn't seen:

• Measure accuracy, precision, recall, F1-score
• Identify where model struggles
• Compare against baseline and business requirements

Step 7: Deployment

Put model into production:

• Integrate with business systems
• Set up monitoring and alerting
• Plan for updates and retraining

Step 8: Monitoring

Track model performance over time:

• Watch for accuracy degradation (model drift)
• Monitor data quality and distribution changes
• Retrain when performance drops

Types of Machine Learning

1. Supervised Learning

Learning from labeled examples (you provide correct answers).

How it works:

• Training data includes inputs and correct outputs
• Model learns to map inputs to outputs
• Goal: Make accurate predictions on new, unseen data

Common algorithms:

• Linear Regression: Predicting numbers (house prices, sales)
• Logistic Regression: Binary classification (yes/no, pass/fail)
• Decision Trees: Tree-like model of decisions
• Random Forest: Many decision trees voting together
• Neural Networks: Complex patterns through layers of connections
• XGBoost/LightGBM: Advanced boosting algorithms (very popular in competitions)

Business applications:

• Customer churn prediction
• Fraud detection
• Credit scoring
• Demand forecasting
• Image classification

2. Unsupervised Learning

Finding patterns in unlabeled data (no correct answers provided).

How it works:

• Model identifies structure and patterns on its own
• Discovers groupings, relationships, anomalies
• No explicit goal—exploratory in nature

Common algorithms:

• K-Means Clustering: Grouping similar items
• Hierarchical Clustering: Creating hierarchy of clusters
• Principal Component Analysis (PCA): Reducing data dimensions
• Association Rules: Finding items that occur together

Business applications:

• Customer segmentation
• Market basket analysis ("customers who bought X also bought Y")
• Anomaly detection
• Data exploration and visualization

3. Reinforcement Learning

Learning through trial and error with rewards and penalties.

How it works:

• Agent takes actions in an environment
• Receives rewards (positive) or penalties (negative)
• Learns strategy that maximizes total rewards
• Explores vs. exploits trade-off

Common applications:

• Game playing (Chess, Go, video games)
• Robotics and autonomous vehicles
• Resource allocation and scheduling
• Personalized recommendations (learning from user feedback)

Feature Engineering: The Art of ML

Features are the input variables your model uses to make predictions. Good features are crucial for model performance.

What Makes a Good Feature?

• Relevant: Actually predictive of the target
• Available: Can be computed from available data
• Stable: Meaning doesn't change over time
• Independent: Not redundant with other features

Common Feature Engineering Techniques

• Aggregations: Sum, average, count over time periods
• Ratios: Revenue per customer, conversion rate
• Time-based: Day of week, hour of day, time since last event
• Categorical encoding: Converting categories to numbers
• Text features: Word counts, TF-IDF, embeddings
• Interactions: Combining features (price × demand)

Example: Customer Churn Prediction

Raw data: Customer ID, transactions, support tickets, login times

Engineered features:

• Days since last login
• Average transaction value
• Number of support tickets in last 30 days
• Trend in activity (increasing or decreasing)
• Ratio of support tickets to transactions
• Day of week for typical activity

Model Evaluation Metrics

Different metrics for different problems:

For Classification Problems

Metric	What It Measures	When to Use
Accuracy	% of correct predictions	Balanced datasets, equal importance of all classes
Precision	% of positive predictions that were correct	When false positives are costly (spam detection)
Recall	% of actual positives found	When false negatives are costly (fraud detection)
F1-Score	Balance of precision and recall	Imbalanced datasets, balanced costs
AUC-ROC	Trade-off between true/false positives	Comparing models, understanding thresholds

For Regression Problems

• Mean Absolute Error (MAE): Average size of errors
• Mean Squared Error (MSE): Penalizes large errors more
• R-squared: % of variance explained by model
• MAPE: Mean Absolute Percentage Error (useful for comparing across scales)

Common ML Challenges

Overfitting vs. Underfitting

• Overfitting: Model memorizes training data, fails on new data. Too complex for available data.
• Underfitting: Model too simple to capture patterns. Performs poorly on all data.
• Just Right: Model learns generalizable patterns, performs well on new data.
• Solutions: More training data, regularization, cross-validation, simpler models

Imbalanced Data

• Problem: Much more of one class than another (e.g., 99% legitimate transactions, 1% fraud)
• Impact: Model learns to predict majority class always
• Solutions: Resampling (over/under-sampling), class weights, different evaluation metrics, anomaly detection approaches

Data Quality Issues

• Missing values: Impute, drop, or flag
• Outliers: Understand if real or errors, handle appropriately
• Inconsistent data: Standardize formats and definitions
• Biased data: Ensure training data represents reality

Model Drift

• Problem: Model performance degrades over time as reality changes
• Causes: Changing customer behavior, new fraud tactics, market shifts, seasonal patterns
• Solutions: Regular monitoring, scheduled retraining, automated alerts, A/B testing

ML Best Practices

When to Use ML (and When Not To)

Good ML Use Cases

• Pattern recognition in large datasets
• Predictions where historical data exists
• Problems too complex for explicit rules
• Tasks requiring personalization at scale
• Automating repetitive decisions

Poor ML Use Cases

• Simple rules-based problems
• Insufficient or poor-quality data
• Critical decisions without explainability options
• Rapidly changing environments without retraining plan
• High cost of errors without human oversight

The Future of Machine Learning

Emerging trends in ML:

• AutoML: Automated feature engineering and model selection
• Few-Shot Learning: Learning from very few examples
• Transfer Learning: Reusing knowledge from one task for another
• Federated Learning: Training on distributed data without centralizing it
• Explainable AI: Making ML decisions more interpretable
• Neurosymbolic Integration: Combining ML with symbolic reasoning for more reliable, explainable systems

Looking Ahead

Understanding ML fundamentals prepares you to evaluate AI solutions and make informed decisions. The next modules explore specific industry applications, starting with iGaming—one of Malta's key sectors where AI is driving significant innovation.