Skip to content
YA
All work
CourseworkMLResearch

DoS Detection & Response

A machine-learning intrusion detector tested honestly — including on an attack it had never seen.

Status
Applied Machine Learning for Security Analytics — course project with paper, slides, code and trained models.
Period
2026
Platforms
Python
Repository
View source
Live
—
Generalisation gap chart: recall on seen vs unseen attacks

01Overview

Classifies network flows from CIC-IDS2017 as benign or DoS, evaluates the classifier on held-out and out-of-distribution traffic, and drives a simulated automated response — flow blocking, alerting and firewall IP bans — from the model's predictions. Delivered with a research paper and presentation.

By the numbers

F1 on held-out test
0.9989
F1 on held-out test
README §5.1
false-positive rate
0.07%
false-positive rate
README §5.1
unseen DDoS flows stopped
92%
unseen DDoS flows stopped
README
innocent hosts banned (simulated IPs)
0
innocent hosts banned (simulated IPs)
README §6
  1. 02The problem

    Intrusion-detection papers often report near-perfect scores on a random split of the same day's traffic — which says little about the next attack. The goal was a detector whose limits are measured, not hidden.

  2. 03The solution

    Cleaning rules chosen from Wednesday only and applied unchanged to Friday; Destination Port dropped to prevent shortcut learning; duplicates removed to prevent leakage; a stratified 80/20 split with tuning inside training only; Friday's DDoS used once as an out-of-distribution test. A separate ALERT band in the response engine reflects the model's uncertainty.

04Key features

  1. 01

    Held-out results

    Random Forest on 116,996 never-seen Wednesday flows: precision 0.9985, recall 0.9993, F1 0.9989, false-positive rate 0.07% — 86 errors in total.

  2. 02

    Generalisation, stated plainly

    On the unseen Friday DDoS, recall per flow drops to 0.633 (precision 0.9994). XGBoost — marginally better on Wednesday — collapses to 0.113, which is why Random Forest was chosen.

  3. 03

    Response that still works

    Even with per-flow recall of 0.63, strike-based IP bans let the response layer stop 92.3% of DDoS flows and ban all 30 attacker IPs, with no innocent host banned. Source IPs are simulated — the dataset contains none.

  4. 04

    Error analysis

    Almost every error is a flow where the server sent no payload back — genuinely ambiguous. The model is uncertain on its errors (mean attack probability 0.29 for misses, 0.83 for false alarms).

05Screens

06System architecture

  1. L1Data
    • CIC-IDS2017 CSV
    • Cleaning → Parquet
    • 584,980 flows × 66 features
  2. L2Modelling
    • Dummy
    • Logistic Regression
    • Random Forest (3-fold CV)
    • XGBoost
  3. L3Evaluation
    • Held-out 20%
    • Unseen Friday DDoS
    • Per-attack recall
    • Error profile
  4. L4Response
    • Flow blocking
    • ALERT band
    • Firewall IP bans
    • Generated rules
Three reproducible scripts plus notebooks; models and a model card are saved with the results.Diagram — derived from the source code

07Lessons learned

Report the test you might fail. The out-of-distribution number is the one that matters in production.

Technology

  • Python
  • scikit-learn
  • XGBoost
  • pandas
  • Parquet
  • Jupyter
  • Random Forest
  • Machine Learning

How this was verified

Code, README and results reviewed. Figures and terminal screenshots were produced by the project's own runs and are shown unaltered.

Related projects

Next project

Athar