{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Module - Cross Validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import to be able to import python package from src\n",
    "import sys\n",
    "sys.path.insert(0, '../src')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ontime as on\n",
    "from ontime.module.processing.cross_validation import cross_validation, evaluate_cross_validation\n",
    "from ontime.module.benchmarking.benchmark_metric import BenchmarkMetric\n",
    "from darts.metrics import mae"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Load data\n",
    "\n",
    "For this notebook, we generate a simple synthetic time series so the notebook runs quickly and without external downloads."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "index = pd.date_range(start='2020-01-01', periods=120, freq='D')\n",
    "values = np.arange(len(index), dtype=float) + np.random.normal(scale=0.5, size=len(index))\n",
    "df = pd.DataFrame({'consumption': values}, index=index)\n",
    "ts = on.TimeSeries.from_dataframe(df)\n",
    "ts.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Why cross-validation for time series?\n",
    "\n",
    "Unlike i.i.d. data, time series can't be split randomly: shuffling would leak future information into the training set. `ontime.module.processing.cross_validation` implements the standard strategies used to evaluate forecasting models while respecting temporal order:\n",
    "\n",
    "- **expanding** window (anchored, growing train set) \u2014 setting `horizon=1` gives *walk-forward validation*, while `horizon > 1` evaluates multi-step forecasts.\n",
    "- **sliding** window (fixed-size, rolling train set).\n",
    "- **blocked** cross-validation (non-overlapping contiguous blocks).\n",
    "\n",
    "All strategies also support a `gap` parameter to leave a buffer between train and test (purged / embargoed cross-validation), which helps avoid leakage caused by autocorrelation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Expanding window (walk-forward, `horizon=1`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "folds = cross_validation(\n",
    "    ts, n_splits=5, strategy='expanding', initial_train_size=80, horizon=1\n",
    ")\n",
    "for train, test in folds:\n",
    "    print(len(train), len(test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Expanding window, multi-horizon (`horizon > 1`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "folds = cross_validation(\n",
    "    ts, n_splits=5, strategy='expanding', initial_train_size=80, horizon=5\n",
    ")\n",
    "for train, test in folds:\n",
    "    print(len(train), len(test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sliding window"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "folds = cross_validation(\n",
    "    ts, n_splits=5, strategy='sliding', initial_train_size=40, horizon=5\n",
    ")\n",
    "for train, test in folds:\n",
    "    print(len(train), len(test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Blocked cross-validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "folds = cross_validation(ts, n_splits=5, strategy='blocked')\n",
    "for train, test in folds:\n",
    "    print(len(train), len(test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Purged / embargoed cross-validation (`gap`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "folds = cross_validation(\n",
    "    ts, n_splits=5, strategy='expanding', initial_train_size=80, horizon=1, gap=3\n",
    ")\n",
    "train, test = folds[0]\n",
    "print(train.pd_dataframe().index[-1], '->', test.pd_dataframe().index[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Evaluating a model across folds\n",
    "\n",
    "`evaluate_cross_validation` fits (or refits) a model on the train set of each fold, predicts the test horizon, and aggregates a metric across folds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from darts.models import NaiveDrift\n",
    "\n",
    "model = on.Model(NaiveDrift())\n",
    "metric = BenchmarkMetric(name='mae', metric_function=mae)\n",
    "\n",
    "results = evaluate_cross_validation(\n",
    "    model, ts, metrics=metric, n_splits=5, strategy='expanding',\n",
    "    initial_train_size=80, horizon=5,\n",
    ")\n",
    "print(results['folds'])\n",
    "print(results['mean'])\n",
    "print(results['std'])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}