{ "cells": [ { "cell_type": "markdown", "id": "c9f31421", "metadata": {}, "source": [ "# Choosing a forecast horizon and a training window with Optuna\n", "\n", "Two questions come up in every forecasting project: **how far ahead** should the model predict\n", "(the *horizon*), and **how much history** should it be trained on (the *window*)?\n", "\n", "Both are model selection problems, just like a learning rate or a number of layers. So we treat\n", "them as such: we score each candidate with a proper time series cross-validation, and we let\n", "[Optuna](https://optuna.org/) search the space and keep track of the results.\n", "\n", "**What you will do here**\n", "\n", "1. Score a configuration with `ontime`'s cross-validation helpers.\n", "2. Sweep a list of candidate horizons and read the results table.\n", "3. Sweep the horizon and the training window together.\n", "4. Look at the horizon / accuracy trade-off as a Pareto front.\n", "\n", "**Requirements:** `optuna`, which ships with the `dev` dependency group of onTime\n", "(`pip install optuna` otherwise)." ] }, { "cell_type": "code", "execution_count": 1, "id": "44b913af", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:44.022745Z", "iopub.status.busy": "2026-07-28T13:28:44.022571Z", "iopub.status.idle": "2026-07-28T13:28:44.028123Z", "shell.execute_reply": "2026-07-28T13:28:44.027499Z" } }, "outputs": [], "source": [ "# Import to be able to import python package from src\n", "import sys\n", "sys.path.insert(0, '../../src')" ] }, { "cell_type": "code", "execution_count": 2, "id": "9d54a56e", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:44.030842Z", "iopub.status.busy": "2026-07-28T13:28:44.030707Z", "iopub.status.idle": "2026-07-28T13:28:50.541939Z", "shell.execute_reply": "2026-07-28T13:28:50.541028Z" } }, "outputs": [], "source": [ "import warnings\n", "\n", "import numpy as np\n", "import pandas as pd\n", "import optuna\n", "\n", "import ontime as on\n", "from ontime.module.processing.cross_validation import evaluate_cross_validation\n", "from ontime.module.benchmarking import BenchmarkMetric\n", "\n", "optuna.logging.set_verbosity(optuna.logging.WARNING)\n", "warnings.filterwarnings(\"ignore\")\n", "\n", "RANDOM_STATE = 42\n", "np.random.seed(RANDOM_STATE)" ] }, { "cell_type": "markdown", "id": "b5b540ed", "metadata": {}, "source": [ "## 1. Setup\n", "\n", "### The data\n", "\n", "A synthetic hourly series with a daily seasonality, a weekly seasonality and a mild trend, so\n", "the notebook runs fast and offline. Replace it with your own `on.TimeSeries` to reuse the code\n", "as is." ] }, { "cell_type": "code", "execution_count": 3, "id": "694e927f", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:50.543842Z", "iopub.status.busy": "2026-07-28T13:28:50.543626Z", "iopub.status.idle": "2026-07-28T13:28:51.231408Z", "shell.execute_reply": "2026-07-28T13:28:51.230769Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1440 points, from 2023-01-01 00:00:00 to 2023-03-01 23:00:00\n" ] }, { "data": { "text/html": [ "\n", "\n", "
\n", "" ], "text/plain": [ "alt.LayerChart(...)" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "index = pd.date_range(start=\"2023-01-01\", periods=24 * 60, freq=\"h\")\n", "t = np.arange(len(index))\n", "\n", "daily = 10 * np.sin(2 * np.pi * t / 24)\n", "weekly = 4 * np.sin(2 * np.pi * t / (24 * 7))\n", "trend = 0.002 * t\n", "noise = np.random.normal(scale=1.5, size=len(t))\n", "\n", "df = pd.DataFrame({\"consumption\": 50 + daily + weekly + trend + noise}, index=index)\n", "ts = on.TimeSeries.from_dataframe(df)\n", "\n", "print(f\"{len(ts)} points, from {index[0]} to {index[-1]}\")\n", "ts[: 24 * 7].plot()" ] }, { "cell_type": "markdown", "id": "1483fd6b", "metadata": {}, "source": [ "### The scoring function\n", "\n", "Everything below relies on one function: given a horizon and a training window, return the\n", "cross-validated error of a model.\n", "\n", "`evaluate_cross_validation` does the work. It builds the folds, refits the model on the train\n", "set of each fold, predicts `horizon` steps ahead and aggregates the metric across folds. We use\n", "the **sliding** strategy so the training window stays a real degree of freedom (with\n", "`expanding`, the train set grows from fold to fold).\n", "\n", "The metric is the **MASE**, which divides the error by that of a naive forecast. Being\n", "scale-free, it is easier to compare from one configuration to the next than a raw error." ] }, { "cell_type": "code", "execution_count": 4, "id": "4460249c", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:51.233065Z", "iopub.status.busy": "2026-07-28T13:28:51.232923Z", "iopub.status.idle": "2026-07-28T13:28:51.235635Z", "shell.execute_reply": "2026-07-28T13:28:51.235215Z" } }, "outputs": [], "source": [ "from darts.metrics import mase\n", "\n", "MASE = BenchmarkMetric(name=\"mase\", metric_function=mase)\n", "N_SPLITS = 5\n", "\n", "\n", "def evaluate(model, horizon, window):\n", " \"\"\"Mean and std of the cross-validated MASE for a horizon and a training window.\"\"\"\n", " results = evaluate_cross_validation(\n", " model,\n", " ts,\n", " metrics=MASE,\n", " n_splits=N_SPLITS,\n", " strategy=\"sliding\",\n", " initial_train_size=window,\n", " horizon=horizon,\n", " )\n", " return results[\"mean\"][\"mase\"], results[\"std\"][\"mase\"]" ] }, { "cell_type": "markdown", "id": "e3801c47", "metadata": {}, "source": [ "A quick check with a seasonal naive model, before letting Optuna drive:" ] }, { "cell_type": "code", "execution_count": 5, "id": "9f9f8c67", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:51.237030Z", "iopub.status.busy": "2026-07-28T13:28:51.236872Z", "iopub.status.idle": "2026-07-28T13:28:51.261036Z", "shell.execute_reply": "2026-07-28T13:28:51.260479Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MASE = 1.307 (+/- 0.267)\n" ] } ], "source": [ "from darts.models import NaiveSeasonal\n", "\n", "mean, std = evaluate(on.Model(NaiveSeasonal(K=24)), horizon=24, window=24 * 14)\n", "print(f\"MASE = {mean:.3f} (+/- {std:.3f})\")" ] }, { "cell_type": "markdown", "id": "3493717c", "metadata": {}, "source": [ "## 2. Sweeping a list of horizons\n", "\n", "The simplest sweep: a fixed list of candidate horizons, one cross-validation each, one table of\n", "results.\n", "\n", "In Optuna terms this is a study with a **`GridSampler`**: the search space is enumerated and the\n", "study stops once every combination has been tried. Compared to a hand-written loop we get the\n", "results bookkeeping, the best trial, and later on the pruning and the dashboard for free." ] }, { "cell_type": "code", "execution_count": 6, "id": "0ba61939", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:51.262786Z", "iopub.status.busy": "2026-07-28T13:28:51.262657Z", "iopub.status.idle": "2026-07-28T13:28:51.388618Z", "shell.execute_reply": "2026-07-28T13:28:51.387424Z" } }, "outputs": [], "source": [ "from darts.models import LinearRegressionModel\n", "\n", "HORIZONS = [1, 3, 6, 12, 24, 48]\n", "WINDOW = 24 * 21\n", "\n", "\n", "def horizon_objective(trial):\n", " horizon = trial.suggest_categorical(\"horizon\", HORIZONS)\n", " model = on.Model(LinearRegressionModel(lags=48, random_state=RANDOM_STATE))\n", " mean, std = evaluate(model, horizon=horizon, window=WINDOW)\n", " trial.set_user_attr(\"mase_std\", float(std))\n", " return mean\n", "\n", "\n", "horizon_study = optuna.create_study(\n", " study_name=\"horizon_sweep\",\n", " direction=\"minimize\",\n", " sampler=optuna.samplers.GridSampler({\"horizon\": HORIZONS}, seed=RANDOM_STATE),\n", ")\n", "horizon_study.optimize(horizon_objective)" ] }, { "cell_type": "code", "execution_count": 7, "id": "85f8c736", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:51.391969Z", "iopub.status.busy": "2026-07-28T13:28:51.391810Z", "iopub.status.idle": "2026-07-28T13:28:51.401542Z", "shell.execute_reply": "2026-07-28T13:28:51.401039Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
masemase_std
horizon
10.4926850.251522
30.4318170.045623
60.5232510.068379
120.5083200.102901
240.5507600.083044
480.6706320.230549
\n", "
" ], "text/plain": [ " mase mase_std\n", "horizon \n", "1 0.492685 0.251522\n", "3 0.431817 0.045623\n", "6 0.523251 0.068379\n", "12 0.508320 0.102901\n", "24 0.550760 0.083044\n", "48 0.670632 0.230549" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results = (\n", " horizon_study.trials_dataframe(attrs=(\"params\", \"value\", \"user_attrs\"))\n", " .rename(\n", " columns={\n", " \"params_horizon\": \"horizon\",\n", " \"value\": \"mase\",\n", " \"user_attrs_mase_std\": \"mase_std\",\n", " }\n", " )\n", " .sort_values(\"horizon\")\n", " .set_index(\"horizon\")\n", ")\n", "results" ] }, { "cell_type": "code", "execution_count": 8, "id": "2115f533", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:51.403007Z", "iopub.status.busy": "2026-07-28T13:28:51.402885Z", "iopub.status.idle": "2026-07-28T13:28:51.405356Z", "shell.execute_reply": "2026-07-28T13:28:51.404780Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Best horizon: 3\n", "Best MASE: 0.432\n" ] } ], "source": [ "print(f\"Best horizon: {horizon_study.best_params['horizon']}\")\n", "print(f\"Best MASE: {horizon_study.best_value:.3f}\")" ] }, { "cell_type": "markdown", "id": "38ea1b88", "metadata": {}, "source": [ "### Careful: the shortest horizon almost always wins\n", "\n", "This is the trap of any horizon sweep. Predicting one step ahead is easier than predicting\n", "forty-eight, so minimizing an error over a list of horizons tends to return the shortest one.\n", "That is a true, but useless, answer.\n", "\n", "The table above is the interesting artifact: it shows *how fast* the error grows with the\n", "horizon. Three ways to turn it into a decision:\n", "\n", "| The question you are really asking | How to set up the search |\n", "| --- | --- |\n", "| Which window and model are best **for the horizon my use case imposes**? | Fix the horizon, sweep the rest. |\n", "| How far ahead can I forecast **before the error crosses my budget**? | Sweep horizons, compare to a threshold. |\n", "| What is the best **trade-off** between horizon and accuracy? | Multi-objective, look at the Pareto front. |\n", "\n", "The next cells illustrate the error budget, then section 4 covers the trade-off." ] }, { "cell_type": "code", "execution_count": 9, "id": "9045504c", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:51.406870Z", "iopub.status.busy": "2026-07-28T13:28:51.406725Z", "iopub.status.idle": "2026-07-28T13:28:51.409494Z", "shell.execute_reply": "2026-07-28T13:28:51.408951Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Longest horizon with MASE <= 0.6: 24\n" ] } ], "source": [ "MASE_BUDGET = 0.6\n", "\n", "within_budget = results[results[\"mase\"] <= MASE_BUDGET]\n", "if len(within_budget):\n", " print(f\"Longest horizon with MASE <= {MASE_BUDGET}: {within_budget.index.max()}\")\n", "else:\n", " print(f\"No horizon reaches a MASE <= {MASE_BUDGET}\")" ] }, { "cell_type": "code", "execution_count": 10, "id": "10f2c892", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:51.410744Z", "iopub.status.busy": "2026-07-28T13:28:51.410638Z", "iopub.status.idle": "2026-07-28T13:28:51.618889Z", "shell.execute_reply": "2026-07-28T13:28:51.616487Z" } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm0AAAFiCAYAAACkte/2AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjgsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvwVt1zgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAY41JREFUeJzt3Qd0FOXXBvAbeu+9d+kCUkR6VUCpgiKI0kEUFJAmiAooiFJEUTryUURERDoiXZCOSJVeQw8QSKHNd54bZv+bzW6yCUl2Nvv8zlnITiazs/POzt65b/MzDMMQIiIiIrK0RJ7eASIiIiKKGoM2IiIiIi/AoI2IiIjICzBoIyIiIvICDNqIiIiIvACDNiIiIiIvwKCNiIiIyAswaCMiIiLyAgzaiIiIiLwAgzayvNmzZ4ufn5+cOXPGtqx27dr6iMrGjRv1b/G/lRQoUEDefvttT++GT3r48KEMGDBA8ubNK4kSJZLmzZuLt5z3VjR27FgpVKiQJE6cWMqVKxftvzc/o7/88ovEtStXrsirr74qmTNn1tecMGFCnL+mN8E1tXTp0vHyWjivUQY4z8l9DNos5uTJk9K9e3e9CKZIkULSpUsn1apVk4kTJ0pwcLCnd8+nrFy5Uj755BNP7wbFspkzZ2qggS/vH3/8UT744AOPHuPPP/9cfvvtN/FGa9eu1QAY16hZs2bpe3Fl/vz5Hg+SUNZr1qyRwYMHy//93//JSy+9JAnZtm3b9Bp269YtT+8KxZIksbUhenorVqyQ1q1bS/LkyaVDhw56x3P//n3ZunWrfPjhh3Lo0CGZOnUqD/WTL4v4CNq+++47Bm4JzPr16yV37twyfvx4sQIEOgggHTN+b775prz++ut6PbDysUS2csaMGZIsWbJI10XQdvDgQXn//ffFk/vbrFkz6d+/v/gCBG2ffvqpZvUzZMggVpI/f35NRCRNmtTTu+JVGLRZxOnTp/UCjRMZF5acOXPafterVy85ceKEBnWuPH78WAM8ZOd8QVRfEL7OMAwJCQmRlClTRvgdluP44cs2pu7duyepU6cWb3T16lXLfYE5g+pGPKx+LHGOecvnMbbLPjY+S77YPAHfVzhuvvJ9FZt4plnEl19+KXfv3tU7VvuAzVSkSBHp06eP7TnaArz77rsyb948KVWqlN6Nr169Wn+3b98+adSokVatpkmTRurVqyd///13uO09ePBA78CKFi2qHxy08ahevbr88ccftnUuX74sHTt2lDx58uj2sV+4S42sjQ3apWDfNm3aFOF3U6ZM0d/hbhsOHDigd4BmVXCOHDmkU6dOcuPGjSiPl7M2bRcuXNBsBYKJbNmyaVVIaGhohL/dsmWLZjTz5cun7wttm7CuffUz9gtZNvNYmw8TLjqo6sGxx75nz55dq7UDAgIiBE8jR47UY5gqVSqpU6eOZkzd5e7roI3cyy+/rFU/FStW1C9SHG+zvdBPP/0kQ4cO1QwT9uPOnTv6d4sWLZLnnntO18+SJYu0b99eLl68GG7bOBY4j1B137hxY0mbNq20a9fO5T6fPXtW3nnnHXnmmWd0uzi3cLwdzxt3zkFnbt68qZmSMmXK6H7hPMf5/s8//7jVhmbDhg1aBmaZ4hi5avvorN2NeTxwnHC+4eesWbPqPj169ChC+aFpA/YV7xHroUpu9+7d+ntsGwEwqmnN/THbOrpq0zZ58mTbZz5Xrlx6U+dY/WW2TTp8+LCecyhzlD2uM+5+sY4YMUIKFy6sr4Pza8iQIeE+T9g3VIli/819d9U+CfuDm06cG+a62KbjsRo1apR+VnCscN3CzaqjHTt26DFMnz69vq9atWrJX3/9Fen7MY8lPo/4XDt+nk+dOqXnaKZMmXSbzz//fISb5Kg+S+7uF86bzp07a9nh2BYsWFB69uypN93RPb8nTZqk5wJeL2PGjPrZR0YTUC2KGhrAa5jv2Z02ku6cNwiA8T5wTUJ5Pfvss3oeO/v8fPXVV3odM88nbN/xs2UeX2cPx3Nlcjx8BqyKmTaLWLZsmQYvL7zwgtt/g4zczz//rMEbvnBxYuPLqEaNGvpBR1sTpJ7x5Y0TGIFUlSpVbB/oL774Qrp06SKVK1fWCw++SPbu3SsNGjTQdVq1aqXbe++993Tb+JDiC/XcuXMRPkSmJk2a6IUG+4WLlr2FCxfqB81s6Ipt4WKJwBABm1n9i/8RZNpfVKOCgAsXeexb79699YOMNis4Ro4QqAQFBemFEoHCzp079eKHoA+/AwRGly5d0n3Edhzh97jYYN/xesiUfvvttxow40Jtpvw//vhjDdoQ7OCB49uwYUPbBToq7r4OHDt2TNq2bat/07VrVw2aTPgCxp0tvgzwxYufze1WqlRJzwU00kaAge1i+/YZCXyJv/jiixpU4QKMC6Aru3bt0moZZI7xBYyL8/fff6/nIC6g5t+6cw46g3MGbcDwJYsvI+w3znGcb9g+yt4ZBEwoSwQGuEHCa0OJEiXkyJEjEh0IznA88HnC8Vi3bp18/fXX+qWE88qELzUcZ3zp4n3iOOKmAec3vmCxP+b779atm/4NtuEKjhkC3fr16+vroMxxbHHMHc8HBPYIIlq2bClt2rTRG6qBAwdqMID9iQz2CV/AqLbt16+fBiQ4XjhOS5Ys0XWw7/i84vMzffp0Xebq+vXRRx/J7du39TNmVkvjOmFv9OjRmrHCOYp18eWKmwO8tgmfZ+w7bjSGDx+u6yNwrFu3rh5XHEdnatasqfuLKmecW2h+YsL5g/3GNQGfMVwT8N6bNm2qx6xFixbhtuXss+TufuGagp8RYKC8ixcvrkEcXgevj225e35PmzZN9xdlhBt6ZP1wI4zj9cYbb2i5//fff7JgwQI95viOMD8HkXHnvMH1Fp9nBNX4/sF+4tqJGw68N/sEA+BYYP/wnhFoIThGkG4Pn0PHay221bdvX70Jj+/PgGUZ5HG3b982UBTNmjVz+2+wfqJEiYxDhw6FW968eXMjWbJkxsmTJ23LLl26ZKRNm9aoWbOmbdmzzz5rNGnSxOX2AwIC9DXGjh0b7ffTtm1bI1u2bMbDhw9ty/z9/XV/P/vsM9uyoKCgCH+7YMECfd3Nmzfbls2aNUuXnT592rasVq1a+jBNmDBB1/n5559ty+7du2cUKVJEl2/YsCHS1/3iiy8MPz8/4+zZs7ZlvXr10r91tGXLFl0+b968cMtXr14dbvnVq1e1LHCcHz9+bFtvyJAhut5bb73l4ghG73Ugf/78ugy/s4f3jeWFChUK977v37+vZVS6dGkjODjYtnz58uW6/scff2xbhv3EskGDBhnucHZ8t2/frtuYM2eO2+egKyEhIcajR4/CLcO5kTx58nDnlys4b0qVKuX0ONmfJ+Z2sRznoOPxcHyt8uXLG88995zt+fr163W93r17R9gH+/MhderUTs8Fx/PePJ8aNmwY7v1/++23ut7MmTPDvUfH4x0aGmrkyJHDaNWqVaTHZ//+/fq3Xbp0Cbe8f//+uhzvy/5YYP/dgbLGeerIPPYlSpTQfTRNnDhRl//777+2Y1a0aFHjxRdfDHf8cL4VLFjQaNCgQZT7gO3hc23v/fff1+X4vJkCAwN1mwUKFLAda1efpejsV4cOHfQ6uGvXrgj7Zv6tu+c3vi8cz2NHuH47Xjsj4+55Y15v586dG+6aUrVqVSNNmjTGnTt3bPuN9dKlS6fnb1SfLcfj8fLLL+v2zO+5q/H0GbAyVo9agJleR7VTdODOq2TJkuHu/tFAH1U2yNqZUK2JOy90aDBfC1kUZLSOHz/udNtmOxWkrB2r4qLy2muvaVbOvqoJdzi4s8Lv7F/DhLuw69eva7UEINsS3U4DeJ+46zQho2NmLxzfmwlVO3hd3Gnjmo4MU1RwR4kqENyx42/NB+6ykT1A9Rsg+4KMGjKV9llDdxtiu/s6JtztIvvjzFtvvRXufSOjhTJCNaZ9uxJkSnH376z9pH0GKTL2r4MqUFR3o3of55x9uUZ1DrqCO3WzDRHOeWwfxwOZxeieN0+jR48e4Z4jw40siWnx4sVa7si8OIpOFtlknk84f+zbUCGrisy6Y5nhmKC624TPM7I89vvo6rMEyHDYQ8YNImtb+zSQ9bVvG4fjCeb+7t+/X88VXMtQ5ubnAZ9hZNk3b94cIXvjDrxfHBdkke2PHa4dyBIjuxXZZ8nd/cIDGbRXXnlFs6yuzgl3z298fpC5RIYpNrlz3uCYoXYEmX0TMlzI/CGL7dg8BrU2UWX4HCGjuXz5cs1Um99z6+LpM2BlDNosACcbBAYGRuvv8CVt79q1a5pit68Ws08946Jx/vx5ff7ZZ59p6rlYsWKaKkbbB6TWTbhwjBkzRlatWqVtFlC9gOoKtHMzoQoDz80H2mKA2a4D1aEm/IwxnPB6JqyPNDq2j4sgPtTme8K2owNtZRAYOH4ZOjsWqEJFGh8perM9klmV687r4gKN9ZCyx9/aP3DBQjBk7hOgzZY9rIf2J7H1Oq7Oh8h+Z+6bs+ODoM38vSlJkiRa1ekOVJ2gWhhtBXEeoVoG+4zzzf74RnUOuoLzGNU9OK7228ffRve8iSmzfZo9lKn9DQ7aAKIqC+dZbHBVZvgiwk2aY5mhvBw/D4776Op18IWIz5M9fEkjUHB8ndiCNqaO+wrm/prBPYImx88DqmdRVRmT8sf7cXXNNH8f2WfJ3f3C9Rk3zVGNg+bu+Y1qPly/EIRgXbTriqptnzvcOW9wTPCajh0w3D1mUUH7bFSBYmgWBHz2rxsfnwErY5s2iwRtuLibDfTd5axnoLsQhOFLZenSpZqdw8UFF4offvhB27MA7mZwV4i7QzRwHzZsmLZrQfuN8uXLa8Bl3/AUgQ+ya7jQINuHti9oMIo2GbiYOI7hhDYGaPuEL2sEdLgA4YKFoC8md8zuwJ0rMlcIGHHRQ4CCjgtoV4JAzp3XxToIpNAJxJno3lHG1utEdj48zbniePcfFWQW0YYF50/VqlU1gMeFE23c7I+vO+egMziPcC6i0wruxhEUYd/wejE9b1xlvhw7Fpis3qszsn0MqyWMWkyygXG5v2bZYow9V4P4OraTiwuOnyV398u8qY2Ku+c3AiS050I2CkEOMru43uKGCQGPp86bp73+oN0u2jLiOo32wE8jcRy8F09j0GYR6PmHRr3bt2/XL7qYwJc4qgTxQXZ09OhR/eAj+2HCxQBVEnggc4MvUTTytP/CRKNoVIvggTtKXJTQ4Hru3Lna0cE+9WyfPUI1KAK6P//8Uxsv40NiXzWKOx38DhcXXGRM0a0qM2GoFAS9eB37LxvHY/Hvv/9q41zsm31jZGc9Fl19aeGYIE2PAUUjuxhhn8z3ZF9djTtud+703H2dmDD3DccHjaXtYZn5+5hAVTiyDjhP7Ku/nQ3w6c456Gz76A2Gntb2sH2zsXV0meeu4z4+TVYJ5YebHXxZR5Ztczc4si8z+/MJ1UX4okPD7NiA10FwgPPWzJwAbr5wfGJ6bjxtEGh20MBNbmy9V8D7cXXNNH8fG/uF6zPWiermPDrnN244cV3FA+cBGtyjow0yVMgGx1XgjWOCzB/OE/ubOXePWWRZerwHZHTRgcLxRjF/PH0GrIzVoxaBAAgfQHxZ4eLoCBkJ9OyL6q4CPRORubDv1o3toRs42myYVbGOw2rgThDVIWaXflSz4ovW8eKEdnfmOmhngA+J+UBbKxOe44sK1aJ4IIVvnyI374Ac73hiOmI6emaiZ5b9VDh4D46DETt7Xfzs7Nia45A5fpEjQ4gMDO6CHaF3oLk+jgHaeaBnqv3rufse3X2dmECbGmTxkNWyH8YB1eEIstG2LaZwjB3LFcfAMWsV1TkYne2j/Z/jUCXRgS8DbBftj+whcxFTqNbBfjrLetjvP84zd8oS5xOqgb755ptwf48vd1SbPU2ZOX6WnJ2n48aN0/9j+jp4n09TfY3rC65B6K2LAN8RboZi+n7RAxY3zCa0R8O1A73k7dsNP81+mVOmYaQAc8gXe2aZunt+O35+cG5gX/G3aEsa2TXsaeGYoUmMfRMYXJPwOcfn2HHkgOi0E8VNNWppnDUhqR9PnwErY6bNIvChR2CFOybc3drPiIAqRLM7dVSQTkbWCAEaGpmjLRK6i+OL0H58Gny40WUbFxwEV7iIIOBB923ABweNaBE4YF1sBx8kBICo5ooKghXcMWFMI1wAcUGzh+DRbCeHCwzGz0EVGe6WYgINUTEUBo7bnj17tFMCuo87Dk2B6lAca3TXx0UQ+4FqBWeZLzMIReNaNPDHxRTvHRckDKuBqmI0QkagjPeLzATKCQEgOkSYY3dhPWRScaFDRwcERu5khNx9nZjAdtBmERkuvA4aFJtDfuCL6mmmdsJ7xbFHtSjOHXwZImOIoRTsRXUORrZ9tIfDvqMDCbKnqEK2v/OOLuwrhljAlw6yEzhHUO3k2G4wOpAtwRAT+IJBmZnV/hgCAr8z3yfeP44PgiI0k8DNjTk0jz2cT8igIAjEtjAkBTIOCCwxbIt91vtpYLwtZEoRtODLHucHghpkpxF0YN9jAu8TX/Lo4ID9xZc7ml+4C0EPqtAxVAOGDkL547qBzzE65eCzjIAougYNGqRZHWwXn3Wci3ivuBbh2hBVs4Do7BeqPnGdwzFFRwdc6/39/fXzjI5iyDC5e37jeoB2hsjEo10wbrZwDUTgYnZqM69hGHIF1y587nHMn3ZgbOw7vlfwnYTrLa4Z+OyiGQyC/eh2qgN0IpgzZ47e7CCLZ9++FecKzr2s8fQZsDRPd1+l8P777z+ja9eu2tUcXZsxVEe1atWMSZMmaVfwyLqum/bu3avdz9FVOlWqVEadOnWMbdu2hVtn5MiRRuXKlY0MGTIYKVOmNIoXL26MGjVKu23D9evXdftYji796dOnN6pUqRJuSI2o/PHHH7qfGErj/PnzEX5/4cIFo0WLFroP2H7r1q11eBL8zfDhw6M15AdguI6mTZvqe86SJYvRp08f2/AY9kM5HD582Khfv74eH6yH4/3PP/9E6H6OIUvee+89I2vWrPoeHD8uU6dO1SEecPxQTmXKlDEGDBig78GEbumffvqpkTNnTl2vdu3axsGDB3Xog6iG/IjO62B7zobPMIcpWLRokdNtL1y4UIeqwHACmTJlMtq1a6flYi86wzqYw8V07NhRjy2OMc7Fo0ePRnjPUZ2DruBz0K9fP9sxxecDQ4o4OyfcHfIDrl27pkMB4PzJmDGj0b17dy0rZ0N+ODseOGcdzxGcQxh2Ae8Nn2ecS40aNTL27NljWwfHBsPx4L3YDwXj7Lw3hzfA9pImTWpkz57d6Nmzpx5zd94jtu1s2A1HDx480PMWQ1bgdfLmzWsMHjw43DUosmPhzN27d4033nhDyxvvy9wPV+eoqyEh9u3bZ7Rs2dLInDmznrfYTps2bYw///wzyn1wdd3EEEmvvvqq7luKFCn0vMTwN9H5LLm7X7hOYegPnAtYD0OIYJ/M4U7cPb+nTJmi5435eoULFzY+/PBDHULK3ogRI4zcuXPrUCNRDf8RnfPmypUrts85zm1clxzLyixDZ0NHOZaveb47ezi+9rfx8BmwKj/84+nAkYiIiIgixzZtRERERF6AQRsRERGRF2DQRkREROQFGLQREREReQEGbURERERegEEbERERkRdg0EZERETkBRi0xRBGNseI2XE1sTk9PZaR9bGMrI9lZH0sI98pIwZtRERERF6AQRsRERGRF2DQRkREROQFGLQREREReQEGbURERERegEEbERERkRdI4ukdICIiIrKCR48eyZYtW8Tf319y5swpNWrUkMSJE4tVMGgjIiIin/frr79Knz595MKFC7ZjkSdPHpk4caK0bNnSEseH1aNEREQkvh6wvfrqq+ECNrh48aIux++tgEEbERER+XSVaJ8+fcQwjAi/M5e9//77up6nMWgjIiIin7Vly5YIGTbHwO38+fO6nqcxaCMiIiKf5e/vH6vrxSUGbUREROSTHj16JEeOHHFrXfQm9TT2Ho0nVu9GTERE5CsePnwoCxYskFGjRsmxY8ciXdfPz097keJ729OYaYsH6HVSoEABqVOnjrzxxhv6P55bpTcKERGRL7h//77MmDFDihcvLh06dJD//vtPe4d+9dVXGpzhYc98PmHCBEskWhi0JdBuxLVr15b33ntPe7xkzJhRsmfPLtOmTZN79+5Jx44dJW3atFKkSBFZtWqVLRPYuXNnKViwoKRMmVKeeeYZHZvG3saNG6Vy5cqSOnVqyZAhg1SrVk3Onj1r+/3SpUulQoUKkiJFCilUqJB8+umnejdDRETkSSEhITJ58mQpWrSodOnSRU6fPq1JlIMHD8qiRYukX79+8ssvv0ju3LnD/R0ybFjOcdp8gKe7Ef/444+SJUsW2blzpwZwPXv2lNatW8sLL7wge/fulYYNG8qbb74pQUFB8vjxYz05cfIePnxYPv74YxkyZIj8/PPPui0EX82bN5datWrJgQMHZPv27dKtWzfbXQiqfnHXgveLv58yZYrMnj1bU89ERESeEBQUpAmIwoULS69evTRh8vbbb2s7tnnz5knJkiVt6yIwO3PmjGzYsEHmz5+v/yO4s0rABn6Gs4iCIvXss8/K1atXNdiKLF0aGhoqAQEBUR5NZMKSJ0/u8vfZsmWTf/75J9qZNrMdHeDn9OnT68k3Z84cXXb58mVtX4cA7Pnnn4+wjXfffVfXwV3GzZs3JXPmzJptQ+DmqH79+lKvXj0ZPHiwbdncuXNlwIABcunSJfEEBKLIBObPn18SJWJS2YpYRtbHMrI+llFEd+/ele+//16rPfF9nTRpUq1lGjRokNYoeWsZsSNCHBdSbK4XXWXLlrX9jOASQVeZMmVsy1BlCjih4bvvvpOZM2fKuXPnJDg4WOv+y5Urp7/LlCmT3p28+OKL0qBBAw3S2rRpY+tNg6Dyr7/+CpdZQ6CIlDTudFKlShUn75GIiMh0+/Zt+fbbb2X8+PFy48YNTYggAYEEQt68ecXbMWiLAQQo7kTNyEqh00FUfvvtN82MxTbcWdhDVab9MrNqE+/lp59+kv79+8vXX38tVatW1TZvY8eOlR07dtjWnzVrlvTu3VtWr14tCxculKFDh8off/yhWTrc1aANm7M0Mtq4ERERxZWbN29qNSgeCNyQKOjbt69+r1lhqI7YwqAtDqF7MNqJoQ7dWS20lboRI0uGtm7vvPOObdnJkycjrFe+fHl9oBoUwR3q/RG0oQMCuk2jcwMREVF8uHbtmowbN06za0gepEmTRqtAEbBlzZo1wRUCg7Y4hCpJRP3oJYoAzT5ws1o3YvSoQVu3NWvWaH3///3f/8muXbtsdf9ojDl16lRp2rSp5MqVSwO048ePa+cDQMeFl19+WfLly6fvF9lHZCTRM2fkyJEefndERJSQ+Pv7a3u1H374QZvgoM02vofQGQ7NeRIqts6OY6gu9IZuxN27d9d9ee2116RKlSraFsA+64ZU89GjR6VVq1ZSrFgx7TmKnjj4O0Bbt+XLl8vatWulUqVKmn1DmwJUHxMREcWG8+fP62gISCggw4bmN0gMoLkSmugk5IAN2Hs0nnqCcEaE+MceVdbHMrI+lpH1+UIZobZn9OjR2rb6wYMHOqoC2qthKCtUiVode496GVSBxkVnAyIiooQKzXA+//xzbbLz6NEjbZ6DnqBdu3b1yVEJ2KaNiIiILAWDtGMIKYxsgCwVhusYPHiwjrXmyyMSMGgjIiIiS0AHNrRRW7x4sXbew5SICNbQ6S1ZsmTi6xi0ERERkUft3r1bRowYIb///rs+x/zXmEoR84MmScJQxcQjQURERB6xbds2DdYwaDuULl1aB27H0FFWGA7LajzazQTzcmJMlerVq+twE5jY3JVly5ZJixYtdCBaFOaFCxds0TmGmMBy87Fv3754fBdERETkLlR7YsYgzFddrVo1DdgwaDuqRFE9iqGnGLBZMNM2ZswYnQ9z3bp1Ol0S6q1//fVXHSTP3tatW3XkfUyxhLFZELClS5fO9nuMgYapoIiIiMi6wRqmPkRmDd/rULlyZRk2bJg0adLENug8WTDThhGMEWljcFb0BKlVq5YULlxYNm3aFGHdadOmyQcffKANElGo6EViH7QRERGRdYM1DL6OQdcxEDsCNtSwYQaev//+W2fTYcBm8UzbuXPndIyV7Nmz25Zh3spTp06FWw/jsmAkfsyDidGO0SDxlVdekc6dO9sK+cqVK9KgQQMdYK9x48bSqVMnl6nV+/fv68MethndXinogmz/P1kPy8j6WEbWxzKyPquWEfYHtWAYZ81stlS3bl1ts4ZEjRnQOZub2xfKKFEMBkL2WNAWHBwsqVOnDrcMz2/fvh1u2c2bNzVwQzSO8VoCAwPl3XfflZw5c2o6tUCBArJgwQKd8/LMmTM6UWzKlCmlffv2Tl8Xoykjc2evdevW0qZNmxhPqWFFbdu2lRIlSuhcbLEJVdM1a9bUu6aSJUuKN7BqGdH/sIysj2VkfVYpI3xnr1y5Ur777jv577//dBmCNHx3P/fcc/ocMzj4ehkVfDK3t1cEbQis7t27F24ZnjuOcJw8eXL9H2O0pE2bVh/otPDXX39p0JYlSxZ9AKpPkYFbuHChy6ANA/O1a9cuVjJtOPioqrXitCGockYVcmzP/WneESFojs1to+zQKQWP2OKsjEJCQnTqE5wjoaGh0rBhQ72w2Gd8nTly5IjeEGzevFkePnyoAeuiRYv0ZoFit4zIWlhG1meVMsK1Ee3PMd3UsWPHdBlqxj766CPtMOjLHsdSGXksaMOXHdq1Xb16VecQA1SBIhCzh8Aja9asbtd3R3UwEJzF5gB9eD2rftngmMX2vpnbi4v3HRf767iv/fr1kxUrVmjAhQ4vuPNDb2TcBLiC8xLZRdwQfPbZZ3pOHjp0SG8wrFr23sbKnyMKwzKyPk+VEZoczZkzR7744gtt4oRrOa6rqAZ99tln431/EnIZeewqiS88pEunTJmi2Y8tW7bIiRMnbPXc9tBIEScEMnFov7ZkyRJtxGgO+XH58mVbO7kZM2boFyyF3fUgKEFwgmwkeujYtx3AB8ux122GDBlk9uzZtucYhgVdsZG5q1ixotPhVDAYYtGiRXWdOnXqyI8//qjbvnXrlm0dNDzFcCzIsOJOo3fv3rZMK+ZkRaocnU3wd3HVIBVV7zg/xo0bp+0qkKZHdTnGCUL1uyu4S0RbyS+//FKPBTrMNG3a1HazQUTki/DdPXnyZL3+Yy5QNFHCYLgHDx7UG2MGbLHPo7e2qG66du2ajtUyfvx4bayIAGPVqlXh2ph169ZNgw58caJ686WXXtKfAZ0UsAxBHAIUBACuqkZ9DYInVP0i8Jo4caIGK9OnT3f77+/evasBM6oC9+zZI5988olWLdo7ffq03lE1b95cx9dBb2AEOY6ZKpRZq1at5MCBA1o1iSAO5QUY5iVPnjyaxfL399eHK40aNdIOJ64epUqVcvm3eA8PHjyQ+vXr25YVL15cs77bt293mdJGZq5YsWLa6wmBWpUqVTjEDBH5LNSSTZgwQW9ge/XqJRcvXpS3335bv4/nzZvnNe2dvZJBMfI4Rw7jQdas+r9h/+jY8X8rTZsW/nf2j7/++t96zz/vfJ06dWJcOrVq1TJKlChhPH782LZs4MCBusyE4l+yZEm4v0ufPr0xa9Ys/XnKlClG5syZjeDgYNvvv//+e/27ffv22bZZunTpcNv46KOPdJ2AgAB93rlzZ6Nbt27h1tmyZYuRKFEi27bz589vjB8/Psr3deHCBeP48eMuH2fOnLGt++jRI+PUqVP6P8ybN89IlixZhG1WqlTJGDBggNPX8/f31/eSKlUqY9y4cfq+v/jiC8PPz8/YuHFjlPtLkXMsI7IelpH1xVcZ3blzxxgzZoyRLVs2vS4mTZpUr+14bYqfMuI0VgkYxsSxr2qsWrWqDlCMnj3ujDaNxvdly5bVak/7bdhDY1PHBqYYLNEeMnDIsOEOzISYEVksZOrQy9VdGEg5Ppnds5s1a6bVt1CuXDmtUv3hhx+cVucTESUkaFoyadIkrRHDiA7oIIiakgEDBmhzF4o/DNpiyLh4Uc6fPas9KP1cNSrs0iXsERUXVXNxDQGd4/g4qD6MbahmRbUp2rE5im7vS1SPov2jKygPdBJwJkeOHNpgFm3t0HbPhHaS+J0zqJZHFbNjuh+BpjmiNxFRQoQADdWg33zzjQZuaIvet29fbSaDEQQo/jFoS8AwNZg9NLZHg1Ezy4Zeufbtx44fP65tFewDk//7v//TxqZmts2xwf4zzzyj4/HY27VrV7jnFSpUkMOHD+vgya6gRy8ygFFBmzyM8edK0qRJXf4OHQ/w+z///FPb15mZQnRgccwg2u8XMolm93UTxh6K7eFUiIisAKM6oA00hkPCTTfaC6MNOgI2fG+QBz1V5aoPs3o7D7RpS5MmjfHBBx8YR48eNebPn2+kTp3a+OGHH2zrvP7669rGbe/evcauXbuMunXrahsFs01bYGCgkSVLFqN9+/bGoUOHjBUrVhhFihQJ16YNxwB/gzZhx44dMxYuXGjkyZNH17l165au888//xgpU6Y0evXqpX/333//Gb/99ps+NzVo0MBo2rSptlm7du1anJVRjx49jHz58hnr1683du/ebVStWlUf9p555hnj119/tT3Hz3iPU6dO1XZzkyZNMhInTqzt8ij2y4ishWXkO2V06dIl/c7A9RrXcLRx/vjjj40bN27E2r76qkexVEYM2jxcAHEZtL3zzjsapKRLl87ImDGjMWTIkHAdEy5evGg0bNhQg7miRYsaK1euDNcRAbZv3248++yz2oC/XLlyxuLFi8MFbbB06VIN5pInT27Url3b1lnBvgPDzp07NTBDIInXK1u2rDFq1Khwr4Nl2EZs3Us4KyPsE44Ljgc6F7Ro0UI7G9jD69sfA5gxY4a+xxQpUujxQNBJcVNGZC0so4RfRufOndObaPP6iw5oI0eOtN14k3U+R374x5OZPm+FBuoYWwxVZBwUNLxRo0ZpI31PT6nCMrI+lpH1sYwSbhmhIxgGxMXYnGjPjCGN0F6tZ8+eWiVK1vscsU0bPTUMroh2X5kzZ9aZBcaOHWsbg42IiKwFbXIxLurcuXO1LXGuXLm0JygGyHWcSpKshUEbPTV0YBg5cqT2NEJvUEwVNXjwYB5ZIiILQc961IRggHNkfnC9RgcDDFBvP7QTWReDNnpqGLsHDyIisp79+/frjfXixYv1eaFChWTIkCHy5ptvxupc3BT3GLQRERElQBh+acSIEbJs2TLbEE2YZrBt27Y6/iR5H5YaERFRAoK2xQjW1qxZo89Lly4tQ4cO1Xmi3ZkNh6yLQRsREZGXw0AQGzZs0DZr+B/Kly8vw4YN02n4OMpBwsCgjYiIyIuDNWTUEJzt2bNHl1WpUkWfN27cONz80+T9GLQRERF5YbC2fPly7WCwc+dOXVajRg0N1urXr89gLYFi0EZEROQlMFTHkiVLNFhDr1CoV6+edOnSRdq0acNq0AQu5sPyEhERUbzAILgLFiyQsmXLaocCBGyNGjWSbdu2ydq1a7VKlBI+ZtqIiIgs6uHDhzJv3jydwQAzGQA6FqA3aMWKFW3ZN/INDNqIiIgs5v79+/Ljjz/q3KCYIxQdClq3bq3jrD377LOe3j3yEAZtREREFhESEiIzZsyQMWPGyPnz57WNWrt27XQGg5IlS3p698jDGLQRERF5WFBQkEyZMkXGjh0r/v7+OmMB5gTFPM5Fixb19O6RRTBoIyIi8pDAwECZPHmyfP3113Lt2jVJmjSpdO/eXSdyL1CgAMuFwmHQRkREFM9u3bolkyZNkgkTJsjNmzclRYoU8t5778mAAQMkT548LA9yikEbERFRPLlx44YGagjYbt++LalSpZJ+/fpJ//79JUeOHCwHihSDNiIiojh29epVrQJFVejdu3clbdq02l7tgw8+kKxZs/L4k1sYtBEREcWRS5cuaecCdDIIDg6W9OnTy/Dhw6V3796SKVMmHneKFgZtREREsezcuXM6bAeG7wgNDZXMmTPrgLi9evXSwI0oJhi0ERERxZJTp07pgLgYGPfBgweSPXt2ba/Wo0cPSZMmDY8zPRUGbURERE/p2LFjOtUUppzCPKG5cuWSgQMHSteuXSVlypQ8vhQrGLQRERHF0KFDh2TkyJHy888/6xyg+fLl0zHWMDAuhvEgik0M2oiIiKJp//79GqwtXrxYnxcuXFinmmrfvr0kS5aMx5PiBIM2IiIiN+3cuVODtWXLlunz4sWL6yTur7/+uk49RRSXeIYRERFFYevWrTJixAhZu3atPi9durQMGzZMWrVqJYkTJ+bxo3jBoI2IiMgJwzBk48aN8tlnn+n/UL58eQ3WmjVrJokSJeJxo3jFoI2IiMghWFuzZo1Wg/7111+6rEqVKhqsNW7cWPz8/Hi8yCMYtBERET0J1tBWDcHarl279JjUqFFDg7X69eszWCOPY9BGREQ+DUN1/Prrrxqs/fPPP7qsXr16GqzVqlXL07tHZMOgjYiIfBIGwcX4aqNGjdLx1qBRo0YarFWtWtXTu0cUAYM2IiLyKZheCjMXYAaD48eP6zJ0LMDcoBUrVvT07hG5xKCNiIh8wv3792X27NkyevRoOX36tLZRa926tQZrZcuW9fTuEUWJQRsRESVoISEhMmPGDA3WLly4oEN1tGvXTgfFLVGihKd3j8htDNqIiChBCgoKkilTpsjYsWPF399fZyzAnKCDBw+WokWLenr3iKLNoyMDBgQESJ8+faR69erSsmVLnR7EFXTDbtGihXa/fvXVV/Vuyf53GDsHvXw+/fRTba9ARES+KTAwUMaMGSMFChSQvn37yvXr16V79+7afm3mzJkM2MhreTTThg9V5syZZd26dbJjxw69+0G36/Tp00eYPmT+/Pny9ddfS8GCBTVgS5cunf7uxIkTMm7cOPn2228lf/78MmDAAJk+fbr07NnTQ++KiIg84datW/LNN9/IhAkTNCmQIkUKee+99/R7IU+ePCwU8nqJPJm2xrQguPvBBwtZssKFC8umTZsirDtt2jT54IMPpFChQtpwNG/evLagbfXq1VK3bl0pVaqUpEmTRjp16iQrVqzwwDsiIiJPuHHjhg7TgRv34cOHS2hoqPTr1087GyCIY8BGCYXHMm3nzp2TVKlSSfbs2W3LihQpIqdOnYowjs7Ro0fl5MmTWvWJNgmvvPKKdO7cWQM4rF+5cuVw27h8+bIGhdi+s95DeNhL7aptw0sviTFjRtjP06eL3/Dhtl9hEpO8jx6JX+LE8njRIpEXXghbXq2ayJkzEbdVooQY69aF/bxunfi99ZbTlzS+/FKkXbuwbb3xhoiTIFZSphTjxImwn0+cED8Xgz8a774rMnhw2LaGDBH58Ufn6+3dK4JyePhQ/PLnd34smjQRY+rUsJ+nTBG/zz5zvq1ff8V8L2Gvif/tqrFtSpcWY82asJ/XrBG/Tp2cb2vcOJHXXgvbFv7fujXiSmnSiHHsWNjPx46JX926zsuod2+RgQPDlg8YIDJvnvPXPHBAJHNmkdBQ8StUyPmxaNZMjMmTw37+/nvxGznS+baWLhV5MnyAH/7394+4UtmyYqxaFfbzypXi17Wr821NnCjy6qth22rVSuTvvyOulCGDGE/GmpLDh8WvQQPn2+rbV6Rfv7Bt9e8vsmCB8/UOHhTJmBF3WOLn6jPSsqUYkyaF/TxpkviNHu18W7iRKlcu7DXLlxe5ejVCGRkVKoixbFnYHyxbJn49ejjf1rffirRoEfb3zZuLPBm5PpxMmcT499+wnw8cEL9GjZxvC+dCnz5h23r/fRF8lp2td/SoSNq0qHcTv+LFnR+L1q3FmDAh7OeJE8UPn2Vn20J5P+mp6FemjMjNmxFXqlRJjN9+C/t5yRLxw2fZ2bZ++EHklVfCtoX/8Vl2lC2bGPv2hf28f7/4NWnifFuDBom8917YtvA/PsuOZYT1MEQGrq0BAeJXurTzY9G2rRhffRX289dfix8+y85e848/REqWDHudUqWQKou40vPPi7F4cdjPv/wifk/KyxwU9969e/IgKEj2GIYYadPKoEGD5JMdOyQZPuPz5uk+2+TMKcbu3WE/794tfs2aOd+voUNFntTW+L3zjgg+y87Ww/dV8uSIGsXPVe/Tdu3CruswZoz4ffON822tXy/yzDNhr4n/796NuFL16mIsXBj288KF4ofPsrMymjlT5MUXw36H//FZdpQnjxg7doT9vGOH+LVs6Xy/Pv5YpHv3sG116ybiIilinD0rkiSJyJUr4lehgvNj8dZbYnz+edjPX3whfvgsO9sWvveKFAl7TfwfHBxxpVq1xJg/P+znefPCruvOtoXvvfr1w7aF/48cibhSgQJiPJmuTLZtE7/WrZ1v69NPRbp0CdtW587IGjlf7+LFsB8uXRK/SpWclpF06iR+o0aJ1wRtwcHBkjp16nDL8Pz27dvhlt28eVMDt7///lt++uknbavw7rvvSs6cOaVJkyYRtoNsG7gK2mbNmqWZO3sb793TbF+Efbx7V67jRMR2b96UjI8eRVgH+3b18mUJfbJeztBQSeJkvQchIXL5yToprlyRrE7W0fd7/brce7JeVuyXk/WMhw/lwpN1kly8KDldbOtOQIDcfrJextu3JY2L9S6ePy+PQ0I0aMNJ5Yz9sUh786ZkcLGe/bHIFRoqiaM4FimvXpUs7hyLoCDnx+LBA9uxSHrpkuRwUUbhjsWdO5EfC1wsQ0NdHougwEC5YR6LGzdcHosr/v5y3zwW9+87PRb3Q0LkihvH4sa1axL0ZL1sQUGS3Ml6jx88kItRHAu4HRAgd56sl+nOHUkd2bG4c0f8goMljxvHIt3Nm5I+smOBANDFsUAZ3Q8O/t+xuHbNvWMRHPx0x+Lmzf8di8BAl8fiwrlzYqRJI35377o8FvcCA+WmG8fi8qVL8uBJE5DcDx5IIifrhQYHy9Un20p17ZpkdrGt69euSfCT9bIHB0syZ+f//fty6ck6yfz9JbsbxyJzYKCkclJGtmORMqUkun1bcrs6Fnfu/O9YBAREfiyeXL9dHougoAjHAsEarvG4/gNu4Js1bSqjPvlEMmTIII+2bbPtb0yOxa0bNyQwkmNhOo91kieXRAEBLo/F3Tt3JODJttIHBEi6yI7Fk++hPA8eiJ+T9UKCguTak22lvn5dMrkoo+tXr9rOixwhIZLU2bEIDbUdi+SXL0s2V8fi5k3bschy966kjOxYJEkiia5dc30sbt9261j4X7woD5MmDTsWSCY4Oxb37kV6LEzXrlyRkCiOxcPQUPF341gE3Lwpd909FiKS+MoVyeWijHAsMkj0+RmYbM0DkD175513ZD3uLp748ssvJVmyZPI+7nifuHPnjlZ/ogfQc889p8vmzp0rhw8f1oER0cgUmbbXX3/d1qYBc8Rt3rzZ7Uwbsnd43ejAReP8+fNaVYvu42Q9LCPrYxlZn5XKCDU0+J7A8B24jqNNNJrO4LvEsS20L7FSGZH7ZRSTsvJYpi1fvnx6p3T16lXJli2bLkMVKLJn9tB2LWvWrC4n6kU7N3RGMGEbOXLkcBqwAYKz6AZokcFB54fE2lhG1scysj5PlhGawXzxxRfy448/6ugAaFbTv39/6dGjh612hfg58oXPkcdCcgRV6HyADBoGPtyyZYsGX84m53355Zdlzpw52nbhypUrsmTJEh0mBF566SXN1h05ckTu3r2r3bkdAz8iIvI+x44dk7feekuKFSumowLgBn/ixInawQBBGwM28jUezaOiwei1a9ekXr16Mn78eK3uRIp71apV0qZNG9t63bp1kyxZsuhYbBgYEYEafjY7HiA9jmpSLENWDp0UiIjIOx08eFDatm2rsxXghh29P7///nutSendu7ekTJnS07tI5BEea9OWEOqnz549q13MWT1qTSwj62MZWV98ltG+fftk5MiROl4nYBioIUOGyJtvvilJnzRMJ8+WEXm2jDiNFREReRRmwxkxYoQsX75cnxcvXlznBUUHM3QUI6Iw/DQQEZFHYLYbBGtr167V52XKlJGhQ4dKq1atJHHixCwVIgcM2oiIKN6gRc6GDRs0WMOsOFChQgWd0aBp06as3iOKBIM2IiKKl2BtzZo1Gqxt27ZNlz3//PMarDVq1MjlsE5E9D8M2oiIKE6DtWXLlmmwtvvJFFI1a9bUYA0jBzBYI3IfgzYiIoqT3nLoBYreoP/8848uw2w1CNYQtBFR9DFoIyKiWIO5FRcuXCijRo3S6QYBY2iig0HVqlV5pImeAoM2IiJ6apheat68eTpI+vHjx3VZ8+bNNVgz540moqfDoI2IiGIsNDRU5wTF3KBnzpzRNmqY0QbjrJUtW5ZHligWMWgjIqJow5zRmA90zJgxcuHCBR2qo3379jqDAaafIqLYx6CNiIjcdu/ePZkyZYqMHTtWLl++rDMWdOrUSQYPHqxzQRNR3GHQRkREUQoMDNRJ27/++mu5fv26JEuWTHr06CEDBw6UAgUK8AgSxQMGbURE5NKtW7fkm2++0XZrAQEBkiJFCundu7d8+OGHkidPHh45onjEoI2IiCK4ceOGjB8/XiZNmiR37tyRVKlSSf/+/aVfv36SI0cOHjEiD2DQRkRENleuXNEq0MmTJ2v7tbRp08o777wjw4cPl2zZsvFIEXkQgzYiIpKLFy9q54KpU6dKcHCwZMiQQT755BN59913NdOWJUsWHiUiD2PQRkTkw86ePavDdsyYMUPu37+vwRmmmurVq5ekS5dOp6NC0EZEnsegjYjIB508eVIHxEUHg4cPH0r27Nm1cwF6hKZOndrTu0dETjBoIyLyIUePHtWppubPn6/zhObOnVuH7ejSpYukTJnS07tHRJFg0EZE5AMOHjwoI0eOlJ9//lkMw5D8+fPLoEGDpGPHjpI8eXJP7x4RuYFBGxFRArZv3z4ZMWKELFmyRJ9j1gJMNYUpp5ImTerp3SOiaGDQRkSUAO3YsUODtRUrVuhzzAeKSdxfe+01nXqKiLwPP7lERAnIli1bNFj7448/9HmZMmW0N2jLli0lceLEnt49InoKDNqIiLwc2qitX79eg7VNmzbpsgoVKmiw1rRpU0mUKJGnd5GIYgGDNiIiLw7WVq9ercHa9u3bddnzzz+vwVqjRo3Ez8/P07tIRLGIQRsRkRcGa7///rv2Bt29e7cuq1WrlgZrdevWZbBGlEBFK2ferl07+fTTT23Pp02bJkuXLrU9xwja9erVi909JCIihdkJFi1aJOXKlZPmzZtrwFa/fn2tEt24caNef5ldI0q4opVp+++//8KN54M56tDItVmzZvo8KChIAgMDY38viYh8GGYsWLhwoYwaNUqOHDmiyxo3bqyZNVSHEpFvYPUoEZFFPXjwQObOnaszGJw4cUKXIcM2dOhQee655zy9e0QUzxi0ERFZTGhoqMyePVtGjx4tZ86c0SrPNm3a6DhrZcuW9fTuEZG3BG0BAQE6WCMawprPly9fbvuZiIhiJjg4WKZPny5ffvmlXLhwQYfqwMwFmMEAg+MSkW+LdtB28eJFW2cE3P3h+WeffabPEcixESwRUfTcu3dPfvjhB/nqq6/k8uXLOmNBp06dZPDgwTrtFBFRjII2M8NGRERP586dO/Ldd9/JuHHj5Pr165IsWTLp0aOHDBw4UAoUKMDDS0QxD9pwJ0hERE8HTUm++eYbmThxov6cIkUK6d27t3z44YeSJ08eHl4ievqgjb2ViIhiDtm08ePHy7fffqtZttSpU0v//v2lX79+kiNHDh5aIorb3qPnzp2Tbdu2SZo0aXS8oCxZsjztJomIEpQrV65oe7Xvv/9e26+lTZtWOxd88MEHvGYSUdwEbRhMF5MSv/vuu1K9enU5fPiwdO3aVccSgvTp08ukSZOkePHi0dksEVGChI5a6AmKa2dISIhkyJBBPvnkE60KzZgxo6d3j4gS8jRWf//9t5w+fVpnQTCnsbp//752TsDj1q1bMmXKlLjaVyIir3D27Fnp2bOnFCpUSNuuoSYCA+Ri+fDhwxmwEVHcZ9pw15gtWzbNqGFalT179ugQH++9955UrVpVOnToIP/++2/M9oSIyMth1oIvvvhC5syZo9fI7Nmza+cC9AhF+zUiongL2jCvaOHChfXnU6dO6UCQGPwRc4+mS5dOChYsKCdPnnyqHSIispJHjx7Jli1bxN/fX3LmzCk1atSQxIkTh1vn6NGjOi/o/PnzdVL33Llz67AdXbp0kZQpU3ps34nIh4M2BGYYpRsNadH5APLnz6/LAVWlyMIRESUEv/76q/Tp00eveyYMyYGhOlq2bKk1CyNHjpRFixZpExFcDzEg7ttvvy3Jkyf36L4TkY8HbZhG5a+//pImTZpolg1VozVr1tTfBQUFyfnz56VYsWJubw/jE6FRLqpZUe06aNAgqVy5coT1sM6aNWt0lHDA3e7PP/+sP+/evVvbjmCcIxPakJQvXz46b42IKELA9uqrr0YYUBzNRFq1aqXXqp07d+oyzFqA3qCYcipp0qQ8kkTk+aCtW7dusm/fPs20QaZMmaRdu3b6M3qVohqhQoUKbm9vzJgxkjlzZlm3bp3s2LFD71BxoXSWrevcubNWNTiDqojffvstOm+FiMglXMuQYXM2A4y5DAEbesoPHTpUXnvtNdtNJRGRZTJtP/30k2bbcOGqW7eurRcUxmdzlSlzBpm5jRs3ytKlSzVLVqtWLW0vt2nTJmnatGnM3g0RUSxAGzb7KlFXMAUVroNERPEh2reGGLUbVQOOMLAu/PPPP5I3b163BuVNlSqV9q4yoYoBHRycWbBggT7QZqRXr17hZmfAwJUNGjTQbvWNGzfWiZYdGwqb0O4OD3u4Q8acf9GBxsb2/5P1sIysz6pl5E7ABuicYLV995Uyov9hGXlnGaEjZ3TFSj7/6tWrsnz5cn2gvQeqOqOCNnGOXeDx/Pbt2xHWff3116Vv377aCwtVqfgZGT+0bcOkygjm8uXLJ2fOnNFsH9ZD2xJnZs2apePL2WvdurW0adNGYgLt+MjaWEbWZ4UyQu3BgQMHZNmyZbJkyRK3/gYXXYy95gusUEYUOZaRd5URRtyIt6AtNDRUNmzYoBc4dAYwB9hF5wR3ILAy28aZ8BzZN0f2Myw0atRIVq5cqQP9tmjRQqtlzamzMJAl2r4tXLjQZdDWsWNHWzu8p8204eAjqxiTaJniHsvI+qxQRgcPHtSbQFw3zEw/en7iGoWbS2dwnUMvUnRUcJXVTyisUEYUOZaR75RRtIM2VH8io4aMF4Iss1EuLmK1a9eWOnXquLUdZMbQrg1ZOvQcBYzxhp6pUcFrOWsgDFEdDARn0Q3QIoPX44XM2lhG1hffZYRrDQI1PBC0mTdvuP60bdtW29X+8ccfGpSB/fXGvDGdMGGCT/UU5efI+lhGCb+MohW0YVwis60HLmK4w8RMCOhFhflHx44d6/a2kFFD5wNMe4URw3ft2qWjiWOZoz///FNeeOEFvUCil+r+/ftlwIAB+jtk+XDHi7Z2aCc3Y8YMeemll6LztojIB6DpBoYKQnMKXG/MAAw3mgjUcH1Db3YTnv/yyy9Ox2lDwIbfExHFp2gFbUjt4SKHXp6ommzYsKFOgIyLnjlpfHSg/Rnm4atXr552SMDcfBjuY9WqVdr2zByLDaOMf/bZZ/oz2rB99dVXeuE0RyIfNmyYztaAIUjQEcFV1SgR+Zbr169r4IWM2ubNm20ZM3ScQltZtGVF21hXEJhhxpeoZkQgIooPfoarekYnKlWqpEEbMl4YVBdVCci01a9fX6tKzYEmfaV+Gg2Q0ZuV1aPWxDLyzTK6c+eOjtuIjBqqODHmGpQtW1YzahhTLSYNgH0VP0fWxzLynTKKVqYNQRqqJ9E4F1WWeGCcNleNdYmI4gOuQWhri4zaihUrtKOUOYwQAjVk1UqWLMnCICKvFq2gDdNJYRJkdEJAr1G0Lbt586atYS7akqEzAqo9iYjiEsZbRCYNGTUM0n337l1djqYTCNLwwAwt7vZoJyKyumj3HkU3+FdeeUUfly5d0uANQ3Dg5xs3bug0VAzaiCguoKoTs6Ygo7Z48WK9aYSsWbNKhw4dNKuGTktsskBECdFTDa6bK1cu6d69uz4w6TuqJ1B9SkQUW9DsFgN2I6OGzkmXL1/W5ei0hHEXkVHDVFKc+5OIErpoBW2YZy8yGOSW3eCJKLZmJzDHUsNsJ2amHx0JkFFDcwwMgktE5CuiFbTNnj07yvYh+D3GNSIiiq7jx49rRg2B2pEjR3QZeqtjsFtk1NAsA3MMExH5ohhVj0Y2Skg0RhAhItL2sOZYanv37tUjgjZpGEoIGTWMCYle6kREvi5aQRumf0KPLdz5YkBcVFOULl067vaOiBIkTF+HQA1Zta1bt9qWoxMBAjVMH4VZToiIKIZBG3qJLlmyRHttrV69WtasWSMlSpTQ4A2zI7AhMBG5cuvWLb1+IFDDGI8YbBJKlSolb775plZ/YuBJIiKKhRkRTLjYots9enJh7k+0Y0P1BS7ImFPUF3AEautjGXkeZkpBr3IEapieDpl6eOaZZzSjhmmkUqRIwZlFLIyfI+tjGVmfR2ZEMOEFUS2KhsKHDx+WoKAgCQgIsE0XQ0S+C7MRIAuPNmoY9BbXB8iXL59m0xCsPfvss3qzZ17IiIgoDoI2zIKwcOFC2bhxozx8+FC74GOYD9wxp02bNrqbI6IEANcCXBOQUcMA26gKhezZs0vnzp01WMMk7Rz0logonoK2N954Q06cOGEbWBeBGrrisws+ke9Blmz79u2aUUNTCXQugAwZMmighoxarVq12NaViMgTQRvGUEKVBjoc4MKMOUjxcDRz5szY2j8ishA0gUW2HRk1ZNzPnTuny1OnTq03dciovfjii9rTnIiIYleM2rQ9ePBA27KBYz8GTs5MlPAcPXrUNjvBsWPHdBkCs+bNm2tGrUmTJhq4ERGRRYK28uXLMygj8hHoIGAGasiuQeLEiXV4H3PQW8z/SUREFgzapk6dGnd7QkQeh8nYFy1apNWfaK9mqlGjhgZqrVq1kmzZsnl0H4mIfFWMqkfJd2FYly1btoi/v7/kzJlTv8yRfSHvheF6MGA2MmobNmywDXr73HPP2cZSy5s3r6d3k4jI5zFoI7dhKIc+ffrIhQsXbMvy5MkjEydO1GFfyHvcvXtXfv/9d82oYUw1tFOFkiVLaqCGWU6KFi3q6d0kIiI7DNrI7YAN80E6djy5ePGiLsc8kgzcrC0kJESnn0OgtmzZMgkODtblBQsWtA16i0Gz2ZmIiMiaGLSRW1WiyLA5m/EMy/Al//7770uzZs1YVWrBQW8xzyeqPhF437lzR5ejartbt24aqFWuXJmBGhGRF2DQRlFCGzb7KlFngdv58+c1OEDPQvIstEn766+/NKOGTgXXr1/X5ZkyZdJADVm1mjVrMsAmIvIyDNooSuh04I6XX35ZqlWrpgEBHpi2iGN3xQ8Eznv27NGMGga9NYNszFbSvn17zajVr1+fg94SEXkxBm0UJVSluSNz5sw6/yQeenIlSSKVKlWyBXEI6DiuV+zCINfIqCFYM6eYS548uQ7NgYwaBr3F/MBEROT9GLRRlDCsB3qJotOBs3ZtaNOG358+fVouXbokmzdvtj0w1hceY8aM0cnCy5UrpwEc5qSsXr26ZMmShSUQTadOndJsGoK1f//9N+yDnCSJNGrUSDNqaFuYLl06HlciogSGQRtFCeOwYVgP9BJ1ZPY0nDBhgq6H8bzatWunD7hy5Yq2iTODuH379snevXt1fShVqpQtiENwmCtXLpaIEwiGMSk7Mmo7duywHfvatWtrRg2ZNQbAREQJm5/hLHVCbjX2xjQ/+fPn1wySLxgyZIh88cUX4ZYhSEMA5u5wHzdv3tRG8mYQh3ZY6J1qKlKkiC2Iw/84vjEdgsLby+jGjRs66C0yaps2bbJlOdHbExm11q1bS+7cucWbeXsZ+QKWkfWxjHynjJhpI7eZjdu//PJLrQ6NyYwI6MH4yiuv6AMCAwO1+hQBHAKTnTt3ysyZM/VhBoVmAIdHsWLFEvTwFDgev/32m2bU1q5dq0N2QJkyZTSjhkehQoU8vZtEROQBzLTFkK/d2WBg1uzZs0vSpEm1Nyn+jwsY8BWBmxnEbdu2zTYILGAfzAAODwwG6+r4e0sZ4f2tXLlSM2orVqzQYw2FCxfWjBoCNVQjJ0TeUka+jGVkfSwj62OmjeIVsj4YmLVr165xFrABejois4bHsGHD5P79+9oGDgEcArmtW7fq2GN4QMaMGTXbZwZx5cuX10b5Vodpo9atW6eBGjJryLABqjsxhRSCNcz9mZCzikREFD3W/3YjS0AjeMDk4fEpWbJkOt4bHgMHDtT2bwcOHLAFcXhgDk08zHHJzLHi0Ds1W7ZsYhXYd3TKQNUnpv1CmzVzqJQePXpooIZ9ZsaJiIicYfVoDPlSOhrVdwh+UqRIoVWjVspkoRyOHj1qC+Lwv/1gwBizrGrVqrbODQj+UqVKFW/7h84Du3bt0owaAl/0AgUMydGiRQut+qxXr16cZi+tzJc+R96KZWR9LCPrY/UoxZs1a9bI3bt3dRgPKwVsgC/6kiVL6qNnz54aJJ08edIWwK1fvz7cgL8IjipWrGjr3ICsnLtjmpmZMgSFUXXCwPhpyKjhgXHVAEEvenwio4Yx1fCciIjIXdb6BiZL8lTVaEygDRiGDcHj7bff1iwOlpnDjCCQMwf8HT16tG3AXzOIczXgLyZb79OnT7g5WNGDFuPXmcOdYEYCM1A7dOiQLUjE9F7IqDVt2lTSpk0bj0eDiIgSElaPxpCvpKPNqlFUKWJGBKtl2mJSRvYD/iKIQ1bMfrhC9NS0H2YEAR4GFnYc0tDsJNChQwcN0nbv3m1bXqdOHc2oIaDDMCcUvTIi62AZWR/LyPpYPUrxYvXq1Vo1+uabb3pVwBYZDBuCIMyc4cEc8NdsF4feqgjCJk+erL/H+3Y2BrW57Mcff9T/0XYOGTVUgbo7XysREZG7Esa3MMUZb6oajSlXA/4iiEOv1IMHD0a5DXQ0QMBGREQUVxi0kUtBQUGybNkyzUyh0b2vQLuzhg0b6gOD977xxhtR/g1ngyMiorjGRiTk0qpVq+TevXs6GXl0pqpKSNyt5mR1KBERxTUGbeTTVaNRQYYRvURdzUyA5Zgf1ZcykURE5BkM2sgpZNiWL18uOXLk0GEwfBUyjBjWAxwDN/P5hAkTfDYTSURE8YdBGzmFCczRpg09LH09IMGwHZh2CvOC2kMGDsvNcdqIiIgSbEeEgIAA+eSTT2TPnj06FtigQYOkcuXKEdbDOhiV3xxyAu2HzKo7QGP577//XrNDdevWlSFDhvjstECxhVWj4SEwa9asmdszIhARESWooG3MmDE6Wfa6detkx44dMnjwYB15Pn369BHW7dy5s3Tp0iXCcoxCP27cOPn22291gM4BAwbI9OnTdUojihmMy7ZixQoNTDDNE4VBgFa7dm0eDiIi8q3qUVS9YT7I7t276xyMGIG+cOHCOjZWdAd/RXYNo9inSZNGOnXqpAEHxRyOH2ZCwCCxHKWeiIjIxzNt586d06mRMAaYCfNFmpNrOxu8FA9k03r16iXPPfecLsf69lWq2Mbly5c1KMT2Hd2/f18f9lDtmixZsmhPSWH/f0KycOFC/R/t2bz5/SXkMkooWEbWxzKyPpaRd5ZRTJIiHgvakMlJnTp1uGV4fvv27QjrYqT5vn37SsqUKbUqFT9jUm5U3zluB9k2cBW0zZo1S6ZNmxZuGTJKMR3W4vz585KQoF0gOiGg12iuXLl0Xkhvl9DKKCFiGVkfy8j6WEbeVUYFCxb0nqANARgCBHt47izQKl68uO3nRo0aaVDx999/S4sWLSJsB+2xwNl2oGPHjtKuXbtYybTh4GOMroRUhYhgODQ0VF577bUYnVBWklDLKCFhGVkfy8j6WEa+U0YeC9ry5cun2bCrV69qz1E4efKkNGnSJMq/xfhY5rRBhQoV0s4IJmwDWSJXQRuCs+gGaJHBwY+sAB49euRVPQ4xhAUgaEsogU5UZUSexzKyPpaR9bGMEn4ZeeybDEEVOh9MmTJFQkJCNLBB8IVljv7880+tBn348KGsXbtW9u/fb2vH9tJLL8n69evlyJEjmmWbOXOmW4FffEBP2AIFCkidOnV0/kr8j+dYbkWYKB1ZTNwJVKlSxdO7Q0RERHY8mn7AuGzXrl2TevXqyfjx4+Xzzz/X4T4w56V9G7P58+drcIb15s2bJ1999ZUObGp2PPjggw+0nVvjxo0la9asOjyIpyEwQ0P+CxcuhFt+8eJFXW7FwA3j3aFqFPvHzBQREZG1+BlmPSNFu34ajfTRm9UxwEGVKDJqjgGb7aD7+WnQefr0aUtVlTZv3lyWLl0q27dvl+eff14SchmRNbCMrI9lZH0sI98pI36TxQFU9boK2ABxMhokYj2ruHPnjmY40daQVaNERETWw6AtDqDTQWyuFx9+//13Hb8Ow584ToxOREREnsegLQ6gl2hsrhcfONcoERGRtTFoiwMY1gNt1lxlrLAcPTSxnhXcunVL1qxZo3XtlSpV8vTuEBERkRMM2uIAOhdMnDhRf3YM3MznEyZMsEwnBLNqFD12WTVKRERkTQza4kjLli11oNrcuXOHW54lSxZdjt9bBatGiYiIrI9BWxxCYHbmzBnZsGGD9OvXT5c1bdrUUgEbqkYxYDGmrHruuec8vTtERETkAoO2OIYq0Nq1a8vIkSMlXbp08ttvv8mDBw/EKjAuG/aHVaNERETWxqAtnqRIkUKaNWsmN27c0MybVbBqlIiIyDswaItH5tRcZqDkaQEBAVo1WrhwYSlfvrynd4eIiIgiwaAtHjVo0EDnVsW8o1aoIkVV7cOHDzmgLhERkRdg0BaPkidPrvN7IsP1559/iqexapSIiMh7MGjz0SpStK1bt26dFClSRMqVK+fRfSEiIqKoMWiLZ/Xr15cMGTLIkiVLdEBbT1eNstcoERGRd2DQFs+SJUsmLVq00PHRkOnyFFaNEhEReRcGbT5YRXr9+nVtU1esWDEpW7asR/aBiIiIoodBmwfUq1dPMmbMqFWUoaGh8f76qJp99OgRq0aJiIi8CIM2D0iaNKlOZXX79m35448/4v31Fy1aFC7jR0RERNbHoM3HqkivXbsm69evl+LFi0vp0qXj9bWJiIgo5hi0eUidOnUkc+bMOvdnSEhIvL0uq0aJiIi8E4M2D1aRtmrVSu7cuaNTScUX9holIiLyTgzafKiK9OrVqzpZfcmSJaVUqVLx8ppEREQUOxi0eVCtWrUka9asWkUaHBwc56+HOU8fP37MDghEREReiEGbByVJkkSrSO/evStr1qyJ89czM3qtW7eO89ciIiKi2MWgzUeqSC9fviybNm3SalFUjxIREZF3YdDmYTVr1pRs2bLJ77//LkFBQXH2OqwaJSIi8m4M2jwsceLE8uqrr8q9e/dk1apVcfY6rBolIiLybgzafKCK1N/fXzZv3ixlypSREiVKxMlrEBERUdxi0GYB1atXlxw5csjy5cs14xbbFi9eLIZhsNcoERGRF2PQZqEqUrRpW7lyZZzNNcpeo0RERN6LQVsCryK9dOmSbNmyRZ599ll55plnYnXbREREFH8YtFlEtWrVJGfOnLJixQodty22sGqUiIgoYWDQZhGJEiXS6kvMjIDALbaw1ygREVHCwKAtAVeRXrx4UbZu3Srly5eXokWLxso2iYiIyDMYtFlI1apVJXfu3NoZITAw8Km398svv4QLBomIiMh7MWizYBVpSEiIDv/xtFg1SkRElHAwaEugVaTnz5+Xbdu2SYUKFaRw4cKxtHdERETkKQzaLKZKlSqSN29endLqzp07Md4Oq0aJiIgSFgZtFq0iDQ0NlWXLlsV4O6waJSIiSlgYtCXAKtKzZ8/K33//LRUrVpRChQrF8t4RERGRJzBos6DKlStLvnz5ZPXq1XL79u1o/z2rRomIiBIejwZtAQEB0qdPH50wvWXLlrJz584op2TCzAEjRoywLdu9e7dUqlRJatSoYXvs27dPvJmfn59m2+7fvy+///57tP+ec40SERElPB4N2saMGSOZM2eWdevWafA2ePDgSDNL48aNczp/JsY2w/ya5gODyfpqFemZM2dkx44dmq0rUKBAHO0dERER+UzQFhQUJBs3bpTu3btLihQppFatWjo0xaZNm5yuv337djEMQ3tX+gK0R0PQtWbNGrl165bbf8eqUSIiooQpiade+Ny5c5IqVSrJnj27bVmRIkXk1KlTEdZ98OCBTJw4Ub766iun83JeuXJFGjRoIGnSpJHGjRtLp06dJHHixE5fF1WOeNhLkiSJJEuWLFr7//jx43D/xwX0Ih07dqwsWbJE3nrrLbf+xszMobo5LvfNG8RHGdHTYRlZH8vI+lhG3llGGC3Ca4I2TIyeOnXqcMvw3Fn16Lx587QtW548eSL8DtmoBQsWaMN9VA0OGjRIUqZMKe3bt3f6urNmzZJp06ZFCI5iOtUTBrGNK2jrh6Btzpw5Urt2bbf2ZdeuXbbqYfQipbgtI4odLCPrYxlZH8vIu8qoYMGC3hO0IbC6d+9euGV4juybvatXr2pj/Llz5zrdTpYsWfQBGN6ic+fOsnDhQpdBW8eOHaVdu3axkmnDwcdAuDGJlt2BQBTvCZO+p0uXTjJmzOhWlg3vPX/+/OLr4qOM6OmwjKyPZWR9LCPfKSOPBW0ISNCuDUFZtmzZdNnJkyelSZMm4dY7fPiwVn+2aNFCn+Nv8Ob9/f1l8uTJEbYb1cFAcBbdAC0yeL24DAiQARw9erQGrgg43e01yiAl/sqInh7LyPpYRtbHMkr4ZeSxbzJk1ND5YMqUKTpBOnp9njhxQpfZe+GFF2Tp0qVaRYpHq1atpE6dOvL555/bhvy4fPmyrZ3cjBkzpGbNmpJQuNuLFAHvnj17pGrVqhrJExERUcLisUwboP3Z8OHDpV69etohAYFY+vTpdd5NtD1DoIKsmFn9aVarJk+eXDJkyKDPjx49KsOGDZPAwEDJlCmTdkRwVTXqjcqVK6cdNDAsyo0bN3SIlMiybDFtm0dERETW5mdgHA2KNlTRoqE/2o7FddXb0KFDZdSoUTJ9+nRts+dMhQoVdFBh1Jk767Dhi+KzjChmWEbWxzKyPpaR75QRv8kSQBUpqpURsLnqYUtERETej0GbFyhTpozOBPHnn3/K9evXI/yeVaNEREQJH4M2L5qL9NGjR/Lrr79G+D0ycFgHnTSIiIgoYWLQ5uVVpP/995/s379fB+LFHKxERESUMDFo8xKlSpWSEiVKyIYNG3RsOxOrRomIiHwDgzYvqyJFDxT7KlJWjRIREfkGBm1eBDMd2FeRYoy6AwcO6GDCOXPm9PDeERERUYIdXJeiX0WKx8aNGzXbtmDBAl3+6quv8lASERElcAzavEzp0qXl0KFD4XqKYiaJXLlyScuWLT26b0RERBR3WD3qRZBdczbALuZeRbbN2XAgRERElDAwaPMSGKOtT58+4mzWMXPZ+++/r+sRERFRwsOgzUts2bJFLly44PL3CNww7yjWIyIiooSHQZuX8Pf3j9X1iIiIyLswaPMS7g7pwaE/iIiIEiYGbV6iRo0akidPHh1k1xksz5s3r65HRERECQ+DNi+ROHFimThxov7sGLiZzydMmKDrERERUcLDoM2LYBy2X375JcLE8MjAYTnHaSMiIkq4OLiul0Fg1qxZM+0lik4HaMOGKlFm2IiIiBI2Bm1eCAFa7dq1Pb0bREREFI9YPUpERETkBRi0EREREXkBBm1EREREXoBBGxEREZEXYNBGRERE5AUYtBERERF5AT/DMAxP7wQRERERRY6ZNiIiIiIvwKCNiIiIyAswaCMiIiLyAgzaiIiIiLwAgzYiIiIiL8CgjYiIiMgLMGgjIiIi8gIM2oiIiIi8AIM2IiIiIi/AoI2IiIjICzBoi4GAgADp06ePVK9eXVq2bCk7d+6M/ZKhaPnll1+kXbt2UqVKFZkyZUq43y1btkwaN24stWrVkk8//VQePHjAo+sB9+/f1+PfpEkTLYu3335bDhw4YPv97NmzpX79+lK3bl2ZOHGicIY9zxg1apS8+OKLWkavvfaabN68mWVkQfjsVKpUSaZPn25bxs+QNXTr1k1eeOEFqVGjhj569+4de2WEuUcpegYOHGh8+umnRnBwsLFx40ajbt26xq1bt3gYPWjDhg1aFoMGDTJ++OEH2/Ljx48btWvXNg4ePGgEBgYaPXv2NCZPnsyy8oCgoCBj6tSphr+/v/Ho0SNj9erV+tm5d++esWXLFqNx48bG+fPnjWvXrhlt2rQxlixZwnLygNOnTxuhoaH6Mz43NWvWNAICAlhGFoLPz1tvvWV06NDBmDZtmi7jZ8g6unbtaqxYsSLC8tgoI2baoikoKEg2btwo3bt3lxQpUujdaOHChWXTpk3R3RTFotq1a2tZpE2bNtzy1atX6x1NqVKlJE2aNNKpUydZsWIFj70HpEyZUrp27So5cuSQRIkSaTYnadKkcvbsWVm5cqW0aNFC8uTJI1myZJH27dvrMop/BQoUkGTJkunPfn5+8vDhQ7l27RrLyEJ+/fVXKV26tBQsWNC2jJ8h64uNMmLQFk3nzp2TVKlSSfbs2W3LihQpIqdOnYrupigeoFyKFi0arqwuX76swTd5/rN0584dyZs3r5w+fTpCOZ08edKj++fLRo8eLdWqVZMOHTpIxYoVtTxYRtZw69YtWbBggSYO7LF8rGXcuHFaDfrOO+/I8ePHY62MksT6niZwwcHBkjp16nDL8Pz27dse2ydyv7yQbQMEbQi+yTNCQkJk2LBh2q4NZYLysC8n/IyyI88YNGiQfPjhh7Jnzx79UkHGjWVkDZMnT5a2bdtGqFVg+VgH2rAVKlRIaxQWLlyoz9HuOjbKiJm2GFTx3Lt3L9wyPGcA4B3ldffuXf2f5eU5qG5DUIAMG6pLzfKwLyf8jLIjz0mcOLFUrlxZO1pt3bqVZWQBR48elcOHD2sVmyN+hqwDVdcoDzSheuutt/Tnf//9N1bKiJm2aMqXL59Gy1evXpVs2bLpMtyJokccWQ/udk6cOGF7jrJCmyoGbZ7x+PFjzbAhc/PJJ5/o/4C2OSgntEs0ywltRcnzHj16JBcuXGAZWcDevXu1DSh6w5s3oQiuL168yPKxMGTcYus6x0xbNOHLHgccw0qgimfLli3hCoE8l70JDQ3VoABfMvgZ/7/00kuyfv16OXLkiF7gZs6cyQDbgz7//HO5ceOGtplKkuR/94z4EkLjagQH+P28efNsX0wUf/AZQecd3JjiM7Vu3TrZvXu3lC9fnmVkARhiasmSJfr5wKNmzZrSunVr6du3L8vHIgIDA+Xvv//WIY4wvBTKCW13kX2LjeucH7qQxtneJ+Bx2oYPH67tPdAhYeDAgTo+GHkOguhp06aFW4YyeuWVV3ScNrQDQSoaPUmHDBli6x1H8cff31/LI3ny5LY7T/jmm280KJg1a5bMnTtXA+/mzZtrOxAzE0fxF7T169dPjh07puNHoQobPa7xuQGWkbUgW42eiF26dNHnLB9rxAe4diEjihvTYsWKyfvvvy/FixePlTJi0EZERETkBVg9SkREROQFGLQREREReQEGbURERERegEEbERERkRdg0EZERETkBRi0EREREXkBBm1EREREXoBBGxHFKgygW7FiRR3wOC5h+3gdvJ7VfPTRR7pvv//+uz6/dOmSPscDMwx4A+w79hfvhYisgUEbEXklzEaCqWGeeeYZsZJTp07JH3/8IZkzZ5ZGjRqJt8K+4z3gveA9EZHnMWgjIq+C6ZUwLyamgJk9e7Z89dVXYiWLFy/WKWrq1asnSZMmFavBfIjuwL7jPeC9YL5EIvI8Bm1EFCcQWI0bN06/+Bs0aKDBFZaZQkJC5LvvvtPg6/nnn9f5LTHv5YkTJ2zrYN5Ys1px27Zt0qZNG113//79EapHUe1oruv4wHbg0aNHOu8fJtmuWrWq1KpVS9555x3Zt2+f7TXtt7Nx40bp2rWrVKtWTVq1aiVbtmyJMqBcu3at/ozJvJ25du2a9O/fX6pXry5NmzaV3377LdzvL1++LB9//LG8+OKLOqcxJpQePXq03L59O9yck9i/bt26OT1WJvwez4cNGyYTJ07UcsD7gK1bt0rHjh2ldu3a+v5QDoMHD9bJrU01atTQ/833RESexaCNiOLE/PnzZcWKFTpBPCZR/umnn2zBE/Tt21cnT7548aJOeo2AbtOmTTpB+ZkzZyJsDwEdAj1UizqTJk0arS41H1mzZrX9DvsAn3/+uUyYMEFOnz4tOXLk0Amdd+7cKT169JA9e/ZE2OagQYPkxo0b+jMmgB46dGi44MkR1sF7hZIlSzpdZ+TIkfLff/9J4sSJta0b9sl8vzdv3tRAauXKlRIYGCj58+fX1//ll180AAsNDZWYWLdunSxYsEAyZcokqVOn1n388MMP5d9//9Xjli9fPn1fqArFpPEm8z1gv5yVCRHFLwZtRBQnsmTJIkuXLpUlS5bYAigESGY2y/z5gw8+0KBk0aJFkipVKgkKCtJgztEbb7yhjePxKF++fITfFy9eXKtL8RgxYoQtq1enTh2pX7++XLhwwdYxoG3btrpf2L+cOXNqBs5Zx4nXXntNqwYRWMG9e/fk0KFDLt/z+fPn9X8ERunSpXO6DjJseN3p06frc1Q/mp0TcAyQiUuUKJHMnDlTfv75ZxkzZoz+7uTJk7JmzRqJqTlz5sjChQs104hsHqpJsZ849gjo1q9fLz/++KNkzJjR9jfp06fXdQDHj4g8i0EbEcUJVA8ii4MsV65cuWwZG7APfF566SX9Hxm0cuXK6c9HjhyJsD0EWiZkqVy5deuW9O7dW7NJ2B4yWwiCsE1UXwKqHgH7h6pBOHz4cIRtoWoSChUqZFtmvgdnzCyVGei4auDv5+cnBQsWjLBN87ggw4YgFFB9mSJFCpfHxR2oIi1WrJjt2OH95M6dW4PQhg0bSrt27bTK9fr165IyZcpwf2u+F/sMHBF5BoM2IooTCIgcgywzaIoJ9GSMCqpPkblDxgtBEdrUmVWjMZE2bdoIQWJk78EMcJAtjOq4oGrWnW06g6DPzNKZIguqUC1qD8cEGTe0rUO7PkCVLKqgUZVqD4Gd/XsjIs9h0EZE8a5UqVK2n1evXq3/X7lyRTsYQIkSJVwGKq4ggMGYYminherYSZMmhauixDbNbZjVjAh0/vrrr0jboEUH2oaZ20WbtJgeF7SNO3r0qP6MzhAIRs33AGYVJtrEoRoY1bsbNmxwuV3HY4f9Q7s+VP+iKnnevHnawQP27t1rWw/t3MygLW/evNF+P0QUu/53q0dEFE9QXVe5cmVt1zZ+/HhtX3b16lXNUKFdGxrjRxca0aMjA6A6FJ0ITF26dLH11kR7MrThQu9J9JREYIJMmn1PzJhCtSYCKlTNoioT7zE60KsVxwLVlOiQgQ4aCOCgcOHCtmpdbBdt1HDM0NYPmTpzPXdg/7B9BLXZsmXT9m3m3xctWtS2nllljExdgQIFovVeiCj2MdNGRB6BqksEZ2jvdu7cOQ2cUFWHBvgxCRDu379v+xlZu4MHD9oeZo/OIUOGSJ8+fbTqFI3xkaVCAPTDDz+EGyojppDRQhsxiGp4EGcQHKEjBdrSoWoWgRSqhTFMx9SpU21VvciK9ezZUzOK/v7+GtDhubvQwQBDpeD1kK3D8cIx79Wrlw79YTLfg/meiMiz/IynaWRCREThYPaA119/XTNuy5cvt+QAu+5A9u3ll1+2Dddi3xmDiDyDmTYioliE4AaD2GJ8tVWrVnntsUXHBLwHZNkYsBFZAzNtRERERF6AmTYiIiIiL8CgjYiIiMgLMGgjIiIi8gIM2oiIiIi8AIM2IiIiIi/AoI2IiIjICzBoIyIiIvICDNqIiIiIvACDNiIiIiIvwKCNiIiISKzv/wEmU9refnk7/QAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "ax = results[\"mase\"].plot(marker=\"o\", figsize=(7, 3.5))\n", "ax.axhline(MASE_BUDGET, color=\"red\", linestyle=\"--\", label=f\"budget = {MASE_BUDGET}\")\n", "ax.set_xlabel(\"horizon (hours)\")\n", "ax.set_ylabel(\"MASE\")\n", "ax.set_title(\"Cross-validated error as a function of the forecast horizon\")\n", "ax.legend()" ] }, { "cell_type": "markdown", "id": "311fdb09", "metadata": {}, "source": [ "## 3. Sweeping the horizon and the window together\n", "\n", "The training window is the other half of the question, and the two interact: a longer horizon\n", "often needs more history to stay predictable. A nested loop would work, but the space grows\n", "quickly, so we switch to the default **TPE sampler**, which samples promising regions instead of\n", "enumerating everything.\n", "\n", "Two details worth noting:\n", "\n", "- `optuna.TrialPruned` discards configurations that make no sense or do not fit in the series,\n", " which keeps the study clean without aborting the sweep;\n", "- the number of lags of the model is thrown into the same search, since it is tied to the window." ] }, { "cell_type": "code", "execution_count": 11, "id": "d3875b31", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:51.622375Z", "iopub.status.busy": "2026-07-28T13:28:51.622128Z", "iopub.status.idle": "2026-07-28T13:28:52.254580Z", "shell.execute_reply": "2026-07-28T13:28:52.254025Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Best params: {'horizon': 1, 'window': 552, 'lags': 24}\n", "Best MASE: 0.397\n" ] } ], "source": [ "def horizon_window_objective(trial):\n", " horizon = trial.suggest_categorical(\"horizon\", HORIZONS)\n", " window = trial.suggest_int(\"window\", 24 * 3, 24 * 30, step=24)\n", " lags = trial.suggest_int(\"lags\", 24, 96, step=24)\n", "\n", " if lags >= window:\n", " raise optuna.TrialPruned(\"lags must be shorter than the training window\")\n", "\n", " model = on.Model(LinearRegressionModel(lags=lags, random_state=RANDOM_STATE))\n", " try:\n", " mean, _ = evaluate(model, horizon=horizon, window=window)\n", " except ValueError as e:\n", " raise optuna.TrialPruned(str(e))\n", " return mean\n", "\n", "\n", "joint_study = optuna.create_study(\n", " study_name=\"horizon_window_sweep\",\n", " direction=\"minimize\",\n", " sampler=optuna.samplers.TPESampler(seed=RANDOM_STATE),\n", ")\n", "joint_study.optimize(horizon_window_objective, n_trials=30)\n", "\n", "print(f\"Best params: {joint_study.best_params}\")\n", "print(f\"Best MASE: {joint_study.best_value:.3f}\")" ] }, { "cell_type": "markdown", "id": "15ffa534", "metadata": {}, "source": [ "Since the horizon dominates the error, the global best is once again the shortest horizon. Group\n", "the trials by horizon instead: each row is then the best window and lags **for that horizon**,\n", "which is what you actually deploy." ] }, { "cell_type": "code", "execution_count": 12, "id": "414a98c1", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:52.256042Z", "iopub.status.busy": "2026-07-28T13:28:52.255936Z", "iopub.status.idle": "2026-07-28T13:28:52.264021Z", "shell.execute_reply": "2026-07-28T13:28:52.263359Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
windowlagsmase
horizon
1552240.397
3288240.436
6528480.552
12264720.644
24264480.546
48720480.599
\n", "
" ], "text/plain": [ " window lags mase\n", "horizon \n", "1 552 24 0.397\n", "3 288 24 0.436\n", "6 528 48 0.552\n", "12 264 72 0.644\n", "24 264 48 0.546\n", "48 720 48 0.599" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "trials = joint_study.trials_dataframe(attrs=(\"params\", \"value\", \"state\"))\n", "trials = trials[trials[\"state\"] == \"COMPLETE\"]\n", "\n", "best_per_horizon = (\n", " trials.sort_values(\"value\")\n", " .groupby(\"params_horizon\")\n", " .first()[[\"params_window\", \"params_lags\", \"value\"]]\n", " .rename(columns={\"params_window\": \"window\", \"params_lags\": \"lags\", \"value\": \"mase\"})\n", " .round(3)\n", ")\n", "best_per_horizon.index.name = \"horizon\"\n", "best_per_horizon" ] }, { "cell_type": "markdown", "id": "8ae74b08", "metadata": {}, "source": [ "## 4. The horizon / accuracy trade-off\n", "\n", "The most honest phrasing of the problem: *forecast as far as possible, as accurately as\n", "possible*. These two goals conflict, so there is no single best horizon, there is a **Pareto\n", "front**. Optuna handles multi-objective studies natively, just return a tuple and give one\n", "direction per objective." ] }, { "cell_type": "code", "execution_count": 13, "id": "dd19ad8d", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:52.265315Z", "iopub.status.busy": "2026-07-28T13:28:52.265210Z", "iopub.status.idle": "2026-07-28T13:28:53.247014Z", "shell.execute_reply": "2026-07-28T13:28:53.246488Z" } }, "outputs": [], "source": [ "def pareto_objective(trial):\n", " horizon = trial.suggest_categorical(\"horizon\", HORIZONS)\n", " window = trial.suggest_int(\"window\", 24 * 7, 24 * 30, step=24 * 7)\n", " lags = trial.suggest_int(\"lags\", 24, 96, step=24)\n", " if lags >= window:\n", " raise optuna.TrialPruned(\"lags must be shorter than the training window\")\n", "\n", " model = on.Model(LinearRegressionModel(lags=lags, random_state=RANDOM_STATE))\n", " try:\n", " mean, _ = evaluate(model, horizon=horizon, window=window)\n", " except ValueError as e:\n", " raise optuna.TrialPruned(str(e))\n", " return mean, float(horizon) # minimize the error, maximize the horizon\n", "\n", "\n", "pareto_study = optuna.create_study(\n", " study_name=\"horizon_pareto\",\n", " directions=[\"minimize\", \"maximize\"],\n", " sampler=optuna.samplers.TPESampler(seed=RANDOM_STATE),\n", ")\n", "pareto_study.optimize(pareto_objective, n_trials=40)" ] }, { "cell_type": "code", "execution_count": 14, "id": "62c82a37", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:53.249138Z", "iopub.status.busy": "2026-07-28T13:28:53.249023Z", "iopub.status.idle": "2026-07-28T13:28:53.256701Z", "shell.execute_reply": "2026-07-28T13:28:53.256216Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
horizonmasewindowlags
010.24767224
130.38133648
2120.50850448
3240.55150448
4480.59967248
\n", "
" ], "text/plain": [ " horizon mase window lags\n", "0 1 0.247 672 24\n", "1 3 0.381 336 48\n", "2 12 0.508 504 48\n", "3 24 0.551 504 48\n", "4 48 0.599 672 48" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "front = (\n", " pd.DataFrame(\n", " [\n", " {\n", " \"horizon\": int(t.values[1]),\n", " \"mase\": round(t.values[0], 3),\n", " \"window\": t.params[\"window\"],\n", " \"lags\": t.params[\"lags\"],\n", " }\n", " for t in pareto_study.best_trials\n", " ]\n", " )\n", " .drop_duplicates()\n", " .sort_values(\"horizon\")\n", " .reset_index(drop=True)\n", ")\n", "front" ] }, { "cell_type": "markdown", "id": "fd1648d0", "metadata": {}, "source": [ "Each row is a non-dominated configuration: nothing tested forecasts further *and* more\n", "accurately. Choosing one is now a business decision rather than a statistical one." ] }, { "cell_type": "markdown", "id": "693b203f", "metadata": {}, "source": [ "## What to remember\n", "\n", "- Score a configuration with `evaluate_cross_validation`, let Optuna choose which ones to try.\n", "- Use `GridSampler` for an exhaustive sweep, the default TPE sampler for larger spaces.\n", "- Never pick a horizon by minimizing an error alone: use a budget, a fixed horizon, or a\n", " Pareto front.\n", "\n", "## Going further\n", "\n", "- **Several models at once.** Suggest the model as a categorical parameter and build it inside\n", " the objective. Optuna supports conditional spaces, so each model can have its own\n", " hyperparameters, and you get a model x horizon comparison in a single study.\n", "- **Persistence and dashboard.** Pass `storage=\"sqlite:///horizon_sweep.db\"` and\n", " `load_if_exists=True` to `optuna.create_study` to resume a sweep, then inspect it with\n", " `optuna-dashboard sqlite:///horizon_sweep.db`.\n", "- **Parallelism.** `study.optimize(..., n_jobs=-1)` runs trials in parallel, and with a shared\n", " storage several processes can feed the same study.\n", "- **Pruning.** For deep models, stop hopeless trials early with a pruner, as shown in\n", " [Optimize model hyperparameters using Optuna](1_optimize-model-hyperparameters-optuna.ipynb).\n", "- **Other validation schemes.** `evaluate_cross_validation` also accepts\n", " `strategy=\"expanding\"` or `\"blocked\"`, and a `gap` for purged validation. See the\n", " [cross-validation guide](../user_guide/1_module/1-processing/1.1-cross-validation.ipynb).\n", "- **Parameter importance.** How much does the horizon really matter compared to the rest?" ] }, { "cell_type": "code", "execution_count": 15, "id": "a3ec1eef", "metadata": { "execution": { "iopub.execute_input": "2026-07-28T13:28:53.258220Z", "iopub.status.busy": "2026-07-28T13:28:53.258116Z", "iopub.status.idle": "2026-07-28T13:28:53.396338Z", "shell.execute_reply": "2026-07-28T13:28:53.395844Z" } }, "outputs": [ { "data": { "text/plain": [ "{'window': 0.7039159264526447,\n", " 'horizon': 0.20179455288155612,\n", " 'lags': 0.09428952066579922}" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "optuna.importance.get_param_importances(joint_study)" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.0" } }, "nbformat": 4, "nbformat_minor": 5 }