Context - Common

# Import to be able to import python package from src
import sys
sys.path.insert(0, '../src')

import pandas as pd
from darts.datasets import EnergyDataset

import ontime as on

The `LightGBM` module could not be imported. To enable LightGBM support in Darts, follow the detailed instructions in the installation guide: https://github.com/unit8co/darts/blob/master/INSTALL.md
The `Prophet` module could not be imported. To enable Prophet support in Darts, follow the detailed instructions in the installation guide: https://github.com/unit8co/darts/blob/master/INSTALL.md

---

Load data

ts = EnergyDataset().load()

Process the data

df = ts.pd_dataframe()
df = df.interpolate()
cols = ['generation biomass', 'generation solar', 'generation nuclear']
df = df[cols]

ts = on.TimeSeries.from_dataframe(df)

ts_uni = ts['generation solar'].slice(pd.Timestamp('2015'), pd.Timestamp('2016'))
ts_multi = ts.slice(pd.Timestamp('2015'), pd.Timestamp('2016'))

train, test = ts_uni.split_after(pd.Timestamp('2015-09-01'))

---

Load Common Context

from ontime.context import common

Profiler

profiler = common.Profiler()

Daily Aggregation

day_mean = profiler.profile(ts_uni, profiler.Period.DAILY, profiler.Aggregation.MEAN)
day_median = profiler.profile(ts_uni, profiler.Period.DAILY, profiler.Aggregation.MEDIAN)

day_mean.plot();
day_median.plot();

Weekly Aggregation

week_mean = profiler.profile(ts_uni, profiler.Period.WEEKLY, profiler.Aggregation.MEAN)
week_median = profiler.profile(ts_uni, profiler.Period.WEEKLY, profiler.Aggregation.MEDIAN)

week_mean.plot();
week_median.plot();

Generic Predictor

model = common.GenericPredictor()

model.fit(train)

<ontime.context.common.generic_predictor.GenericPredictor at 0x2c52d7f10>

What does the future looks like ?

pred = model.predict(48)

on.plots.prediction(train[-96:], pred, test[:48])

Generic Detector

model = common.GenericDetector()

model.fit(train)

<ontime.context.common.generic_detector.GenericDetector at 0x2c53e4b50>

Does the current signal has problem ?

detected_test = model.detect(test)

on.plots.anomalies(test[:72], detected_test[:72])

What if we want to have an idea about the future problems ?

predetected = model.predetect(72)

on.plots.anomalies(test[:72], predetected[:72])

Data Quality Detector

Detect values above an absolute threshold

detector = common.DataQualityDetector(
    threshold_type='threshold',
    upper_threshold=3000
)

# Fitting on some data
detector.fit(test)

# Detecting on other
detector.detect(test[:72]).plot();

Or between two thresholds

detector = common.DataQualityDetector(
    threshold_type='threshold',
    upper_threshold=3000,
    lower_threshold=1000
)

# Fitting on some data
detector.fit(test)

# Detecting on other
detector.detect(test[:72]).plot();

Or within a statistical range

detector = common.DataQualityDetector(
    threshold_type='quantile',
    upper_threshold=0.8
)

# Fitting on some data
detector.fit(test)

# Detecting on other
detector.detect(test[:72]).plot();

Missing Data Detector

Creating data with NaNs

import numpy as np

def add_random_nans(series, n=1):
    """
    Randomly add NaN values to a pandas Series.

    Parameters:
    - series (pd.Series): The pandas Series to modify.
    - n (int): The number of NaN values to add. Default is 1.

    Returns:
    - pd.Series: The modified pandas Series with NaN values.
    """
    n = min(n, len(series))
    nan_indices = np.random.choice(series.index, size=n, replace=False)
    series[nan_indices] = np.nan
    return series

ts_w_nans = on.TimeSeries.from_series(add_random_nans(test.pd_series(), 300))

Detecting the NaNs

detector = common.MissingDataDetector()

detector.detect(ts_w_nans[:72]).plot();