Autoencoder

Disclaimer: The autoencoder is a work in progress, stability and results are not guaranteed.

Ontime offers an implementation of an autoencoder made specifically for anomaly detection in periodic time series. The autoencoder is trained to compress and reconstruct signals. The reconstruction can then be used for anomaly detection. The way it detects anomalies is by evaluating the error between the reconstructed signal and the original; since it learned the “usual” shape of the signal, the error should be significantly higher in case of anomaly.

This notebook shows how to parameter and use ontime’s autoencoder through an example of training, testing and anomaly detection.

# Import to be able to import python package from src
import sys
sys.path.insert(0, '../../../../src')

from ontime.core.time_series.time_series import TimeSeries
from ontime.core.modelling.registry.autoencoder import Autoencoder
from ontime.module.processing.pytorch.sliced_dataset import SlicedDataset
import torch.utils
import matplotlib.pyplot as plt
import pandas as pd

Preparing data

The autoencoder will be able to train on any dataset with periodic data that can be put in TimeSeries format. Here we will use one of the datasets provided by the Dataset class.

from ontime.module.datasets.dataset import Dataset

data = Dataset.SunspotsDataset.load()

#subset = data.columns.tolist()[:3] # for the sake of this guide, we will use a subset of the available features
#data = data[subset]
data.pd_dataframe().plot(figsize=(20, 5))
data.pd_dataframe().head()

component	Sunspots
Month
1749-01-01	58.0
1749-02-01	62.6
1749-03-01	70.0
1749-04-01	55.7
1749-05-01	85.0

Period of the signal

Ontime does not provide any period detection tool, but here is an example using the periodicity_detection package.

import periodicity_detection as pyd

def get_period(data):
    periods = []
    for col in data.columns:
        periods.append(pyd.findfrequency(data.pd_dataframe()[col].to_numpy(), detrend=True))

    return max(periods)

print(get_period(data))

125

period = 125 # Since the period is very small, we will take a multiple of it as a base

Data format

The autoencoder uses a particular data format available as a SlicedDataset. We will use a common train/test method for this example.

from ontime.module.processing.common import train_test_split
train, test = train_test_split(data, train_split = 0.75)

tdataset = SlicedDataset(train, period)
print(tdataset)

<ontime.module.processing.pytorch.sliced_dataset.SlicedDataset object at 0x7fbc2759d2d0>

The autoencoder uses pytorch’s DataLoader during the training phase. The batch size can be adjusted as you wish.

tdata = torch.utils.data.DataLoader(
            tdataset,
            batch_size=100,
            shuffle=False)

Preparing the autoencoder

The autoencoder is made from an Encoder and a Decoder with symmetrical structures.

An autoencoder needs to be parametrized depending on the dataset it will be trained on. Namely, the number of entry dimensions is the number of features times the length of the period. The number of output dimensions is the same as entry.

Since one sample is a data slice of period length, and the error is computed by sample, we can only detect anomalies over a window of period length.

Now, the period given to the SlicedDataset can be any multiple of the actual period. Giving the autoencoder a smaller window will make the detection more precise, but it also makes detecting patterns more difficult.

As for the latent space, its size is freely determined by the user.

columns = train.columns.tolist()
input_dimensions = len(columns) * period #we give (period) samples at once, meaning the AE receives nb features * period data points at once
latent_dimensions = input_dimensions//6 # an arbitrary value, can be anything smaller that input_dimensions

The implementation is based on pytorch, which offers to run the code on GPU if available. As of now, only running on CPU is supported.

autoencoder = Autoencoder(input_dimensions, latent_dimensions)
device = 'cpu'
autoencoder = autoencoder.to(device)

print(autoencoder)

Encoder: 2 layers
                layer 0: 125 -> 72
                layer 1: 72 -> 20

Decoder: 2 layers
                layer 0: 20 -> 72
                layer 1: 72 -> 125

Alternatively, Autoencoder offers a class method that will automatically create a new Autoencoder with the correct entry dimensions. This Autoencoder’s latent space will have (entry dimensions)/4 dimensions.

autoencoder, period = Autoencoder.new_encoder_for_dataset(train, period)
print(autoencoder)

Encoder: 2 layers
                layer 0: 125 -> 78
                layer 1: 78 -> 31

Decoder: 2 layers
                layer 0: 31 -> 78
                layer 1: 78 -> 125

autoencoder

Autoencoder(
  (encoder): Encoder(
    (linear1): Linear(in_features=125, out_features=78, bias=True)
    (linear2): Linear(in_features=78, out_features=31, bias=True)
  )
  (decoder): Decoder(
    (linear1): Linear(in_features=31, out_features=78, bias=True)
    (linear2): Linear(in_features=78, out_features=125, bias=True)
  )
)

Training

Training is done by invoking the method train().

autoencoder.train(tdata, device, epochs = 100)

The method get_reconstructed() returns a reconstructed version of the signal given as parameter as a TimeSeries, with a new column containing the loss and another one with the y value (labels) if provided in the dataset.

reconstructed_dataset = autoencoder.get_reconstructed(test, period, verbose = False)

print(type(reconstructed_dataset))
reconstructed_dataset.pd_dataframe().head()

<class 'ontime.core.time_series.time_series.TimeSeries'>

component	Sunspots	loss
Month
1925-04-01	46.331925	10.793975
1925-05-01	46.186287	10.793975
1925-06-01	52.945164	10.793975
1925-07-01	75.413064	10.793975
1925-08-01	57.793451	10.793975

Evaluating reconstructed data

The reconstructed data has the exact same structure as the original dataset, except for the two additional columns “loss and “y” (if applicable). Applying a threshold on the loss function will create an anomaly detector.

Here is a visualization of the reconstructed signal.

# normalizes a Series of values to fit into df's range
def scale_to_df(series, df):
    scale = df.max().max() - df.min().min()
    return scale * ((series - min(series)) / (max(series) - min(series))) + df.min().min()

# print one column and its reconstructed version
c = test.columns.tolist()[0]
plot_df = pd.DataFrame()
plot_df[c] = test.pd_dataframe()[c]
plot_df[f'{c} reconstructed'] = reconstructed_dataset.pd_dataframe()[c]

#let's scale the loss signal to make the display easier
plot_df['loss']  = scale_to_df(reconstructed_dataset.pd_dataframe()['loss'], plot_df)

plot_df.plot(figsize=(20, 5))
plt.title(c)
plt.show()

Anomaly detection

This dataset is not labelled, so we can’t measure the performance of this autoencoder an anomaly detector. However, here is a way to turn the loss signal into an anomaly detector.

import numpy as np

threshold = 20
anomaly = np.where(reconstructed_dataset['loss'] > threshold, 150, 0)

c = test.columns.tolist()[0]
plot_df = pd.DataFrame()
plot_df[c] = test.pd_dataframe()[c]
plot_df[f'{c} reconstructed'] = reconstructed_dataset.pd_dataframe()[c]
#let's scale the loss signal to make the display easier
plot_df['loss']  = scale_to_df(reconstructed_dataset.pd_dataframe()['loss'], plot_df)
complete_end = len(plot_df.index) - len(anomaly)
plot_df['anomaly'] = np.append(anomaly,np.zeros(complete_end))
plot_df.plot(figsize=(20, 5))
plt.title(c)
plt.show()