Create an Anomaly Detection Workflow#

[1]:

# Import to be able to import python package from src
import sys
sys.path.insert(0, '../src')

[2]:

import pandas as pd
import numpy as np
import ontime as on
from IPython.display import Image

The `LightGBM` module could not be imported. To enable LightGBM support in Darts, follow the detailed instructions in the installation guide: https://github.com/unit8co/darts/blob/master/INSTALL.md
The `Prophet` module could not be imported. To enable Prophet support in Darts, follow the detailed instructions in the installation guide: https://github.com/unit8co/darts/blob/master/INSTALL.md

Prerequisite#

Install Prefect within your project

[3]:

!pip install prefect

Requirement already satisfied: prefect in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (2.19.8)
Requirement already satisfied: aiosqlite>=0.17.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.20.0)
Requirement already satisfied: alembic<2.0.0,>=1.7.5 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (1.13.2)
Requirement already satisfied: apprise<2.0.0,>=1.1.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (1.8.0)
Requirement already satisfied: asyncpg>=0.23 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.29.0)
Requirement already satisfied: click<8.2,>=8.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (8.1.7)
Requirement already satisfied: cryptography>=36.0.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (42.0.8)
Requirement already satisfied: dateparser<2.0.0,>=1.1.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (1.2.0)
Requirement already satisfied: docker>=4.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (7.1.0)
Requirement already satisfied: graphviz>=0.20.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.20.3)
Requirement already satisfied: griffe>=0.20.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.48.0)
Requirement already satisfied: jinja2<4.0.0,>=3.0.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (3.1.4)
Requirement already satisfied: jinja2-humanize-extension>=0.4.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.4.0)
Requirement already satisfied: humanize>=4.9.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (4.10.0)
Requirement already satisfied: kubernetes<30.0.0,>=24.2.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (29.0.0)
Requirement already satisfied: pytz<2025,>=2021.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (2024.1)
Requirement already satisfied: readchar<5.0.0,>=4.0.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (4.1.0)
Requirement already satisfied: sqlalchemy!=1.4.33,<3.0.0,>=1.4.22 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from sqlalchemy[asyncio]!=1.4.33,<3.0.0,>=1.4.22->prefect) (2.0.31)
Requirement already satisfied: typer!=0.12.2,<0.13.0,>=0.12.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.12.3)
Requirement already satisfied: anyio<4.0.0,>=3.7.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (3.7.1)
Requirement already satisfied: asgi-lifespan<3.0,>=1.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (2.1.0)
Requirement already satisfied: cachetools<6.0,>=5.3 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (5.4.0)
Requirement already satisfied: cloudpickle<4.0,>=2.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (3.0.0)
Requirement already satisfied: coolname<3.0.0,>=1.0.4 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (2.2.0)
Requirement already satisfied: croniter<3.0.0,>=1.0.12 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (2.0.5)
Requirement already satisfied: fsspec>=2022.5.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (2024.6.1)
Requirement already satisfied: httpcore<2.0.0,>=1.0.5 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (1.0.5)
Requirement already satisfied: httpx!=0.23.2,>=0.23 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from httpx[http2]!=0.23.2,>=0.23->prefect) (0.27.0)
Requirement already satisfied: importlib-resources<6.2.0,>=6.1.3 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (6.1.3)
Requirement already satisfied: jsonpatch<2.0,>=1.32 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (1.33)
Requirement already satisfied: jsonschema<5.0.0,>=4.0.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (4.23.0)
Requirement already satisfied: orjson<4.0,>=3.7 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (3.10.6)
Requirement already satisfied: packaging<24.3,>=21.3 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (24.1)
Requirement already satisfied: pathspec>=0.8.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.12.1)
Requirement already satisfied: pydantic!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.10.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from pydantic[email]!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.10.0->prefect) (2.8.2)
Requirement already satisfied: pydantic-core<3.0.0,>=2.12.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (2.20.1)
Requirement already satisfied: python-dateutil<3.0.0,>=2.8.2 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (2.9.0.post0)
Requirement already satisfied: python-slugify<9.0,>=5.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (8.0.4)
Requirement already satisfied: pyyaml<7.0.0,>=5.4.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (6.0.1)
Requirement already satisfied: rfc3339-validator<0.2.0,>=0.1.4 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.1.4)
Requirement already satisfied: rich<14.0,>=11.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (13.7.1)
Requirement already satisfied: ruamel.yaml>=0.17.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.18.6)
Requirement already satisfied: sniffio<2.0.0,>=1.3.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (1.3.1)
Requirement already satisfied: toml>=0.10.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.10.2)
Requirement already satisfied: typing-extensions<5.0.0,>=4.5.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (4.12.2)
Requirement already satisfied: ujson<6.0.0,>=5.8.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (5.10.0)
Requirement already satisfied: uvicorn!=0.29.0,>=0.14.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.30.1)
Requirement already satisfied: websockets<13.0,>=10.4 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (12.0)
Requirement already satisfied: itsdangerous in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (2.2.0)
Requirement already satisfied: python-multipart>=0.0.7 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (0.0.9)
Requirement already satisfied: pendulum<3.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from prefect) (2.1.2)
Requirement already satisfied: Mako in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from alembic<2.0.0,>=1.7.5->prefect) (1.3.5)
Requirement already satisfied: idna>=2.8 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->prefect) (3.7)
Requirement already satisfied: exceptiongroup in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->prefect) (1.2.1)
Requirement already satisfied: certifi in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from apprise<2.0.0,>=1.1.0->prefect) (2024.7.4)
Requirement already satisfied: requests in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from apprise<2.0.0,>=1.1.0->prefect) (2.32.3)
Requirement already satisfied: requests-oauthlib in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from apprise<2.0.0,>=1.1.0->prefect) (2.0.0)
Requirement already satisfied: markdown in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from apprise<2.0.0,>=1.1.0->prefect) (3.6)
Requirement already satisfied: async-timeout>=4.0.3 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from asyncpg>=0.23->prefect) (4.0.3)
Requirement already satisfied: cffi>=1.12 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from cryptography>=36.0.1->prefect) (1.16.0)
Requirement already satisfied: regex!=2019.02.19,!=2021.8.27 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from dateparser<2.0.0,>=1.1.1->prefect) (2024.5.15)
Requirement already satisfied: tzlocal in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from dateparser<2.0.0,>=1.1.1->prefect) (5.2)
Requirement already satisfied: urllib3>=1.26.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from docker>=4.0->prefect) (2.2.2)
Requirement already satisfied: backports-strenum>=1.3 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from griffe>=0.20.0->prefect) (1.3.1)
Requirement already satisfied: colorama>=0.4 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from griffe>=0.20.0->prefect) (0.4.6)
Requirement already satisfied: h11<0.15,>=0.13 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from httpcore<2.0.0,>=1.0.5->prefect) (0.14.0)
Requirement already satisfied: h2<5,>=3 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from httpx[http2]!=0.23.2,>=0.23->prefect) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from jinja2<4.0.0,>=3.0.0->prefect) (2.1.5)
Requirement already satisfied: jsonpointer>=1.9 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from jsonpatch<2.0,>=1.32->prefect) (3.0.0)
Requirement already satisfied: attrs>=22.2.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from jsonschema<5.0.0,>=4.0.0->prefect) (23.2.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from jsonschema<5.0.0,>=4.0.0->prefect) (2023.12.1)
Requirement already satisfied: referencing>=0.28.4 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from jsonschema<5.0.0,>=4.0.0->prefect) (0.35.1)
Requirement already satisfied: rpds-py>=0.7.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from jsonschema<5.0.0,>=4.0.0->prefect) (0.19.0)
Requirement already satisfied: six>=1.9.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from kubernetes<30.0.0,>=24.2.0->prefect) (1.16.0)
Requirement already satisfied: google-auth>=1.0.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from kubernetes<30.0.0,>=24.2.0->prefect) (2.32.0)
Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from kubernetes<30.0.0,>=24.2.0->prefect) (1.8.0)
Requirement already satisfied: oauthlib>=3.2.2 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from kubernetes<30.0.0,>=24.2.0->prefect) (3.2.2)
Requirement already satisfied: pytzdata>=2020.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from pendulum<3.0->prefect) (2020.1)
Requirement already satisfied: annotated-types>=0.4.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from pydantic!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.10.0->pydantic[email]!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.10.0->prefect) (0.7.0)
Requirement already satisfied: email-validator>=2.0.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from pydantic[email]!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.10.0->prefect) (2.2.0)
Requirement already satisfied: text-unidecode>=1.3 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from python-slugify<9.0,>=5.0->prefect) (1.3)
Requirement already satisfied: markdown-it-py>=2.2.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from rich<14.0,>=11.0->prefect) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from rich<14.0,>=11.0->prefect) (2.18.0)
Requirement already satisfied: ruamel.yaml.clib>=0.2.7 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from ruamel.yaml>=0.17.0->prefect) (0.2.8)
Requirement already satisfied: greenlet!=0.4.17 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from sqlalchemy[asyncio]!=1.4.33,<3.0.0,>=1.4.22->prefect) (3.0.3)
Requirement already satisfied: shellingham>=1.3.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from typer!=0.12.2,<0.13.0,>=0.12.0->prefect) (1.5.4)
Requirement already satisfied: pycparser in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from cffi>=1.12->cryptography>=36.0.1->prefect) (2.22)
Requirement already satisfied: dnspython>=2.0.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from email-validator>=2.0.0->pydantic[email]!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.10.0->prefect) (2.6.1)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from google-auth>=1.0.1->kubernetes<30.0.0,>=24.2.0->prefect) (0.4.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from google-auth>=1.0.1->kubernetes<30.0.0,>=24.2.0->prefect) (4.9)
Requirement already satisfied: hyperframe<7,>=6.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from h2<5,>=3->httpx[http2]!=0.23.2,>=0.23->prefect) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from h2<5,>=3->httpx[http2]!=0.23.2,>=0.23->prefect) (4.0.0)
Requirement already satisfied: mdurl~=0.1 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich<14.0,>=11.0->prefect) (0.1.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from requests->apprise<2.0.0,>=1.1.0->prefect) (3.3.2)
Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes<30.0.0,>=24.2.0->prefect) (0.6.0)

Generation of random time series#

[4]:

ts = on.generators.random_walk().generate(start=pd.Timestamp('2022-01-01'), end=pd.Timestamp('2022-12-31'))

Create a Prefect Flow and Tasks#

[5]:

from prefect import task, flow, get_run_logger
from ontime.context import common

@task
def task_detect(ts):
    det = on.detectors.quantile(low_quantile=0.1, high_quantile=0.9)
    return det.fit_detect(ts)

@task
def task_predict(ts):
    model = common.GenericPredictor()
    model.fit(ts)
    return model.predict(48)

@flow
def detection_flow(ts, log_prints=True):
    ts_ano = task_detect(ts)
    ts_pred = task_predict(ts)
    ts_pred_ano = task_detect(ts_pred)
    return ts_ano, ts_pred, ts_pred_ano

Display the graph#

[6]:

res = await detection_flow.visualize(ts)
Image(filename=f'./{detection_flow.name}.png')

/Users/fred.montet/Library/Caches/pypoetry/virtualenvs/ontime-P4y4iRhi-py3.10/lib/python3.10/site-packages/prefect/flows.py:1301: UserWarning: `flow.visualize()` will execute code inside of your flow that is not decorated with `@task` or `@flow`.
  warnings.warn(

[6]:
../_images/tutorials_0_create-an-anomaly-detection-workflow_11_1.png

Run the flow#

[7]:

ts_ano, ts_pred, ts_pred_ano = detection_flow(ts)

14:33:46.487 | INFO    | prefect.engine - Created flow run 'vengeful-corgi' for flow 'detection-flow'

14:33:46.522 | INFO    | Flow run 'vengeful-corgi' - Created task run 'task_detect-0' for task 'task_detect'

14:33:46.523 | INFO    | Flow run 'vengeful-corgi' - Executing 'task_detect-0' immediately...

14:33:46.566 | INFO    | Task run 'task_detect-0' - Finished in state Completed()

14:33:46.579 | INFO    | Flow run 'vengeful-corgi' - Created task run 'task_predict-0' for task 'task_predict'

14:33:46.580 | INFO    | Flow run 'vengeful-corgi' - Executing 'task_predict-0' immediately...

14:33:47.241 | INFO    | Task run 'task_predict-0' - Finished in state Completed()

14:33:47.252 | INFO    | Flow run 'vengeful-corgi' - Created task run 'task_detect-1' for task 'task_detect'

14:33:47.253 | INFO    | Flow run 'vengeful-corgi' - Executing 'task_detect-1' immediately...

14:33:47.283 | INFO    | Task run 'task_detect-1' - Finished in state Completed()

14:33:47.296 | INFO    | Flow run 'vengeful-corgi' - Finished in state Completed()

Show results#

[12]:

ts.plot(label='truth')
ts_ano.plot(label='anomalies')
ts_pred.plot(label='predictions')
ts_pred_ano.plot(label='predicted anomalies');
../_images/tutorials_0_create-an-anomaly-detection-workflow_15_0.png 
[ ]:
On this page
Show Source