Update documentation (#133)

* Use AICc for BOCPD instead of likelihood.

* Update feature table.

* Set maximum maxlags to 500.

* More stringent failure conditions for tests.

Test failures weren't causing the test job to fail.

* Make ETS prediction intervals more flexible.

* Add more robust documentation for spark-on-k8s

* Add a basic Dockerfile.

* More info on specifying a Spark app.

* Add additional levels of organization to API docs.

* Note which models support exogenous regressors.

* Remove last 'submodules' in doc.

* More systematic exclusion of dashboard from pytest

* Improve implementation of moving average inverse.

Author: aadyotb

.dockerignore

@@ -1,7 +1,6 @@
 # package
 __pycache__
 *.egg-info
-data
 docs
 tmp
 ts_datasets

.github/workflows/tests.yml

@@ -39,15 +39,17 @@ jobs:
         PYTHON_VERSION: ${{ matrix.python-version }}
       with:
         max_attempts: 3
-        timeout_minutes: 40
+        timeout_minutes: 60
+        retry-on: error
         command: |
+          set -euxo pipefail
           # Get a comma-separated list of the directories of all python source files
           files=$(for f in $(find merlion -iname "*.py"); do echo -n ",$f"; done)
-          script="import os; print(','.join({os.path.dirname(f) for f in '$files'.split(',') if f and 'dashboard' not in f}))"
+          script="import os; print(','.join({os.path.dirname(f) for f in '$files'.split(',') if f}))"
           source_modules=$(python -c "$script")
 
           # Run tests & obtain code coverage from coverage report.
-          coverage run --source=${source_modules} -L -m pytest -v -s
+          coverage run --source=${source_modules} --omit=merlion/dashboard/* -L -m pytest -v -s
           coverage report && coverage xml -o .github/badges/coverage.xml
           COVERAGE=`coverage report | grep "TOTAL" | grep -Eo "[0-9\.]+%"`
           echo "coverage=${COVERAGE}" >> $GITHUB_OUTPUT

README.md

@@ -39,34 +39,46 @@ across multiple time series datasets.
 
 Merlion's key features are
 -  Standardized and easily extensible data loading & benchmarking for a wide range of forecasting and anomaly
-   detection datasets.
--  A library of diverse models for both anomaly detection and forecasting, unified under a shared interface.
-   Models include classic statistical methods, tree ensembles, and deep
+   detection datasets. This includes transparent support for custom datasets.
+-  A library of diverse models for anomaly detection, forecasting, and change point detection, all
+   unified under a shared interface. Models include classic statistical methods, tree ensembles, and deep
    learning approaches. Advanced users may fully configure each model as desired.
 -  Abstract `DefaultDetector` and `DefaultForecaster` models that are efficient, robustly achieve good performance,
    and provide a starting point for new users.
 -  AutoML for automated hyperaparameter tuning and model selection.
+-  Unified API for using a wide range of models to forecast with
+   [exogenous regressors](https://opensource.salesforce.com/Merlion/tutorials/forecast/3_ForecastExogenous.html).
 -  Practical, industry-inspired post-processing rules for anomaly detectors that make anomaly scores more interpretable,
    while also reducing the number of false positives.
 -  Easy-to-use ensembles that combine the outputs of multiple models to achieve more robust performance. 
 -  Flexible evaluation pipelines that simulate the live deployment & re-training of a model in production,
    and evaluate performance on both forecasting and anomaly detection.
--  Native support for visualizing model predictions.
+-  Native support for visualizing model predictions, including with a clickable visual UI.
+-  Distributed computation [backend](https://opensource.salesforce.com/Merlion/merlion.spark.html) using PySpark,
+   which can be used to serve time series applications at industrial scale.
 
 The table below provides a visual overview of how Merlion's key features compare to other libraries for time series
 anomaly detection and/or forecasting.
 
-|                     | Merlion | Prophet | Alibi Detect | Kats | statsmodels | GluonTS | RRCF | STUMPY | Greykite |pmdarima 
-:---                  | :---:     | :---:|  :---:  | :---: | :---: | :---: | :---: | :---: | :----: | :---:
-| Univariate Forecasting | ✅      | ✅      | | ✅    | ✅          | ✅       |      |      |✅        | ✅ 
-| Multivariate Forecasting | ✅ | | | ✅ | ✅ | ✅ | | | | |
-| Univariate Anomaly Detection | ✅ | ✅ | ✅ | ✅ | | | ✅ | ✅ | ✅ | ✅ | 
-| Multivariate Anomaly Detection | ✅ | | ✅ | ✅ | | | ✅ | ✅ | | | |
-| Change Point Detection | ✅ | ✅ | ✅ | ✅ | | | | | ✅ | |
-| AutoML | ✅ | | | ✅ | | | | | ✅ | | ✅ 
-| Ensembles | ✅ | | | | | | ✅  | | | | 
-| Benchmarking | ✅ | | | | | ✅ | | | | | 
-| Visualization | ✅ | ✅ | | ✅ | | | | | ✅ | ✅ | | 
+|                     | Merlion | Prophet | Alibi Detect | Kats | statsmodels | nixtla | GluonTS | RRCF | STUMPY | Greykite |pmdarima 
+:---                  | :---:     | :---:|  :---:  | :---: | :---: | :---: | :---: | :---: | :---: | :----: | :---:
+| Univariate Forecasting | ✅ | ✅| | ✅ | ✅ | ✅ | ✅ | | |✅| ✅ 
+| Multivariate Forecasting | ✅ | | | ✅ | ✅| ✅ | ✅ | | | | |
+| Univariate Anomaly Detection | ✅ | ✅ | ✅ | ✅ | | | | ✅ | ✅ | ✅ | ✅ | 
+| Multivariate Anomaly Detection | ✅ | | ✅ | ✅ | | | | ✅ | ✅ | | | |
+| AutoML | ✅ | | | ✅ | | | | | | | ✅ | | ✅ 
+| Ensembles | ✅ | | | ✅ | | | | | ✅ | | | | 
+| Benchmarking | ✅ | | | | ✅ | ✅ | | | | | 
+| Visualization | ✅ | ✅ | | ✅ | | | | | | ✅ | ✅ | | 
+
+The following features are new in Merlion 2.0:
+
+|                     | Merlion | Prophet | Alibi Detect | Kats | statsmodels | nixtla | GluonTS | RRCF | STUMPY | Greykite |pmdarima 
+:---                  | :---:     | :---:|  :---:  | :---: | :---: | :---: | :---: | :---: | :---: | :----: | :---:
+| Exogenous Regressors   | ✅ | ✅ | | | ✅ |  | | | | ✅ | ✅
+| Change Point Detection | ✅ | ✅ | ✅ | ✅ | | | | | | ✅ |
+| Clickable Visual UI    | ✅ | | | | | | | | | |
+| Distributed Backend    | ✅ | | | | | ✅ | | | | | 
 
 ## Installation
 

docker/Dockerfile

@@ -0,0 +1,14 @@
+FROM python:3.9-slim
+WORKDIR /opt/Merlion
+# Install Java
+RUN rm -rf /var/lib/apt/lists/* && \
+    apt-get clean && \
+    apt-get update && \
+    apt-get upgrade && \
+    apt-get install -y --no-install-recommends openjdk-11-jre-headless && \
+    rm -rf /var/lib/apt/lists/*
+# Install Merlion from source
+COPY *.md ./
+COPY setup.py ./
+COPY merlion merlion
+RUN pip install "./"

docker/spark-on-k8s/Dockerfile

@@ -12,6 +12,7 @@ COPY merlion merlion
 RUN pip install pyarrow "./"
 
 # Copy Merlion pyspark apps
-COPY apps /opt/spark/apps
+COPY spark_apps /opt/spark/apps
+COPY data/walmart/walmart_mini.csv .
 RUN chmod g+w /opt/spark/apps
 USER ${spark_uid}

docs/source/index.rst

@@ -11,7 +11,7 @@ Merlion is a Python library for time series intelligence. It features a unified
 point detection on both univariate and multivariate time series, along with standard
 :doc:`pre-processing <merlion.transform>` and :doc:`post-processing <merlion.post_process>` layers.
 It has several modules to improve ease-of-use,
-including :ref:`visualization <merlion.plot>`,
+including :doc:`visualization <merlion.plot>`,
 anomaly score :ref:`calibration <merlion.post_process.calibrate>` to improve interpetability,
 :doc:`AutoML <merlion.models.automl>` for hyperparameter tuning and model selection,
 and :doc:`model ensembling <merlion.models.ensemble>`.
@@ -44,14 +44,14 @@ Note the following external dependencies:
    If using ``conda``, please ``conda install -c conda-forge lightgbm``
    before installing our package. This will ensure that OpenMP is configured to work with the ``lightgbm`` package
    (one of our dependencies) in your ``conda`` environment.
-   If using Mac, please install `Homebrew <https://brew.sh/>`_ and call ``brew install libomp`` so that the
+   If using Mac, please install `Homebrew <https://brew.sh/>`__ and call ``brew install libomp`` so that the
    OpenMP libary is available for the model.
    This is relevant for the
    :py:class:`LGBMForecaster <merlion.models.forecast.trees.LGBMForecaster>`,
    which is also used as a part of the :py:class:`DefaultForecaster <merlion.models.defaults.DefaultForecaster>`.
 
 2. Some of our anomaly detection models depend on having the Java Development Kit (JDK) installed. For Ubuntu, call
-   ``sudo apt-get install openjdk-11-jdk``. For Mac OS, install `Homebrew <https://brew.sh/>`_ and call
+   ``sudo apt-get install openjdk-11-jdk``. For Mac OS, install `Homebrew <https://brew.sh/>`__ and call
    ``brew tap adoptopenjdk/openjdk && brew install --cask adoptopenjdk11``. Also ensure that ``java`` can be found
    on your ``PATH``, and that the ``JAVA_HOME`` environment variable is set.
    This is relevant for the :py:class:`RandomCutForest <merlion.models.anomaly.random_cut_forest.RandomCutForest>`

docs/source/merlion.evaluate.rst

@@ -13,27 +13,24 @@ of time series models on different tasks.
     anomaly
     forecast
 
-Submodules
-----------
-
-merlion.evaluate.base module
-----------------------------
+merlion.evaluate.base
+---------------------
 
 .. automodule:: merlion.evaluate.base
    :members:
    :undoc-members:
    :show-inheritance:
 
-merlion.evaluate.anomaly module
--------------------------------
+merlion.evaluate.anomaly
+------------------------
 
 .. automodule:: merlion.evaluate.anomaly
    :members:
    :undoc-members:
    :show-inheritance:
 
-merlion.evaluate.forecast module
---------------------------------
+merlion.evaluate.forecast
+-------------------------
 
 .. automodule:: merlion.evaluate.forecast
    :members:

docs/source/merlion.models.anomaly.change_point.rst

@@ -1,5 +1,5 @@
-merlion.models.anomaly.change\_point package
-============================================
+anomaly.change\_point
+=====================
 
 .. automodule:: merlion.models.anomaly.change_point
    :members:
@@ -9,11 +9,8 @@ merlion.models.anomaly.change\_point package
 .. autosummary::
     bocpd
 
-Submodules
-----------
-
-merlion.models.anomaly.change\_point.bocpd module
--------------------------------------------------
+anomaly.change\_point.bocpd
+---------------------------
 
 .. automodule:: merlion.models.anomaly.change_point.bocpd
    :members:

docs/source/merlion.models.anomaly.forecast_based.rst

@@ -1,5 +1,5 @@
-merlion.models.anomaly.forecast\_based package
-==============================================
+anomaly.forecast\_based
+=======================
 
 .. automodule:: merlion.models.anomaly.forecast_based
    :members:
@@ -15,59 +15,56 @@ merlion.models.anomaly.forecast\_based package
     lstm
     mses
 
-Submodules
-----------
-
-merlion.models.anomaly.forecast\_based.base module
---------------------------------------------------
+anomaly.forecast\_based.base
+----------------------------
 
 .. automodule:: merlion.models.anomaly.forecast_based.base
    :members:
    :undoc-members:
    :show-inheritance:
 
-merlion.models.anomaly.forecast\_based.arima module
----------------------------------------------------
+anomaly.forecast\_based.arima
+-----------------------------
 
 .. automodule:: merlion.models.anomaly.forecast_based.arima
    :members:
    :undoc-members:
    :show-inheritance:
 
-merlion.models.anomaly.forecast\_based.sarima module
-----------------------------------------------------
+anomaly.forecast\_based.sarima
+------------------------------
 
 .. automodule:: merlion.models.anomaly.forecast_based.sarima
    :members:
    :undoc-members:
    :show-inheritance:
 
-merlion.models.anomaly.forecast\_based.ets module
--------------------------------------------------
+anomaly.forecast\_based.ets
+---------------------------
 
 .. automodule:: merlion.models.anomaly.forecast_based.ets
    :members:
    :undoc-members:
    :show-inheritance:
 
-merlion.models.anomaly.forecast\_based.prophet module
------------------------------------------------------
+anomaly.forecast\_based.prophet
+-------------------------------
 
 .. automodule:: merlion.models.anomaly.forecast_based.prophet
    :members:
    :undoc-members:
    :show-inheritance:
 
-merlion.models.anomaly.forecast\_based.lstm module
---------------------------------------------------
+anomaly.forecast\_based.lstm
+----------------------------
 
 .. automodule:: merlion.models.anomaly.forecast_based.lstm
    :members:
    :undoc-members:
    :show-inheritance:
 
-merlion.models.anomaly.forecast\_based.mses module
---------------------------------------------------
+anomaly.forecast\_based.mses
+----------------------------
 
 .. automodule:: merlion.models.anomaly.forecast_based.mses
    :members: