1. MLflow Dashboard#

../_images/infrastructure.png

The concept of MLOps a set of practices that aims to deploy and maintain machine learning models in production reliably and efficiently. We adopt MLFlow, an open source platform for the machine learning lifecycle. In our using practice, it’s essential to keep on track of every experiments. MLflow Tracking provides such functionality.

1.1. Components#

The tracking server is hosted on our workstation, composed by 3 containerized services,

  1. A backend storage (database)

    We use MySQL instance.

  2. An artifact store

    We use MinIO, a high-performance, S3 compatible object storage.

  3. Mlflow server itself

1.2. Dashboards#

There are 2 frontend user-facing dashboards,

  1. MLflow dashboard

    The MLFlow Tracking dashboard allows you to visualize, search and compare runs, as well as download run artifacts or metadata for analysis in other tools.

  2. MinIO dashboard

    MinIO itself provides 2 accessing methods. One can view artifacts on dashboard, or use API endpoint for programmatic control.


    For programmatic access, one can refer to

    mbl.analysis.run_searcher

1.3. The internet#

To protect the services, an nginx load balancer is put in front of the real services. Internally, services are running in Docker virtual network, and are separated from the host network.

  1. Frontend network

  2. Backend network