uk analysis dashboard

Internal dashboard for OCF to track forecast statistics, log the historical data of the forecast performance and to manage Data Platform resources.

The analysis dashboard is a tool that was developed for OCF’s internal use and continues to evolve.

Built with Streamlit, a Python-based framework made specifically for creating data apps, the dashboard tracks and displays Quartz Solar and other data model statistics, such as mean absolute error (MAE), normalized mean absolute error (nMAE) for all the client sites. The database provides the error statistic using comparing the live generation with the forecast provided. Internally it has the option of chosing the forecast horizion to check the performance with generation. The larger the error, the less accurate the forecast.

Thanks to the analysis dashboard, OCF has a valuable feedback tool for understanding the accuracy of the forecasts being provided to it's clients.

Installation

Using uv (Recommended)

This project uses uv for fast and reliable dependency management.

Prerequisites:

Install uv if you haven't already:

curl -LsSf https://astral.sh/uv/install.sh | sh

Steps:

1. Clone the repository and navigate to the project folder:

git clone https://github.com/openclimatefix/analysis-dashboard.git
cd analysis-dashboard

2. Install dependencies:

uv sync

3. Create a login secret:

echo 'password = "example"' > src/.streamlit/secrets.toml

4. Run the app:

cd src && uv run streamlit run main.py

Manual Installation (Legacy)

You can also install using pip, though uv is recommended for better performance:

In the main project folder, install from pyproject.toml:

pip install -e .

Create a login secret:

echo 'password = "example"' > src/.streamlit/secrets.toml

Run app:

cd src && streamlit run main.py

Data Platform Toolbox

The Analysis Dashboard now includes a Data Platform Toolbox, for interacting with the OCF Data Platform via gRPC.

The toolbox allows internal users to:

• Manage Organisations
  • Create, fetch, and delete organisations
• Manage Users
  • Create users and view user details
  • Add or remove users from organisations
• Manage Locations
  • Create locations
  • Fetch location details
  • List locations with filters (energy source, location type)
• Manage Policies
  • Create location policy groups
  • Add or remove policies from groups
  • Attach or detach policy groups to organisations

Database connection

To run the app locally, you'll need to connect it to the forecast development database

OCF team members can connect to the forecast development database using these Notion instructions. Add DB_URL= (db_url from notion documents) to a secrets.toml file. Follow the instructions in the Notion document to connect to the database v.

To connect to the database platform, use DATA_PLATFORM_HOST and DATA_PLATFORM_PORT.

Run app:

cd src && uv run streamlit run main.py

Using Docker Compose**

This method uses Docker Compose to set up the app and its environment automatically.

Prerequisites:

You need to have Docker and Docker Compose installed on your machine. If you don't have them, you can download them from the Docker website.

Steps:

1. Clone the repository and navigate to the project folder:

git clone https://github.com/openclimatefix/analysis-dashboard.git

cd analysis-dashboard

2. Create a .env file in the root directory and add the following environment variables:

# DB_URL=your-database-url      # Optional, if not available, you can skip this line
REGION=india                  # Choose 'india' or 'uk'
ENVIRONMENT=development       # or 'production'
password=example              # Set your password here
SHOW_PVNET_GSP_SUM=0          # Set this to 1 if you want to show pvnet_gsp_sum model

3. Create a secrets.toml file in the src/.streamlit directory and add the following line:

echo 'password = "example"' > src/.streamlit/secrets.toml

4. Build the Docker image and start the app:

docker-compose up --build

5. Open your browser and go to http://localhost:8501 to view the app.

6. To stop the app, press Ctrl+C in the terminal, and then run:

docker-compose down

Files

main.py

main.py contains functions for the home page of the app, which focuses on MAE for the OCF Quartz Solar forecast.

main_india.py

main_india.py contains functions for the home page of the app, which focuses on MAE for the OCF Quartz Energy forecast.

forecast.py

forecast.py contains functions for the forecast page. The forecast page looks at how well each of OCF's forecast models is performing compared to PVLive updated truth values.

status.py

status.py contains functionality for the status pagwe and allows the OCF team to update the forecast status in the database. This is one of the advantages of using an interface like Streamlit, facilitating status updates in a database.

auth.py

auth.py contains code for the basic authenticaion that's been put in place.

pvsite_forecast.py

pvsite_forecast.py contains the formulas and the metrics used to calculate MAE, nMAE and penalty incured against all sites.

site_toolbox.py

site_toolbox.py is a page on the dashboard that can be used to get details of any particular site that OCF provides forecast to.

plots/make_pinball_and_exceedance_plots.py

Function to make pinball and exceedance plots. This shows how good the probabilistic forecasts are doing.

plots/ramp_rate.py

Function to make ramp rate plots.

dataplatform/toolbox/

• clients.py – Creates and manages async gRPC clients for interacting with the Data Platform services.
• organisation.py – Organisation management for creating, fetching, and deleting organisations.
• users.py – Manages users and retrieving user details.
• user_organisation.py – Handles adding and removing users from organisations via the UI.
• location.py – UI for creating locations, fetching location details, and listing locations with filters.
• policy.py – UI for managing location policies, policy groups, and their organisation mappings.

🛠️ infrastructure

.github/workflows contains some CI actions.

1. docker-pipeline.yml: Creates and publishes a docker image.

With any push to main, in order to deploy changes, the Terraform Cloud variable is updated with the commit reference and deployed to AWS Elastic Beanstalk.

Environmental Variables

• DB_URL: The database url which will be queried for forecasts
• password: The password for accessing the code
• SHOW_PVNET_GSP_SUM: Option to show pvnet_gsp_sum model or not. This defaults to zero
• REGION: Option can be UK or India. This effects the default values on the NWP and Satellite pages
• ENVIRONMENT: Option can be development or production. This effects the default values on the NWP and Satellite pages

Develop

Currently this repository is only used by OCF for internal metric calculations, as it contiains client information. We hope to make it more freely useable in the near future.

Tests

To run the tests, make sure you have pytest installed

pip install pytest

and then you can run

pytest

Contributors and community

• PR's are welcome! See the Organisation Profile for details on contributing
• Find out about our other projects in the OCF Meta Repo
• Check out the OCF blog for updates
• Follow OCF on LinkedIn
• OCF templete: (https://github.com/openclimatefix/ocf-template?tab=readme-ov-file#contributing-and-community)

The following folks have contributed to this repo.

Suleman Karigar 💻	Peter Dudfield 📆	devsjc 💻	rachel tipton 💻	braddf 💻	James Fulton 💻	Aditya Sawant 💻
MAYANK SHARMA 📖	Anuj Nayak 💻	Ramkumar R 🚇

This project follows the all-contributors specification. Contributions of any kind welcome!

Part of the Open Climate Fix community.