Public Utility Data Liberation Project

Stream data with DDA:

from dagshub.streaming import DagsHubFilesystem

fs = DagsHubFilesystem(".", repo_url="https://dagshub.com/DagsHub-Datasets/catalyst-cooperative-pudl-dataset")

fs.listdir("s3://intake.catalyst.coop")

Description:

The Public Utility Data Liberation Project (PUDL) provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.

PUDL is an open source data processing pipeline that makes US energy data easier to access and use programmatically. Hundreds of gigabytes of valuable data are published by US government agencies, but it's often difficult to work with. PUDL takes the original spreadsheets, CSV files, and databases and turns them into a unified resource. This allows users to spend more time on novel analysis and less time on data preparation.

This information allows users to explore the operating costs of individual power plants, and see how fuel costs impact the viability of different types of generation. It can highlight the competitiveness of renewable electricity in the market today. It can show how the generation mix of different utilities has evolved over time, and how the usage of individual power plants has changed as fuel prices have changed and more renewable generation has been brought online.

The data hosted on Amazon Web Services is intended to be accessed through the PUDL Intake Catalog. The catalog allows users to access the data via a uniform API for each data type (parquet, SQL), handles local caching and provides rich metadata about the data.

Contact:

The Public Utility Data Liberation Project (PUDL) provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.

PUDL is an open source data processing pipeline that makes US energy data easier to access and use programmatically. Hundreds of gigabytes of valuable data are published by US government agencies, but it's often difficult to work with. PUDL takes the original spreadsheets, CSV files, and databases and turns them into a unified resource. This allows users to spend more time on novel analysis and less time on data preparation.

This information allows users to explore the operating costs of individual power plants, and see how fuel costs impact the viability of different types of generation. It can highlight the competitiveness of renewable electricity in the market today. It can show how the generation mix of different utilities has evolved over time, and how the usage of individual power plants has changed as fuel prices have changed and more renewable generation has been brought online.

The data hosted on Amazon Web Services is intended to be accessed through the PUDL Intake Catalog. The catalog allows users to access the data via a uniform API for each data type (parquet, SQL), handles local caching and provides rich metadata about the data.

Update Frequency:

The federal agencies that publish the raw data PUDL processes release new data, monthly, quarterly and yearly. PUDL is continuously improving the data and tries to release new versions of the data monthly.

Managed By:

https://catalyst.coop/

Resources:

1. resource:
   • Description: All PUDL data outputs.
   • ARN: arn:aws:s3:::intake.catalyst.coop
   • Region: us-west-2
   • Type: S3 Bucket

Tags:

climate, climate model, energy, environmental, government records, infrastructure, open source software, electricity, energy modeling, utilities

Tutorials:

1. tutorial:

   • Title: PUDL Intake Catalog Setup
   • URL: https://catalystcoop-pudl-catalog.readthedocs.io/en/latest/
   • AuthorName: Catalyst Cooperative
   • AuthorURL: https://catalyst.coop/

2. tutorial:

   • Title: PUDL Examples
   • URL: https://github.com/catalyst-cooperative/pudl-examples
   • AuthorName: Catalyst Cooperative
   • AuthorURL: https://catalyst.coop/
   • NotebookURL: https://github.com/catalyst-cooperative/pudl-examples/blob/main/notebooks/03-pudl-parquet.ipynb