Register
Login
Resources
Docs Blog Datasets Glossary Case Studies Tutorials & Webinars
Product
Data Engine LLMs Platform Enterprise
Pricing Explore
Connect to our Discord channel
Integration:  dvc git github
Wei Ji 0f405ba3dc
:whale: Migrate Dockerfile from conda to mamba and link to Pangeo BinderHub (#341)
1 year ago
38949ad6a1
:boom: Let subglacial lake integration tests use dvc tracked parq files
2 years ago
0f405ba3dc
:whale: Migrate Dockerfile from conda to mamba and link to Pangeo BinderHub (#341)
1 year ago
38949ad6a1
:boom: Let subglacial lake integration tests use dvc tracked parq files
2 years ago
b8392c5a59
:sparkles: Crossover track analysis over Subglacial Lake Conway
3 years ago
f6d14adcd4
:package: Download ATL06 files up to 20210715
2 years ago
c7fc42ac23
:seedling: Run dvc init and configure dvc remote origin
3 years ago
38949ad6a1
:boom: Let subglacial lake integration tests use dvc tracked parq files
2 years ago
1512910bf4
:robot: Add stickler style checker configured for black code style
4 years ago
f5d5713878
:package: Refresh ATL11 Zarr data up to 20210715
2 years ago
120676aaf6
:package: Download and refresh ATL11 Zarr data up to 20210421
2 years ago
0f405ba3dc
:whale: Migrate Dockerfile from conda to mamba and link to Pangeo BinderHub (#341)
1 year ago
6b5fb8344c
:tada: Initial commit
4 years ago
0f405ba3dc
:whale: Migrate Dockerfile from conda to mamba and link to Pangeo BinderHub (#341)
1 year ago
ca79a32ac2
:package: Detect active subglacial lakes up to 20201224
2 years ago
ca79a32ac2
:package: Detect active subglacial lakes up to 20201224
2 years ago
f6d14adcd4
:package: Download ATL06 files up to 20210715
2 years ago
f6d14adcd4
:package: Download ATL06 files up to 20210715
2 years ago
f5d5713878
:package: Refresh ATL11 Zarr data up to 20210715
2 years ago
f5d5713878
:package: Refresh ATL11 Zarr data up to 20210715
2 years ago
6bbd5831b2
:package: Detect active subglacial lakes up to 20201111
3 years ago
6bbd5831b2
:package: Detect active subglacial lakes up to 20201111
3 years ago
1db62e93e2
:chart_with_upwards_trend: Ice surface hrange and dhdt up to 20201224
2 years ago
1db62e93e2
:chart_with_upwards_trend: Ice surface hrange and dhdt up to 20201224
2 years ago
f716373463
:memo: Refresh Whillans IX figures on main README.md up to 20201224
2 years ago
f716373463
:memo: Refresh Whillans IX figures on main README.md up to 20201224
2 years ago
f716373463
:memo: Refresh Whillans IX figures on main README.md up to 20201224
2 years ago
f716373463
:memo: Refresh Whillans IX figures on main README.md up to 20201224
2 years ago
895c6d080f
:arrow_up: Bump llvmlite from 0.36.0 to 0.38.0 (#336)
1 year ago
d5ccdf4c29
:arrow_up: Bump gmt to 6.2.0, cuml and cuspatial from 21.06 to 21.10
2 years ago
120676aaf6
:package: Download and refresh ATL11 Zarr data up to 20210421
2 years ago
817e147ebe
:arrow_up: Bump notebook from 6.4.4 to 6.4.10 (#333)
1 year ago
d2470d8e29
⬆️ Bump jupyterlab from 3.1.0 to 3.4.2 (#337)
1 year ago
Storage Buckets
Data Pipeline
Legend
DVC Managed File
Git Managed File
Metric
Stage File
External File

README.md

You have to be logged in to leave a comment. Sign In

DeepIceDrain [poster]

Mapping and monitoring deep subglacial water activity in Antarctica using remote sensing and machine learning.

Zenodo Digital Object Identifier GitHub top language Code style: black Test DeepIceDrain package Dependabot Status License

Ice Surface Elevation trends over Antactica Active Subglacial Lake fill-drain event
ICESat-2 ATL11 rate of height change over time in Antarctica 2019-03-29 to 2020-12-24 dsm_whillans_ix_cycles_3-9.gif

DeepIceDrain Pipeline Part 1 Exploratory Data Analysis DeepIceDrain Pipeline Part 2 Subglacial Lake Analysis

Along track view of an ATL11 Ground Track Elevation time-series at Crossover Points
alongtrack_whillans_ix_1080_pt3 crossover_anomaly_whillans_ix_2019-03-29_2020-12-24

Getting started

Quickstart

Launch in Binder (Interactive jupyter lab environment in the cloud).

Binder

Alternative Pangeo BinderHub link. Requires a GitHub account and you'll have to install your own computing environment, but it runs on AWS uswest2 which allows for cloud access to ICESat-2!

Pangeo BinderHub

Usage

Once you've properly installed the deepicedrain package (see installation instructions further below), you'll have access to a wide range of tools for downloading and performing quick calculations on ICESat-2 datasets. The example below shows how to calculate ice surface elevation change on a sample ATL11 dataset between ICESat's Cycle 3 and Cycle 4.

import deepicedrain
import xarray as xr

# Loads a sample ATL11 file from the intake catalog into xarray
atl11_dataset: xr.Dataset = deepicedrain.catalog.test_data.atl11_test_case.read()

# Calculate elevation change in metres from ICESat-2 Cycle 3 to Cycle 4
delta_height: xr.DataArray = deepicedrain.calculate_delta(
      dataset=atl11_dataset, oldcyclenum=3, newcyclenum=4, variable="h_corr"
)

# Quick plot of delta_height along the ICESat-2 track
delta_height.plot()

ATL11 delta_height along ref_pt track

Installation

Basic

To just try out the scripts, download the environment.yml file from the repository and run the commands below:

cd deepicedrain
mamba env create --name deepicedrain --file environment.yml
pip install git+https://github.com/weiji14/deepicedrain.git

Intermediate

To help out with development, start by cloning this repo-url

git clone <repo-url>

Then I recommend using mamba to install the non-python binaries. A virtual environment will also be created with Python and poetry installed.

cd deepicedrain
mamba env create --file environment.yml

Activate the virtual environment first.

mamba activate deepicedrain

Then install the python libraries listed in the pyproject.toml/poetry.lock file.

poetry install

Finally, double-check that the libraries have been installed.

poetry show

Advanced

This is for those who want full reproducibility of the virtual environment, and more computing power by using Graphical Processing Units (GPU).

Making an explicit conda-lock file (only needed if creating a new virtual environment/refreshing an existing one).

mamba env create -f environment.yml
mamba list --explicit > environment-linux-64.lock

Creating/Installing a virtual environment from a conda lock file. See also https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#building-identical-conda-environments.

mamba create --name deepicedrain --file environment-linux-64.lock
mamba install --name deepicedrain --file environment-linux-64.lock

If you have a CUDA-capable GPU, you can also install the optional "cuda" packages to accelerate some calculations.

poetry install --extras cuda

Running jupyter lab

mamba activate deepicedrain
python -m ipykernel install --user --name deepicedrain  # to install virtual env properly
jupyter kernelspec list --json                          # see if kernel is installed
jupyter lab &

This work would not be possible without inspiration from the following cool open source projects! Go check them out if you have time.

Citing

The work in this repository has not been peer-reviewed, but if you do want to cite it for some reason, use the following BibLaTeX code from this conference proceedings (poster presentation):

@inproceedings{LeongSpatiotemporalvariabilityactive2021,
  title = {{Spatiotemporal Variability of Active Subglacial Lakes in Antarctica from 2018-2020 Using ICESat-2 Laser Altimetry}},
  author = {Leong, W. J. and Horgan, H. J.},
  date = {2021-02-10},
  publisher = {{Unpublished}},
  location = {{Christchurch, New Zealand}},
  doi = {10.13140/RG.2.2.27952.07680},
  eventtitle = {{New Zealand Antarctic Science Conference}}},
  langid = {english}
}

Python code for the DeepIceDrain package here on Github is also mirrored on Zenodo at https://doi.org/10.5281/zenodo.4071235.

Tip!

Press p or to see the previous file or, n or to see the next file

About

Mapping and monitoring deep subglacial water activity in Antarctica using remote sensing and machine learning, with ICESat-2!

Collaborators 1

Comments

Loading...