dagshub-docs/docs/feature_guide/discovering_experiments.md

Experiments Tab

DagsHub users can easily log their experiments to their DagsHub project, using either DagsHub Tracking or MLflow Tracking, and visualize them interactively. View your experiments in a table, or compare them in various ways, such as coordinate plots, chart plots, and a simple side-by-side presentation of the logs.

What is the Experiments Tab?

The experiments tab is where you can find all your project's experiments. Each row in the table represents one experiment that was either created by Git or MLflow. A column can be a parameter, metric, or meta-data for that experiment.

What information does the experiment table hold?

 _{Experiment Table}

??? info "Experiment Table Columns"

- <u>Status of the run</u>:
    - Successful.
    - Failed.
    - Still Running.
- <u>Code</u> - Link to the Git commit:
    - Git experiment - the Git commit that produced this experiment.
    - MLflow experiment - the last Git commit before executing the experiment.
- <u>Name</u> - The name of the experiment will be generated randomly. It can be changed manually in the singel experiment view. 
- <u>Source</u> - the tool that was used to produce the experiment - [Git](../feature_guide/git_tracking.md)  or [MLflow](../integration_guide/mlflow_tracking.md).
- <u>Commit</u>:
    - Git experiment - the Git commit that produced this experiment.
    - MLflow experiment - the last Git commit before executing the experiment.
- <u>Created</u> - How long ago the experiment was created.
- <u>Lable</u> - The label of the experiment.
- <u>Blue background</u> - the experiment's parameters.
- <u>Pink background</u> - the experiment's metrics.

What are the experiments tab capabilities?

Label experiments

• What: To help you manage many different experiments, DagsHub enables you to label them as you wish. For example, you can label them based on the hypothesis that they belong to (e.g., epoch testing, model testing, parameter testing, etc.) or by their relevance. Also, there is a special label that allows you to hide experiments named "hidden". You can use labels to group experiments together, and then filter them to show only specific groups.

• How: Hover over the label column in the row of the experiment and click on the plus sign. A drop-down menu will open with labels you can choose from or create a new one by typing it and clicking on "create label".

??? illustration "Label Experiments"

<br/>
<center>
<video controls autoplay loop muted playsinline width="100%">
  <source src="../assets/label.webm" type="video/webm">
  <source src="../assets/label.mp4" type="video/mp4">
</video>
<sub>Label Experiments</sub></center>
<br/>

Filter experiments

• What: To show only the relevant experiments, you can filter each column by different parameters.

• How: Hover over the column header and click on the sign. It will enable you to choose the filter method (e.g., created column - by date, label - labels to include and exclude, etc.).

!!! info

You can restart all filters by clicking on the “Reset filters” button.

??? illustration "Filter Experiments"

<br/>
<center>
  <video controls autoplay loop muted playsinline width="100%">
    <source src="../assets/filter.webm" type="video/webm">
    <source src="../assets/filter.mp4" type="video/mp4">
  </video>
<sub>Filter Experiments</sub></center>
<br/>

Hide experiments

• What: Some experiments are not relevant anymore or didn't provide valuable insights. To keep them in the DB but not to show them in the table, you can hide them.

• How: Hover over the label column in the row of the experiment, click on the plus sign, and choose the "hidden" label. To filter the experiments labeled as hidden, make sure that you exclude them in the filtering option of the label column (default option).

??? illustration "Hide Experiments"

<br/>
<center>
  <video controls autoplay loop muted playsinline width="100%">
    <source src="../assets/hide.webm" type="video/webm">
    <source src="../assets/hide.mp4" type="video/mp4">
  </video>
<sub>Hide Experiments</sub></center>
<br/>

Add / Hide columns

• What: Presenting all the information all the time can be confusing. DagsHub enables you to choose which columns to show or hide in the table.

• How: Click the Columns button, which will open the column selection menu. There you can click a column name to select or de-select it or drag and drop to change the order of columns.

??? illustration "Add / Hide Columns"

<br/>
<center>
<video controls autoplay loop muted playsinline width="100%">
  <source src="../assets/col-slow.webm" type="video/webm">
  <source src="../assets/col-slow.mp4" type="video/mp4">
</video>
<sub>Add / Hide Columns</sub></center>
<br/>

Sort experiments

• What: DagsHub enables you to sort experiments based on single column values.

• How: Click the arrows on the relative column header. You can sort in ascending or descending order.

??? illustration "Sort Experiments"

<br/>
<center>
<video controls autoplay loop muted playsinline width="100%">
  <source src="../assets/sort.webm" type="video/webm">
  <source src="../assets/sort.mp4" type="video/mp4">
</video>
<sub>Sort Experiments</sub></center>
<br/>

Present a single experiment

• What: The single experiment view helps you understand an experiment at a deeper level. It includes a more detailed view of the experiment parameters and metrics, as well as interactive graphs of metrics over time.
• How: Click on the name of the experiment.

??? illustration "Single Experiment"

|       |                           |
| :---------: | :----------------------------------: |
| <video controls autoplay loop muted playsinline width="100%"><source src="../assets/single-experiment.webm" type="video/webm"><source src="../assets/single-experiment.mp4" type="video/mp4"></video>       | ![Single Experiment](assets/single-experiment.png) |

How to compare experiments?

Check the checkbox of 2 or more experiments in the experiment table. After checking the experiments, you'd like to compare, click the "compare" button to go to the view.

!!! info

Clicking the blue box in the table header will automatically check or uncheck all experiments.

 _{Experiment comparison view}

The experiment comparison view looks similar to the single experiment view but is geared towards showing differences in performance between the compared runs.

Comparison table

The comparison table shows the different metadata, parameters, and latest metric values in a table view. This can help show how different parameter choices affected performance, and is probably more suitable when you compare a relatively small amount of experiments.

 _{Comparison Table}

!!! info

The colors next to each commit ID represent the color that will correspond to its lines in the metric chart view.

Parallel coordinates plot

To dive deeper into the relationships between parameter choices and metrics and look at a very broad set of experiments, you can use the parallel coordinates plot.

!!! info

Only parameters and metrics that appear in all compared experiments will be available in the parallel coordinate plot view.

 _{Parallel Coordinates Plot}

In the example seen in the image above, you can see that the parameter determining the avg_val_loss metric is the learning rate parameter. Perhaps un-intuitively, you can see that the lowest and highest learning rates produce higher losses than the middle learning rate. This is a simple example of the insights that might be gained from this plot.

Metric charts view

The last section of the comparison view is a list of charts that include all metrics that exist for the compared experiments. Each chart includes lines for the relevant compared experiments.

 _{Chart View}

Each view is interactive, and you can download the graphs as png files, zoom in on relevant parts by dragging a rectangle on them (double click to zoom back out), or remove experiments from a plot by clicking the experiment name in the legend.

How to create experiments?

DagsHub supports creating experiments in multiple ways. One option is to use MLflow to log the experiments to the repository's MLflow server and view them in the experiment table. Another option is to use Git to log the experiments by saving the information to open-source format files that end with params.yml for parameters and metrics.csv for metrics, track them with Git, and push to DagsHub. The information will be parse and shown in the experiment table.