Loading parrot. Har Har.

Dashboarding Extension

Preview of a full Jupyter Dashboard

Overview

During my internship at Project Jupyter, I designed a dashboarding platform that leverages JupyterLab's rich environment for data visualization and exploration. Design work was shared among engineering teams working on initiatives surrounding dashboards.

Role

User Experience Design Intern

Duration

March 2019 — June 2019

01. Background

JupyterLab as a platform for dashboard creation

JupyterLab is Project Jupyter's flagship integrated development environment where users can write, run, and present code in the same interface. JupyterLab is an environment for producing rich output, including 3D images and data visualizations. JupyterLab is great for producing and presenting data work in a linear, notebook-like format. JupyterLab is not great for presenting data work in a grid-like layout that summarizes main-points.

A Jupyter notebook containing code and data visualizations.
A Jupyter notebook containing code and data visualizations.

Unlike notebooks, Dashboards offer a single point of access for disparate data. Instead of going through various sources of data, a viewer only needs to scan the dashboard to get an overview of important points. Dashboards provide easier, faster, and more comprehensible data by taking the best of several worlds (markup, code, and visualizations) and placing them in a layout for quick consumption.

As JupyterLab already can produce rich output, it makes an ideal candidate as a platform for dashboard creation.

02. Problem

No Easy Way to Make Dashboards

JupyterLab users are forced towards other Python and R libraries to create dashboards, rather than use Jupyter notebooks already containing dashboard elements within them. This also limits dashboard creation to technical users with high programming skills, leaving out more casual users.

The goal became to provide an accessible way for all users to create dashboards using JupyterLab.

03. Research

Conversations and analyzing the domain space

I reached out to members of the open-source community via Github and Gitter to understand dashboarding use cases and expectations. I also came across a team that had previously worked on dashboards creation for Jupyter's previous IDE. Some key points from my conversations:

  • Users want top to bottom, full-width dashboards with some cells hidden and others removed.
  • There needs to be a way to save dashboard layouts and their components for sharing with viewers. This was a limitation in the previous project.
  • Users expect a low-bar of entry for dashboard creation, as well as the ability to create dynamic dashboards.
  • Dashboards typically are limited to 5–9 visualizations, otherwise they are noisy.

Competitive Analysis

Referencing products in the same domain space, I found that:

  • Most dashboarding software follows a persistent grid layout.
  • Qlik, Tableau, and Datadog use a left sidebar to drag and drop pre-built visualizations onto a "canvas" area.
  • Options to add, remove, edit, resize, rename, and modify Dashboards are standard.
Breaking down the designs of other dashboarding software.
Analyzing down the layouts and features of other dashboarding software.

Why JupyterLab Dashboards?

Empathy exercises helped me see Dashboarding through a bird's eye view.

Use Case (As described in Jupyter Dashboard READ.me)

“Alice is a Jupyter Notebook user. Alice prototypes data access, modeling, plotting, interactivity, etc. in a notebook. Now Alice needs to deliver a dynamic dashboard for non-notebook users. Today, Alice must step outside Jupyter Notebook and build a separate web application. Alice cannot directly transform her notebook into a secure, standalone dashboard application.”

Jobs to be Done

I hire JupyterLab Dashboards to:

Present complicated work in a consumable way.
Provide real-time insights into constantly changing data.
Consolidate important parts of my work into a single source of truth.

04. Solutions

File Format & Empty States

After consulting Zach Sailer (lead software engineer), I found that it would be feasible to create a new file type (.dshbrd) that could be connected to notebook widgets. This new dashboard file would contain metadata preserving dashboard layouts when saved.

On launching the dashboard app in JupyterLab, an empty state prompts the user to begin the dashboard creation process. As we are introducing a new interaction style (drag and drop) not seen anywhere else in JupyterLab, training the user in this interaction is important to prevent workflow statis.

Empty state of the JupyterLab Dashboarding software.
The final iteration of the empty state and two previous version I A/B tested for fine-tuning.

Composing a Dashboard

Users drag and drop pieces from open notebooks, or by dragging and dropping from the Table of Contents extension in the left sidebar. The Table of Contents (ToC) as a means for dashboard creation has several benefits:

  1. ToC provides easy access to all parts of a notebook. The ToC is a condensed view of an entire notebook that exists in the left sidebar.
  2. More space for your canvas. By using the sidebar, we are saving real-estate in the main canvas area that would have otherwise been taken up by a notebook tab.
  3. Reusing established conventions from other dashboarding software. Rather than retraining users who have experience creating dashboards, we let their mental model of dashboard creation work for us (left sidebar as a means).

When dragging and dropping notebook pieces into the dashboard canvas, docking and tab splitting follows phosphor.js drop zones and auto-resizing logic. The first piece fills the entire canvas, then the next fills up half (depending on where it's dropped, either horizontally or vertically), and so on.

A gif video showing a prototype of the drag and drop interaction from the Table of Contents
Prototyping the drag and drop interaction from the Table of Contents, along with phosphor docking and drop zone previews.

Some design changes to the ToC were made for purposes of dashboarding:

Diagram outlining the design changes made to Table of Contents. This includes additions of images in your Table of Contents, a dropdown menu for switching between notebooks, and hover states.
Highlighted changes made to the Table of Contents extension for the dashboarding use case.

Dashboard "Tiles"

Tiles are widgets within your dashboard that represent pieces from your notebook. I experimented with low-fidelity layouts before iterating to higher-fidelity.

Diagram outlining the design changes made to Table of Contents. This includes additions of images in your Table of Contents, a dropdown menu for switching between notebooks, and hover states.
Diagram outlining the design changes made to Table of Contents. This includes additions of images in your Table of Contents, a dropdown menu for switching between notebooks, and hover states.
Before and after explorations of tile layouts. Each tile consists of the following parts: drop-down menu, title, notebook connectivity status, source path, and tile content.

The final tile design consisted of four elements:

  1. Notebook status indicator. For a dashboard to be dynamic (updating based on real-time changes) it must be connected to a notebook. The indicator surfaces whether or not the tile is connected to the notebook (circle not filled), or needs reconnecting (circle filled).
  2. Notebook Cell. This is the main content of the tile, either markup, code, or a visualization.
  3. Dropdown Menu. For further actions on the tile, such as saving the tile as an image or making the source path of tile visible or not.
  4. Source Path. Initially toggled off, the source path is needed to identify which notebook the tile came from if the status indicator is disconnected.

Previewing, Publishing / Packaging Dashboards

Dashboards are meant to be viewed and presented. The preview button in the toolbar strips away unnecessary UI elements for a presentable view of the dashboard.

Highlighting the preview button in the right side of the toolbar.
Preview of your JupyterLab dashboard, ready to present.
The preview button strips away unnecessary UI elements from JupyterLab for a presentable view of your Dashboard, in full screen.

Dashboards are connected to Notebooks, which are connected to kernels (what runs and checks the code). There needed to be a way to "package" all of these components together. After a discussion with software engineers, I decided to go with the .zip packaging route, where a .zip file is created containing the dashboard, notebook, and layout metadata for sharing.

Bringing it all together

After 3 rounds of iteration, feedback from the open-source community, my final deliverable was a working prototype of the user flow of creating a dashboard, editing and resizing dashboard tiles, packaging a dashboard, previewing a dashboard, and launching a dashboard.

Overview of the final prototype of the Dashboard extension
Overview of all the screens of the final prototype.

06. Reflection

Sharing my Work with Developers

On finishing this project, I shared my work with engineering teams working on several initiatives, such as Voila, surrounding dashboarding and notebooks.

Takeaways

  1. It's important to check blue sky ideation with technical realities to produce feasible designs that serve user needs.
  2. Github is a great way to post and receive feedback on on-going design work.
← Status Bar Xplore