10.2. Faster, better, stronger

Students often ask about how they can grow their skills after this semester.

There is no one path forward, but I think people learn skills best when skill-building is the byproduct of doing something fun/exciting that scratches your curiosity.

Your projects don’t have to be finance-related. (My initial python projects weren’t!) Maybe it’s about your love of some game (e.g. Sim City?), tv/movies/books, or sports. Just pick something that sounds fun.

Dashboards - A Really Fun Project Idea

Tip

One awesome thing you can do with data: Build a dashboard/”app”/webpage!

Not only are dashboards cool as hell, but building them is also a good way to show off your skills (e.g. put them on your resume). Dashboards can be really useful for real-world decision-makers in business and government settings.

How do you think this amazing page was built? Or the interactive visualizations 538, NYT, WaPo, WSJ, and others frequently publish? And look at all these freaking awesome examples!

What do you need? Mostly: Data acquisition and formatting skills - which you have!

The main hiccup: GitHub Pages can create a website out of a Jupyter notebook, but it’s just a static image of the file. People looking at the site will just see whatever it looked like at the time you last saved the file, so there is no interactivity and visitors can’t adjust inputs and rerun it. I am not a web dev, and you probably aren’t either, so below I discuss (1) free options (2) that reduce the web dev pain (3) as of late 2022.

Part 1: Make it work on your computer

Note

To make your website faster, you want it to load the smallest dataset possible and do as little data processing as required. This means you do all the preprocessing steps (everything right up until user input is needed) in a separate file that saves this as a dataset. Then, your app can start by loading this and continuing.

Also, because we want free, the hosting service you choose in Part 2 will impose limits on how much data you can load and how computation intensive your app can be. So try to imagineer your dashboard with that in mind.

The first thing is to build your dashboard/app so it works on your computer. Basically, you’ll make a notebook file and then convert it to a webpage.

I suggest looking around to see if anyone has made an app/dashboard like what you want, using a given package I list below. (Meaning: Layout choices, similar element types, interactivity.) Then emulate the code for that and tweak it to your needs.

Many packages are designed to produce full featured dashboards, apps, and widgets. My current rough pecking order:

  1. streamlit - see some examples here. Deploying the app to a website is easy as heck and free as long as traffic to the site is low.

  2. dash - highly flexible, uses plotly for visuals

  3. gradio

  4. shiny - deployment is easy, but because the package is still in development, things might change.

For simpler applications and dashboards, you can write a Jupyter notebook file, and make it so that the file itself produces what you want shown on the website. This lets you output plotly and seaborn graphs you’re familiar with. To add interactivity, choose one route below:

Part 2: Deploy it so others can find it on the web

Note

Note: If you Google, you’ll see lots of walkthroughs using “Heroku.” This service is no longer completely free.

Now, you need to “deploy” it so that it is a website people can visit and interact with. This step depends on how you built your app:

  • If shiny: shiny handles it!

  • If streamlit: Can use streamlit’s hosting (easy, and free as long as traffic to the site is low)

  • If streamlit or gradio: Walkthrough You can upload the app to huggingface.co, which probably has better performance than streamlit’s free service. After you have your app on huggingface, you can easily embed the app inside a GitHub page, which you know how to create and customize to your liking!

  • If mercury: You can also take the huggingface route, but slightly different steps.

  • If dash1:

  • If ipywidgets or ipywidgets + voila:

    • binder makes notebooks on GitHub an interactive file anyone on the web can use. You just get the binder link and put it on your person site so people can find your app.

    • PythonAnywhere

    • Any of the suggestions under dash can work too.

If you ever build one, PLEASE send it to me!


1

My previous suggestion to deploy dash was Heroku, but it is not completely free anymore. (It offers a free plan for students for 12 months.) A written walkthrough to build this page. And a good video showing different approaches based on the data needs of your site.