10.2. Faster, better, stronger¶
Students often ask about how they can grow their skills after this semester.
There is no one path forward, but I think people learn skills best when skill-building is the byproduct of doing something fun/exciting that scratches your curiosity.
Your projects don’t have to be finance-related. (My initial python projects weren’t!) Maybe it’s about your love of some game (e.g. Sim City?), tv/movies/books, or sports. Just pick something that sounds fun.
A fantasy draft wizard, powered by data scraped from multiple sites and organized by ML techniques. (This repo, written in R, might be inspiring.)
Build a dashboard (see below)
Our resources page has lots of good stuff to explore
I’d also recommend meandering the links on this page, including “fecon”
Dashboards - A Really Fun Project Idea
One awesome thing you can do with data: Build a dashboard/”app”/webpage!
Not only are dashboards cool as hell, but building them is also a good way to show off your skills (e.g. put them on your resume). Dashboards can be really useful for real-world decision-makers in business and government settings.
How do you think this amazing page was built? Or the interactive visualizations 538, NYT, WaPo, WSJ, and others frequently publish? And look at all these freaking awesome examples!
What do you need? Mostly: Data acquisition and formatting skills - which you have!
The main hiccup: GitHub Pages can create a website out of a Jupyter notebook, but it’s just a static image of the file. People looking at the site will just see whatever it looked like at the time you last saved the file, so there is no interactivity and visitors can’t adjust inputs and rerun it. I am not a web dev, and you probably aren’t either, so below I discuss (1) free options (2) that reduce the web dev pain (3) as of late 2022.
Part 1: Make it work on your computer
To make your website faster, you want it to load the smallest dataset possible and do as little data processing as required. This means you do all the preprocessing steps (everything right up until user input is needed) in a separate file that saves this as a dataset. Then, your app can start by loading this and continuing.
Also, because we want free, the hosting service you choose in Part 2 will impose limits on how much data you can load and how computation intensive your app can be. So try to imagineer your dashboard with that in mind.
The first thing is to build your dashboard/app so it works on your computer. Basically, you’ll make a notebook file and then convert it to a webpage.
I suggest looking around to see if anyone has made an app/dashboard like what you want, using a given package I list below. (Meaning: Layout choices, similar element types, interactivity.) Then emulate the code for that and tweak it to your needs.
Many packages are designed to produce full featured dashboards, apps, and widgets. My current rough pecking order:
dash- highly flexible, uses
shiny- deployment is easy, but because the package is still in development, things might change.
For simpler applications and dashboards, you can write a Jupyter notebook file, and make it so that the file itself produces what you want shown on the website. This lets you output
seaborn graphs you’re familiar with. To add interactivity, choose one route below:
ipywidgets: Probably a little more coding than mercury and resulting page will have inputs above outputs, but you avoid Mercury’s opinionated formatting for the website. Walkthrough here.
Part 2: Deploy it so others can find it on the web
Note: If you Google, you’ll see lots of walkthroughs using “Heroku.” This service is no longer completely free.
Now, you need to “deploy” it so that it is a website people can visit and interact with. This step depends on how you built your app:
shiny: shiny handles it!
streamlit: Can use
streamlit’s hosting (easy, and free as long as traffic to the site is low)
gradio: Walkthrough You can upload the app to huggingface.co, which probably has better performance than streamlit’s free service. After you have your app on huggingface, you can easily embed the app inside a GitHub page, which you know how to create and customize to your liking!
If you ever build one, PLEASE send it to me!