.ipynb .pdf

repository open issue suggest edit

Binder Colab

Contents

1.7. Debugging - Making code work

1.7.1. Bugs

Perhaps the most famous early case of a computer bug was reported by Grace Hopper, who was absolutely a computer science legend and pioneer, and certainly also a bad mama jama. You should definitely go down the wiki rabbit hole on Grace’s life sometime.

Computers are extremely powerful but incredibly stupid. We want to both

1. Fix bugs when they happen

2. Prevent them from happening!

This section is mostly about the former. Good coding habits and defensive coding will help prevent them, and I cover those in the next lecture.

So, to fix bugs, you need to

1. Realize that you have a bug

2. Figure out where it is

3. Make it repeatable (and you’ll understand the bug)

4. Fix it (duh) and test it (the existence of the bug should disabuse you of your coding invincibility!)

Advice that could save (or cost) you thousands: Those steps are general, and work for other things besides code, like plumbing and electrical work on your parent’s house.

1.7.1.1. Read the error codes!

Tip

This website-tutorialsteacher has a nice page listing the most common error types. If you get an error and aren’t sure what it means, refer to this link as a starting point.

Really, they tend to be informative! You can google them for more info, but even without Google, they often point directly at the issue and location.

1.7.1.2. %debug

Covered in nice detail here.

1.7.1.3. Flipping switches / divide and conquer / or: find the bug

After slaving over your computer and a piece of paper (you smartly planned out your code before you went in head first), you’ve found a clever solution to your problem. Your code is beautiful and elegant, like this:

2+2 # imagine this is a bunch of code
2+2 # imagine this is a bunch of code
2+2 # imagine this is a bunch of code
Error # somewhere in the code is an error. But in real programming you don’t know the error is here!
2+2 # imagine this is a bunch of code
2+2 # imagine this is a bunch of code

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-49b639131bdf> in <module>
      2 2+2 # imagine this is a bunch of code
      3 2+2 # imagine this is a bunch of code
----> 4 Error # somewhere in the code is an error. But in real programming you don’t know the error is here!
      5 2+2 # imagine this is a bunch of code
      6 2+2 # imagine this is a bunch of code

NameError: name 'Error' is not defined

But python had other ideas, I guess…

Despite the appearance (his computer is on fire, after all), that guy works in IT. He spends all day taking calls from people with computer problems, typically menial. It drives him and his coworker crazy. One of the true lessons of the show, a profound piece of wisdom, really, and one that is my first method of solving virtually any technical issue, comes from that coworker:

I don’t mean turning the computer off and on again. (Well, sometimes.) … But you can turn parts of your code off:

2+2 # imagine this is a bunch of code
2+2 # imagine this is a bunch of code
# 2+2 # imagine this is a bunch of code
# Error # somewhere in the code is an error. But in real programming you don’t know the error is here!
# 2+2 # imagine this is a bunch of code
# 2+2 # imagine this is a bunch of code

4

At least we know the issue isn’t in the first two lines. We can proceed and look elsewhere.

Luckily, python error statements tend to be informative enough. Above, we know the issue is in line 4. But in more sophisticated settings, where the lines above aren’t 2+2 but chunks of code, and the error isn’t simply due to syntax or namespace issues… the on/off method can be useful. Why? Because many “errors” can exist even when the code executes.

1.7.1.4. Print statements

Another ugly, possibly brutish method of debugging is: put print statements all over your code.

def silly_func_here_but_in_a_real_situation_its_complicated_and_mysterious_ooooooh():
    return 6
a=silly_func_here_but_in_a_real_situation_its_complicated_and_mysterious_ooooooh()
print(a)
a+=a # += means "add whatever is on the right to the existing value"
print(a) # see, the value changed
b = 'hey'
print (a)
print (b)
a+'hey'

6
12
12
hey

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-2ac5244cfbad> in <module>
      8 print (a)
      9 print (b)
---> 10 a+'hey'
     11 

TypeError: unsupported operand type(s) for +: 'int' and 'str'

In this example, we don’t actually need print statements to realize that a is an int type and b is a str type because the error code is so obvious.

Nevertheless…

1.7.2. Seriously… print your data and objects OFTEN!

Suppose you have a large dataset you want to explore. What can you do to look at it?

1. Print parts of it in Jupyter, along with many summary stats.

2. Output to a csv file and open in Excel.

3. Use the spyder program that came with Anaconda. Spyder has a UI that is more like Matlab or Stata, and so you can view and scroll through objects in memory a la Excel as you build your code. This can be very, very useful for developing code. (In fact, Spyder is how most of my own research code is written.) Spyder does not however, allow for Markdown.

If you have a dataset in Jupyter you want to open in Spyder, you can either

• save the object from Jupyter (via the pickle module) and open it in Spyder,

• or convert the ipynb file to a simple py file which removes the Markdown so that Spyder can run: (File menu > Export Notebook as > Export Notebook to Executable Script )

Then you’d simply execute the code in Spyder up to the point you were at, and continue.

Much of this class will require delivery of ipynb files, and I would recommended using Jupyter exclusively at the beginning. However, if you personally prefer Spyder for bigger projects later on, feel free to use Spyder until the project is ready for write up (at which point you copy the code into a ipynb file and add Markdown elements to build the report).

1.7.2.1. No, seriously, look at your data a lot!

This isn’t even a “debugging” point per se.

You know a 6 is a 6. But we will be handling increasingly large datasets, and it’s easy to make rather large changes without knowing exactly what you have done, … or not done … , if you don’t see into the object. Are you sure that object is exactly what you think it is, containing exactly what you think it does? Thus, the print statement and other ways of “glancing into” datasets are crucial. Even when you’ve all become pythonic pros, you should tend towards examining your objects “too much”.

1.7.3. Are you still stuck?

It’ll happen! We will try to build some ambitious code this semester! (Imagine trying to replicate the “simple program” we saw last class in Excel!) Coding complicated analysis is iterative and debugging can be as tough as having your IT firm audited.

So if you’ve tried the above, what can you do?

• Writing smart code will save us from getting into intractable problems. More on that next class.

• Again, see the resources tab of our website! It’s got some good pointers, along with a 15 minute rule: Once you’ve spent 15 minutes attempting to troubleshoot a problem, you must ask for help!

• Finally, clearing your head and getting a mental break might help you spot the problem:

.

1.7.4. Clear output and rerun from the start!

Code must run from beginning to end and produce the same thing every time

• Restart the kernal and clear output

• Run all cells

1.7.5. Did you follow these tips and fix your code?

1.6. Digging into Py(thon) 1.8. Errors