Humane Guide to Python Errors

Content and code examples of this guide are licenced with Creative Commons CC BY-SA 4.0 licence. The creation of this original work is supported by Spice Program

Introduction

Python is often praised for being a very approachable programming language and I agree with that. However, it has one downside that's very common across different languages that can make learning programming bit harder: its error messages are cryptic.

If you have just learned about lists and loops (still being bit confused about them but being able to write simple things) and you see this:

numbers = [1,2,3,4,5]
for i, num in enumerate(numbers):
	print(f'{num} + {numbers[i+1]} = {num + numbers[i+1]}')

1 + 2 = 3
2 + 3 = 5
3 + 4 = 7
4 + 5 = 9
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
IndexError: list index out of range

you might be justifiably confused. What's a traceback? What's an IndexError? While the example code is not very pythonic, it's very common approach by a beginner and makes sense. However, I'd argue that the last three lines are not the most obvious. Luckily, they are quite well named and very googleable.

The motive of this guide is to give a more humane approach to these errors. It's supposed to work as a reference guide that you can look at when you encounter errors and see what are likely reasons for them to happen.

All the examples in the code are using Python 3.7.0.

Anatomy of an Error Traceback
AttributeError
IndexError
KeyError
ModuleNotFoundError
NameError
SyntaxError
Afterword

Anatomy of an Error Traceback

Traceback is an output that Python gives you when it encounters an error. It is built to help you as a developer to figure out what went wrong and where. Below is a typical traceback for a simple script. Let's look at how we will read it:

Traceback (most recent call last):
  File "moduleexample.py"(1), line 3(2), in <module>
    print(sum([1,2,3]))
  File "code-examples/mod/hello.py"(1), line 4(2), in sum(3)
    _sum += lst[i]
IndexError: list index out of range(4)

First of all, traceback can contain one or more calls in stack, starting from the most recent. In our example, we have a two files, one of them calling a function imported from another one and the error is in the imported function.

(1) is the name of the file where the call happens. In our case, first we have a call print(sum([1,2,3])) inside a moduleexample.py file. On the second line of the traceback, we see that inside code-examples/mod/hello.py we have an expression _sum += lst[i]. Since that is the last file in the stack, it is most propably the cause of our bug. Unfortunately sometimes the bugs seem to happen in one place but actually happen somewhere else. That's when you get to put on your detective hat.

Sometimes the errorenous line of code is in the bottom of the stack but that's not always the case. Sometimes we make a mistake upper in the stack and it just triggers somewhere down the line. That's why it's important to pay attention to the entire stack and start figuring out what happens from the top to bottom.

(2) is the line number inside the file. In our example, both files are small, just a couple of lines but when you have a massive codebase, it really helps find the right place quickly.

(3) shows you in which function the line lives in. In this case, we have a function sum inside hello.py so we can use that to get a better understanding of the situation and then use line in (2) to navigate to right place. In a basic script, the top level will always be <module>.

(4), last but not least will show you the actual error. It starts with the name of the error (which you'll find listed below in the guide), followed by a description. Python's error messages are very consistent and makes searching help from Google and Stack Overflow much easier than in some other languages.

AttributeError

AttributeError is an error that is often caused by a typo or some kind of a masking. It happens when you are referring to a non-existent attribute or property of an object. Let's look at both of these cases:

class Person:

    def __init__(self, name, age):
        self.name = name
        self.age = age

juhis = Person('Juhis', 30)
print(f'{juhis.name} is {juhis.aeg} years old')

Traceback (most recent call last):
  File "attributeerror.py", line 8, in <module>
    print(f'{juhis.name} is {juhis.aeg} years old')
AttributeError: 'Person' object has no attribute 'aeg'

The first example is a simple case of a typo. Instead of writing age, I wrote aeg, which is not an attribute of the Person object. It is one of the easiest errors to fix but not always the easiest to spot — at least I will often not see my own typos when I see what I meant to write instead of the actual text.

import string

string = 'Hello world'
print(f'Letters of the alphabet are {", ".join(string.ascii_letters)}')

Traceback (most recent call last):
  File "attributeerror.py", line 4, in <module>
    print(f'Letters of the alphabet are {", ".join(string.ascii_letters)}')
AttributeError: 'str' object has no attribute 'ascii_letters'

Our second example is bit harder to spot, especially in a larger codebase. Importing a module and then using a variable of the same name is super common. Even more common if someone else has first created a variable, function or object with one name and then you, having not read through all the code, end up importing a module of the same name, it can be difficult to figure out why there is no attribute you're sure the imported module should have.

How to solve?

1. Check for typos

Since there are many things that can cause an AttributeError, you have to play detective. Start by checking the spelling of the attribute in the last line of the error message.

2. Print contents

One of my favorite methods of debugging is to print things out. So in our second example, I would do print(string). If you imported a module and print that, you should see something along the lines of <module 'string' from .... If instead of that you see Hello world, you know that somewhere in the scope a string has been created with the same name.

3. Check that documentation matches module

Another cause of AttributeError can be misalignment of the module version and documentation version you are using. If you are using a 3^rd party library but the attributes it offers have changed from one version to another, you might see this error. If you want to see what attributes your imported module has, you can use print(dir(module_name)). In this case, making sure that you are using the correct version of the library.

4. Make sure you don't have local files with the same name

The fourth option is that you have a local file with the same name as imported third party library in your folder. Let's say you have installed documentation tool Sphinx with pip install sphinx. Now, you would import that with import sphinx and get access to the functionality it offers. Now, if you also happen to have a file called sphinx.py in your local folder, Python will actually import that instead of the third party module. In that case, you can print(sphinx.__file__) to see which file it is importing your module from.

IndexError

IndexError is one of the errors you'll see a lot in your early days (and to be honest, even in the later days) of your Python developer career. It happens when you are trying to access a non-existent index in a list.

An old saying in computer science world is

"There are only two hard things in computer science: cache invalidation, naming things and off-by-one errors."

and it's the off-by-one that we are often dealing when we see an IndexError.

fruits = ['apples', 'oranges', 'avocado', 'banana']

print(fruits[4])

Traceback (most recent call last):
  File "indexerror.py", line 3, in <module>
    print(fruits[4])
IndexError: list index out of range

First example is a situation where you try to directly access an index that is out of range. This might be due to a confusion of how indices in Python work (they start from 0 instead of 1) or because the index comes from input or as a result of some calculation and is not checked to be within the boundaries.

first_names = ['Jen', 'Maurice', 'Roy', 'Richmond']
last_names = ['Barber', 'Moss', 'Trenneman']

for i in range(len(first_names)):
    print(f'{first_names[i]} {last_names[i]}')

Traceback (most recent call last):
  File "indexerror.py", line 5, in <module>
    print(f'{first_names[i]} {last_names[i]}')
IndexError: list index out of range

Second example is another look at the same problem. An experienced Python developer reading that example will frown at how non-pythonic it is but for a beginner it's very common. You try to access the index of one list with a variable that is created by using the length of another, longer list.

# Imagine this list coming from a data file or through API
drinks = [
    ['Cola', '0.5', '2.30'],
    ['Beer', '0.4', '5.50'],
    ['Wine', '0.12', '7.80'],
    ['Whisky', '9.20']
]

NAME = 0
VOLUME = 1
PRICE = 2

for drink in drinks:
    print(f'{drink[VOLUME]} litres of {drink[NAME]} costs ${drink[PRICE]}.')

Traceback (most recent call last):
  File "indexerror.py", line 14, in <module>
    print(f'{drink[VOLUME]} litres of {drink[NAME]} costs ${drink[PRICE]}.')
IndexError: list index out of range

Our third example is using constants for giving a more meaningful name to indices for data. This is a great convention: it helps make the code more readable (imagine down the road seeing drink[1] vs. drink[VOLUME]: which one is easier to understand?) and it helps prevent typos (especially if you are working with string keys).

We expect the list of drinks to be a list of lists with each sub-list being in form of [name, volume, price]. However, sometimes that is not the case. We might have malformed data and our program crashes because it tries to access data from an index that is not present.

How to solve?

While IndexError is one of the simplest errors, finding out the cause is not always trivial. Sometimes you might have a long list of data with the error happening deep inside a loop.

1. Understanding how indexing works

Like in many other programming languages but totally not like our daily life, we start counting positions on lists from 0. For example in first_names = ['Jen', 'Maurice', 'Roy', 'Richmond'], 'Jen' is found at index 0 and accessed with first_names[0]. This means that the last element of a list will always be in index len(lst) - 1.

Compared to many other languages, we have one thing in our advantage with Python: we rarely loop over lists by index. If you have ever programmed in languages like Java, you might be familiar with a for loop like this:

for(int i = 0; i < names.length; i++)

(if you haven't, you can skip the rest of this section). It's very easy to make a mistake in the comparison (intuitively it would make sense to compare with <= instead of <). In Python though, we iterate over the items in a list:

for name in names:

If we need indexes, we use enumerate function:

for index, name in enumerate(names):

So it's much more difficult to end up in a situation inside a loop where you would be accessing something outside the range. Something similar to my first example in Introduction can still happen.

In Python, indexing also works with negative numbers, counting from the end. So index -1 will refer to the last item, -2 to the second to last, etc.

2. Print contents

The error given in traceback is not often very helpful since it doesn't give insights into which part of the line triggered the error. In our example, we have three occasions where we access list by index (drink[VOLUME], drink[NAME], and drink[PRICE]) but the error just states that we tried to access it outside it's index range.

Adding strategically placed print statements can take you far. Print the contents of your data (the list you are accessing) as well as the indices (especially if they are inside variables) and try to figure out where it breaks.

3. try-catch

Python's way of dealing with errors is using try-catch blocks. If your data is coming from sources you don't control, you probably cannot avoid IndexErrors all together. Building exception handling to deal with IndexErrors gracefully will help you build more stable programs.

Sidenote: logging is a great utility to add to your toolbelt with Python and as you can see below, it's incredibly easy to start using.

import logging

logging.basicConfig(filename="error.log", level=logging.ERROR)

# Imagine this list coming from a data file or through API
drinks = [
    ['Cola', '0.5', '2.30'],
    ['Beer', '0.4', '5.50'],
    ['Wine', '0.12', '7.80'],
    ['Whisky', '9.20']
]

NAME = 0
VOLUME = 1
PRICE = 2

for drink in drinks:
    try:
        print(f'{drink[VOLUME]} litres of {drink[NAME]} costs ${drink[PRICE]}.')
    except IndexError:
        logging.error(f'IndexError happened with {drink}, trying to access indices {NAME}, {VOLUME} or {PRICE}')

Now, instead of getting an IndexError, we get the output of

0.5 litres of Cola costs $2.30.
0.4 litres of Beer costs $5.50.
0.12 litres of Wine costs $7.80.

and error.log with contents of

ERROR:root:IndexError happened with ['Whisky', '9.20'], trying to access indices 0, 1 or 2

KeyError

KeyError is a close relative to IndexError. But instead of accessing lists with indices, we use keys to access dictionaries. We won't be getting off-by-one errors here. Let's start with the basic example to understand the error:

maurice_moss = {
    'first_name': 'Maurice',
    'last_name': 'Moss',
    'department': 'IT'
}

print(f'{maurice_moss["name"]} works at {maurice_moss["department"]}')

Traceback (most recent call last):
  File "keyerror.py", line 7, in <module>
    print(f'{maurice_moss["name"]} works at {maurice_moss["department"]}')
KeyError: 'name'

Trying to access a non-existent key in a dictionary will trigger KeyError. The error will show you the key you tried to use to access the dictionary so it's much nicer than IndexError.

Another case when KeyError might pop up is if you're using .format() to format strings:

template = '''
In {programming_language}, lists look like [1,2,3] and dictionaries look like {a: 1, b:2}
'''

print(template.format(programming_language='python'))

Traceback (most recent call last):
  File "keyerror.py", line 3, in <module>
    print(template.format(programming_language='python'))
KeyError: 'a'

In these case, .format() tries to convert everything inside of curly braces ({ }) with an argument given to it. It fails to find the key 'a' – the first key inside our dictionary.

How to solve?

1. Like IndexError

I recommend reading through the steps to approach solving IndexError because many things are similar.

2. Use defaults

Dictionaries have many ways of implementing default values. Here are a few:

Use `.get()`

counts = {
    'A': 10,
    'B': 20,
    'C': 15
}

for letter in ['A', 'B', 'C', 'D']:
    print(f'There are {counts.get(letter, 0)} {letter}s')

There are 10 As
There are 20 Bs
There are 15 Cs
There are 0 Ds

Use `collections.defaultdict`

from collections import defaultdict

counts = defaultdict(int)
counts['A'] = 10

for letter in ['A', 'B']:
    print(f'There are {counts[letter]} {letter}s')

There are 10 As
There are 0 Bs

Use `if else`

counts = {
    'A': 10,
    'B': 20,
    'C': 15
}

for letter in ['A', 'B', 'C', 'D']:
    if letter in counts:
        print(f'There are {counts[letter]} {letter}s')
    else:
        print(f'There are 0 {letter}s')

There are 10 As
There are 20 Bs
There are 15 Cs
There are 0 Ds

3. Double curly braces in format

Solving the last example, using {} inside a format is easy: you double curly brace to escape them. Notice the difference in line 2:

template = '''
In {programming_language}, lists look like [1,2,3] and dictionaries look like {{a: 1, b:2}}
'''

print(template.format(programming_language='python'))

In python, lists look like [1,2,3] and dictionaries look like {a: 1, b:2}

ModuleNotFoundError

Python has a very nice module system. You can import code from other files, modules and external libraries with import keyword.

There are a couple of ways to import things into Python. Let's look at them:

import localfile
import externalpackage

from module import func, klass

Line 1: if you have a file named localfile.py in the same folder as the file you are running, you import it with the filename (excluding the .py suffix).

Line 2: if you have installed a third party library with pip install, you can import it with its name. Refer to the documentation of the package because the name of the package to import might not be the same as the name of the package installed with pip. For example, with a popular DOM parser BeautifulSoup, you install it with pip install beautifulsoup4 but import it with import bs4.

Line 4: instead of importing the entire module, we can import only the functions, variables or classes you need by using the from syntax. If you want to import multiple things from one module, separate them with commas.

Traceback (most recent call last):
  File "imports.py", line 1, in <module>
    import localfile
ModuleNotFoundError: No module named 'localfile'

How to solve?

1. Check spelling

Similar to AttributeError above, one cause of ModuleNotFoundError is that you have misspelled the name of the module.

2. Is the library installed?

If you are importing an external library installed with pip, make sure it is actually installed. If you use pip and run pip freeze, you can see which packages are installed in your current environment.

3. Are you using same versions of python and pip?

Another possible mixup can happen if you have multiple versions of Python and pip installed. Running pip --version in terminal will give tell you which Python's pip it is using. If you install packages with pip belonging to Python 2.7 and run your program with Python 3.7, it will not find the package.

4. Are you using virtualenv or pipenv?

If you are using virtualenv or Pipenv to manage your environment, make sure you have the virtualenv activated and that the package is installed inside that.

With virtualenv, run source [name_of_your_venv]/bin/activate. Replace [name_of_your_venv] with the name of the folder you installed virtualenv in your project.

With pipenv, run pipenv shell. This will open a new sub-shell environment that will have access to your packages.

NameError

NameError refers to a situation where you are trying to access a variable or a function that is not defined in your current scope. This can happen for various reasons, one of the most typical one for beginner developer being forgetting quotes around string.

print(Hello)

Traceback (most recent call last):
  File "nameerror.py", line 1, in <module>
    print(Hello)
NameError: name 'Hello' is not defined

While our first example is a simple one, it's something I see all the time. When you are just starting programming, it's easy to make mistakes like these as you are probably not used to surrounding all your text in quotes in anywhere else.

# This example is Python 2 specific!

name = input('What is your name? ')
print 'Hey ' + name

What is your name? Juhis
Traceback (most recent call last):
  File "nameerror.py", line 7, in <module>
    name = input('What is your name? ')
  File "<string>", line 1, in <module>
NameError: name 'Juhis' is not defined

If you are using Python 2, you migh encounter a situation like this. The code looks fine but when you run it and give it an input, it crashes with NameError. And even more peculiar, it won't always crash but might do very weird outputs.

The reason why this happens is that in Python 2, input evaluates the input as code. So when I gave it Juhis as an input, it evaluated that and tried to find a variable named Juhis.

Now, let's say we have a situation like this:

# This example is Python 2 specific!

juhis = 'Juhis'

name = input('What is your name? ')
print 'Hey ' + name

What is your name? juhis
Hey Juhis

In this case, it would actually find a matching variable from scope and evaluate it as name = juhis. For this reason, even a problematic code might not always crash but almost always will cause behaviour you weren't expecting.

Please note that in Python 3, the behaviour of input() was changed so that it reads the input as a string.

When you are working with classes, you have to keep an eye on how self works. Default values for function arguments are evaluated when function is defined but self will only be available when function is being called. So trying to use an object variable as a argument default will not work:

class Indexer:

    def __init__(self):
        self.default = 0

    def start(self, starting_point=self.default):
        self.current = starting_point
        return self.current

indexer = Indexer()
print(indexer.start())

Traceback (most recent call last):
  File "nameerror.py", line 1, in <module>
    class Indexer:
  File "nameerror.py", line 6, in Indexer
    def start(self, starting_point=self.default):
NameError: name 'self' is not defined

Variables and functions in Python have a scope. In short, it means where they are visible and accessible and where not.

def add(a, b):
    total = a + b

print(total)

Traceback (most recent call last):
  File "nameerror.py", line 4, in <module>
    print(total)
NameError: name 'total' is not defined

In this case, total is not accessible in main program because it is defined inside the function and it's scope lives within that function. After the function ends, it is gone.

How to solve?

1. Check quotes

Make sure you are surrounding your strings with either single quotes (') or double quotes ("). With Python, it does not matter which one you use as long as you start and end your strings with the same type. In general, it's recommended to be consistent with your quote types.

2. Check input()

If you are using Python 2, use raw_input() function instead of input(). raw_input will parse the input as a string.

If you are running a file written by another developer, it might be a Python 3 file that uses new changed behaviour of input while running it with Python 2 will cause NameError.

3. Check your imports

If you are attempting to use a module outside your current file but have not imported it, you will get a NameError.

4. Are you running Python 2 code with Python 3?

When Python had an upgrade from 2.7 to 3.0, bunch of things were changed in the standard library. Many functions had name changes or got merged to each other. If you are running your program with Python 3 and get NameError: name 'xrange' is not defined, it is most likely the case that you are running a program written for Python 2 with Python 3.

If this happens to be the case (making sure requires bit of googling and detective work!), doing a conversion for the entire program from 2->3 might be in order. There's good documentation for that in Python's documentation.

5. Check your scope

Make sure you are not trying to access variables outside their scope. If you define something inside a function, you can only use it outside the function if you return it.

SyntaxError

While most of the other errors occur when something in the data or the usage of variables or loops causes the Python intepreter to break, SyntaxErrors are mistakes made in the language itself.

The unfortunate thing is that Python's error messages don't give you any hints on what might be wrong. Let's take a look at a couple of common examples. I have excluded examples that are simply a cause of mistyping a keyword.

Print needs parentheses in Python 3

print 'Hello'

File "printerror.py", line 1
        print 'Hello'
                    ^
    SyntaxError: Missing parentheses in call to 'print'

If you have started programming in Python 2 or are following a tutorial or running a program built for Python 2, you might face this error. Since Python 3.4, it's been the most helpful SyntaxError since it's telling you what is wrong.

When Python 3 was created, print was changed from a keyword statement into a function. This means that any Python 2 program that includes printing, will break when you try to run it with Python 3.

How to solve?

Add parenthesis around print

print('Hello')

Using non-ASCII characters in Python 2 source code needs extra notation.

With Python 2, the default character set used in Python source codes (your .py files) is ASCII. In Python 3, the source code default encoding was changed to UTF-8 which allows much more flexibility in terms of programming.

print('Héllo')

File "printerror.py", line 1
      SyntaxError: Non-ASCII character '\xc3' in file printerror.py on line 1, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

How to solve?

You can define the character encoding with a special comment string in the beginning of the file. If you are using Python 3, you will most likely not encounter this issue.

# -*- coding: latin-1 -*-

Double semicolon

name = 'Juhis';;

File "syntaxerror.py", line 1
      name = 'Juhis';;
                     ^
  SyntaxError: invalid syntax

In Python, you can use either semicolons or line breaks as a way to separate multiple statements. However, you cannot use two semicolons unless there's another statement between them.

How to solve?

Just remove extra semicolons.

Try without except

user_inputted_age = 'a'
try:
  age = int(user_inputted_age)
print(user_inputted_age)

File "syntaxerror.py", line 5
      print(user_inputted_age)
          ^
  SyntaxError: invalid syntax

This SyntaxError can be one of the sneakiest ones to spot because it does not point to try. But it points to the first line where it's trying to find except statement.

How to solve?

Make sure every try has a matching except.

user_inputted_age = 'a'
try:
  age = int(user_inputted_age)
except ValueError:
  age = -1

print(user_inputted_age)

Assignment in if

if name = 'Juhis':
  print('We set name to Juhis')

File "syntaxerror.py", line 1
      if name = 'Juhis':
              ^
  SyntaxError: invalid syntax

This can be caused by two things: either you meant to do a comparison (if name == 'Juhis':) or you tried to assign a value while doing an if check. In Python, assignments don't return values so you cannot use them inside if condition.

How to solve?

1. If you meant to do comparison, add another =.

if name == 'Juhis':
      print('It is Juhis')

2.If you meant to do assignment, split it to two lines.

name = 'Juhis'
if name:
  print('It is Juhis')

Misplaced parentheses

arr = []
arr.append(f'We can calculate things {int("1") + int("2"})')

File "syntaxerror.py", line 2
      arr.append(f'We can calculate things {int("1") + int("2"})')
                                                                ^
  SyntaxError: invalid syntax

In our example, we misplaced the closing parentheses outside the closing curly braces of interpolation. Python doesn't know what to do when you close things in wrong order.

How to solve?

Check your parentheses. I often keep a mental count of opened parenthesis while reading the code left to right and see where I make a mistake. With many code editors, like VS Code or Atom, if you put your cursor right next to a parenthesis, it will show you where its matching partner is.

arr = []
      arr.append(f'We can calculate things {int("1") + int("2")}')

Missing a closing string quote

print('Hello world)

File "syntaxerror.py", line 1
      print('Hello world)
                        ^
  SyntaxError: EOL while scanning string literal

If you are seeing an "EOL while scanning string literal", you are missing a closing quote. Either it was accidentally deleted or it's not matching the same character as you used to open up a string.

How to solve?

Make sure you end your strings with a matching quote. Single quoted strings close with a single quote and double quoted strings close with a double quote.

print('Hello world')

Afterword

Learning to interpret the error tracebacks is essential for a good Python developer. You will encounter different kind of errors when you are writing your code and the better you understand different possibilities that can go wrong and how to debug them, you'the more successful you're gonna be at writing code that works.

I hope this guide provided some helpful insights into problems that you might be facing. If it did, I'd love to hear about it! Consider sending me a toot at @hamatti@hamatti.org in Mastodon or an email at juhamattisantala [{at}] gmail [{dot}] com. (Also do that if you see a mistake I made!)

There are more error types in Python than what this guide currently goes through. I am going to update the guide with more insights and keep this guide updated with new things.

Humane Guide to Python Errors

Table of Contents

How to solve?

1. Check for typos

2. Print contents

3. Check that documentation matches module

4. Make sure you don't have local files with the same name

How to solve?

1. Understanding how indexing works

2. Print contents

3. try-catch

How to solve?

1. Like IndexError

2. Use defaults

Use .get()

Use collections.defaultdict

Use if else

3. Double curly braces in format

How to solve?

1. Check spelling

2. Is the library installed?

3. Are you using same versions of python and pip?

4. Are you using virtualenv or pipenv?

How to solve?

1. Check quotes

2. Check input()

3. Check your imports

4. Are you running Python 2 code with Python 3?

5. Check your scope

Print needs parentheses in Python 3

How to solve?

Using non-ASCII characters in Python 2 source code needs extra notation.

How to solve?

Double semicolon

How to solve?

Try without except

How to solve?

Assignment in if

How to solve?

Misplaced parentheses

How to solve?

Missing a closing string quote

How to solve?

Use `.get()`

Use `collections.defaultdict`

Use `if else`