Q is for Q - Python A to Z

Python A-Z is a blog series about Python. Each day, I share insights, ideas and examples for different parts of Python development that match with the letter of the day. Blaugust is an annual blogging festival in August where the goal is to write a blog post every day of the month.

Second Django ORM post of the week! First, an important distinction before we start: this blog post is about Django’s Q objects and not about the task queue project Django Q (or its fork Django Q2). Maybe I’ll write about them the next time I need to come up with something for letter Q because that’s not a large pool of ideas.

Basics

Q objects are a way to write Django queries in a more composable way. These two are equivalent in outcome:

from models import Card

Card.objects.filter(name='Bulbasaur')

# and 

from django.db.models import Q

bulbasaur_query = Q(name='Bulbasaur')
Card.objects.filter(bulbasaur_query)

For a simple example like this, there’s not a lot to gain. They are more useful for complex queries. If you want to use multiple fields for lookup, Django combines them with AND operator:

# name = Bulbasaur AND set.name = Mega Evolution
Card.objects.filter(name='Bulbasaur', set__name='Mega Evolution')

With Q objects, we get more logical operators:

name = Q(name='Bulbasaur')
set_name = Q(set__name='Mega Evolution')

name & set_name # AND
name | set_name # OR
name ^ set_name # XOR
~name # NOT

Now, we can combine different queries in many ways that makes it possible to do complex queries while keeping the code more readable and maintainable.

Documentation by giving expressions names

Which brings me to another favourite of mine: documentation.

I really like to give names to implementation details. Rather than reading through code and coming to a complex implementation detail of a query, I can read what it means in the context of the program.

If you run into the following code:

Card.objects.filter(regulation_mark__in=['H', 'I', 'J'])

you’d need to know quite a lot about the details to understand what it does. Sure, you can quickly read that it filters the cards based on their field “regulation mark” being one of “H”, “I” or “J” but what does that mean?

Instead, we can extract the query to a Q object, give it a name and make it reusable across the application (and making it less error prone!):

is_standard_legal = Q(regulation_mark__in=['H', 'I', 'J'])
Card.objects.filter(is_standard_legal)

Now it’s a little bit clearer that this query finds Cards that are legal in the standard format.

This becomes even more valuable when there’s a lot of them. One use case I have for them in my Pokémon TCG toolkit is to combine different pre-determined configurations like format legality as shown above

is_standard_legal = Q(regulation_mark__in=['H', 'I', 'J'])

ace_spec = Subtype.objects.get(type='ACE SPEC')
is_restricted_to_one = Q(subtypes=ace_spec)
is_trainer = Q(supertype='Trainer')

standard_legal_single_trainers = Card.objects.filter(
  is_standard_legal & is_restricted_to_one & is_trainer
)

If you’d only run into the last line, you would have a pretty good idea of what kind of cards we are filtering for. Especially when compared to mental work you need to do to gain the same understanding from:

standard_legal_single_trainers = Card.objects.filter(
  Q(regulation_mark__in=['H', 'I', 'J']) & 
  Q(subtypes=Subtype.objects.get(type='ACE SPEC')) &
  Q(supertype='Trainer')
)

Variables are very good, not only when you need to use them multiple times or in multiple places. I always tell developers, especially the ones newer to the craft, to liberally use intermediate variables to break down their code rather than trying to do too much at once.

It also makes it way easier to debug because you can use those intermediate variables and their values to find where things went wrong.

Giving them good names also helps you understand what you’re trying to achieve and notice logical blunders when the name you give it doesn’t match the effect it has.

Dynamically crafting queries

Like regular queries, Q objects can be crafted dynamically.

# You can start with an empty query!
query = Q()

if set_name:
  query &= Q(set__name=set_name)
  
if supertype:
  query &= Q(supertype=supertype)
  
Card.objects.filter(query)

My project isn’t public but if you want to get an idea for how many search options there are for Pokémon TCG cards, take a look at pkmncards.com’s advanced search page. Q objects make creating these a joy and I can use them everywhere in the app that deals with querying so I can guarantee that my queries are consistent across the app (either all right or all wrong) and need to change them only at one place when I need to fix them or the circumstances change.