Juha-Matti Santala
Community Builder. Dreamer. Adventurer.

Lexing and parsing search queries for Pokémon TCG

This here, is a Pokémon TCG card:

Even by just looking at it, youcan see there’s a lot of information on it. If you were to search for this card on a card database, you might search with any of these different pieces of information.

First, the most obvious part would be to search for it by its name: Pikachu ex. Or just Pikachu. You might want to be specific and use the exact match with “Pikachu ex”. Already, we’re into a territory where we need to figure out how to deal with our search queries.

A bit of background

Over the last couple of years, I’ve been building a collection of tools for my private use for dealing with all sorts of fun Pokémon TCG tools: I have an API with near complete collection of English prints of cards, a proxy printer, a collection inventory, a almost-working deck builder and a few other small trinkets.

I have many reasons for why I’ve been doing this. First one being that I like building tools for my own use and having a thing I can add or change as I need is just a really nice thing to have. For collection inventory, I used to use pkmn.gg but when I realised there’s no way to export the data and I didn’t want my data to be vendor locked, I built my own version.

I used to use Pokémon TCG API but when it turned into expensive Scrydex, I was really happy I had already started building my own API for my collection and turned more focus towards making it great when they announced its closing.

This month, I’ve been working on building a proper search engine with a custom DSL (domain specific language) syntax for search queries. It’s been heavily inspired by and is compatible with the one at https://pkmncards.com/ as that’s been the card database site I’ve been using for years.

Since I’m shit scared of their lawyers, the services I build are not unfortunately publicly accessible. Eventually, I’ll make them open source sans the data and images but before that happens, I think the tooling to add the data needs to be better than it is right now for it to be of any use to anyone.

Building a query syntax engine

When I initially built a quick search functionality, I just used the query to check against card names. That worked for the need I had at the time but especially once I started building more complex tools on top, I needed something more complex.

The lovely people at PkmnCards documented their syntax really well and I chose to pick that up as a basis for mine.

To search for this Pikachu ex without using its name, I could write a query like

stage:basic hp=200 is:tera is:ex set:SSP number:57

If something above resonated with you, let's start a discussion about it! Email me at juhis@hamatti.org and share your thoughts . This year, I want to have more deeper discussions with people from around the world and I'd love if you'd be part of that.