TCG-AR augments a live Pokémon card game, placing animated sprites directly onto each card and showing card information
Motivation and objective
If you have ever watched a Pokémon Trading Card Game stream, you know the experience: a fixed overhead camera, a flat mat, cards being played and moved around. The commentator does their best, but unless you already follow the format closely, it is genuinely hard to know what is going on from the footage alone.What if the Pokémon on each card could just... appear? Live, on stream, as the cards are played?
That is exactly what we built.
We are a team of researchers from the University of Liege, Belgium, and we have spent more than a year building TCG-AR, a free, open-source tool that automatically detects every Pokémon card on the playing field, recognises which card it is, and overlays its animated Pokémon sprite directly onto the card in the live camera feed. Multiple camera angles at once. No chips. No special board. Just ordinary webcams and a gaming PC.
The dream that inspired us
This idea is not entirely new. Many of you may have seen the video by French content creator SuperZouloux, titled "Le reve de tous les fans de Yu-Gi-Oh!" ("The dream of every Yu-Gi-Oh! fan"), which went viral among card game communities. In it, SuperZouloux demonstrates a system that places three-dimensional monster models directly on top of Yu-Gi-Oh! cards during a live game, making the creatures appear as if they are physically standing on the table.
It is one of the most exciting things we had ever seen done with a card game, and it was a direct inspiration for TCG-AR. We wanted to build something like it, but for Pokémon TCG, and make it available to everyone for free.
Why this kind of tool is so rare
Systems like SuperZouloux's work by embedding a tiny NFC chip (the same kind used in contactless payment cards) inside every single card. A sensor mat placed under the play area reads the chips and tells the software which card is where.It works perfectly, but the cost is huge. You need a specially manufactured mat and you need to chip every card in every player's collection. Tournaments and large events can afford this kind of setup, but the average content creator or community organiser simply cannot.
We wanted the same result with equipment (almost) anyone already owns: a webcam and a computer with a graphics card.
How TCG-AR works
We built our system around three ideas working together and leveraging computer vision and AI tools.- Finding the cards. The first step is scanning the camera feed and locating every card on the table. Cards can be placed at any angle and may overlap each other, and the system handles all of this, drawing a tight outline around each one.
- Reading the card. Once it has found a card, the system checks which way it is facing.
- Recognising the card. This is where the magic happens. The system looks at the card and compares its appearance against a database of reference images, one for each of the 20,360 Pokémon TCG cards sourced from the official Pokémon TCG API. It finds the closest match and identifies the card.
The AR overlay running on a real game board. Each card is identified and its Pokémon sprite is composited live across multiple camera views
More than one camera
A streamer typically has more than one camera on the table: a top-down view and one or two side angles to show reactions or the cards in a player's hand. TCG-AR handles all of them.The system uses the top-down camera to do the hard work of recognition, then maps every result across to each additional camera angle. A one-time calibration at the start of the stream (clicking four corners of the board in each camera view) is all it takes. After that, the animated sprite appears in exactly the right position in every angle simultaneously, with no delay and no extra hardware needed.
The broadcast overlay
Beyond the AR sprites, TCG-AR includes a full broadcast overlay designed for commentary streams.
The deck-building screen lets each player select their deck before the match
Before a match, each player selects their deck from the full card database using a built-in search interface. During play, the system tracks which cards have been identified on the board, letting operators and viewers follow the game state at a glance.
The broadcast side panel shows each player's active and benched cards with HP, status, and attacks, updated live during the match
A TV-style side panel displays each player's active Pokémon, their benched cards, HP, status conditions, and available attacks, updated live as the game progresses, just like what you would see at a major tournament broadcast. The augmented feed goes directly into OBS, so the streamer's existing setup works without changes.
Does it actually work?
We tested it on real gameplay footage, with real cards under varied lighting and different board configurations. Identifying the right card from the complete database of 20,360 cards at once is challenging: the system gets this right about 85 times out of 100. But in practice, each player brings a deck of around 60 unique cards, and once those decks are selected in the app, the system only has to distinguish between those. In that mode, it identifies the correct card 96 times out of 100, and can sustain three simultaneous high-definition camera views at smooth 30 frames per second on a single consumer GPU.Try it yourself
TCG-AR is completely free and open-source. It requires a Windows PC with an NVIDIA graphics card.Watch the interface in action before diving in:
Full setup instructions are on our GitHub: https://github.com/ULiege-VIULab/tcg-ar
For a deeper technical look at how it was built, the full academic paper is available here: https://arxiv.org/abs/2607.02090
What comes next
The system has a few known limitations we are working on. Camera calibration assumes the cameras stay fixed, so moving one during a game requires recalibrating. Cards briefly hidden by a hand disappear from the overlay until they reappear in frame. And the sprites are currently two-dimensional; we would love to move to full 3D models in a future version, much like what SuperZouloux achieved.If you try TCG-AR on your own stream or at a local tournament, we would genuinely love to hear about it and see footage.
Citation
If you reference or use this work, please consider citing:About the author: Anthony Cioppa is an assistant Professor at the University of Liege, Belgium, where he co-leads the VIULab on computer vision for sports and broadcast media. TCG-AR was developed together with Antoine Verdonck, Maxim Henry, Marc Van Droogenbroeck, and Raphael La Rocca.
Disclaimer: Pokémon, all Pokémon names, and all associated imagery are trademarks of Nintendo, Game Freak, and The Pokémon Company. TCG-AR is an independent academic research project with no affiliation to or endorsement by these companies. Card images used by the system are fetched at runtime from the Pokémon TCG API (pokemontcg.io) and are not stored or redistributed by us. This project is shared for research and educational purposes only.
- Sources
