Parsing CSV Data Using Python

Introduction

I recently had another use case for writing a script to parse some data. Normally, I would use Powershell for something like this (see Updating a CSV using Powershell), but this time I wanted to challenge myself to do it in Python. I’ve used Python a little, but I’m not as strong with it as I am with Powershell.

The problem I wanted to solve was that I needed to take a pool of digital cards for my MTG Cube, sort them into their color categories, shuffle the color-grouped cards, and deal them into “booster packs” - groups of 15 cards.

Inputs / Outputs

The digital representation of my cards would come in as a CSV, which contained the name and color category information I would need.

For output, I needed to have a text file that listed one card per line, with a blank line to separate each “pack”, like so:

Blade Splicer (nph) 4
Parallax Wave (vma) 37
Ninja of the Deep Hours (mb1) 446
Murktide Regent (mh2) 52
Headless Rider (vow) 118
Rotting Rats (con) 51
Dreadhorde Arcanist (war) 125
Pyrite Spellbomb (mrd) 232
Gaea's Cradle (usg) 321
Lotus Cobra (ima) 174
Monastery Swiftspear (ktk) 118
Bonesplitter (mrd) 146
Prismatic Vista (mh1) 244
Goblin Rabblemaster (j22) 545
Reflector Mage (ogw) 157

Luminarch Aspirant (znr) 24
Anointed Peacekeeper (dmu) 2
Force of Will (2xm) 51
Remand (rav) 63
Liliana, the Last Hope (emn) 93
Ophiomancer (c13) 84
Cemetery Gatekeeper (vow) 148
Chandra, Hope's Beacon (mom) 134
Jolrael, Mwonvuli Recluse (m21) 191
Vengevine (2xm) 185
Zagoth Triome (iko) 259
Overgrown Tomb (rav) 279
Chrome Mox (mrd) 152
Dack Fayden (cns) 42
Xander's Lounge (snc) 260

Getting Started

After some searching, I found that the third-party pandas module looked like the easiest way to handle the CSV. At first I tried doing all of the manipulation using their data frame data type, but eventually found it easier to just convert it all into a dictionary, with lists for each card.

I used a dictionary so that I could sort the cards into separate groups by their color. Here’s what that looked like after struggling to get the right syntax:

import pandas

input = "RickTesting.csv"

cube = pandas.read_csv(input)

colors = ["White","Blue","Black","Red","Green"]
colors_abv = ["w","u","b","r","g"]

cards = {}

## move from pandas data frame to dictionary of lists, organized by color
for color in colors:
    cards[color] = cube[cube['Color Category'].isin([color, colors_abv[colors.index(color)]])].values.tolist()  
cards["Other"] = cube[~cube['Color Category'].isin([*colors, *colors_abv])].values.tolist()

The last line takes cards that don’t fall into the five colors and creates an “Other” group.

I heavily used a python shell to troubleshoot each step along the way. For example, at this point I was able to confirm that the data looked like I expected it to by poking around the dictionary:

screenshot

The next step of shuffling the cards in each color group was quite simple, after adding import random I added:

# shuffle
for color in cards:
    random.shuffle(cards[color])

The rest wasn’t so simple, and took the majority of a day.

Building the “packs”

I wanted to deal two cards of each color to each of 24 packs, then add any leftovers to the “Other” group of cards, and shuffle and deal that to the 24 packs. After a lot of effort, I ended up creating a new list for the “boosters” (packs), added the cards using three nested loops, and kept track of the last card so that I could add the remaining colored cards to the “Other” group. Here’s what all of that eventually looked like:

num_boosters = 24
num_cards = 15
num_each_color = 2

last = {}
boosters = []

# add colored cards to boosters
for booster in range(0, num_boosters):
    boosters.append([])
    for color in colors:
        for i in range(0, num_each_color):
            position = i + booster * num_each_color
            if (position < len(cards[color])):
                boosters[booster].append(cards[color][position])
                last[color] = position

# add rest of colored cards to other
for color in colors:
    for i in range(last[color]+1, len(cards[color])):
        cards["Other"].append(cards[color][i])
random.shuffle(cards["Other"])

# add the rest to boosters
position = 0
for booster in range(0, len(boosters)):
    for i in range(0, num_cards - len(boosters[booster])):
        boosters[booster].append(cards["Other"][position])
        position += 1

This is usually when I think “there must be a more elegant way to do this”, but try not to get lost in the rabbit hole of searching for that. Learning comes incrementally - it’s important to take small bites.

Putting it All Together

Now that I had my boosters, I just needed to output them. I was initially going to write them to a file in the script, but decided that outputting using print() might make more sense. This seemed the same as the typically preferred method in Powershell (write-output). The idea being that it gives greater control at runtime. Either way, here’s what that looked like:

for booster in boosters:
    for card in booster:
        print(f"{card[0]} ({card[4]}) {card[5]}")
    print("")

Then I was able to create a new text file of boosters by running

python3 make-boosters.py > boosters.txt

Conclusion

It was a joy, if a struggle, to dig into a language I’ve been wanting to learn more of. I feel grateful that I had a problem to solve where Python was a good solution. I look forward to using it more in the future.

You can find the entire script on my Github Gists.

Cheers!
Rick

Introduction#

Inputs / Outputs#

Getting Started#

Building the “packs”#

Putting it All Together#

Conclusion#