Jeff Keen

This time it's different

2022-01-17T05:00:00-06:00

Recording an album in an actual recording studio is on my bucket list. I want some songs I wrote on vinyl and to be out in the world far more than I want to be on a stage. And I think this is the year that I do that.

After nearly a two year songwriting slump I’m back in it, and have started organizing songs I have, re-working things that need re-working, and picking out the best ones to put on an album.

This exercise has been reaffirming, and I’d like to share for accountability purposes the excuses I have told myself in the decade re: why I couldn’t record an album.

I don’t have enough songs - lol
The songs aren’t good enough/ready - do it anyway (and yeah, they are)
I should play the songs out in public first - But why
I don’t have the money - I spend more money on less important things
Who would listen to it? - doesn’t matter, me?
I need to get a band together first - eh
I can probably record them well enough myself - you absolute loon

This Time It’s Different

Fun Game

2022-01-11T04:00:00-06:00

Since recording this the memory got wiped on this calculator, so this is my only remaining evidence of Fun Game

In 1997, I was one of the few people in my high school who had a TI Graph Link, which was the little doohickey that connected the calculator to a computer’s serial port, allowing you to transfer programs from the computer to the calculator. This opened up a world of possibilities, letting me download all sorts of games from the internet which I would then distribute.

My friend Matt and I somehow schemed this silly prank idea: create a game called “Fun Game” that we would transfer along with the programs the person actually wanted, and then wait.

When it was launched, the whole sequence you see in the demo version would happen (which at the time we thought was hilarious), and then finally the program would copy a single file over and over and over and over until the calculator ran out of memory.

Turns out basic math functions take several seconds to perform after fun game ate all the memory. Fun, right?

Bob Saget's Unsuccessful Music Career

2022-01-09T05:00:00-06:00

One night in 1989 while I was watching my favorite show (America’s Funniest Home Videos), my Dad explained to me that before Bob Saget was on TV, he had an unsuccessful musical career and in an effort to escape his past he had changed his name from “Bob Seger” to “Bob Saget”.

7 or 8 years later, I’m 15 and flipping through a box of records at my friend’s house when I come across a Bob Seger album. Wouldn’t ya know it, I had a super fun trivia fact about Bob Seger! So I proudly shared it with the room.

My friend’s dad laughed until he cried. At me, not with me.

And so with that in mind, I say RIP Bob Saget

Big Bend National Park

2022-01-02T11:34:16-06:00

Dear Big Bend National Park,

I recently had the pleasure of camping in your wonderful park and really enjoyed the natural beauty of the area. The Rio Grande river is exquisite and catching the sunrise in the valley with the light coming off the rock faces was absolutely breathtaking and a highlight of our trip. I found the trails to be somewhat challenging for a woman my age (but not impossible!) and my kids even behaved themselves and had a good time. Thank you for everything you do!

However, I did have some trouble locating the clock tower. Not to discount the natural offerings of the area, but seeing one of the biggest clocks in the world would have been a nice addition to our amazing vacation. I assume the site has been closed for covid?

Sincerely,
Linda B. Wilson

Ember Stereo

2021-09-04T12:00:00-05:00

Long ago I wrote an open source ember addon for New York Public Radio called ember-hifi that powered their web audio applications, like wnyc.org, wqxr.org, newsounds.org, wnycstudios.org. It was used and maintained consistently for years and years, and as ember evolved it lagged behind a bit. So I started a branch to bring it up to modern Ember standards, and really started thinking about it from the outside in, to try and lower the learning curve.

I find it tremendously useful to start with a interactive documentation site when building something like this, as the usability problems are easy to spot early. And building this one was no different. Naming things is the hardest part, and that stuff really jumps out at you when you’re trying to explain how to use it to someone else.

After all the updates I did to the project, the changes got too big to feasibly do a mega-pull request to NYPR as their apps were still running legacy version of Ember and couldn’t benefit anyhow. So I did a hard fork, renamed it and launched it.

Spotify after signing Joe Rogan

2021-08-23T06:00:00-05:00

I mean, seriously.

Amazon Prime Jump Scares

2021-06-18T01:00:00-05:00

Amazon Prime started putting surprise advertising jump scares into movies that used to be completely free with a membership. This is the last thing I watched, and Prime just dropped to the last place I’ll check for movies.

Aw man, no modern browser?

Wait for it

Texas winter storm

2021-05-01T01:00:00-05:00

This may surprise you, but electricity is a pretty clutch utility, and not having it for three days in February during record below-freezing weather is a real challenge to say the least.

I had been reading about how stressed the grid was getting during the cold snap, and at 2am when the power went out I got up and hung up blankets over the windows to try and keep in the heat in case the power didn’t come back on for a while. Sealing them with plastic like people used to do in drafty houses would’ve been better, but cling wrap wasn’t gonna do it.

Power didn’t come back on for three days. By the end of the power saga it was dipping below 40º in my house… and that is too cold.

At first, it was kinda fun. I’ve been in Austin for thirteen years and I like the heat less and less each year (not to mention each year it seems to be getting worse), so any time we get any sort of inclement weather and the locals flip out I’m usually loving it. So getting a real winter in this town? What a freakin’ dream.

Losing power for days, though? No so much a dream. At the 48 hour mark we lost our damn minds, and just at the right moment my brother’s neighbor lent us a generator which powered a tiny heater, which was just enough heat to thaw out our patience.

This crew made the best of it, for sure

A rare item in this part of the world, we thought we could help out some neighbors with an ice scraper

This beer never tasted so good as it did when it was chilled outside in texas

Macbook air with incredible battery life playing Wonder Boys from my ancient DVD collection.

Power finally came on almost exactly 72 hours after it went off, and it stayed on for daaaayys. And then it went off for hours, on for days, off for hours, on for days, off for hours, etc. What a blast.

I attended a zoom conference with my out-of-patience neighbors, our nearly useless district city council rep, and Austin Energy where the situation was managed. Austin Energy took no responsibility for it and claimed that all the power outages were due to circumstances out of their control—historic weather events, trees crowding the power lines, and the best one: three different different occassions of someone crashing into a utility pole, in our neighborhood. I find that really hard to believe.

But anyway, let’s see what happens next year come winter.

The story of the thong song

2021-02-23T00:00:00-06:00

This was surprisingly fascinating. This song’s roots are… Eleanor Rigby??

The only spin doctors shorts in existence?

2021-01-20T00:00:00-06:00

Spin Doctors shorts, designer unknown. Birkies and Shaka to complete the look.

A friend gave these shorts to me in 2002 or so, and I have no idea where he got them from. Despite their appearance, they’re not made for water activities and I never wear them because they’re so heinous. But I’ve kept them all these years because they they feel like a precious artifact from another time and the mystery of how they got here is intriguing.

Who green-lit this project? Did the designer intentionally make the pockets comically tiny as a tip of their hat to the album we all know? When I came down the stairs in these shorts was my girlfriend laughing at me or with me?

Crest Charcoal Toothpaste

2020-01-21T11:34:16-06:00

Dear Crest,

I recently saw a video on youtube about whitening your teeth using charcoal, and your charcoal toothpaste was one of the recommended brands.

My son has very sensitive teeth, and the Kingsford charcoal “briquettes” I have been using to clean his teeth have not been working as I had expected. He complains of the taste, and his teeth seem to be blacker than they were before. Does your charcoal toothpaste suffer from the same problem?

I am open to switching brands, but we need to get through this bag first. My husband is having a grill out this weekend, so we might be able to switch soon.

Linda

How to Correct 32,000 Incorrect CSV Files in Fewer Than 32,000 Steps

2019-08-07T00:00:00-05:00

Quick, find the extra comma!

CSV feels like the simplest of file formats, where it might seem that there’s not much to know after mentally expanding the acronym - Comma Separated Values. But tell me this: if it’s so simple, then why are there so many CSV parsing libraries, alternative CSV parsing libraries, and CSV parsing libraries that claim to be better or smarter, and a mountain of mangled CSVs in existence?

CSV isn’t so much a file format as it is a loose set of guidelines for converting tabular data into text. The closest thing to a spec for it is this, which deals with vital and often overlooked questions such as:

“What happens if a value has a comma in it?” - oh, you quote it
“What happens if a value has a quote in it?” - oh, you put another quote before it

One question the spec definitely does not cover is one I needed answering: “What do you do with 32,000 files claiming to be valid CSVs but of the 750,000 some lines an unknown number of them have extra unquoted commas hidden in the values, basically making the data untrustworthy?” This is not such a simple problem, but it’s an interesting problem.

Background

I’ve been building a platform for public and community radio stations the last year and have been working with the fantastic 91.7 KOOP in Austin, TX to pilot it. Part of this project involved exporting the data from their old system (some 750,000 tracks) into the system I built for them. And noticing some tracks from an artist named “10” doing a song called “000 maniacs”, I realized the old system’s crusty CSV exporter did not do the right thing when it came to commas in values, leaving me with a real mess.

How many track titles, artists, albums, or record labels have commas in their names? Many.

I initially thought this was an impossible problem to solve (at least in a way that would not drive a person insane) so I tried reaching out to the developer of the old system to fix their ancient CSV exporter, or to send me data in another format that I could sort through myself. This felt like the most straightforward option and didn’t seem like too much of a lift. But after weeks and weeks of back and forth, this path dead ended leaving me with no other option but to fix it myself.

The Problem

Here’s an abbreviated and simplified example of a messed up CSV I was dealing with:

artist	title	album	label
Lester Sterling	Lynn Taitt & The Jets	Check Point Charlie	Merritone Rock Steady 3: Bang Bang Rock Steady 1966-1968
Lester Sterling	Lester Sterling Special	Merritone Rock Steady 2: This Music Got Soul 1966-1967	Dub Store

The way a CSV parser would parse that data currently is like this:

artist	title	album	label
Lester Sterling	Lynn Taitt & The Jets	Check Point Charlie	Merritone Rock Steady 3: Bang Bang Rock Steady 1966-1968
Lester Sterling	Lester Sterling Special	Merritone Rock Steady 2: This Music Got Soul 1966-1967	Dub Store

This is incorrect, and would only be parsed correctly if the artist value were properly quoted, like so:

1
2
3
artist,title,album,label
"Lester Sterling, Lynn Taitt & The Jets",Check Point Charlie,Merritone Rock Steady 3: Bang Bang Rock Steady 1966–1968,Dub Store,
Lester Sterling,Lester Sterling Special,Merritone Rock Steady 2: This Music Got Soul 1966–1967,Dub Store,

Which would be parsed like this:

artist	title	album	label
Lester Sterling, Lynn Taitt & The Jets	Check Point Charlie	Merritone Rock Steady 3: Bang Bang Rock Steady 1966-1968	Dub Store
Lester Sterling	Lester Sterling Special	Merritone Rock Steady 2: This Music Got Soul 1966-1967	Dub Store

But how do we get there without manually looking at 750,000 lines of comma separated text?

Computers, man

First, we can determine if a line is incorrect by parsing it individually as a CSV, seeing how many values we end up with, and comparing that number with the number of headers.

1
2
3
4
5
6
lines = File.read('/path/to/file.csv').lines
header = CSV.parse(lines[0], liberal_parsing: true)

incorrect_lines = lines[1..-1].select do |line|
  CSV.parse(line, liberal_parsing: true).values.size != header.values.size
end

1
2
3
4
5
6
7
8
9
10
11
12
[
  [
    "artist", "title", "album", "label"
  ],
  [
    "Lester Sterling", " Lynn Taitt & The Jets", "Check Point Charlie", "Merritone Rock Steady 3: Bang Bang Rock Steady 1966-1968" , "Dub Store"
  ],
  [
    "Lester Sterling", "Lester Sterling Special", "Merritone Rock Steady 2: This Music Got Soul 1966-1967", "Dub Store"
  ]
]

For this example, the header row has 4 values, the first line has 5 values, and the second line has 4 values. The first line is incorrect, meaning there’s an extra comma that belongs within a value.

Knowing that, we need to figure out all the different possibilities of what it could be. One of those possibilities will be the correct one, and the rest will be wrong. With five values and four fields, we need to join two values together into one (and quote it), making four values.

I think best on paper, so I started drawing out how this might work in a brute force sort of way, to try to get a handle on the algorithm I needed.

Eliminate 1 position, Fit 5 into 4

[1, 2, 3, 4, 5] -> [x, x, x, x] =>

[[1, 2], 3, 4, 5]

[1, [2, 3], 4, 5]

[1, 2, [3, 4], 5]

[1, 2, 3, [4, 5]]

What if we had two commas in there? Then we’d have six values needing to fit into four slots.

Eliminate 2 positions, Fit 6 into 4

[1, 2, 3, 4, 5, 6] -> [x, x, x, x] =>

[[1, 2, 3], 4, 5, 6]

[1, [2, 3, 4], 5, 6]

[1, 2, [3, 4, 5], 6]

[1, 2, 3, [4, 5, 6]]

[[1, 2], [3, 4], 5, 6]

[[1, 2], 3, [4, 5], 6]

[[1, 2], 3, 4, [5, 6]]

[1, [2, 3], [4, 5], 6]

[1, [2, 3], 4, [5, 6]]

[1, 2, [3, 4], [5, 6]]

This was pretty easy to visualize and draw out on paper manually, but figuring out how to do this programmatically for any situation was a challenge. This sort of combination/permutation type math felt like the type of thing I would have learned in a Probability and Statistics class that I took in college, but you know what I remember from that class?

Having a huge crush on a lil’ cutie and zero game to do anything about it
Drawing this funny comic

This was less dark in 2002

That’s what I remember. Certainly not this practical math.

The Actual Question

Anyway, turns out the question we’re trying to answer is: if v is the number of values we have (5), and h is the number of headers we have (4), what are all the unique permutations of h numbers that add up to v?

The following looks similar to the written out paper breakdowns from before, but now instead of each number representing a position, each number represents how many values will be joined together.

What are all the permutations of 4 numbers that add up to 5?

[2, 1, 1, 1] # join the first two values together

[1, 2, 1, 1] # join the second and third values together

[1, 1, 2, 1] # join the third and fourth values together

[1, 1, 1, 2] # join the fourth and fifth values together

To calculate this programmatically, we can first calculate the unique combinations using the method below that I nicked from a stack overflow post that has since been lost in my browser history. There were a number of them, and this one seemed most performant.

1
2
3
4
5
6
7
8
9
10
11
12
13
def combos(desired_size, count, minimum = 1)
  # determine all combinations of [count] numbers that add up to [desired_size]
  # e.g if we have an array of 6 items and want an array of 4 items
  # we need 4 numbers that add up to 6, => [[1, 1, 1, 3], [1, 1, 2, 2]]

  return [] if desired_size < count || desired_size < minimum
  return [desired_size] if count == 1

  (minimum..desired_size - 1).flat_map do |i|
    combos(desired_size - i, count - 1, i).map { |r| [i, *r] }
  end
end

Given these combinations we can now calculate all the different permutations of those sets, which is basically just getting every ordering of those numbers and then eliminating duplicates.

Here’s a class that does all this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
module CommaSplice
  class JoinCalculator
    attr_reader :from_size, :to_size

    def initialize(value_count, header_count)
      @from_size = value_count
      @to_size   = header_count
    end

    def possibilities
      @possibilities ||= permutations(combos(from_size, to_size))
    end

    private

    def permutations(combinations)
      # get all permutations of those combinations
      # to determine every possibility of join

      all_permutations = combinations.collect do |combo|
        combo.permutation(to_size).to_a
      end

      # flatten down to a list of arrays
      all_permutations.flatten(1).uniq
    end

    def combos(desired_size, count, minimum = 1)
      # determine all combinations of [count] numbers that add up to [desired_size]
      # e.g if we have an array of 6 items and want an array of 4 items
      # we need 4 numbers that add up to 6, => [[1, 1, 1, 3], [1, 1, 2, 2]]

      return [] if desired_size < count || desired_size < minimum
      return [desired_size] if count == 1

      (minimum..desired_size - 1).flat_map do |i|
        combos(desired_size - i, count - 1, i).map { |r| [i, *r] }
      end
    end
  end
end

# CommaSplice::JoinCalculator.new(6, 4).possibilities
#  #=> [[1,1,1,2], [1,1,2,1], [1,2,1,1], [2,1,1,1]]`

Now given all the permutation options, we can loop through them and generate all the value possibilities.

1
2
3
4
5
6
7
8
9
10
11
join_possibilities.collect do |joins|
  values = @values.dup
  joins.collect do |join_num|
    v = values.shift(join_num)
    if v.size > 1
      quote_values(v)
    else
      v.first
    end
  end
end

Here are the value possibilities. Five values made into four.

At this point, we could match these up with the headers and prompt the user with the option, making the task a little less tedious, like so:

And that’s a pretty good worst case scenario! But there’s still something we can do to determine which one of these is most likely correct without dying from boredom after doing this thousands of times.

The Human Element

You might have already have caught this, and if so — nice work, you’re sharp — but this crucial fact only hit me when I was a mile deep into this problem: when people type commas out they generally put a space after it. So if a parsed value starts with a space…it’s probably not the correct choice.

So in this example we can obviously see that option number 4 is the correct one, since every other one has a value starting with a space. So most of the time, we can reject any choice where any value starts with a space.

This doesn’t always work! With 750,000 manually entered tracks, you can bet people made typos, and you can bet there are tracks that have commas without spaces after them, like this beauty:

Deftones,U,U,D,D,L,R,L,R,Select,Start,Saturday Night Wrist,Maverick

(which, by the way, generates 220 possible options with no great way to determine which one is correct without a human and/or an internet connection. Oy vey.)

But this method works well enough to turn what I initially thought to be an impossible task into a very very doable task, which I’m happy to report I completed successfully.

I packaged all this code into a ruby gem (with a command line option) on the off chance that some other poor soul has found themselves in a similar CSV parsing dilemma. If that’s you: the chances of you getting out of that situation alive? Better than average.

Hawaiian Rumble Adventure Golf

2019-06-21T12:34:16-05:00

Dear Hawaiian Rumble Adventure Golf,

My family played your mini golf course last weekend and loved it! It seems warn enough out so I’m wondering when the pool surrounding the course will be open for swimming? Also, is the water around the last hole very deep? It was too black to see clearly, but my son Aaron loves to dive and we’ve now learned that we need to check depths before doing so.

What kind of pool toys do you provide, and do you have an air pump available for customer use? The inflatable white swan we have takes so long to blow up.

Sincerely,
Linda B. Wilson

P.S. How much of a discount do we get for each golf ball we retrieve from the bottom of the pool? Does the color matter?

Everything Is Not Lost

2019-06-01T06:00:00-05:00

Written & Recorded: Spring 2019

I’m pretty proud of this one. This was my first demo recording of it just to get it down, and I hadn’t yet written the bridge or the second half, but I think has the feeling I’m going for in it. Somber lyrics paired with a little skip in your step, ya know?

This version is the full song, adding a bridge and another verse, but I think at this tempo with just a piano it sounds way too sad. Pretty, but sad.

FCC Ruby Gem

2019-06-01T00:00:00-05:00

I just updated a gem I wrote in 2011 (which the FCC actually starred and forked, lol) to use their new API, which apparently knows about caching now. It doesn’t provide all the same data as the old one did, which is kinda weird. No “signal strength”? Why? So the gem can still query the old, horrifically slow and crusty API if you want it to.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
station = FCC::Station.find(:fm, "KOOP")

if station.exists? && station.licensed?
  #Basic attributes, available quickly because the FCC actually caches these in a CDN: 
  station.id #=> 65320
  station.status #=> LICENSED
  station.rf_channel #=> 219
  station.license_expiration_date #=> "08/01/2021"
  station.facility_type #=> ED
  station.frequency #=> 91.7 
  station.contact #=> <struct FCC::Station::Contact>
  station.owner #=> <struct FCC::Station::Contact>
  station.community #=> <struct FCC::Station::Community city="HORNSBY", state="TX">

  # Extended attributes, takes several seconds to load initially because the FCC is running this endpoint on a 1960s era mainframe operated by trained hamsters. 
  station.station_class #=> A
  station.signal_strength #=> 3.0 kW
  station.antenna_type #=> ND
  station.effective_radiated_power #=> 3.0 kW
  station.haat_horizontal #=> 26.0
  station.haat_vertical #=> 26.0
  station.latitude #=> "30.266861111111112"
  station.longitude #=> "-97.67444444444445"
  station.file_number #=> BLED-19950103KA
end

Blame

2019-02-01T05:00:00-06:00

Written & Recorded: Early 2019

2021 feelings about it

(Nobody to blame but myself for the one line that’s embarassingly off pitch. Randy Jackson would have a field day on me.)