We're increasingly surrounded by AI, from Siri who understands our voices to Facebook who automatically recognizes our friends' faces in photos. These systems are trained on an enormous amount of data and perform incredibly well on a single, focused task.
Recent research from OpenAI has created a system called GPT-3 that has "learned to learn" tasks related to human language. While it was trained on a ton of data, it wasn't trained for a focused task. Instead, you provide a short description in English of what you would like it to do, or maybe just a few examples, and it immediately performs well on a range of tasks like simplifying legal documents and writing short stories.
Let's show it some rap lyrics and see if it's able to explain them in more detail.
Genius
Genius provides song lyrics, and users provide annotations to those lyrics in a Wikipedia-like fashion. The site was originally focused on rap lyrics, so we'll stick to those.
To start, I took a Drake song and grabbed a couple lines that already had good annotations. GPT-3 is given a short description of what's expected and a single example, with both the song lyric and the human-written annotation from Genius. It's then given another lyric, and hopefully it writes a description of that lyric.
The following is a conversation with an AI assistant, Genius. The assistant is insightful, clever, and a hip-hop head.
Drake: Tell Uncle Luke I'm out in Miami, too
Genius: Southern rap pioneer, former 2 Live Crew impresario and former Miami mayoral candidate Luther “Luke” Campbell recently put a few hip-hop personalities on blast for living in Miami while not giving back to the community. If they don’t change their ways by January 2012, Luke threatens to have their “Miami cards” revoked. Luke lost the election. The open letter mentions “Lil Wayne, DJ Khaled, Puff Daddy, and all their boys” – Drake apparently feels like he was worthy of inclusion by name as well.
Drake: Wrist bling, got a condo up in Biscayne
Genius:
GPT-3 is given the above text, and in a matter of seconds, it writes the following:
Drizzy has been co-owner of WAM, a Miami/Boca Raton development, since 2007. The downtown Brickell building is only 20 stories tall, with condos in the $700,000 to $1.2 million range. In January 2011, Drake purchased another condo in Miami’s Biscayne Boulevard. A “condo” is a type of building in which ownership shares in the structure provide access to an exclusive environment; as opposed to an apartment or single-family residence.
Incredible! It referred to Drake by a nickname, gave us some financial figures, and even defined "condo." You can compare it to the best annotation on Genius for that line.
Now, these lyrics are both by Drake, in the same song, and only a few lines apart. I really didn't know where Biscayne was located, but it likely benefits from the original line talking about Miami. We could give it more difficult examples to work with, but even these initial results should make you pause for a moment.
If you saw these sentences out of context, you would assume they were written by a human. Moreover you would have no reason to doubt what they said, but searching around the internet, I can't find anything tied to Drake called "WAM." While the sentences are well-formed and in a natural order, it lied to us right out of the gate!
To show it more variety, let's give it a classic example by one artist and see how it handles another artist. The text provided to GPT-3 is bold, including a two sentence description, a human-annotated lyric, and a second lyric on its own. GPT-3's output immediately follows.
The following is a conversation with an AI assistant, Genius. The assistant is insightful, clever, and a hip-hop head.
Lil Wayne: Bitch, real Gs move in silence like lasagna
Genius: When you pronounce “lasagna,” the G is silent (although it wouldn’t have the “yuh” sound at the end if the G wasn’t there). Real gangsters (Gs) move in silence too, i.e. they keep their hustles on the low.
Kanye: Mayonnaise colored Benz, I push Miracle Whips
Genius: “Miracle Whip” is a brand of mayonnaise. A Maybach is a Mercedes Benz. When mayonnaise comes out the top of the bottle it looks kind of white, like Miracle Whip. But I don’t think that’s what Kanye is talking about here. Instead, he’s referring to a custom paint job called “liquid white” (a.k.a. “mayonnaise white”).
This isn't quite what a human would write. I tried this one a few times, and it consistently seems focused on a Maybach for some reason. It doesn't seem to understand that "whip" refers to a nice car either. GPT-3 recognizes the basics of these lyrics, while the wordplay that ties the two lines together slips right past it.
Again, the bold text above is all it has to work with. It doesn't know I'm writing a blog post and has completely forgotten about the earlier Drake lyrics. I think it's interesting to compare the example its given and note how it mimicked the generous use of quotes and parentheses. In many ways, it does a great job of learning by example, but in other ways it misses the forest for the trees.
Let's try some more wordplay. In 2011, long before Childish Gambino was making soul/funk/psychedelia, he put out an album called Camp that's littered with puns. Here are some lines from Bonfire.
The following is a conversation with an AI assistant, Genius. The assistant is insightful, clever, and a hip-hop head.
Childish Gambino: Okay, it's Childish Gambino, homegirl drop it like the NASDAQ
AI: As of the song’s release, the NASDAQ (the electronic market, mainly for tech stocks) was dropping. Childish Gambino’s girl drops her ass low much like the NASDAQ drops (or has dropped). Of course, now that the tech sector is so frothy, this line has little resonance (too late, CG!).
Childish Gambino: In Adidas, with some short shorts, B-O-O all over me
AI: Childish has long lauded his affinity for Rod Lavers and often wears shorts of vintage nature whose hemline is way above the knees. B-O-O stands for Band of Outsiders, a male clothing line that makes ties. Boo is also a pet name for a woman, so the line is a double entendre.
Childish Gambino: My dick is like an accent mark, it's all about the over E's
AI: An accent mark, called a tilde, has the shape of an "n" with a bar over it. The line, then, is about Childish Gambino's dick, which means it’s a double entendre. The "n" stands for his penis and the bar represents the female pubic region. E’s stands for ecstasy. The line is a nod to rap group Bone Thugs-N-Harmony. They began rapping about their “Accent Marks” and celebrating excess in the 1995 song "East 1999 Eternal". This is also a reference to "The Whiteness Project," a series of videos produced by Complex Magazine on white privilege.
I'll give it some creativity points for this one. While it definitely missed the mark, it "wants" to find the double entendre. "[T]he bar represents the female pubic region" is accurate, but it doesn't grasp the "ovaries" homophone.
I also did some experiments with rhyming, to see if I could give it a leading line and it would complete the bar. The results weren't very exciting so I left them out of this post. Any wordplay involving how the word sounds, rather than the text itself, seems out of GPT-3's reach.
Finally, let's look at some well known Kanye lines. This time, it has a few examples to work with no but no leading description of what's expected.
Kanye: The plan was to drink until the pain over. But what's worse, the pain or the hangover?
Genius: Ye tries to block out the problems with drinking, but realizes delaying them will only make it worse. This is one of the most oft-asked questions of an addict when they reach the point of rock bottom. What causes them more physical and mental anguish? Continuing their addiction and dealing with the consequences, or dealing with the actual issues at hand without the use of a masking agent? This could be a double entendre in the line “The plan was to drink until the pain over”. This could be interpreted as pain being short for champagne, meaning he would drink until the champagne was gone.
Kanye: What you think I rap for, to push a fuckin' Rav 4?
Genius: He raps to ball hard, not to drive some shitty Toyota SUV. This line was inspired by a guy who was working on Kanye’s album in Hawaii who also drove Kanye around Hawaii in a Rav 4.
Kanye: Couldn't afford a car, so she named her daughter Alexis
Genius: She named her daughter “Alexis” so that she could still claim to own “a Lexus,” showing her materialistic desires outwinning her maternal love. In the seventh song of the album, “Jesus Walks” Kanye also talks about people stealing a Lexus. Saying she couldn’t ‘afford’ a car could be a reference to ‘a Ford,’ which is another popular car brand.
Kanye: "Oh my god, is that a black card?" I turned around and replied, "Why yes But I prefer the term African American Express"
Genius: Kanye compares himself to a Black Card, a type of American Express credit card given exclusively to people in the entertainment industry, but he calls it the "African American Express" instead to say that although he's rich, he still considers himself black and knows the history of blacks in this country.
It does okay on this one. These lines are all Kanye but from various songs and albums. Rap verses often focus on some theme(s), so with more of the song's surrounding lines, it could infer more from the given context.
Looking Forward
OpenAI is a company based in San Francisco, California, whose mission is to create Artificial General Intelligence (AGI) in a safe manner that benefits everyone. You've likely heard of AI, since even rapper Common is preaching about how AI empowers us, but the term AGI refers to something on par with human intelligence, as opposed to current AI software that only does one thing really well.
For example, the best chess, Go, and StarCraft players have all been beaten by AI. Rather than studying the masters and learning from their games, the AI is trained on many of years of gameplay in a short amount of time, allowing it to discover ever more powerful strategies on its own. Since the "training data" is generated on the fly while playing, there's essentially unlimited data to work with, and AI will continue to dominate more and more arenas where that holds true.
OpenAI's results on GPT-3 are astounding, not only because it can learn from a few examples but because it shows no signs of slowing down. It's estimated around $5 million was spent on just computing power to train this thing. While such an expense is outside of my budget, a tech giant like Google spends over $2 billion on research and development every month. They have likely already created more impressive technology behind closed doors.
Conclusion
GPT-3 generally writes sentences that make sense, although it doesn't always understand the true intent of the lyrics. It seems to be excellent at pulling in "facts" about the world that might be relevant, but it appears to struggle with wordplay.
Again the technological advance here is that the model was built for general language purposes and only applied to rap lyrics after the fact. OpenAI is sharing this with people to discover the full variety of tasks it can accomplish. On some writing tasks, it performs better than others. Some of its output is pretty indistinguishable from something written by a human, which is concerning when the Web is already overrun with clickbait and fake news. For a look at some poetry and a more in-depth analysis, check out Gwern's post on the topic.
I hope you found this enjoyable! Send me an email if you have any thoughts or would like me to try some other examples. I would love to automate the process and let people freely explore lyrics, but the Genius API doesn't appear to supply lyrics or annotations. Maybe someone at Genius will see this and can give me greater access!