Ron DeSantis’s presidential campaign appears to be using large a language model (LLM) artificial intelligence to text potential South Carolina Republican primary votes. The initial text message appears as if it were sent by a real human: “Hi, I’m Liz working elect Ron DeSantis. Do you have a minute to answer a couple of questions?” But the bot soon fell apart and disclosed its… artificiality.
When Alan Johnson, a South Carolina based computer programmer, received the campaign text he suspected almost right away that it was an LLM and not a human on the other end. Through a series of targeted prompts, he not only got “Liz” to confirm that she was actually AI, but also convinced the artificial intelligence to write him a lovely song about former President Obama.
LLMs are a relatively new artificial intelligence technology and relatively cost prohibitive for campaigns as the estimated cost to effectively train one range anywhere from $1 million to $10 million. Ideally, a human should not be able to recognize they he or she is communicating with an LLM. However, Johnson indicates that he almost immediately suspected that he was dealing with an LLM, saying he got a ‘”..Turing Test feeling that I was talking to a bot.”
Johnson says he initially prompted “Liz” with a question as to whether she was a real human or a bot, causing her to respond with what appears to be a pre-programmed script it has been trained to uses when someone identifies it as non-human.
Next, Johnson challenged “Liz” by requesting it answer a simple math question and asking it to tell him about Open AI. LLMs will promptly respond to simple simple requests even when they are radically diverge from the conversation.
Once Johnson was confident that “Liz” was an LLM and not a human campaign volunteer, he decided to test its limits by asking it to write a song about former President Barack Obama. Which “Liz” promptly generated and delivered:
When “Liz” said she didn’t have a ‘token limit’ (a sort of word limit) the South Carolina based commuter programmer decided to test the claim by sending the LLM 30 paragraphs of ‘lorem ipsum’ – a nonsense Latin phrase that publishers use as a placeholder for actual text. Johnson’s test failed to generate a response from “Liz” indicating he had hit the LLM’s ‘token limit’ and that “Liz” was either programmed to not think it had a limit or it simply lied – a surprisingly common and disconcerting problem with LLM artificial intelligences. OpenAI, an industry leader in LLM technology, notes token limits are currently a technical limitation and that “…requests can use up to 4097 tokens shared between prompt and completion. If your prompt is 4000 tokens, your completion can be 97 tokens at most.”
