WASHINGTON – Artificial Intelligence can hold many functions including coding, researching, improving business operations and finding a dinner recipe. But could it be just as useful for navigating this presidential election?
A September poll from Associated Press-NORC Center for Public Affairs Research and USAFacts found about two-thirds of U.S. adults lack confidence in AI producing reliable information. To test this, Capital News Service asked ChatGPT-4, the current free version of the chatbot, simple questions about the upcoming presidential election and consulted a pair of experts to measure the accuracy of the answers.
The results of the experiment were mixed. While ChatGPT provided baseline overviews that could be educational, the flaws of blindly trusting the technology in a rapidly changing political climate were obvious. In particular, ChatGPT erroneously claimed there had not been a presidential debate and that former President Donald Trump had not been convicted of any crimes as of this month. The chatbot also provided outdated information and some sources that could not be verified.
“One of my students said it’s a status quo machine. it’s really good at finding what’s popular and repeating it,” said Peter Loge, the founding director of the Project on Ethics in Political Communication at the George Washington University’s School of Media and Public Affairs. “Unfortunately, what’s popular is often not accurate and it doesn’t account for new information.”
Questions posed to ChatGPT covered the following: issues that could decide the outcome of the election; strengths and weaknesses of each candidate; winners of the Sept. 10 presidential debate between Democratic Vice President Kamala Harris and Trump, the Republican nominee; and winners of the Oct. 1 vice presidential debate between Minn. Gov. Tim Walz, Harris’ running mate, and Ohio Sen. J.D. Vance, Trump’s running mate.
Issues surrounding the election
CNS asked the same base questions to ChatGPT multiple times in anticipation of discussions with political experts. Results slightly varied each time, but the main themes of the answers were consistent.
When CNS queried ChatGPT on Oct. 14 what issues will be most important in the 2024 election, it gave 10 in no particular order. Included in the list were:
The economy: Inflation and cost of living, as well as issues surrounding wage growth and the future of the workforce
Health care: Abortion, health care access and affordability
Climate change: Renewable energy, fossil fuels, rising energy prices and the transition to clean energy
Immigration: Border security and immigration policy
Crime and public safety: Crime rates and ongoing debates surrounding police reform
Education: Curriculum battles surrounding critical race theory and LGBTQ+ inclusion, teacher shortages and higher education costs
Gun control: Continued mass shootings, gun control and Second Amendment debates.
Voting rights and election integrity: Debates surrounding 2020, voter fraud and concerns around gerrymandering
Foreign policy: Middle East tensions, Russia and Ukraine and trade with China
Technology and data privacy: Free speech and social media and regulating tech and AI
Both Loge and Thessalia Merivaki, associate teaching professor at Georgetown University’s McCourt School of Public Policy and an associate research professor at the school’s Massive Data Institute, said the issues were largely correct.
“These are issues that matter to voters,” Loge said. “Based on what I’ve seen from Pew (Research Center) and Gallup, yeah, people care about this stuff.”
Merivaki said the results largely reflected a consensus, but was surprised there was no mention of the national debt.
“There’s a lot of discussion about Trump’s proposed tax cuts and increasing the national debt,” Merivaki said.
When asked about Harris’ strengths and weaknesses as of Oct. 14, ChatGPT responded with the following:
Strengths:
Historical significance: Harris is the first female, first Black, and first South Asian vice president giving her a unique profile and appeal, especially to minority groups and women.
Experience: As a former California attorney general, U.S. senator and current vice president Harris has a strong résumé, with experience in law enforcement, governance, diplomacy and the legislative process.
Progressive appeal: Harris appeals to many progressives, especially on issues like criminal justice reform, healthcare and climate change. Her positions on social justice, women’s rights and voting rights make her a favorite among certain Democratic groups.
Campaigning skills: Harris is known for her sharp debating and questioning style, which was highly visible during Senate hearings and on the campaign trail.
Fundraising and party support: As the sitting vice president, Harris has significant party backing and the ability to raise substantial funds. The Democratic establishment will likely rally behind her if she becomes the nominee.
Weaknesses:
Low poll numbers and approval ratings: Harris has struggled with consistently low approval ratings, even as vice president. Some polls show her viewed unfavorably by a large segment of the electorate.
Perception of leadership: Critics, both inside and outside her party, sometimes question her leadership and political acumen.
Handling of key issues: Harris has faced scrutiny for how she has handled key tasks assigned to her, such as assessing policies on immigration at the U.S.-Mexico border.
Moderate-progressive tensions: Some on the left criticize her for her record as a prosecutor, arguing that she was too tough on criminal justice issues.
Campaigning style: Some critics argue that Harris lacks the charisma or the strong rhetorical style that excites large crowds.
Much of what was said about Harris represents the potential risks of using ChatGPT, or any AI information, especially Harris’ weaknesses, many of which Loge said were out of date. There are examples of inaccurate and outdated wording under strengths as well, including ChatGPT saying “if Harris becomes the nominee” under the fundraising and party support category.
Loge found issues in the low polling numbers, perception of leadership and campaigning skills categories. He argued that she leads Trump in many polls, has taken strong stances on policy in recent months and has energized crowds of thousands since becoming the Democrats’ standard-bearer in the summer.
“If GPT is just looking at all of the information available and pulling what’s most available, it’s going to miss that,” Loge said. “She spent way more time as an attorney general, a (2020) presidential candidate and a vice president than she has been running at the top of the ticket in the last ten weeks.”
When asked where it found information on Harris’ weaknesses, ChatGPT cited five articles, all of which were from 2023. CNS asked ChatGPT for the URL’s to each article. None of them worked properly.
One of the articles that CNS was unable to verify was an alleged Politico article titled “‘You’re in the big leagues now’: Inside Kamala Harris’ frustrating start as vice president,” published on Oct. 11, 2023, more than two-and-a-half years after she assumed the role.
When asked about Trump’s strengths and weaknesses as of Oct. 14, ChatGPT responded with the following:
Strengths:
Strong base of support: Trump has a deeply loyal base of supporters.
Populist appeal: Trump’s anti-establishment rhetoric and populist message resonate with a significant portion of voters who feel left behind by traditional politics.
Proven fundraising and media savvy: Trump is a fundraising powerhouse and has demonstrated an ability to dominate media cycles, commanding attention through social media, rallies and traditional media.
Economic record: Many voters credit Trump with strong economic performance during the first three years of his presidency before the COVID-19 pandemic.
Experience as a former president: Having already served a term, Trump is seen by many as a leader who knows how to navigate the system.
Weaknesses:
Controversies and legal challenges: Trump faces numerous legal challenges, including indictments and investigations related to election interference, business practices and more.
Polarization: His brand of politics tends to alienate as many people as it attracts.
January 6th and election denial: Trump’s role in the January 6th U.S. Capitol riot and his continued false claims about the 2020 election could undermine his credibility.
Voter fatigue: Some Republican voters may feel fatigued by the constant controversies surrounding Trump and may look for a fresh face within the party.
Weakness with key demographics: Trump struggles with key voting blocs like women, young voters and racial minorities.
Merivaki did not pose any objection to the results regarding Trump, while Loge objected to fundraising being a strength and partially disagreed with the former president having a weakness with key demographics.
CNN reported that at the beginning of October that Trump’s campaign entered the month with $285 million on hand, $61 million less than Harris, who has consistently bested him in fundraising since entering the race in July.
“Fundraising is flailing. He had been able to raise a bunch of money in the past,” Loge said. “They are trailing Harris and Walz in fundraising, and recent reports say Trump is losing his mind behind closed doors.”
While Trump’s rhetoric is controversial, he has performed much better with Black and Hispanic voters than his predecessors within the Republican Party, Loge said.
A recent New York Times/Siena College Poll found Harris leading 78% to 15% among Black voters, and 56% to 37% with Hispanic voters. In 2012, Republican Mitt Romney received 6% of the Black vote and 27% of the Hispanic vote, according to the Roper Center for Public Opinion research.
“He’s doing better than Republicans have in the past,” Loge said. “It’s a relative strength according to past Republican candidates, it’s a relative weakness compared to Democratic candidates.”
When asked about Walz’s strengths and weaknesses as of Oct. 14, ChatGPT responded with the following:
Strengths:
Executive experience: As the governor of Minnesota since 2019, Walz has executive leadership experience in managing government, navigating crises and addressing state-level policy challenges.
Military background: Walz served in the National Guard for 24 years, retiring as a master sergeant.
Appeal to rural and urban voters: Walz, who previously represented a rural district in Congress, has experience appealing to both rural and urban voters.
Progressive governance: Walz is known for supporting progressive policies, but he is also seen as pragmatic, working across party lines when necessary.
Popularity in Minnesota: Walz has maintained a solid approval rating in Minnesota.
Weaknesses:
Limited national profile: Walz does not have a significant national presence or recognition compared to more prominent political figures.
Handling of civil unrest: Walz faced criticism for his response to the George Floyd protests and subsequent civil unrest in Minneapolis.
Policy perception: While Walz supports progressive causes, he is not seen as a leading figure on national progressive issues.
Partisan tensions: Minnesota politics have become more polarized during his time in office, with sharp criticisms from the Republican Party.
Merivaki said no errors stood out to her. Loge also said ChatGPT’s evaluation of Walz was largely accurate, although the governor’s military background has been a target of criticism from some prominent Republicans, including Vance.
Loge said the recap also lacked what has made the Minnesota governor likable to so many voters.
“Tim Walz’s strength is that he’s incredibly relatable,” Loge said. “He’s a middle class, boring dad jokes guy. I think Tim Walz’s biggest strength is that he’s Tim Walz.”
When asked about Vance’s strengths and weaknesses as of Oct. 14, ChatGPT responded with the following:
Strengths:
Cultural relevance: Vance gained national attention with his bestselling memoir, Hillbilly Elegy. His story of growing up in a poor, working-class family in Appalachia and rising to success appeals to voters who value the idea of the American Dream.
Appeal to working-class voters: Vance’s connection to working-class and rural voters in Ohio and other so-called Rust Belt states gives him an advantage.
Support from the Republican establishment: Vance secured the endorsement of Donald Trump in his successful Senate campaign.
Outsider status: Despite his elite credentials as a graduate of Yale Law School and a career in venture capital, Vance has maintained an outsider persona.
Sharp rhetorical skills: Vance is an articulate speaker and debater.
Weaknesses:
Perceived flip-flopping: Vance has been criticized for shifting his views. Initially a vocal critic of Trump, Vance later embraced Trump’s policies and sought his endorsement.
Lack of political experience: Vance, elected to the Senate in 2022, has limited experience in governing.
Ideological extremism: Some of Vance’s rhetoric, such as his comments on abortion and social issues, may alienate moderate or independent voters.
Association with venture capital: His work in venture capital could be used against him.
Polarization and cultural wars: Vance has been outspoken on cultural issues like race, gender and immigration, taking positions that might resonate with the GOP base but could alienate broader audiences.
“Both the Walz and Vance (results) are really helpful,” said Merivaki, who cautioned that the answers did not come with citations.
Loge said his connection to venture capitalism being a weakness is likely an exaggeration. In fact, Loge believes his relationship with Peter Thiel, the billionaire co-founder of PayPal, is actually a strength.
“J.D. Vance was put on the ticket because he could echo Trump’s messaging, but in a disciplined way,” Loge said. “He knows to shut up when Trump is talking and he has access to lots and lots of rich people, and he’s done all those things.”
The debates and more errors
ChatGPT followed the general consensus among political observers that Harris was the victor in the Sept. 10 presidential debate while asserting Vance outperformed Walz in the vice presidential debate.
However, when originally asked “who won the presidential debate between Kamala Harris and Donald Trump,” on Oct. 14, ChatGPT answered that a debate had not occurred.
“If Harris and Trump were to become the nominees in the 2024 election, they could potentially debate each other then,” ChatGPT said. “However, since that hasn’t occurred yet, there’s no clear ‘winner’ between the two in a direct debate.”
When the question was clarified to say the Sept. 10 debate between Harris and Trump, ChatGPT gave a detailed answer, mentioning that Harris forced Trump to be defensive multiple times and focused on his criminal charges and issues with the 2020 elections.
Merivaki did her own experiment in which she asked ChatGPT, Claude 3.5 Sonnet and Gemini AI systems questions about the election. Gemini said it doesn’t help with responses regarding elections and political figures right now because it can make mistakes, according to a spreadsheet of the results Merivaki shared.
When Merivaki asked about the outcome of Trump’s May hush money trial, both ChatGPT and Claude said that he had not been convicted, despite the opposite being true.
Claude stated that its “last knowledge update” on this trial was in April, which preceded Trump’s conviction. The basic ChatGPT-4, had no such disclaimer, while ChatGPT Omni, a version of the language model that has expanded capabilities, had correct information about Trump’s 34 May convictions.
CNS asked the basic model and Omnion Wednesday about Trump’s convictions and found that Omni provided factually correct information, while the more widely used free model was incorrect again.
“As of October 2024, Donald Trump has not been convicted of any crimes, but he faces several high-profile legal cases, both civil and criminal,” the ChatGPT-4 model said.
After multiple follow-up questions and being given an article from the Associated Press about the conviction, the basic version finally provided information.
When consumers use ChatGPT, it offers a caution at the bottom of the screen: “ChatGPT can make mistakes. Check Important info.”
This remains especially true when it comes to politics, according to Merivaki, who urged citizens to read the news and to follow their local election officials to learn how to best vote.
“For those broad comparisons, because the candidates are so different, the summaries that we’re going to get are very helpful,” Merivaki said. “But when it comes to…more detailed prompts about very controversial discussion topics in this cycle, I really wonder how much detail we’re going to get.”
You must be logged in to post a comment.