Is ChatGPT Useful for Creating Website Content?
You’ve probably heard a lot about ChatGPT lately. This new AI chat tool is capable of rendering complex mathematical equations in seconds, answering difficult questions in plain English, making personalized restaurant recommendations, and even drawing up lesson plans for university professors and K-12 teachers.
But can it be useful for creating website content? More importantly, can it create content that will rank well in search engines? Is ChatGPT really capable of producing rankable content on autopilot? We put it to the test to see what this cutting-edge tool has to offer.
Our verdict: ChatGPT is impressive in terms of its research capabilities but it is not a replacement for real, expert-driven insights and creative writing. Google can also detect AI-generated content, so even if the content ranks, it is only temporary.
As an agency that prides itself on our content marketing savvy, we’ve found that certain qualities are imperative for content that ranks. It needs to be original, properly optimized, full of useful insights, and intelligent in covering its topic. It must cover the topic more clearly, more fully, and in a more engaging manner — while providing undeniable value to the reader.
You’re not going to get that with AI. Here are a few of the key things we noticed when experimenting with ChatGPT.
ChatGPT Copy Is Generic
We entered several queries for common SEO blog topics, and the results were impressive for machine-generated content but generic in terms of overall substance and value.
For instance, when we asked ChatGPT to produce an article about obesity prevention, the result was an accurate yet simplistic set of recommendations.
The above article looks like the type of content that might have been rank-worthy in 2008. But today’s SERPs are hyper-competitive, and it’s no longer enough to simply cover the basics.
Content needs to be unique, comprehensive, and engaging, and it needs to make readers think about the topic in a whole new way. Tropes like “drink 8 glasses of water a day” and “aim for 7-9 hours of sleep” aren’t going to move the needle.
These are all things we’ve heard a billion times. Quality, rank-worthy content must offer unique insights from a trustworthy authority in the field.
So while the above might be useful for a pamphlet in a doctor’s office, it offers little to no value as SEO content.
ChatGPT Fails the Plagiarism Test
When using a tool like ChatGPT, you have to watch out for plagiarism. The tool consolidates information from across the web, and while it tries to incorporate unique phrasing as much as possible, it often fails.
We fed several ChatGPT-generated articles through our plagiarism detector, and all of them had multiple sentences flagged. The above obesity article generated a 20% plagiarism score—meaning that one-fifth of the article was duplicated from sources already on the web.
Duplicate content is likely to become an even bigger concern as more people turn to ChatGPT and similar AI tools to generate content. Over and over, we watched the tool generate articles with strikingly similar verbiage, which presents a serious problem for anyone looking to create unique, rankable content.
For example, we created several queries asking ChatGPT to tell us “the best songs by [band] and why.” We did it for the Beatles, the Rolling Stones, Coldplay, Save Ferris and others (more on that below). And in almost every instance, the intro contained some version of “the best songs by ___ are subjective and vary according to personal taste.”
As AI tools become more popular for generating copy, we’re going to see a lot more repetitive, cookie-cutter narratives online.
Lack of Accuracy
Accuracy is one of the biggest problems for anyone looking to create AI-generated web content. Perhaps you read about the recent controversy involving Bard, Google’s own AI chatbot. In a promotional ad, Google Bard was asked “What new discoveries from the James Webb Space Telescope (JWST) can I tell my 9-year-old about?"
Its response suggested that the telescope captured the first pictures of a planet outside the Earth's solar system. As numerous viewers pointed out, this was false information.
We found similar types of subtle inaccuracies in our ChatGPT queries. Sometimes they’re easy to miss.
If you’re running a classic film blog, the above narrative might land you in hot water with movie buffs. At a glance, the list looks legitimate. Fred Astaire and Ginger Rogers did make exactly 10 movies together, and numbers 1 through 9 are accurate examples. However, eagle-eyed film historians would quickly point out that Ginger Rogers does not appear in “Damsel in Distress;” the above list is missing “The Story of Vernon and Irene Castle” which does star Astaire and Rogers.
Another great example of factual inaccuracies that our team was able to quickly uncover came from the breakdown of Utah’s drinking laws.
First, Utah does limit the alcohol content of beer sold in grocery stores, but the current limit is 5% ABV (not 4% as generated by ChatGPT). AI is using pre-2019 data.
Additionally, the statement that alcohol can’t be sold on Sundays is factually incorrect. While state-owned liquor stores are closed on Sundays, any legal adult 21 and over can purchase any type of alcohol — including hard liquor — 7 days a week from licensed restaurants, bars, and distilleries. Grocery stores can also sell beer on Sundays.
So if you use ChatGPT for research or content creation, make sure to fact-check every sentence.
AI Requires a Lot of Available Information
ChatGPT does best with topics that have already been covered at length. The more information it can pull from, the more accurate a response you’re likely to get — even if it is still generic.
As an example, we tried a couple of queries for a popular band and a lesser-known band.
The Beatles remain one of the most influential bands of all time, so it’s no surprise that there are countless articles on the web that rank and debate the best Beatles songs.
This makes ChatGPT’s job easy. It was able to easily spit out a list of popular Beatles songs, and it even gets the general trivia correct (“Something” is, in fact, George’s only #1 Billboard hit with the Beatles).
Now let’s try the same query, but with a more obscure indie band:
Here we see the AI’s limitations in full effect. Because there isn’t nearly as much information online about Save Ferris’s best songs, ChatGPT spit out a dialogue that’s rife with inaccuracies.
As dedicated ska-punk fans will attest, the song “Spam” has nothing to do with spam email, “Goodbye” is not a ballad (it’s a neo-swing song), and — most egregious of all — there is no Save Ferris song called “Misery.”
The line “I’ve overcome the misery” does appear a single time in their song “Modified,” but that’s as close as it gets. If a Save Ferris fan read the above article, they would immediately assume that it was written by either an AI or an amateur copywriter with sloppy research skills and no knowledge of the band.
So here’s the catch 22: To get accurate and reliable information with ChatGPT, you need a subject that has been covered extensively online. But those heavily covered subjects are also the hardest to rank for as they have the most competition, meaning that you need to publish content that’s far more useful and comprehensive than what an AI can currently provide.
ChatGPT Doesn’t Always Work
Another quirk of ChatGPT is that it tends to lock people out during periods of heavy use.
This means that it won’t always be available when you need it. The only work-around at the moment is to pay $20 a month for ChatGPT Plus, which gives you priority access.
The company has said that ChatGPT will remain free to the general public, but if bandwidth continues to be an issue, the company may ultimately limit or restrict its non-paid access.
Given its popularity, and the fact that every company eventually must become profitable, you can expect to pay for the tool in the not-so-distant future.
This is a significant factor to consider before you come to rely heavily on ChatGPT.
Its Body of Knowledge Only Goes Up to 2021
As of right now, ChatGPT can only pull information as recent as 2021. If you’re looking to generate content about the Eagles vs Chiefs Super Bowl, or about the recent quiet quitting trend, you’re out of luck for right now.
The 2021 cut-off can also be problematic for anyone researching current data. If you want to know how many people have contracted COVID-19 worldwide, you’re only going to get figures as recent as 2021.
ChatGPT Might Violate Google’s Terms of Use
Perhaps the biggest issue with using ChatGPT for website content is that it might place you in violation of Google’s “spammy automatically generated content” rule. This one is a bit of a gray area because, while Google doesn’t forbid AI per se, it does forbid the inclusion of auto-generated content that serves only to manipulate search engines as opposed to providing value to users.
Ultimately, it’s at Google’s discretion whether your AI content provides value or not. But at the very least, you could be putting yourself in a vulnerable position by populating your website with AI content that offers little in terms of uniqueness or quality.
ChatGPT Does Have Potential
While most of our initial impressions of ChatGPT have been underwhelming (at least in terms of its rankable content creation capabilities), the tool does have real value when used in the right way. Ultimately, we see ChatGPT as being an effective virtual research assistant.
For instance, if you’re drafting an article about how to train for a marathon, you can use the tool to quickly look up information like how to calculate the maximum heart rate of a 30-year-old or how to convert the distance of a marathon from miles to kilometers. The tool can also be used to pull up any data you might need (as recently as 2021) and even make spelling and grammar suggestions. Just make sure to fact-check any information that ChatGPT provides and use your own informed judgment when drafting your narratives.
There’s a place for some automation in content creation, but rank-worthy content will always require a human touch—and an in-depth knowledge of the most important content metrics. If anything, the collective push toward AI will only increase the demand for human-driven content that’s unique, compelling, and logically crafted. So, by all means, embrace automation technologies—but keep them in the passenger’s seat. You can’t automate the human experience.
