Comparing AI vs. Human Generated Content Writing
If you search for “Ai content writing” on Twitter, there’s a nearly endless stream of results. Most of them are marketing tweets talking about whatever the hot new artificial intelligence content writing tool is; a few are discussing what it was like to use one. And it’s definitely a hot topic in search — just look at this five-year graph from
Exploding Topics:
The idea behind Ai content writing is simple, and appealing: don’t spend time writing copy or paying someone to do it. Instead, pay one flat fee a month (or nothing! Some are free!) and have a computer write your content for you.
It sounds almost too good to be true...and it is. Kind of.
Ai writing can be a real mixed bag
It’s 100% true that Ai content creation tools exist, and they do work on a functional level. If you give the Ai writer a prompt, it will produce content for you. Whether or not it’s high-quality content you want, though, is another story.
Now, you may be thinking to yourself, “of course you're going to say that, you’re a writer. You don’t want to find your job outsourced to a machine.” And you aren’t wrong, it’s true. I wouldn’t be delighted to find myself replaced by GPT-3 or LaMDA (which, despite recent news reports, is not actually sentient).
But, I like technology and I’m willing to admit when I’m wrong. So I tested some Ai content writers. And I gotta say . . . by and large, they aren’t ready for prime time. I can definitely see that they will be ready for widespread content marketing use one day, and I’ll need to learn to play nicely with the robots. But we aren’t there yet.
My first experience with an Ai content writer
My curiosity about Ai writing began to pick up when I heard about a company that claimed it could create fully unique content outlines for writers using its Ai technology. That piqued my interest. I’m not ready to be fully be replaced by a machine, sure, but I can recognize the value in using Ai to support my work. I already use
Clearscope, which is a tool that analyzes SEO content and evaluates its ability to rank against competitors. It’s really handy.
I signed up for the company’s Ai content generator tool. I’m not going to name names, but . . . if you’re in the content world you’ve definitely seen their ads (or you will after reading this — thanks, algorithm!)
To test this program out, I prompted the Ai to create a long-form content outline based off of a topic I’d already outlined for one of my clients. I wanted to see how it compared to what I put together.
A big problem
What I found was that the program directly copied an article from my client’s largest competitor, including pulling text and headers verbatim from that competing content. I knew this was the case because I had very recently read all of that competing content as part of my research process.
If I hadn’t realized it though, and actually relied on the program to do my blog article outlining, I would have wound up with something probably bordering on plagiarism. Not great. Admittedly, the tool was accurate, but to use it would still require a heavy dose of human involvement and intervention.
Testing two freemium Ai writing tools
That experience left me even more curious as to how some of the other Ai content writing software tools I hear about work in practice. While the original tool I tried was specifically geared toward outlining, I wanted to see what some other programs marketed for full-on website copy creation could produce.
So, we’re going to do a little test. Here’s how it will work:
- I’m using a topic I’ve already written about for clients and am familiar with: campaign tagging.
- I’ll feed keywords on this topic — ones I know I can get clients to rank for when creating organic content — into each Ai tool.
- I’ll adjust the Ai tool’s settings and request that it continues to produce content until I hit whatever the free or trial plan maximum is for that particular product.
- We’ll compare it to some original content I’ve written on the same topic, keeping in mind adjustments for length.
My point of reference for this test will be an article about campaign tagging that I wrote for a client. The article went live in January 2022. At the time of writing this comparison piece, my client’s original article is ranking on the first page of Google for “campaign tagging,” “url tagging,” and five other keywords. The draft I submitted to my client was over 3,500 words long and had a Clearscope score of A-, which was recommended in order to be competitive.
My inaugural Ai content experience was with a paid tool. Typically, I expect a paid service to offer a higher degree of accuracy than a free one, but that isn’t guaranteed. As such, I’m going to test two other Ai writing programs that use a free or freemium model.
Test subject #1
I learned about our first test program while looking up Ai content writing keywords in Ahrefs — it’s definitely getting increasing attention in search. The company behind this tool says that it can create blogs, ad copy, and other types of content using “state of the art language Ai” in more than 30 languages and 20 tones of voice.
Interestingly, I noticed several typos (mostly around the use of possessives) on the product landing page. I have no way of knowing if they used their own product to create the content, but it gave me pause.
This product does bill itself as an Ai writing assistant and the landing page repeatedly states that it “helps” writers create content faster. I do appreciate the relative honesty here — it’s not touted as a complete writer replacement tool.
- When setting up my test, I got to tell the Ai:
- What language to write in
- What tone to use
- My intended use case for the content
- Whether I wanted one or more variations on the result
- How creative or factual I’d like to be (represented as a sliding scale)
After setting up my parameters, I could enter up to 125 characters to explain the topic and another 125 characters’ worth of keywords. Here’s what I wound up with:
- Language: English
- Tone: Informative
- Use case: Blog sections
- Variations: 1
- Creativity: 0 (i.e., most factual)
- Topic: How to do campaign tagging to track URLs for digital marketing
- Keywords: Campaign tagging
I asked the Ai to return two paragraphs of content. This is what I got:
Campaign tagging is a process of adding a unique identifier to each URL in a campaign. This helps marketers track the performance of their campaigns and identify which URLs are performing better than others.
Campaign tagging is an important part of digital marketing because it helps marketers track the performance of their campaigns and identify which URLs are performing better than others. Campaign tagging is also important for SEO purposes because it helps search engines index content more efficiently.
This is all factually accurate, but it’s pretty top-level and a little repetitive. If I was going to use this intro to start an article on campaign tagging, I’d flip the two paragraphs, add more content and a subhead in between, and try using it as an introduction.
Without changing any variables, I asked the text generator to produce more content:
Campaign tagging is a process of adding metadata to URLs to track the performance of a campaign.
Campaign tagging is a process of adding metadata to URLs to track the performance of a campaign. This can be done by using Google Analytics or other marketing tools.
Campaign tagging is an important part of digital marketing because it helps marketers understand how their campaigns are performing and what they need to do in order to improve them.
Again, this is all technically correct but very repetitive. It’s clearly picking up on keywords and concepts beyond the sentence I entered during setup, but it’s not able to dive deeper.
Next, I adjusted the creativity level to medium. The Ai returned:
Campaign tagging is a powerful way to track URLs for digital marketing. It helps marketers to know which channels are generating the most conversions and can also be used to track offline conversions. In this article, we will take a look at how campaign tagging works and how you can use it in your digital marketing campaigns.
The first step is to set up your campaign tags on your website’s CMS system or CRM. You will need to create a tag for each of the channels you are using, such as Google Adwords, Facebook Ads, LinkedIn Ads, etc. Next up is to create a URL that has the campaign tag appended onto it. For example if you have a URL that looks like https://www.example-website/blog/post/, then you would make the
(Author’s note - where is the rest of this sentence?)
Campaign tagging is a technique that helps digital marketers to track the URLs of their marketing campaigns. Campaign tagging is a technique that helps digital marketers to track the URLs of their marketing campaigns. This ensures they know which campaign leads to which URL. Campaign tagging also enables them to measure the performance of each campaign and compare it with other campaigns.
With this technique, marketers can identify where they need to improve and what areas they need to focus on for future campaigns.
Once I gave the Ai more creative freedom, it began to vary the sentence structure. That was good, but the content was still very repetitive, top level, and even cut some sentences off midway. At no point did the Ai actually begin to explain how campaign tagging worked, which would be essential for an article about the topic. Instead, it hovered on the edges, giving various statements about what campaign tagging is, who uses it, and why it’s useful.
I continued to use this Ai assistant to generate just over 800 words of content and plugged it into Clearscope after adding subheads. It got a grade of C-. Granted, it would need to be longer in order to get a full A++ score, but, given the limited keywords used in what it did produce, I think it would take a fair amount of human intervention to get a rankable piece of content.
It’s my opinion that this Ai tool really is just an assistant (as they do advertise). It might be useful to help you get over some writer’s block, but the final product will require a lot of human intervention.
Test subject #2
Next, I tried a free Ai article writer that didn’t hedge its bets as much as the first one. This tool said it was suitable for “all your content writing needs” and all levels of writing skill. Interesting! Of course, it also says it can be used to spark new ideas.
Needless to say, I wasn’t quite clear on whether I was going to get (potentially) a full piece of content or just reworded prompts.
I set up this test similarly to the first using the program’s available fields:
- Topic: Campaign tagging
- Prompt: How to do campaign tagging to track URLs for digital marketing
- Creativity level: Balanced
Honestly, the Ai writing tool’s first output was pretty good. It seemed to understand search intent and guessed correctly that I’d be writing a tutorial given my “how to” prompt:
In this tutorial you'll learn how to tag campaign URLs and other campaign data with your campaign ID or tag code. You'll also learn some basic campaign tracking tips, such as how and when to use campaign tracking code.
This particular tool allows the user to continue generating content from the end of the previous output, and rate it with a thumbs up or a thumbs down. Not necessarily full-on machine learning, but still interesting.
I was curious if that Pandora-style rating system would help to fine tune the results, so I asked it to generate some more content:
To create a campaign, you will create an account and select a URL. This page will tell you how you can use the campaign to send digital ads to people who have joined your list. The campaign will be created for you and you should select and set a date and a number of impressions. Campaigns create the ad and then automatically collect the impressions and generate the unique number.
Okay, still not too bad. Campaign tagging isn’t only for serving ads (and it’s very vague about what “list” it’s talking about—email? Doesn’t make sense). However, it’s not terrible. You could feasibly work with this given just a little tweaking.
I decided to try another paragraph, and see how long this Ai could keep going before running off of the rails.
Answer: it immediately ran off the rails. This is what the program returned for its third paragraph:
For example, if you want to create campaigns that target an audience of 2,000, the default campaign size is 500 impressions for each campaign. Use the Campaign Manager to select campaigns, and click the Create Campaign button. (Refer to Figure 5-18.) Click the Advanced button in the top-right corner of the screen to enable advanced features. Click Create. On...
O-kay. Here we run into some problems. While this program provides generally more readable and linguistically diverse content than the first Ai service I tried, it gets way too specific here. The fact it calls out “figure 5-18” indicates to me that it is directly pulling content from other websites and repurposing it here. That’s a big problem if you’re writing content for your business site or that of a client.
This isn’t to say it’s generating purely plagiarized content — I did run it through a plagiarism checker and whatever rewording the Ai did was enough to pass — but you’d need to be very careful to eliminate any text that doesn’t directly align with the way your webpage and platform are set up.
Given the success of the first two paragraphs, though, this particular Ai tool might have some application for writing short e-commerce product descriptions or social media post captions. I’m still not sold on Ai content writing tools for blogging, long-form marketing copy, or business copywriting, though. You’re still gonna need a human marketing team, or at least a freelance writer who knows their stuff.
What could you really do with this Ai content?
More productive ways to include Ai in a writing workflow
Instead of relying on Ai tools to generate actual content, it’s best—in my opinion—to use Ai and computer algorithms to make your human-centered content writing workflow a little easier. This could be in the form of automation, research assistance, or scheduling, for example.
I happily use Clearscope, as mentioned above, and
CoSchedule’s Headline Analyzer to help me speed up the process of comparing my content, and keywords against other pages. I’m still doing all of the drafting, revising, editing, and creating, but I’m able to get some help with the information collection side of things.
There are a ton of Ai content writing startups hitting the market, and there may still be a golden goose out there that really can produce optimized content that is on par with what human bloggers create. I haven’t found it yet, but I’ll keep poking around . . . gotta have a heads up when the robots come for my work, you know.
