A copy editor sizes up her A.I. competition

An illustration of a robot reading a book.

The Grammarly plug-in promises to free your writing of mistakes. Can it replace a human copy editor? A biased investigation.

Like a taxi driver warily eyeing a fleet of self-driving Waymos, I didn’t exactly welcome Grammarly’s arrival on the scene. The San Francisco–headquartered company creates software that detects common spelling, grammar, and punctuation errors in text. You can install a free extension in your web browser to automatically flag these errors as you type up emails, Google Docs, and many other editable text fields.

Admittedly, I have my reasons to be leery. I’m a professional copy editor and have been for over a decade, correcting copy for freelance clients including Apple and The New Yorker. But when friends tease me that A.I.-based technology like Grammarly will put me out of business, I confidently scoff.

My defense goes something like this: Copy editors do so much more than move commas around. They check for issues of tone, accuracy, and logic. They point out redundancies, inconsistencies, and insensitive language. They enforce a publication’s particular house style. (Many of the time-honored stylistic idiosyncrasies that The New Yorker hews to would be flagged as errors by Grammarly: the hyphen in teen-ager, for instance, or the double s’s in focussed.)

But as Grammarly’s ads followed me around the internet, I couldn’t help but feel curious. I don’t think twice about running spell check before I send off copy. Is the Grammarly extension really so different? If it’s essentially spell check but better, then what did I have to lose? Was I secretly scared it might be too competent for this copy editor’s comfort?

Face with that kind of self-challenge, I had to give it a try.

A promising start

According to a stats email sent by Grammarly, I look at a lot of words.

Grammarly stats showing 93,000 words edited.

First, a disclaimer. I only tried the free individual Grammarly plan. Upgrading to a Premium or Business tier gives you access to additional features, like the ability to set rules for house style and advanced readability suggestions, which are not reviewed here.

The free extension is simple to install and intuitive to start using. Much like the built-in grammar and spelling features in Microsoft Word, it underlines issues in red. All you have to do is hover your cursor over them to see Grammarly’s suggestions and either accept or reject them.

Some issues that the extension regularly caught and correctly fixed:

  • Basic subject/verb agreement
  • Repeated adjacent words
  • Typos and misspellings
  • Missing articles and prepositions

I expected it to be able to cover those bases, but it also caught some more nuanced issues that surprised me—things like recommending singular pronouns for businesses and correcting slightly off prepositions.

Correcting pronouns for business entities

Grammarly suggesting a singular pronoun for a business.

Preposition suggestion

Grammarly suggesting an alternative preposition.

Not quite a foolproof solution

Not every suggestion was quite so astute. The program can not only miss mistakes, but it can also actively lead writers astray by suggesting they change something that isn’t incorrect to begin with. Most of the misguidance I noted fell into the following categories.

Overzealous matchmaking

This is by far Grammarly’s biggest category of mistaken suggestions. Its algorithm wants to turn adjacent words into a single closed compound whenever it thinks it spots a chance—but there’s a big difference in meaning between “checking in frequently” and “checking infrequently,” or between “meaning less” and “meaningless,” even if only a space separates them. These matchmaking misfires showed up again and again.

Word-combination mistakes

This slideshow requires JavaScript.

 

Overlooked errors

Not every writing error fits into a neat Strunk & White edict. Some of the mistakes Grammarly missed were highly contextual, which confirmed my suspicion going in—there’s really no way a program can catch everything without the ability to truly understand the content. Although it did pick up on a few homonym errors (e.g., “flair” instead of “flare”), others went unflagged, as did cases when the entirely wrong word was used, but it fit the grammatical structure of the sentence.

Other overlooked errors included dangling and misplaced modifiers, non-parallel structure, technically correct but just plain awkward phrasing, and some good old-fashioned misplaced or missing punctuation. The free version doesn’t account for publication-specific style preferences, like capitalizing after a colon or spelling out numbers. And the plug-in won’t be able to flag even the most basic of larger content issues, like if an entire sentence or paragraph is missing, a person’s name changes halfway through an article, or a top-ten list only contains eight items.

Unflagged mistakes

Incorrect word (should be “in”)

Text showing "is" instead of "in"

Missing period

A paragraph with a missing period.

Incorrect period placement

Text showing a period inside of parentheses.

Incorrect word (should be “parry”)

Text showing "party" instead of "parry."

Hyphen havoc

In a similar vein to its word-condensing zeal, Grammarly has a tendency to suggest hyphenated compound modifiers where none should exist. What is driving the program’s deep-seated need to force unnatural connections? Who hurt you, Grammarly?

Grammarly suggesting adding a hyphen to "single window."

Opposite Day

Once in a while, the program picks out a correctly chosen word and suggests that you change it to its opposite … maybe just for kicks?

Text showing "an OnePlus" instead of "a OnePlus"

 

Text showing Grammarly incorrectly changing "everything" to "anything."

“WTF”

More often than not, I could tell exactly what “if this, then that” rule the plug-in is trying to enforce—it just hasn’t been trained on the all the exceptions or is applying a blanket rule without context. Other times I was left truly baffled—the program seemed to be editing a completely different text than what was on my screen. Whether they’re caused by coding glitches, syncing issues, or something else entirely, these types of bizarre suggestions are, fortunately, rare.

“Killing yourself to sleep” is, admittedly, an angle I hadn’t considered.

Text showing Grammarly suggesting a change from "lulling" to "killing."

Reading between the lines (or spaces), in a bad way.

Text showing Grammarly suggestion to change "striking design" to "employees."

A copy editor’s verdict

Will I continue using the Grammarly extension? I absolutely will. It’s particularly useful at flagging the most basic kinds of mistakes—repeated words, typos—that even a trained editor’s eyes can slide right past when grappling with a larger issue within a sentence, such as a dangling modifier or illogical construction.

That said, it shouldn’t be over-relied on. Each Grammarly suggestion should be considered individually and not blindly accepted. Nonprofessionals or anyone who doesn’t have a solid grasp of English grammar should be especially wary of accepting suggestions without research.

I’ve seen this play out while editing for a tech news site. Whenever one particular author made updates to an already published post, I would spot new, strange mistakes in the previously clean copy. With each mistake I corrected, I noticed the Grammarly app incorrectly flagging my updated text as a mistake. Evidently, the writer was running the Grammarly extension and accepting all its suggestions without another thought, thereby inserting a handful of mistakes into every article.

I have no doubt that the algorithms will only improve, and it’s not hard to imagine a future where the software will be able to handle a lot of the issues I’ve listed with ease. But other issues will continue to require the trained eye of a human, at least for a while to come—things like tone, consistency, logic, advanced house style, wordiness, accuracy, and sensitive language.

In short: It’s another tool in my arsenal, the same as spell check on my word-processing programs and the copy of Garner’s Modern English Usage on my desk. Yes, I’m wildly biased, but I’ll consider my job safe for now.

Tags: , ,