I spent a fair chunk of my weekend playing around with various A.I. tools, and, for the first time ever, I came away genuinely impressed by some of its outputs.
I’m as surprised as you are.
If your first impression of ChatGPT calcified a year or two ago as you watched it spew out weird hallucinations in a tinny robot voice… you might want to give it another whirl. It gives me no pleasure to report to you that much of it has gotten really good.
But for writing, has it gotten useful?
That’s where I’m still skeptical.
I’m by no means the world’s foremost expert here, and I don’t have all the time in the world to try every new A.I. gizmo under the sun (let’s face it: no one does). But I thought I’d peel back some of the layers of my weekend journey into A.I. to give you a sense of where I think things stand for writers.
A word on A.I. and ethics
Now. Is it even ethical to be using A.I. in the first place, given its innards were purloined from countless uncompensated authors and its increasing demands on a warming planet?
Believe me, I don’t feel great about it. I still want our A.I. overlords to pay me for their theft and our climate bill is long past due.
But at the end of the day, I just don’t feel like I, Nathan Bransford, pissed off author, can stop the A.I. freight train chugging along to who knows where.
It’s left the station. The powers that be keeping it running are vastly more powerful than the skeptics. I’d rather understand the beast rather than just stand aside as it gathers steam. And, yes, full disclosure, I’m now giving OpenAI $20 per month because I want to see what’s coming. I’m too scared of being caught flat-footed in my life and business.
But even engaging with A.I. should, at minimum, be preceded with a pause to gut check what we’re supporting. The upshot: I don’t feel great about it.
What A.I. now does well
I spent quite a chunk of my Saturday and Sunday fiddling around with two main tools: OpenAI’s ChatGPT (including the “Creative Writing Coach” GPT) and Google Gemini Canvas. Both purport to be able to help writers assess and improve their craft.
But before I get to writing, I experimented in a bunch of different directions. Here are some of the things A.I. now does surprisingly–even shockingly– well.
- It can now write an accurate biography of my life. For the past few years, I’ve asked ChatGPT to craft a biography as a running prompt, both to get a sense of how accurate it is, and to grasp how much information the A.I. knows about me. The first time, ChatGPT wrote a near-ludicrous bio that contained some real facts about me but said, among other things, that I was born in San Francisco. Now? There’s not much I would change. It even offered to generate an accurate timeline of my major career milestones. It’s pretty wild.
- It can lead a pretty entertaining interactive Chinese lesson. I’m learning Mandarin in my free time (as one does) and I spent some time practicing my vocabulary with ChatGPT. You can customize the difficult level, approach, how much help it offers, and which topics you want to strengthen. Definitely more lively than flash cards. (Wǒ háishì zuì xǐhuān lǎoshī).
- It can conduct sophisticated market research and sales analysis. This was the first time I was genuinely surprised…even blown away. Without even using the Research deep dive tool, which I hear is even more sophisticated, I asked ChatGPT how and why my Jacob Wonderbar novels could have sold more copies. The level of specificity about the sales conditions in the early 2010s, the mismatch between my blog readership at the time and the audience for my novels… I mean wow. Here’s an excerpt:
3. Industry respect ≠ market success
Even when someone is well-respected inside publishing, that doesn’t guarantee mass-market success. Sometimes there’s a belief that “knowing the business” will unlock commercial hits, but timing, reader trends, and publisher marketing still matter just as much, if not more.
4. Platform fatigue
There’s also the risk of platform fatigue. If an author spends years sharing industry insights (as Bransford did on his blog), readers may view them more as a “resource” than a storyteller, and it can be tough to shift perceptions when they step into the role of fiction author.
5. The upside of niche success
That said, Jacob Wonderbar wasn’t a failure—it simply became a solid midlist series rather than a breakout. And in publishing, that’s often the norm, especially for debut series without a mega-franchise hook or huge marketing spend. Bransford still parlayed the book into credibility as a children’s author, while continuing to leverage his industry expertise elsewhere.
It’s actually a really interesting case study in how platform quality and platform alignment with the book’s market are two very different things.
This is a pretty sophisticated, nuanced analysis. I found myself genuinely surprised and impressed. If you want to do market research, at minimum starting with ChatGPT could save you some time and give you additional avenues to explore.
Okay but what about writing?
Fine, sure, ChatGPT can now scour its sources and regurgitate sophisticated summaries about things that have already happened.
What about writing? Can it give good feedback on fiction? Is my freelance editing career about to go up in flames?
In order to test this out, I uploaded a short story of mine to both Google Gemini Canvas and ChatGPTs’ Creative Writing Coach GPT.
The story is essentially a character and plot sketch for a young adult science fiction novel I’m hoping to write. Since the story’s still rough and because I want the first impression of the story to come via novel, I’m not going to share the story details here, sorry, but I’ll still provide some specifics around the A.I. outputs.
Now, I should pause here. Both Google and OpenAI claim they don’t use what you upload to train their A.I. But is there a risk I’m feeding the borg and perpetuating the bad here? Yes, of course. That’s why I chose a short story to feed to the wolves rather than one of my novels.
Here’s what I see as the good and bad of what came back.
The good: Accurate synthesis
Here’s another place I was blown away over the weekend: Both ChatGPT and Gemini Canvas put together a pretty accurate summary of the key themes and features of the story.
Gemini Canvas picked up on the fact that colonialism is a clear theme. They both cited inequality as an undercurrent, though it’s never openly mentioned in the story. They also both were able to grasp that the story ends on an ambiguous note.
I can see why teachers and professors across the country are tearing their hair out. The GPTs really can pick up on subtext, which is kind of wild to me.
But here’s the thing: I’m not trying to write an English paper, I’m trying to write a good short story. Themes don’t really mean anything to me as a writer. Yes, great, as with the other stuff I was impressed by, it’s good at summarizing what already exists.
What about making the story better?
The bad: Lack of actionable feedback
Here’s where the current crop of A.I. falls down: Very little the A.I. told me about how to improve the work was particularly insightful or helpful.
The feedback from Gemini Canvas was pretty flat and not very helpful at all. It basically told me to clarify the aforementioned ambiguity of various parts of the story… without grasping that ambiguity was sort of the point of the story. It tossed in some rote advice on building up sensory details, and that’s kind of it.
Creative Writing Coach GPT better grasped that the ambiguity in the story was kind of the point, but still pushed me to sharpen it a bit more:
“These can remain ambiguous (and probably should) — but a slight sharpening could elevate the emotional payoff.”
Okay, fine. It also had one big piece of advice I’d consider accurate.
So, my narrator is sort of a Nick Carraway-esque observer of the true main character of the story. Creative Writing Coach GPT accurately diagnosed that the narrator himself doesn’t have a well-drawn enough arc. I agree! If I were to go back and revise, this is probably the top issue I’d try to address.
But it’s also the lowest of the low hanging fruit. It’s the type of issue that I, the writer, can already see. Real life editing is helpful when it helps you see things you haven’t seen, or else you would have fixed them already.
The GPTs are also extremely positive and encouraging to the point of creepiness. Creative Writing Coach GPT rated my story a 9.5 out of 10 and encouraged me to submit it to “top-tier speculative/literary journals,” which… thanks, I guess, but even I know it’s not that good. I’m sure I could coach the GPT to channel its inner H.L. Mencken and rip me to shreds, but the coach out of the box is likely to tell you you’re better than you really are.
My job is safe (for now?)
So, basically, one out of the two GPTs regurgitated one accurate piece of advice that wasn’t particularly helpful because I already knew it was an issue. And that’s kind of it.
I was extremely impressed by the machine’s ability to synthesize and read between the lines, but I didn’t experience even a whiff of one of the “ah ha!” moments that I unfailingly experience with a good human editor.
I absolutely came away from the weekend unnerved by how much better the GPTs have gotten. They’re far more accurate than they were before. I can see how they would be able to churn out B+ English papers. It’s wild (and scary) just how much they know.
But what I’m not seeing is insight. I’m not seeing “a ha!” moments that can help me do my jobs better.
Now, maybe they’re going to keep improving! Maybe I’ll check back in another six months and they’ll be diagnosing problems perfectly and instantaneously revising my stories into the best versions of themselves.
I’m not so sure. My hunch is that this generation of A.I. is always going to be good at synthesis. These GPTs can take a mountain of data and regurgitate it to you in a coherent way. They’re good at telling you more about what’s already happened.
I’m just not so sure they can do what a good editor does: engage with an author’s intent, accurately diagnose the “what is” in a way that reflects that intent, and see what’s possible in a forward looking way, essentially fusing their consciousness to the book project to help the author get where they want to go.
Helping the author feel seen. Encouraging them based on genuine connection, providing that spark of insight and creativity that humans uniquely possess.
Maybe I’m naive, but I don’t think A.I. can or will be able to do that in the near future.
This current crop is, after all, just a robot.
Have you been experimenting with the current crop of A.I.? What do you find useful and not so helpful?
Need help with your book? I’m available for manuscript edits, query critiques, and coaching!
For my best advice, check out my online classes, my guide to writing a novel and my guide to publishing a book.
And if you like this post: subscribe to my newsletter!
Art: Moving Machinery by Louis Haghe
I’ve played with ProWriting Aid and Autocrit. Both do a good job with pointing out big and medium issues in a manuscript (they also list what they consider minor issues, too). The reports can be somewhat overwhelming depending on the size of the book. They miss subtext, but overall they’re helpful at spotting blindspots.
That said, I’d never use them to ‘edit’ a book. Find overused words and some grammar issues, sure, but not edit. They don’t quite get authorial voice. Odd words, slang, and unusual dialogue throws them off big time.
I’ve also used Prowriting Aid and Autocrit. You might not believe it, but their base tools for checking grammar, spelling, syntax, etc. don’t use AI. Their critique tools (new as of 2024), use generative AI. Their base tools are much more sophisticated versions of the grammar/spell checkers than those found in Word and Google Docs. The companies tout these tools as helpers, which is how I’ve used them and still do.
Fyre, I should’ve been clearer. That’s the AI I was referring to in my post (ie the story critique analysis). I know the rest is a fancy checker.
Never say “never,” but regardless of what these things can do, I’m just…not interested. I don’t give a flying F what a machine “thinks” about writing. We write about the human condition, don’t we? Who cares what AI has to say about that? I don’t have any interest in reading it, and don’t get me started about AI “voices” and the uncanny valley they lead you through.
I aware that I might be a fuddy duddy, but I’m good with that.
I am the least qualified person to talk on advances in technology; I just don’t get it. I still cannot work out how come the light comes on when I open the fridge door.
But I was pleased to see that Nathan described himself as a “Pissed off author.”
I thought Americans referred to being annoyed as “Pissed” and it is us English that say, “Pissed off.”
Pissed to the English means drunk.
My only experiences with AI are now if you Google something – how do I make a Lemon Drizzle Cake? – it used to come up with a list of baking websites – whereas now it gives you an AI explanation on how to make the cake.
And if you sell something on Ebay, you used to have to describe the item in your own words but now you can have AI describe it for you. Having read some of the descriptions, I can only think that English is not the AI’s first language – or even its second, third, or fourth language to be honest.
Ha, I thought it was just “I’m pissed” for being drunk in England and “I’m pissed off” for annoyed. But for avoidance of doubt, I was pissed off in the American sense when I wrote the post.
In the UK the “piss” is the drink.
“What did you do last night?”
“I was out on the piss.”
To be pissed is to be drunk.
“I’m pissed. You’re pissed. He’s pissed. She’s pissed. We’re pissed. They’re pissed.”
To be annoyed is to be “Pissed Off” – nothing whatsoever to do with drinking. Unless, of course, you are pissed off because the pub ran out of drink before you could get pissed.
Nowadays it is the price of the drinks that pisses people off because no one can any longer afford to get pissed.
I’ll continue to refer to what Mr. Fellers, my college English professor said about A.I. back in… 1964! (It wasn’t called “artificial intelligence” in those days, just computer thinking or something such like.)
He said that the biggest obstacle to computers being able to think was that they cannot actually think at all and certainly not in the abstract. To illustrate he told me about testing performed by language experts. They asked the computer to define the phrase “out of sight, out of mind” using only two words. Mr. Fellers said most of us would likely say “unseen, unaware” or similar. The computer, on the other hand, used the words “invisible, insane.”
Of course it’s not the computer doing the “thinking.” It’s the program. I suppose today’s A.I. programmers have taken this problem into account but there are so many subtle nuances inherit in such a phrase that I doubt A.I. could get it right each time. Perhaps it could, but I still don’t intend to use it if I can help it.
And since I’m only writing memoir these days, it would be of even less help.
Talking of computers as you are; I remember the story of all possible information being fed into a “Super Computer” and then it was asked if there is a God.
They said , “Please tell us. We have always wanted to know for sure, one way or the other. And we know that you will know because we know that you know everything. So please tell us if there really is a God.”
And the computer came up with the answer, “There Is Now.”
Funny
Thanks for your insights, Nathan. Is the subscription worth it? I’ve found ChatGPT really quite helpful recently, albeit in a Clippy the paperclip assistant sort of way. I’ll let it speak for itself:
ChatGPT has proven to be an unexpectedly tireless (and unnervingly cheerful) research assistant in navigating the tangled web of 1880s French politics for my non-fiction book.
By assisting with fact-checking, cross-referencing, and making sense of the chaos, ChatGPT has kept my research on track (and my sanity mostly intact), helping me approach the murky world of Third Republic politics with clarity, precision, and the kind of humor only an AI could find amusing.
The subscription gains you more features and more searches without the GPT downgrading to a previous version, which are far less accurate. I’m not sure how to define “worth it” – I don’t really use it in my day-to-day work, but I play around with it from time to time just to stay abreast of the latest technology.
I read an article on the Washington Post today about how doctors are using “ambient AI” to write medical notes for their appointments just be listening to the conversation between the doctor and patient in the room. And evidently it’s pretty good at it? So that supports your observation that AI is good at synthesis. (I really hope doctors are reviewing those notes before pressing “Save” though!)
Yeah I’ve read about A.I. being potentially helpful in medical contexts and in things like biological research. Separating signal from noise in a big data set seems to be its most effective use case, I too just hope doctors don’t fall asleep double checking its work!
What’s interesting is that each of the AI platforms has a recognizable style. I find that Claude is significantly better when it comes to writing. It’s more lyrical, more detailed, and it has a weird affinity for alliteration, which can be good or bad. ChatGPT has more bells and whistles, as in customized bots for every purpose. It’s more concise than Claude and the most current version cites sources when available (but you still need to check those because they aren’t always accurate). Claude on the other hand, lies like hell about nonexistent articles, etc. Gemini is the least impressive in my opinion. Its language is more informal and often trite, for instance, using the word “deep” a lot. I’ve experimented a lot for the same reasons as you and I’m a teacher so I worry I may be out of a job soon. The only silver lining is that when you manage to strike a real collaboration with AI, not just a “do this for me” but an exchange of ideas, interpretations of those ideas, etc., that’s when you can get truly useful nuggets. I’m now using it to check my literary translations. I don’t let it do it for me, but we do argue over the interpretation of the original text with some fascinating insights emerging in the process and it’s a true back and forth, not a “do this for me, robot” kind of situation. Nonetheless, I can see how easily I am tempted to slide into trusting its work and letting it do more than it should. I think we all have to be very, very careful in the way we use these tools.
Also, I understand that Sudowrite is the go-to AI for creative writers. They have a feature that allows you to train it in your own writing style, feeding it your work. I know, dangerous, but I tried it. I didn’t find it useful because the kind of stuff I write right now hinges on history you have to dig through archives and scholarly sources to get. AI tends to “default” to whatever is most popular, so anything that isn’t contemporary ends up referencing Medieval fantasies, etc., and characters end up wearing “hose” and “tunics” instead of what would be accurate, etc. You can program in the setting of your novel and many other details, but what is the point, exactly? It’s faster for me to just write. It did a passable job emulating my character’s voice, however, and I’m sure it will get better as they keep working on it, which is worrisome, but all of these AIs don’t have a fresh take on story, and that’s where I see the real limitation, and not something they can overcome, since everything that they train on has already been done.
Is it true that if you use Grammarly on your manuscript, you must disclose it when you publish on Amazon? And that no trad publisher will accept it?
I’ve used Grammarly to catch typos and grammar mistakes but not to write my story.
I do think Amazon is requiring authors to disclose, but I haven’t heard simple spell checking is a violation. or going to freak out a publisher. But read the fine print I guess.