The hallucinating ghost in the machine

My personal and professional opinion on gen-AI can mostly be summed up with "ugh".

I don't use gen-AI to brainstorm, write, or edit. I don't trust it for research purposes1, on account of all the well-documented cases where it just makes shit up. I will always pay artists rather than generate AI "art".

No. No, thank you. Ugh.

I will admit that telling the chatbot to "respond to this scenario in the manner of Murderbot" is fun. But there are better uses for my time, and online media already sucks enough of my attention away without me having conversations with an imaginary friend. Actually, if you must interact with AI, telling it beforehand to do so as Murderbot cuts down on the irritating cheery sycophancy by a lot. Do recommend!

(I don't really need to talk about the potential psychological and cognitive dangers, the massive rise in cheating in our educational institutions2, the part where LLMs scraped all of my books without permission, acknowledgement, or payment, or the horrendous environmental impacts, because people have already done that.)

However, the other day I thought I had finally found a good use-case for Gemini!

Due to some behind-the-scenes stuff, the rights for my witchy romance novellas Bespoke & Bespelled and Savory & Supernatural have returned to me. This is great for you, because now they can be cheaper, and you can be sure that I am getting 60-70% of the royalties. This makes you a worthy patron of the arts, like a Renaissance Italian banker, hopefully without the assassinations and Papal election rigging.

Because the final files were pdfs, and my ebook formatter (Atticus) only works with docx, I converted the pdfs to docx, whereupon the formatting promptly and predictably broke. The big problems were lots of weird line/paragraph breaks, incorrect dialogue paragraphing, and a bunch of hyphenated words where the text had been justified.

Going through line by line was possible, but tedious, and would take a long time, and I wanted to get those books back out pronto. Also, I would absolutely miss something. So, possibly swayed by AI: Your Automatic Functions Hero rhetoric, I gave Gemini very specific instructions, tested it a number of times, and then fed chapters through to unfuck the formatting.

Gemini did, mostly, unfuck the formatting.

It also, despite frequent instruction not to, did the following:

  • Removed all my italics for emphasis, and about half of the italics for book/movie titles.
  • Added italics for Māori words (yuppp) and internal monologue, apparently at whim.
  • Twelve chapters in, randomly decided to censor all my swear words - "fuck" became "f***", "shit" became "s***", and "Oh, God," became "Oh, gosh."

The most egregious error? It just made shit up.

The original document had a text message conversation about Kingston trying to choose an outfit for his second date, which had been rendered graphically - little text bubbles on the page. Gemini evidently couldn't read this conversation. Instead of saying "Ooops, I can't do this", or alerting me to the gap, Gemini invented what it thought the conversation should be.

This new version of that conversation started with a link to a Pinterest board with some outfit choices, and then developed into an at-first-glance lucid, but in context nonsensical conversation.

The real dialogue puts Kingston in pink for his date, because a) it looks good with his dark skin, and b) I am deliberately signaling that this is not a guy who concerns himself with rigid gender stereotyping. But I suspect that Gemini has internalised "blue is for boys", and so, in this weird Gemini version, my characters blithely and blandly suggest he wears dark blue.

I have been a high school teacher for over a decade, and I am accustomed to a) repeating myself and b) rephrasing instructions in ways that will hopefully improve clarity. But AI, unlike teenagers, can't learn in the moment. It doesn't have any notion of truth, and thus, like a confident narcissist, lies constantly and convincingly, unless you are already knowledgeable in the subject.

No. No, thank you. Ugh.

Anyway, should you wish to read my delightful witchy novellas Bespoke & Bespelled and Savory & Supernatural, featuring magical mischief, sweet romance, and all the glamour (??) of the movies, check them out! I made them up, with my very own brain.

Available at all your favourite e-retailers at just 2.99 USD!

1 ALSO. I love research! I love noodling around and finding weird things! I do not want a glib summary, especially when they're so often wrong!

2 I object to cheating on the grounds of justice but also, holy shit, you can't outsource learning. I'm sure we all know people with degrees who scammed and scraped their way through university, but having whole cohorts of them seems like a slightly bad idea. We can't buy curiosity. We can't fake process. We have to practise thinking. The pattern-recognition robot can't do it for us, because it doesn't think.