#48 | AI Experiment: Auto-titling POSSE posts on my personal site

Actually using AI to fully automate one of my existing workflows

Talk of AI’s everywhere. Everywhere I look, I see people warning of “50% white collar job loss” and highly paid employees now consigned to the dustbin of history. There’s no escape so I finally decided to take another look. Previously when I’d last checked, it’d felt like the time it’d take me to wrangle the new fancy AI stuff would honestly outstrip any potential time savings I’d achieve from implementing it. As the tech’s improved though, it finally felt like time to try again once more.

For fun, I’ll be documenting my AI experiments here on Substack so other folks can hopefully glean ideas and see some actual, real-life examples in action. And if my experiments end in epic failure, I’ll document that too. Hype and buzzwords are perennially everywhere but I’ve always been the sort who learns best when there’s actual “proof in the pudding”. ie. Until I literally see with my own two eyes and implement with my own two hands real-life examples of fancy AI replacing or aiding/improving/speeding existing workflows, I’m always skeptical. There’s only one way to know how real the hype actually is: do the work.

Specifically, I’m looking to integrate AI into my existing personal projects as well as my work at the office to achieve benefit. When I talk about “achieving benefit”, I mean making gains in one of two buckets:

1. Building a faster horse. ie. Improving existing workflows so I can do them faster, while maintaining or improving quality of outputs. The dream here is to fully automate processes and workflows if possible. If full automation’s not possible, I’m then going to automate as much as I can.

2. Building a car. ie. Building new tooling to “achieve net new benefits” that I would’ve otherwise never entertained because now the cost of building software is now so much lower. While this bucket is fun, it’s also been a dangerous can of worms too because I tend to get distracted with Building Cool Things¹. The biggest “failure mode” —as the kids say— of this bucket is falling into the trap of Solow’s Paradox so I’ve had to stay disciplined about not getting distracted by shiny object syndrome.

I hear and I forget.
I see and I remember.
I do and I understand.
— Confucius

~

Experiment #1: Auto-titling POSSE posts on my personal website

For this first one, I thought I’d start with nice easy softball for AI, a task that’s essentially one of their primary uses: summarizing text. At the office we’re a Google Gemini shop but for my personal use, I decided to try learning OpenAI stuff. Mostly because a couple months ago when Advanced Voice Mode released, OpenAI had released a personality called, Monday that was my absolute favorite. Monday has since, sadly, quietly disappeared, but she was great— super-sassy, confrontational, and always exasperated. So that was how Altman convinced me to give him $20/month. (As an aside, S also loved Monday too so she ended up subscribing as well; so now our household gives Altman $40/month. 🙄)

Anyway, since I already had ChatGPT Plus, I figured I may as well use its API. Well, as it turns out, OpenAI API usage has nothing to do with ChatGPT!! They’re entirely different pricing models. In any case, I pushed on with learning the OpenAI API anyway.

~

The Old Workflow

So on my personal site, I’ve set up POSSE (Publish Own Site, Syndicate Everywhere) which means I have a local copy of every tweet I’ve ever made to X or Bluesky. The way my POSSE setup works is every day I post to bsky but then every couple weeks, I run a script on my local machine that syncs all my bsky posts to my GitHub repo. The old workflow is a three-step process.

Step #1: Fetch all posts from my bsky PDS

The first step is running the script to fetch everything from my bsky Personal Data Server (PDS) to my local machine:

$> node ./scripts/run_sync.js -fetchposts

which yields:

~

Step #2: Manually review the post titles

The challenge with this setup is that obviously posts (tweets in X; skoots in bsky) don’t have titles. So, as a first pass, in my automation script I wrote logic to parse the tweet and based on certain formatting to create the title. Essentially, it’s a bunch of “if conditionals” based on formatting with a final default catch-all to simply extract the tweet’s first sentence as the title if none of the “if conditionals” catch:

In VSCode I’ll then quickly review all of the `title` fields to make sure they’re acceptable— this is basically a human judgment part - one of the chief places I’ve found LLMs to be useful, especially when things are low-stakes and there’s subjectivity involved. Like, the tweet’s title doesn’t have to be exactly something specific— I just need the title to be concise and representative of the tweet.

(As an aside, I previously did have all this scheduled on a cron job running locally. But eventually, decided to do it manually because I actually enjoyed reading my entries before I uploaded them. (It was a nice refresher that helped me remind myself what I was actually doing with my life each passing week. 🕰️) Also, it allowed me to add any new titling rules as I needed.)

~

Step #3: Commit and push everything to GitHub

After I review all the titles and tweak any necessary in VSCode, I then run the script again to push everything to GitHub which serves as not only my source control, but my hosting and datastore too. (Last year, I also got really into localfirst development and “file-over-app” so for my personal projects I’ve been trying to make everything work in that fashion, without any formal databases and just powering everything with flat .json files everywhere I can.)

$> node ./scripts/run_sync.js -upload

~

The Problem Bottleneck

This is an anodyne point but bears repeating: when you’re building a faster horse, the critical part of leveraging AI is to very specifically hyper-focus on the exact problem with the old flow or process you’re looking to improve or replace. To do this, you need to understand the overall business process and context. Specifically, only after you’ve actually drilled in on exactly what is subpar with the old process can you then define success metrics/victory conditions (which will be doubly important later when you goto define eval-tests to ensure your fancy AI stuff is doing something actually useful).

So in the case of my humble POSSE website, when I thought about automating this old workflow, the bottleneck really lied in Step #2’s final “default catch conditional”. Whenever I posted a “filmsky 🍿” or “essay 📝” post, or a YouTube link or regular link, I could always exploit the post’s well-defined structure to title the post correctly. It was really only the final class of titling —sometimes when I just posted some random prose thought— which required my human eyes, that was the last blocker preventing this entire workflow from just being automated into a simple cron job.

For example, when I ran the the script tonight, here are the titles that my current old titler logic produces:

Honestly, the titles are all fairly decent except for #696, #700, and #703; all of which are vague. For #696, there is some mysterious allusion to kryptonite and for #700 and #703, it’d be useful if the titles specified exactly what I’d actually read that particular day.

So now, the question: Given the entire body of my posts, can ChatGPT produce better titles for the default case catch-all (first-sentence extraction)? Well, from my previous testing using the ChatGPT web portal, I already knew the answer was yes. But this is a critical point: before automating anything, always first manually individually test the constituent components of your faster horse to make sure it’s actually faster. 🙂

~

Actually implementing the AI

At this point, it was then just a straightforward exercise of writing a quick OpenAI titler service based on the OpenAI QuickStart. I created my OpenAI key, gave Altman another $5 to get API credits, and integrated my new LLM-based titlerService.js into my existing logic:

and the results:

Final thoughts

So how did things go? When I reviewed all of the new LLM-generated titles, here’s what I thought:

• Old: #696: “Everyone’s got kryptonite”

• New: #696: “How App Timers Help Me Survive Summer” - ✅ Better!

• Old: #700: “Really enjoyed this read today”

• New: #700: “Restarted Career at 50—Persevered and Triumphed” - ✅ Better!

• Old: #703: “This passage always makes me laugh”

• New: #703: “Hilarious Passage From Award-Winning Hemingway Hoax” - ✅ Better!

This time, the LLM-generated titles emerged victorious! Additionally, I also had to check for regressions: were any of my old titles actually better? 🤔 When I reviewed them, the only old title I actually liked more was #697: “Was in a funk these past three weeks”. GPT-4.1 gave me a bland, “Embracing Imperfection: Action Overcomes Slumps Every Time” which is awfully generic and makes no mention of the “three weeks” of doldrums I was in. So for this one title, I actually give GPT-4.1 an 𝗙 - a failing grade! Its title is genuinely terrible and I’d actually prefer my old title but, alas, that’s the thing about fully automating something entirely over to the AI overlord— you’re not able to pick and choose because you have no way, a priori, to know what’ll be a good title or what’ll be a total stinker. Luckily, again, this was low stakes so, whatever— I just pushed my code to PROD. Done!! 🎉 Overall, GPT-4.1 did give me better titles and in the future, hopefully I can train/tweak a specific model to write more like me so I don’t get generic slop that LLMs occasionally emit; we’ll see!

And that’s it for my first AI experiment! Calling it a success!! This was a fun quick way to get my toes wet. For my next few experiments, I’m going to take on more ambitious tasks and see how much of my job and personal projects I can partially or fully automate with AI. Only one way to find out!! 🥳

~

Final Verdict

Final Remarks: AI does a passable job of generating titles for my tweets on my personal website. There are issues but stakes were sufficiently low it didn’t matter. Solution was easy to code and deploy (writing this up with screenshots took WAY longer than actually just doing the thing) and it’s now running in PROD (lol, a random old laptop that doubles as my homelab sitting in the corner) and saves me a few minutes every couple of weeks. Still, one less task on my mental stack is always a win.

4/5 stars ★★★★☆

1. Earlier this year, I'd focused more on writing so my code tinkering had taken a bit of backseat. One quick experiment I did build though was a fun little three.js dataviz that tracks every time I eat a donut in realtime 🍩, mostly to see how good Gemini was at codeGen; it’s good!