ChatGPT, The Worst Summer Intern Ever

Back when I used to work in the pharma industry, I had the opportunity to hire summer interns. This was a long time ago, long enough that the fresh-faced college students who applied for the gig are probably now creeping up to retirement age. The idea, as I understood it, was to get someone to help me with my project, which at the time was standing up a distributed data capture system with a large number of nodes all running custom software that I wrote, reporting back to a central server running more of my code. It was more work than I could manage on my own, so management thought they’d take mercy on me and get me some help.

The experience didn’t turn out quite like I expected. The interns were both great kids, very smart, and I learned a lot from them. But two months is a very tight timeframe, and getting them up to speed took up most of that time. Add in the fact that they were expected to do a presentation on their specific project at the end of the summer, and the whole thing ended up being a lot more work for me than if I had just done the whole project myself.

I thought about my brief experience with interns recently with a project I needed a little help on. It’s nothing that hiring anyone would make sense to do, but still, having someone to outsource specific jobs to would be a blessing, especially now that it’s summer and there’s so much else to do. But this is the future, and the expertise and the combined wisdom of the Internet are but a few keystrokes away, right? Well, maybe, but as you’ll see, even the power of large language models has its limit, and trying to loop ChatGPT in as a low-effort summer intern leaves a lot to be desired.

The specific task I needed help with was simple: a latch on the automatic door of my chicken coop. The details of why I need this are better left unsaid, but I will say that the latch is my attempt to deal with the cruel equation “11 Chickens + 1 Raccoon = 8 Chickens.” The first part for the latch that I had in mind was a simple bracket — something with two arms rising from a base that could be attached to the coop door (a sheet of 2-mm aluminum). This bracket will support a spring-loaded arm to engage a crossbar in the frame of the coop door, and hopefully prevent anything with clever little claws and a taste for take-out chicken from working under the lower edge of the sliding door and getting into the coop.

From my experience with Fusion 360, I knew it would be pretty easy to whip up a couple of sketches, extrude the arms up from the base, drill some holes, and tidy things up with the proper chamfers and fillets. The trouble with that is that I moved to Linux quite a few years ago, so Fusion 360 isn’t really a viable option. Yes, I know there are workarounds using Wine or VMs, but I’m also not interested in playing in Autodesk’s cloud. Of course, there’s FreeCAD, too; in fact, just prior to this project I had started fiddling with it, but I’m still in the early part of the learning curve and not terribly productive with it yet.

That final point gets to the crux of the matter, though. When I was learning Fusion 360, I found that it quickly became the case that perfecting the 3D model was mentally more important to me than what I was designing. I would spend hours getting everything just right, only to find out that something didn’t quite line up, or I had some constraint wrong, and I’d have to start all over again. I have every expectation that my mind will treat FreeCAD the exact same way, not as a means to an end but as the end itself. That’s a slippery slope, especially when there’s so much else I have to do.

This is when I had the idea to outsource my design to someone — or something — else. Credit where it’s due; I got the idea to enlist ChatGPT after seeing [ROFLhoff]’s Hackaday.io project about repairing a broken Toyota speedometer. The project required a knob to easily spin the speed sensor that drives the speedo for bench testing. With just a brief description of the part needed, [ROFLhoff] was able to prompt ChatGPT to generate OpenSCAD code to create a model of the knob.

I thought that was pretty slick and decided to do the same. The bracket seemed like the best place to start, so I gave ChatGPT the following prompt:

I know I wasn’t terribly specific — I didn’t say how far in from the corners the mounting holes should be, for instance — but I figured that was something I could tweak later. And besides, when you hand a job over to a subordinate, you should be able to just vaguely describe what you want, and let them run with it. Right?

Maybe not, because here’s what I got back from ChatGPT:

Now, it’s important to note that at the time, I had even less experience with OpenSCAD than I did with FreeCAD. And by less, I mean zero — I’d never used OpenSCAD before. So the code ChatGPT created didn’t mean all that much to me. But it sure looked reasonable, especially to someone who used to code for a living — I could see all the parameters up at the top like constants, the calculations used to figure out where to put the arms, and the creation of the cylindrical holes. But I had no idea what translate was, and I couldn’t really see how the holes would be punched through the base.

Strictly speaking, though, that wasn’t my problem. In the ideal world, I would just copy the AI-generated code into OpenSCAD and get a 3D model ripe for tweaking, right? Perhaps, but in the reality-based world, I had no such luck. The code ChatGPT generated didn’t work in OpenSCAD — it just threw a syntax error on line 21. I told ChatGPT about the error and asked to have it fixed, but the code that came back was essentially the same and threw the same error. Disappointing.

At this point, I figured I had two choices: either keep fiddling with the ChatGPT prompt, or start learning enough OpenSCAD to figure out what ChatGPT was doing wrong. All things considered, the latter seemed like the smarter route, so that’s the way I went. It was quickly apparent that ChatGPT didn’t have the slightest idea how to program OpenSCAD — for example, note the complete lack of difference() commands, which are needed to make the holes, from the AI-generated code. This led me down the very rabbit hole I was trying to avoid, and I spent a few hours learning enough OpenSCAD to do the job myself:

Granted, my OpenSCAD code has both the bracket and the latch, but even accounting for the added complexity, I feel like ChatGPT’s first pass at doing what I wanted was half-assed at best. This is about the point where I realized that ChatGPT was behaving like an earnest and eager-to-please but somewhat clueless summer intern. Given the same instructions, such an intern might go off and spend a few hours Googling how to do the job, maybe hitting StackExchange or r/openscad for some help, and coming up with some code that they figured would do the job. That they didn’t even bother to see if OpenSCAD would interpret the code would be grounds for a Stern Talking-To™ with a meatspace intern; but with a virtual intern, what recourse did I have?

Honestly, although I didn’t get the magic help from ChatGPT that I was hoping for, this experiment in AI-assisted design wasn’t a complete loss. The original code may have been crap, but it at least gave me the impression that OpenSCAD programming wasn’t entirely foreign territory for me; flawed though it was, I could see that it’s just a combination of coding and 3D modeling, and intuitive enough to take a stab at. So now I know just enough OpenSCAD to be dangerous, all thanks to ChatGPT — the worst summer intern ever.

And as for the coop door latch, that project went in an entirely different direction once I realized that I could make the bracket and the latch from bent sheet aluminum — which forced me to learn not only FreeCAD but the Sheet Metal workbench as well. So ChatGPT actually forced me down two rabbit holes I’d hoped to avoid.