AI & Software Development: Should We Be Scared?

It was November of last year when I first got genuinely worried that AI was coming for my job.

I've been consulting for a long time — helping companies get products into the field — and I'd played with these tools for years without being all that impressed. Then I tried a new model and watched it one-shot a working piece of firmware. Basic firmware, sure, but real firmware for a real product, from a spec. That shocked me. It felt like a turning point, where the thing finally crossed from "neat toy" into "actually useful."

So I did the only sensible thing: I dove in. I've spent the last nine months driving these tools hard on real customer work — mostly firmware support, which is emphatically not what they're best at. Somewhere north of 25 billion tokens in the first six months, across several providers. (I've written about how that token usage breaks down — and what it actually costs.)

Here's where I landed: I'm not worried anymore. Not even a little.

The two camps

You've probably met both. One camp is all-in on the hype — every white-collar job is going away, and they're lost in the version numbers, Sonnet versus Opus, GPT-5.5 versus 5.4. The other camp thinks it's all nonsense and smoke. The reality, like most things, sits somewhere in the middle. I've been building products since the late 90s, and as far as I can tell this is just another tool everyone's going to have to learn. It isn't the end of embedded firmware. It's a new set of tools and practices, same as every shift before it.

What it actually changes day to day

Let me give you a concrete example. I had a customer — I can't get into the details — who needed a way to author a pile of configuration files. JSON, basically. Normally I'd spend several days building an interface to create and edit those, with all the loading and validation that goes with it. Once I'd specced it out, the AI built a fully usable version in about two hours. Then I spent maybe five minutes debugging the result. Days of work, offloaded.

That's the shape of the win. AI is exceptional at boilerplate, scaffolding, and the tedious, run-of-the-mill stuff we all have to write to get a product out the door. It thinks it's good at the rest, too — but that's where it gets you. The trick is to keep your system broken into simple pieces. Lots of small, well-defined parts is exactly what it handles well.

What it doesn't change

It's not good at architecture. Its judgment isn't there. And in firmware specifically, it doesn't understand the hardware. It can parrot a datasheet back at you — it's a great search tool for a datasheet — but that's not the same as understanding how your system actually has to work. The theory of operation of the whole product is something your engineers hold in their heads in a way the model simply doesn't.

The thing that bites people is how confident it is when it's flat wrong. It doesn't understand data faults. It doesn't understand interrupt priorities, or hard real-time deadlines. Give it room and it'll happily pull in a pile of external libraries and tell you, with total confidence, that that's the right way to do it.

So treat it as what it is: a tool. It doesn't do the work for you. If you're a good engineer, it'll make you a better one. What it won't do is turn someone with no engineering background into an engineer.

If your entire role is translating requirements in English to source code in such an unconstrained environment that efficiency, timing, and resource limitations don't matter... then yes, your job just disappeared.

Where I think it has real, underrated value is a little different. It can take your product people — folks who are technically sharp but aren't engineers — and let them interact with the code. With the right tooling, a product person can make a request, try a configuration change, and see the result without learning to compile the whole project. That's powerful.

This can enable more people trying out things in firmware to make better products. Everyone wants that!

So — should you be scared?

No. Your job isn't going anywhere. The honest exception is the company that decides it wants to learn to survive without engineers — that's a choice some will make, and it's a different conversation. For the engineers themselves: you're safe. What's actually being asked of you is to learn a new kind of tool.

These tools change the job. Like all tool changes, some people will like them more than others. If you enjoyed translating English specifications to source code, your job will change in ways you won't enjoy.

If you enjoyed solving hard problems and building cool things — these tools will allow you to get more done. They enable certain types of software to be developed that would have cost too much in the past.

We've all had a project manager or analyst ask for some mundane change... if you set up your systems correctly, they can try these things out independently of you!

This does require a lot more setup than a project used to take — since you need more guardrails for these agents. Even including that, velocities are up all over. And companies that neglect engineering and solely rely on the models are going to fail in spectacular ways.

I believe team makeup is going to change. If you have a very small team, it means your next hire may be a product manager instead of an engineer. It means your engineers are doing more system and quality work than just typing code all day.

I'll admit I struggled with it at first, mostly because I started treating it like a junior engineer who worked for me. That's the wrong mental model — but that's another essay.

Where to start

If you make firmware products and you want to put your feet in the AI water without betting the product on it:

Build out a really nice diagnostic tool for your embedded system.
Stand up a clean cloud build framework (CI is amazing for all software development).
Have it write unit tests against your existing firmware (sorely missing in most firmware).
Use it for rubber-duck debugging — talking a problem through out loud is something it's genuinely good at.

None of that is about replacing people. It's about getting more done, being more aggressive with timelines, and shipping higher-quality products in less time. That's the whole game.