Why “Show Me Your Harness” Beats Any Coding Test: Adam Witanowski, AI Architect

Most technical interviews still ask a candidate to write a function on a whiteboard. Adam Witanowski says that question is already obsolete, and what should replace it says everything about how AI is changing who gets hired.

In this episode of Building Tech Teams, James MacDonald sits down with Adam Witanowski, AI Architect at nib, where he has spent the past several years building the insurer’s AI engineering practice from the ground up. They get into why most enterprise AI projects underperform (nobody is measuring the right things), what “back pressure” means for AI-generated code, why Australian salaries can’t compete with the US for senior AI talent, and why the technical interview needs a full rebuild.

It’s a hands-on, practitioner’s take on what’s actually happening inside enterprise AI teams, not the conference-keynote version. They cover the real dollar gap between hiring an AI engineer in Australia versus the US, why “the whiteboard should come back” as an interview tool, how Adam built his own AI agent to interview for jobs on his behalf, and the testing harness his team uses to check whether AI-generated work is actually any good.

ABOUT THE GUEST

Adam Witanowski is an AI Architect at nib in Sydney, where he has built Prometheus, an integrated suite of agentic AI tools now running in production across the business. He works at the intersection of senior engineering and AI leadership, and has spoken publicly on enterprise AI engineering discipline, including harness design, testing and evals, and AI governance.

Building Tech Teams is produced by DayOne.

Listen to the episode

Transcript

Auto-transcribed with AI.

Adam Wintonowski [0:00]: AI agents are good at generating code. They’ll do that happily all day long. But is it good code for your environment, for your context? Backpressure is the only thing which is going to shape that.

We’ve taken a bunch of learner drivers, and then given them an F1 car and said, drive around the city, have fun. No — you need to start to think about Jan’s whole role. And that’s what the startups are doing, and that’s what enterprises need to do: start to re-engineer whole processes, not thin slices. And until they do that, they’re going to be really hamstringing themselves. So I think the first thing that any organisation that wants to go down this route of increasing their AI capability is, they need to figure out how to measure. Because what you don’t measure, you can’t change.

James MacDonald [0:40]: I think that’s the best line you’ve said today.

That’s Adam Wintonowski. In his previous role, he was AI architect at NIB, where he helped build their AI delivery capability. It was 18 systems, a knowledge graph across 3,600 repos, AI agents shipping production code inside a regulated health insurer. There aren’t many people in Australia that have done that — AI in production at scale, highly regulated.

But more recently — and this was the reason I wanted him on the podcast — he’s been on the job market, and he’s in the process with nine different companies. At the time of recording, he had seven potential offers. That’s what I wanted to quiz him on. What does a good interview look like for this senior AI talent? How do companies actually interview people that are more technical than them, further along the AI journey than them?

As well as that, we discussed what backpressure is, why your agents are writing code that doesn’t fit your business without it, why most companies see no ROI from AI, and why Adam says that’s a measurement problem, not a technology problem. And what senior AI talent actually costs in Australia at the moment — real numbers on the offers that he’s been receiving.

I interview AI engineers for a living, and I still learned a lot from Adam. If you lead a tech team, you’re planning your first senior AI hire, or looking to upskill your interview game, looking for this senior AI talent — this is the episode to steal some tips from. I’m James MacDonald. This is Building Tech Teams. Here’s Adam.

Welcome to another episode of Building Tech Teams. On today’s episode, we have Adam Wintonowski. Did I get that? Wintonowski.

Adam Wintonowski [2:06]: Wintonowski is fine. You were close enough.

James MacDonald [2:20]: Mate, most recently AI architect at NIB. You’re one of the few people I know who’s actually built AI in production in enterprise. Talk me through what that journey was like.

Adam Wintonowski [2:29]: Yeah. So that started probably a little over 12 months ago. I was brought in to build the AI SDLC capability within health insurer NIB. And obviously a highly regulated environment, FSI.

And FSI loves to talk about how hard it is to do things in their space because of the regulation. And 14 months ago, there really wasn’t — and there still isn’t, in a lot of respects — the tools you need to be able to effectively build an AI SDLC. Like, really genuinely build it for a team of 300 engineers plus, the tools just don’t exist. I was really looking at it from scratch and thinking, what does re-engineering what it looked like to build software at scale in enterprise, or really anywhere, look like?

So it started with dismantling the key components of the SDLC, came up with what are the appropriate backpressure that you would need to enable AI agents to build effectively, and the various tools that you would need throughout that full lifecycle — including cyber, architecture, QA, actually building the thing, shipping the thing. And then it started to morph later on into a conversation of, what does it look like to enable org slop, where you’re going to have non-engineers building and shipping as well. So it was a fairly wide scope.

And that’s something that I think NIB has done well in the past — really think three or four steps ahead. They were first on the cloud for insurers in Australia, and really had quite a mature data space. So they were really quite well set up to move into this sort of AI journey.

So that started with 11 systems that ended up bloating out to about 18, each fairly large in their own right. Giving everything from the ability to ship, which was the smallest piece, through to giving the AI agents a 360-degree view of the environment around them, through an application called SkyNet — which ingested all 3,600 repos, created a knowledge graph and a vectorisation and a code intelligence system that could be queried, and giving the AI agents a view of upstream and downstream impact of changes. So from the really complex down to the really quite basic, you had to think of it all the way through.

James MacDonald [5:12]: And you mentioned backpressure before, and I know you’ve mentioned that online before. Talk to me about that, because I think it’s come up in some of the interviews as well — just talk to me about the backpressure part of what you just mentioned.

Adam Wintonowski [5:23]: Right. So backpressure comes out of fluid dynamics. So you go to the pump, fill up your car with petrol, and it knows to stop because of the backpressure of the fluid. Engineering has a similar concept, and engineers would be familiar with this even if they’re not familiar with the term. So this is things like testing harnesses, gold standards in data.

And so things like — is the agent able to test its own work? Is it able to get signal from CI, which triggers a piece of rework within the system? Now, most people, when they build a harness — or most companies are probably just getting GitHub Copilot, if I’m honest, which is functionally useless, or going to Claude or Codex, which is way better — they’re just prompting, and then it’s writing a bunch of things, and then they’re shocked that it’s not fit for purpose in their environment.

AI agents are good at generating code. They’ll do that happily all day long. But is it good code for your environment, for your context? Backpressure is the only thing which is going to shape that. And so this is about creating as much good steer as possible. The prompt that you put in is part of good steer. But then it’s the rest of the context of your organisation, how you deploy, the environment you’re deploying into, its peer repos, upstream and downstream consumers. Is it taking too much bandwidth within your server environment? Backpressure is the only thing which is going to tell it whether it’s actually fit for purpose in your context.

And I think we need to be thinking, when we build these systems, about how do we re-inject backpressure into the agent, so it understands when it’s failed or succeeded or needs to rework something?

James MacDonald [7:20]: That’s the difference between building software. Everyone can build now, to an extent. But that’s the difference between everyone can build, everyone should build — or putting software, AI, in enterprise. It’s these things where we’re going that next level down. It’s not just, as you said before, creating an environment for people like myself to start to vibe code solutions or build things. I need to have wrappers around me, I need to have these testing environments set up, I need to have the right environment for me to be able to build it. Is that a big challenge that you’re seeing for organisations — to allow their employees to move forward, allow them to use these tools, but in a safe and secure environment?

Adam Wintonowski [7:59]: Yeah. So I think we’ve taken a bunch of learner drivers, and then given them an F1 car and said, just drive around the city, have fun. Whereas I think if you look at — I’m taking F1 as an example, not that I’m any fan of F1 — but that’s a team race. It’s a team sport. It’s not a single driver. So I think when you get to enterprise, you need to be thinking about these from a multiplayer standpoint, not just a single engineer working alone at his desk in his hoodie.

You need to allow, say, cyber to inject cyber policies into that engineer’s coding harness. He shouldn’t have to think about it. Those sorts of policies — legal, risk, compliance, cyber, cloud, et cetera — those shouldn’t live in a document somewhere, and then we hope an AI agent finds it and maybe potentially incorporates that into what you’re building. This is someone who knows better. This is someone who, if they open up that code and do a code review — which is also not really sensible in this day and age — they know what to look for, but they shouldn’t have to. The agent should have forward pressure of knowing that this is the policy. We always ship on lambdas, for example.

That shouldn’t be something that the engineer has to think about. That should just be intrinsic knowledge within a context, and that should be provided by a multiplayer system where cloud is injecting that into your coding harness. I think we need to start thinking about these things much more, rather than at an individual scale, and start thinking of it more as an enterprise scale.

We’ve spent the last two years teaching agents how to work collaboratively, and they can work in swarms of thousands now. We’ve stuck all of that in front of a single user and said, steer this whole ship on your own. We’ve done nothing to think about the multiplayer aspect of that harness. Because really, that’s where enterprises are — an organism of multiple intelligences, multiple people working together in this weird harmony. We’ve kept that out of the coding harness. We need to find ways to inject that in.

Everything I was building at NIB — probably 80% of it had multiplayer aspects to it, which allowed multiple teams to have opinions, and they didn’t have to be there next to the engineer to inject those opinions. Those could come from various systems and various backpressure points throughout what I call a meta-harness, or harness of harnesses.

James MacDonald [10:52]: You mentioned the word re-engineer. It’s like taking what’s existing, pulling it apart, re-engineering. There’s talk of AI-native startups building from the ground up with nothing existing, significantly easier to build at AI-native. Going into an environment that’s currently built, working, highly regulated, significantly harder. If you were to go into a brand new company right now, and as simple as we can break this down, they’re not using AI at all — where do we start? How do we break this down?

Adam Wintonowski [11:23]: The advantage of startups is they don’t come with a bunch of preconceived ideas. Startups are able to start from zero and reimagine a process from the ground up. They’re not treading on anyone’s toes. If they make a decision, Jan from accounting isn’t going to come and throw his hands in the air and say, you’re taking my job away.

Enterprises really need to start thinking in the same way. They probably always needed to be thinking in this way, but now the speed of change is so rapid that unless they’re eating their own market, they’re going to find these startups are taking their business away. We’re going to see that, I think, more and more over the next five years.

Enterprises need to start reimagining what the system looks like — not just injecting AI into a process, finding a human thing that we do, and then finding a way for AI to do that one sliver of work. We need to think about the whole process as a unit and start to replace. Not just go, okay, well, how does Jan from accounting manage this set of spreadsheets, let’s stick an AI to manage those spreadsheets. No. You need to start to think about Jan’s whole role. That’s what the startups are doing, and that’s what enterprises need to do — start to re-engineer whole processes, not thin slices.

Until they do that, they’re going to be really hamstringing themselves, and they’re not going to see the ROI that they want. They’re going to start to see 5%, 10%, or zero.

James MacDonald [13:00]: There are significant challenges with this, right? Leaders potentially fighting against the board, fighting against the AI — we’ve got to do this, we’re actually going to have to invest a significant amount of money. This is a big decision on, are we changing the nature of our business? Any advice for companies to do that? Because a lot of the advice out there at the moment is, people tackle it from a big perspective, and it takes a while, and they’re not seeing any form of ROI. So the alternative view is, take a process, throw some AI, automation at it, actually see some ROI, and then take another process, and iterate, and just continue to chip away. A completely different approach to what you’re mentioning here — which I agree with, but it comes with a significantly bigger challenge.

Adam Wintonowski [13:45]: Yeah. So I mean, I think this is what everyone’s wrestling with. It’s why they bring in consulting companies. Because, A, they struggle to see. You look at engineering — engineering, we know what engineers do. Companies are measured by the ticket, which could be an hour, it could be half a day, a day, whatever. Call centres is the other part of the organisation that’s being largely eaten by AI, and we know what they do by the minute, because we have every recording.

What we don’t know is what accounting does, what marketing does. We don’t measure them. And so a big part of it is the measurement problem. We don’t know how to measure other parts of the enterprise beyond engineering and call centres. And so that’s why those are in the spotlight at the moment, engineering and call centres, because they’re relatively easy to understand, because we know how to measure them. We don’t know how to measure all of the other parts. So we bring in consulting companies, and we do time and motion studies and all that kind of thing.

But you’re asking people who really don’t know your business to come in and solve a problem, which is problematic. They might understand abstractions, and so they might be able to come in and suggest this thin slice, this whole process.

But I think you’re going to get a redefining of a lot of industries where, previously, winners and losers might have been decided over 10, 15, 20, 30 years. The same compression and compaction that’s happening in engineering, where AI is being heavily applied, you’re going to start to see those compactions happening for any player that throws down heavily in this space. And so winners aren’t going to be decided over 15 years, 20 years. They’re going to be decided over 15 months, 20 months. You’re going to see a completely different scoreboard for this if companies invest.

And this is — there was a study done, or someone wrote a blog which became a book, the thousand-day problem, where we have a thousand days, and we’re currently at day 600 or something. We have a thousand days before AI kills us all, there’s an AI meltdown. I don’t buy it. But the main saving grace of that is enterprises’ relatively slow AI adoption. There is a sense to which enterprises have always been relatively slow to adopt new technologies. In the insurance space, you go to conferences in insurance, and they’re still talking about their cloud migration strategy. I mean, that’s very early 2000s. You go a little bit further back and you’ve got Marc Andreessen from Andreessen Horowitz saying software’s going to eat the world. That’s true now more than ever.

And so I think the more businesses are able to cannibalise themselves and really understand the processes that they have internally — which a lot of companies aren’t doing, having that sort of deep introspection — the more they can understand their processes, understand what actually adds value now, the more they’re going to find value out of AI adoption.

James MacDonald [17:10]: I think what you mentioned right there as well — it’s the roles like marketing and these other roles that aren’t as process-driven, do this then do this then do this. Breaking down that process, having a standard operating procedure — because once you’ve got that, you can actually throw AI at it. Those ones that aren’t as well documented, that’s where a lot of companies struggle.

The other part you mentioned just there, software eats the world. The SaaS apocalypse — everyone’s talking about it. Software is going to eat the world, and we’re continuing down that route. Is it, all companies become software companies and all companies are building their own software? Or is it, there will be a rise again of small, native SaaS companies?

Adam Wintonowski [17:51]: Right. So it’s a couple of things there. Is software going to eat the world? Yes. And we’ve been saying for — not we as engineers, we as in people in enterprises, business owners, et cetera — every company is a software company. And Mark from NIB, Mark Fitzgibbon, I think truly believed that it was a software company that sold insurance. And I think the companies that have done that well have been able to operate at lower cost and higher margin over time.

And so, yeah, I think ultimately software does eat the world, and that makes engineers apex predators in this space at the moment. It’s sort of weird to think about, because I know engineers that don’t look like apex predators. But you’re not going to have accounting eating their own portion of the business, because it’s still hard. Like, building with AI is still hard. You still need engineering discipline, otherwise you’re going to be escaping all sorts of data out into the world that shouldn’t be escaping.

Is software going to eat the world? Yeah. Is the SaaS-pocalypse a thing? Yeah, there’s going to be some companies which don’t survive. There’s going to be some practices that we used to use which we don’t need anymore, because AI and a new competitor comes along that does it faster, cheaper, better. Or some companies are going to build those tools in-house and just don’t need the SaaS anymore. However, that creates a high maintenance cost, and something which isn’t your core business.

James MacDonald [19:35]: It’s not your core business, but every business is a software business. Does that mean every business starts to build out internal engineering capability?

Adam Wintonowski [19:44]: On the parts that add value. So there’s not a lot of great, as an example, just off-the-shelf fraud engines. And so that’s a core capability of an insurer or an FSI company. So they’re probably going to need to build out their own fraud detection mechanisms that fit their system, and are able to be understood, tweaked, dials moved.

Now, do they need to build their own ticket management system? Probably not. There’s going to be some things which are just generic, and those should be bought off the shelf. And there’s going to be things which are unique to your business or unique to your industry, and those should be built, not bought. And that can happen faster than ever now, and better than ever now, and create better customer experiences now.

And that becomes — I mean, the customer experience that you’re able to create has always been the hedge, right? Health insurance, there are a thousand of them. There are 20 banks, whatever. But the experience that you get, the functionality they provide — that’s the unique offering. And price is also a way to compete. But if you can do it cheaper and you can create better customer experiences, and AI enables that, which it does, you win. And so I think that’s where it becomes, everyone’s a software company. They need to start thinking at the speed of a software company, not at the speed of an insurance company or a bank that have typically operated over five-year timelines. Those don’t exist anymore.

James MacDonald [21:38]: You mentioned earlier about consultancy companies coming in. We just talked about building internal dev capability, engineering capability. The other one is, if you’ve already got that engineering capability, upskilling the team internally. They’re the three options I look at, the three options I talk with other companies about that are looking to start some form of AI journey. Talk to me about — is there one size fits all? Is there a particular situation where I would bring a consultancy in, this particular situation where I’ve got to build a capability, because that’s not cheap?

Adam Wintonowski [22:10]: Every company’s got blind spots. Every person has blind spots. You have a certain way of operating in companies, and you just think, this is the way we do it, this is the way it’s done. So having an extra set of eyes come in and reimagine what that could look like is, I think, always helpful — as long as you’re willing to listen to it.

And not all consulting companies are equal. Some consulting companies are going to come in and they’re going to sell you unicorn ground-up unicorn-horn magic dust that you’re never going to be able to build out yourself. And they’re probably not capable of building out either, if you’re really honest, because the talent pool is actually surprisingly small. So they may come in with all sorts of promises that they’re not able to fulfil on internally.

James MacDonald [22:57]: We have a look at the big rise of the forward-deployed engineer. A big percentage of them are working for consultancies. Consultancies are trying to hire them. When they try to hire so many of these same type of people that don’t exist — they don’t all exist, right? At the same time, we’re going to get a bunch of underskilled people being presented as forward-deployed engineers.

Adam Wintonowski [23:19]: Oh yeah, for sure. And the forward-deployed engineer obviously comes out of Palantir. And this is a handful of unicorns. Most engineers don’t like spending a lot of time with people. Most engineers like solving problems and really working through those problems. They love mastering their craft and honing skill and being opinionated around which database is best and all this sort of stuff. So engineers are a sort of beautiful and unique stallion.

But what happened with AI was, over the last probably 18 months, two years, it got everyone thinking, oh, our engineers could be product owners, our engineers could talk to customers. And we started to blur about five different roles into one, or imagined that we could, because AI enabled certain things or gave time back. I think what we’re realising — hopefully what we’re realising — is there are some things that a product person does that an engineer is just not good at. And so we’re going to end up, as you say, with a bunch of forward-deployed engineers that actually aren’t fit for purpose. There is a group that totally exists that are extroverted and quite happy to go and talk to customers and are also able to build things. But that’s not every engineer that I’ve met.

James MacDonald [24:46]: My opinion is actually quite the opposite. I think it’s easier to turn a traditional business analyst or product manager, product owner — upskill them from an engineering capacity and turn them into that role — as opposed to the vast majority of engineers. As you said, those really like to get technical, those people definitely have their limitations as well. The ex-PM, the ex-BA have their limitations, but I feel like the opportunity for those people to now upskill technically has changed significantly.

Adam Wintonowski [25:16]: I think that’s true. I think the hard wall they hit is as soon as they want to ship something.

James MacDonald [25:22]: Correct. But I feel like that initial scope — where can we actually throw AI at this problem — is probably better suited to that type of role, backed up by a proper engineer.

Adam Wintonowski [25:34]: Yeah, product people with product thinking. The people who are going to win over the next couple of years are people who can think technically and can think with a human-centric mindset, or a product mindset, where they can imagine applications, they can imagine how this solves the customer problem, they can see pain and understand the pain empathetically for an organisation, and then quickly come up with what’s the sell for that pain.

And that’s been the tension between engineers and product people. And it’s a healthy tension. And it’s definitely something — if you’re talking just up to POC stage, or solution engineering, or solution imagining — yeah, definitely taking a product or a BA into that space and building the first pass is something they can definitely do now.

James MacDonald [26:38]: I mean, so is a 13-year-old shipping software now.

Adam Wintonowski [26:41]: Yeah.

James MacDonald [26:41]: All right, I’m going to go down that route real quick. There’s a tangent — I like tangents. I’ve got eight-, ten- and 13-year-old children. We’re looking at the future of what they want to be when they grow up. Obviously life’s going to change, who knows, just enjoy your life, you’re a kid at this moment. I know you’ve spoken about your child actually building at the moment, and actually building some pretty significant things. Talk to me about that journey. Was that something that they just showed an interest in, obviously seeing you and what you’re interested in, and then it evolved from there? Is it something you’d recommend for others?

Adam Wintonowski [27:18]: Look, we’ve got two kids, my wife and I — a boy, 13, girl, 12. My daughter has basically zero interest in AI at all. The only interest she’s ever shown in software engineering is, they had a mufti day to dress as a future profession, and she wanted to wear a hoodie and jeans, so she went as a software engineer. She will not be a software engineer. So maybe she will — who blew me away. My son, however — so he’s homeschooled, he’s very gifted. He likes solving problems and tinkering. I think he wants to work in space and engineering or something along those lines in the future, and he just sees this as an opportunity to make some money. So he’s picked up the tools and started playing around, and that’s led him to do talks and other things.

James MacDonald [28:09]: Real quick one then — if you would give advice for other parents whose children are showing some form of interest, what are the guardrails you put around them? What are the tools that you provide them with to get started?

Adam Wintonowski [28:20]: So for my son, who showed an interest, wanted to build some stuff — he started off, he wanted to build some things to help him win in video games. So like an auto-shooter, for example, that was able to identify opponents and auto-target their heads and shoot them. Great little starter project.

And from there, I just walked him through how I would build, walked him through some best practice around things like spec-driven engineering, test-driven development, and a few concepts. That started to bloom out, because then he built a thing and it didn’t work that well. It was sloppy. So we went through and did a little bit more of the background, getting into some architectural principles of engineering — really the same thing you would do for a junior. Which, I mean, juniors now, it probably looks more like an apprenticeship, where you’re walking them through some of the things which unis have historically done and are typically now skipping perhaps a little bit more of. Some of that real foundational engineering thinking.

James MacDonald [29:33]: I actually think that’s the most important part of the university degree for software engineers that I’ve met — the understanding of the foundations. Because now, with the tooling, if you’ve got that, then the juniors with that solid understanding of fundamentals, with the new tooling, essentially become mids when they come in, as opposed to juniors. People without that, self-taught, don’t have those fundamentals, are, as you said, more apprentices.

Adam Wintonowski [30:03]: It’s vibing. It’s 100% vibes. Versus someone who is self-taught — they may be able to ship code really quickly, and look, I know some self-taught developers that are fantastic. But they’re fantastic because they’ve taken the time to actually understand the fundamentals. They learnt them in reverse, the hard way, through scar tissue, as opposed to through education. And scar tissue is always going to be the best way to learn anyway. But if you’re not willing to take the time to build the scar tissue, then you’re going to struggle to ship anything meaningful that endures.

James MacDonald [30:43]: Go back to the last question for a real quick one. Companies sitting there at the moment, want to start their AI journey — upskill internally, hire externally, bring in the talent, or use an agency. Do you have an answer? Do you think it’s a practice?

Adam Wintonowski [30:58]: I think building internal capability is always going to be the win. For sure, hire in some people who are good with AI ahead of your team, but use them as a resource to train. I think if you can upskill your team, it’s a gift to you and it’s a gift to them. The future in this space is people who know how to use AI and leverage it for volume. I haven’t written a line of code in probably 11, 12 months, and more and more, that’s going to be the common experience.

And so you’ve got people in organisations all the way from “AI can’t possibly do my job” through to 400, 500x, and then this sort of mushy middle. I think the more you get those 400, 500x to be able to decompose, train — and this is something that engineers have historically done really well: brown bag sessions, taking time out to peer review, PRs, peer programming. Engineers have a long history of training and coaching one another. The more we can leverage that within companies and upskill everyone at somewhat of an equal bar, anyone who doesn’t want to use it will opt out, and they’ll find somewhere else — Luddite or something. But raise the tide, all the boats rise together, I think is a really important idea.

And these people come in with just a huge amount of legacy knowledge that, if you can codify that — and eventually they get an offer and walk out the door. I think AI makes your people more poachable, which makes them, actually, I think counter-intuitively, more loyal, because they’ve been invested in, they know they get the opportunity to experiment. You also get this enduring benefit of, they know the industry, they know your systems, they know how to change them, modify them and uplift them.

Bringing people in — we’ve brought in contracting, consulting companies before, body-shopped them in. Now, there are definitely situations where consulting companies come in, they solve genuine problems, and they do it in an enduring way. But I’d say most engagements from most consulting companies, it’s a matter of them coming in — I think of it like renters, owners and caretakers. They come in as renters. A renter’s not going to uplift the kitchen, they’re not going to tear out the tiles, they’re not going to look for the termites, they’re just going to slowly trash the place over time. Owners are going to genuinely care for it, they’re going to make sure that its value increases over time.

There’s a massive difference between bringing in fly-by-night teams that come in once, quick hit, quick spend, versus the people that have to live with it long-term. So, yeah, of those three categories, I would rather invest in owners than renters — or at least caretakers who are going to lovingly nurture what exists.

James MacDonald [34:13]: So the recommendation is to hire.

Adam Wintonowski [34:16]: Oh yeah, you’re going to love that, NTP.

James MacDonald [34:18]: Well, there are some challenges on my side. There are some challenges for internal recruitment, right? I think I’m reasonably technical. I’ve interviewed a bunch of AI engineers at varying levels of quality. There’s a real challenge for someone like myself, or internal talent teams, to actually know what good looks like. I think the standard software engineering interview has changed. I don’t think what used to work for traditional software engineering roles works for the new AI engineers — or, we’re talking the senior, the principals that are going to come in, as you said, to change a company’s trajectory.

You mentioned, make sure that somebody’s good that comes in. There are a lot of people that are talking about good. They’ve built a POC, they’ve never put something in production. And you’ve recently interviewed for nine different companies. I want to break down what a good interview practice looks like for someone at your level, or even if you want to expand it a little bit down. What does good look like? Because I imagine you’ve seen the best and the worst of it. If you’re going to take all the knowledge from your nine interviews you’ve just been through — start it from the start. What does a good ad look like? Reach out — what does that look like? How do I engage somebody like you? And I want to break it down step by step.

Adam Wintonowski [35:39]: Yeah. So I think, top challenge in AI, I want to know that they’re playing with AI, number one. They’re probably going to want something which is a title which reflects that. Titles at the moment are all over the place. They’re just a red-hot mess. Title used to be representative of comp in some way. That doesn’t seem to be true anymore. You’ve got FDE roles that range from basically what you would have paid a senior previously, all the way up to what would have been considered ridiculous two years ago. So that’s not an indicator anymore.

I think, in terms of people who are in this space, they’ve recognised — they’ve been in a company before, they recognise that they’re probably in the 5% of their organisation, and that they’re relatively rare. And so they know they’re probably worth money. Rare things are worth money.

And so, for someone who’s relatively new — so they know how to drive Claude or Codex or Cursor — they want to know that there’s opportunity to continue to flex that skill, there are going to be opportunities to learn, and opportunities to be around people who are better than them or more advanced than them in AI. For someone who’s been around a little bit longer, and they’re not just using Claude the way it’s built, they’re starting to build their own harness and that sort of thing, they want to know that there’s some room and flexibility to experiment and make a market dent within there. They’re still engineers, they want to solve problems. And they want to understand that the problems are meaty, they’re not going to be shoehorned into one particular thing — because they can probably solve that particular thing in a couple of months now, as opposed to a couple of years.

And the people who are really pushing against the edges of this stuff, they don’t fit all that well into sitting with one team for a really long time, because their mindset is fringe, right? They push against the edge. And if you put them into just one little box all the time, and that’s where they live for 38 hours a week, then they’re probably not going to have that much fun.

James MacDonald [38:01]: So, sell the vision. Obviously the tools — make sure that they’re going to have the full access to actually continue to grow and lean into this. And then actually fixing some big problems, and not just being put in a box.

Mention one other thing then — we’re going to go there right now — comp. If we’re talking about comp, I’m going to have to pay you a significant amount of money, right? Putting you in a box doesn’t make sense. I want to throw you at the biggest problems across the company and have you fix them and continue to chip away at them. Comp in Australia — nine companies you’ve interviewed with, some of them international, some of them Australian. Where does comp currently sit for you? Can Australia actually compete for talent like yourself?

Adam Wintonowski [38:43]: Yeah. So, I’ve applied — or, I’ve interviewed for and applied for, half the roles I didn’t apply for, people reaching out. The other half I did. And the ones that were low, self-filtered, so these aren’t included in that nine, because it’s one of the first questions I asked, because the roles make no sense. I’m like, what’s the comp? Because that’s an indicator of seniority and effort now, apparently.

So the roles here versus overseas — very senior, more senior roles are the ones that I was looking for. Things like distinguished engineer — big grand title, feels like I should wear a top hat — but that was paying less than an engineer role, not even a senior role, in a startup in the US. And the startup in the US was offering 400, 450 to 500 base.

James MacDonald [39:57]: USD?

Adam Wintonowski [39:57]: That’s the converted. Yeah, so Australian. And there were other ones that offered more, obviously. But a lot of that was stock and that sort of thing. So I think offers around the 450 to 500 USD mark were in the mix, which for me just didn’t move the needle for triggering a move to the US and moving my family and all that sort of thing. Because you can get that kind of money here, but it’s a much more senior role, very different role, probably less close to the tools. And so that’s really an appetite — how close do you want to be to the tools?

Those same roles here are probably paying around 200, 220. So 220 versus 500 — which one are you going to choose? Particularly if you’re — I’m married, got kids, got a life here. If you’re an engineer in their 30s, not married, or married with no kids, why would you stay for 220? That makes no sense at all. That’s working against yourself.

I’ve found salaries here to be discounted probably between 60 and 200 to 300%, depending on the role. There were roles with the same title that I had previously at NIB that were paying well over a million, into the two million space.

James MacDonald [41:17]: And those companies were based in the US?

Adam Wintonowski [41:17]: They were based in the US. And I don’t think those are all that common, but they are around. And I didn’t see any AI roles for less than double of what was being offered here.

James MacDonald [41:35]: So for Australia to keep talent like yourself in Australia, to build that talent here, what do we need to do to compete? Obviously you mentioned the vision, you mentioned the tools. But from a pure comp perspective, where do companies need to be to be having the conversations? Are we talking minimum 300s? Are we talking higher than that?

Adam Wintonowski [41:57]: It’s going to depend on the role, obviously. It’s got to be reflective of the value that the person brings. And I think in the US, they’re anticipating, at that pay grade, that you bring a lot of value. But typically they’re also solving more interesting problems.

Like, in Australia, there are a couple of drivers. So one is, in America, you’ve got big tech — companies like Google and DeepMind and Anthropic, OpenAI, et cetera — which are paying just ridiculous salaries, just astronomical. And they do that to attract the best talent. And they’re in a talent war with one another, because they’re in an arms race with one another and with China. So of course they’re paying huge amounts of money — 10 million, 20 million, billions — to acquire really top lab talent. And that’s one thing that puts upward pressure just generally across the market. Because if you’ve got these companies which are actually making things at a large scale, to roll out to billions of people, that drives salaries up.

We don’t make stuff here. Not like that. Google here is largely sales. So we don’t have a Google driving salaries up. On the other end, you’ve got startups. Startup funding here is typically very low as compared to the US, by probably a factor of 10x or larger. And so they just don’t have the money to offer the same sort of ridiculous salaries. And again, that’s an arms race. You’ve got 10 different voice companies competing there for talent that have worked in voice models. Whereas here, you might have one, and that one is funded at a couple of million dollars versus their 20 or 200.

So they’re able to offer way more money. That drives salary up, because now you’ve got two competes for salary — one is well-funded startups, the other is the hyperscalers that are competing in an arms race. Now, that means that if Bank of America wants to get talent, they’re competing against a drain out of two spaces. We’re competing really against — we’ve got enterprises which have never had the sort of inflation of salaries that the US got anyway over the last 10 years. And a lot of enterprises will still complain that engineers are too expensive, because they don’t see themselves as a software company in competition with whoever the hell is going to come and eat their lunch.

So we don’t have the startup pressure. We don’t have the hyperscaler pressure. And we’ve got enterprises that think engineers are too expensive. Salary is one thing. But when you’ve got low investment — and Australia is, I would say, risk-averse in terms of its investment in startups — you don’t get the big swings and the super-exciting problems to solve, typically. You might get one or two of those, but typically they’re probably going to be something derivative of another successful startup in America, released for an Australian context.

And so an engineer sitting there — he’s a five-exer, a 10-exer using AI, knows he’s a bit of a unicorn in his own context. And he looks around the market and he sees, I can go into sales, which is functionally what an FDE role is, and maybe make a little bit more, maybe bump up my salary by 50 grand. Or I can go to a startup and take a massive haircut and work on a problem that’s really not all that compelling. Or I can go to a brand name like Atlassian, whose stock comp is going down. I can go to Google and work, again, in functionally sales. I can go to Canva and get shoehorned into one thin slice of a product. So what are their choices? It’s going to be a lateral move into something which pays slightly more. Or I’ll go to the US, I’ll go to Europe, I’ll go somewhere else, which is going to double my comp.

James MacDonald [46:15]: Are you seeing many of those roles in Australia fully remote?

Adam Wintonowski [46:19]: From the US to here? No. All the roles that I got offers for — and look, there might be one or two — but they’re typically more four or five days in office. They want you there, they want you on the ground, they want you with the team. Especially at that salary.

James MacDonald [46:37]: So, you’re at a company at the moment and I want to hire my first AI architect, AI senior, AI engineer, somebody at your level. What do I need to pay to make things interesting? Where’s my starting point?

Adam Wintonowski [46:50]: Yeah, I mean, you’re probably looking at the three to 500 range. And that might feel ridiculous for a lot of companies.

James MacDonald [47:05]: But it comes down to the mentality though, right? If your mentality is, it’s the same as I’m looking at as a normal software engineer, it’s expensive. If you have a look at it, we’re going to re-architect our solution, we’re going to really aggressively lean into AI and change the nature of our business, provide better customer service, like you said before, take market share from competitors — then it’s not a ridiculous number.

Adam Wintonowski [47:31]: For most enterprises, even the most modest numbers — so they say 15% uplift is sort of a range. You’ve got individual engineers that are capable of — and even at NIB, I saw this in my own experience, other engineers I talked to were seeing way higher than this — but 200% increase in productivity is relatively easy to accomplish on a thin slice of their week.

Now, end to end, if you want to see that, you need to engineer the meta-harness. Otherwise, engineers have typically, historically, spent about 16% of their time actually writing code. That’s the bit that Claude Code replaces, is that 16%. Now, if you want to see that flow end to end, you need more infrastructure, more thinking, an AI architect to help you actually map that out. And if you don’t, you’re going to see between a 0 and 40% increase in productivity. If you do, you can translate almost 100% of that into throughput.

So if you’ve got even a 1% uplift — if you hire this person and they bring a 1% uplift in your company, they’ve paid for their salary multiple times over.

James MacDonald [48:55]: Do you think the big challenge at the moment is most companies aren’t seeing that ROI on AI?

Adam Wintonowski [49:01]: I think most companies aren’t measuring. They don’t know how to measure it. We’ve had DORA stats, for example, DORA metrics, in engineering for years — I would say there’s probably only 5 to 10% of companies that actually look at that. And I think this is why Atlassian bought DX, the developer experience platform, which helps measure both engineer sentiment but also their engineering capability within an organisation. And it plugs pretty well into Jira, and Jira does a pretty good job of pulling out a lot of stats. You probably need to look wider than just Jira. But I think most companies aren’t even measuring it, and then they’re claiming, well, we see no benefit. And I think that’s probably because they’re looking at the wrong things.

We’ve done a relatively good job at measuring how much a token costs. And we’ve done a relatively poor job of understanding what value that token has added. And so I think the first thing that any organisation that wants to go down this route of increasing their AI capability needs to figure out is how to measure. Because what you don’t measure, you can’t change.

James MacDonald [50:11]: I think that’s the best line you’ve said today. I genuinely do, because I think that changes the mentality. Because if you can actually see the measurement, you can actually see the uplift, it changes the mentality about what you might pay for your next hire, as well as the rest of it.

Adam Wintonowski [50:24]: Well, you can also only make change at the level of constraint. So if, number one, you don’t understand what portions of your company does, you don’t actually understand what the constraints are. So you can make a change that makes no difference. And then the other side of it is, if you don’t measure the productivity that you currently have, you can’t measure the delta of what the spend has given you. So, yeah, constraints and measurement are the two key things to do before anything happens.

James MacDonald [50:55]: All right, I’m going back to this interview. How do I get this person in the door? We’ve got the comp sorted, let’s put that to the side. You come in for your first interview. How are you designing that? If you’re sitting in my seat, if you’re taking all your best learnings over these nine different companies, and what you’ve done previously when you’ve hired into your team — what does it look like when I get you in the door for the first interview?

Adam Wintonowski [51:17]: There are some things which are always going to be true. You don’t want to hire impressive jerks, right? So you want to make sure that people are a good culture fit there. And you have to recognise, depending on where this person is in terms of their level of stallion-hood, or rareness, they’re interviewing you as much as you’re interviewing them. They have options.

And so I think, go back five years, it was much more, the upper hand was with the organisation that was hiring. I think that’s shifted for a certain pool of talent. The power is definitely in their hands at the moment. And that’s why I’ve got, like I said, seven offers, I’ve got another one that’ll come this week, and most of them are willing to wait. And one of them even said, we need you more than you need us. So the power dynamic has shifted a little bit.

So they’re interviewing you as much as you’re interviewing them. And the interview processes that I’ve had have been everything from, we’ve heard about your reputation, and we just want you here — here’s an offer, meet some people so you can be comfortable with us as an organisation, and here’s an offer — all the way through to, here’s nine interviews of various things. Some have been old-school coding interviews, write a function which does this. And I’m like, I haven’t written a function in 12 months, man.

So I think a much more compelling thing, particularly at that level — if you’re hiring high-capability engineers, they’re not writing code anymore, they’re building harnesses. And I think if you just sit down with them and say, show me how you use AI to write code, you don’t have to build anything, just show me your harness — some of them are going to open up a Claude window, and they’re going to go, here’s how I prompt and here’s how I give it context. Great, they’re a level one AI engineer, and you get a good idea of what their throughput is going to be. Some of them, they’re going to open up their monitor, and they’re going to have 15 Claudes open, still running. They’ve got Claude handoff, you’re hearing their phone ding — okay, this person’s more like a level three. Some of them are going to be showing you a dark factory producing software while they sleep. And those are very different AI engineers through that journey.

James MacDonald [53:51]: Real quick — I’m Fred, non-technical internal TA team, or a recruiter who’s trying their best to stay up with it. I’ve asked you to come in and show me how you do it. For the good engineers, they’re going to be that couple of levels above. What are the things you look for, for somebody that’s good versus somebody who’s maybe very thin in their expertise but can look flashy from the outside? What are the questions you ask people?

Adam Wintonowski [54:19]: I’m going to be asking them things like, have you ever shipped to production, and what did that look like? Because building — I have a thousand apps. My wife’s a teacher, and she’s going to want to go on this excursion to the art gallery, and I want a thing which does this with AI, and I’ll spin that up, and she’ll ask me that at seven o’clock in the morning and it’ll be live by 8.30. And I could never sell that as a product, because it would just be for a very small group of people, easy to lock down, easy to put in a server.

Now, if that has to be shipped to 100,000 users in a regulated environment, that’s a completely different animal. In fact, to the point which I would say, a month spent building an AI-centric product in an enterprise probably means six months deploying it. It’s a factor of magnitude higher. Now, still, you’re probably still saving two years, because it’s going to be more complete, more feature-rich. But it’s a much slower process, actually productionising something.

So, walk me through how you would productionise this. What hurdles are you going to hit? And I would look for standard language that an engineer would otherwise use — things like, I use test-driven development. Look for backpressure markers. How do you ensure that you have test coverage? How do you ensure that you give your agent the ability to test what it’s built? All these sort of standard things that you would do as an engineer anyway — are they using those practices, or are they just throwing slop out into the world?

And look, there’s nothing wrong with a little bit of slop here and there. The agent can always rewrite it, and it’s where everyone starts, build a sloppy awful thing and watch it fail in production. But if you’re really spending that real money, you’re going to want to ask them some properly engineering questions, and probably look less at the code they write and more at architecture and engineering practice.

James MacDonald [56:45]: Do you think that return of the “you’re actually coming in with your laptop” is going to be a continued trend? Or is it like your old whiteboard discussions — you’re coming in, you’re talking about a theoretical problem, how would you go about this, and you’re looking for their mentality around their architecture, around their problem solving?

Adam Wintonowski [57:04]: I think the whiteboard should come back. Yeah, 100%. Because you’re really looking at the distinguishing marker. You go and get Claude Code, and I get Claude Code, and we sit down and we’re going to build a calculator app — we’re probably going to end up with something which is 90% similar, because of model weights. Model weights know how to build a calculator. And so we’re going to end up with the same code.

Now, that’s testing whether Claude is any good, not whether you’re a good engineer versus me. So I think getting through, how do you think about problems, what questions do you ask, how do we architect this? Okay, so now let’s build a calculator that 10 people can use simultaneously, or it knows when you’re writing dirty words in numbers when you hold it upside down and it can do something interesting with that. So start to think through — now design this feature that’s over and above. Because that comes down to really the distinguishing mark of good engineering now, which is taste. It’s not just, can this person write code — does this person have taste around what’s good code? Because I can build a dark factory harness which will just produce hundreds of apps. Are any of them any good? And that’s a taste issue, not a “can this person generate code” issue.

James MacDonald [58:38]: So, you’ve come in for a first-round interview. We’ve done, let’s call it the behavioural, cultural conversation to start with, making sure you’re the right cultural fit. Got a technical conversation, which is similar to the old whiteboarding session — it’s taking somebody through your actual code. You’ve mentioned before, online, around harnesses and coaching people through the harnesses. Talk to me about that. What does that look like? What does that question look like? What does a great answer look like? What are you looking for?

Adam Wintonowski [59:05]: Yeah. So it depends. Because everything is somewhat a harness now, but in terms of their coding harness — Claude Code is a harness, Cursor is a harness. So, we mentioned this before, but do they just use it vanilla? Or do they run a headless dark factory that’s able to run loop engineering? Or are they thinking more exotically, like graph-based harnesses?

And so there’s various — that’s sort of the graduation, right. You’ve got basic vanilla Claude, then you’ve got, they’ve got a harness which has some skills and plugins and MCPs attached. Then maybe they’ve moved on to headless loop harnesses.

James MacDonald [1:00:03]: I’m somewhere in between two and three at the moment.

Adam Wintonowski [1:00:05]: You are? Yeah. And so at NIB, I built a tool which was able to measure the complexity of someone’s harness. It was built for a particular context at a particular time, but it had a 10-point grading system, and then suggested here’s the next steps you would need to take to graduate to the next phase.

James MacDonald [1:00:27]: Do you think a product like that would be successful in market? I think the technology is just growing so quickly that you’d need to continue to update that product.

Adam Wintonowski [1:00:35]: I think there’s definitely — I mean, I’ve started more and more building in parallel. Anything that I build, I also build a testing harness for it now. Because it’s moving so fast, you need to be able to answer, is this better? Because now it’s doing things so quickly. This is going back to just human-in-the-loop as a concept. We take a context-poor human and put them in front of context-rich agents and say, tell me if it’s good — which is a stupid idea, because they’re just never going to be able to answer that question in any meaningful timeframe.

So we need to be building not just harnesses, or agents and agent harnesses, which do things, but testing harnesses, which can tell us if what it did was any good. So I’ve got three testing harnesses that I’m building at the moment. One over my own coding harness, which I’m building — a recursive contract-based graph harness, which is an interesting thing to build. It runs non-deterministic depth graph loops, so it’s able to build its own sort of roadmap and plan, and build stuff using some deterministic components. Anyway, measuring that — it needs to be able to measure my current harness, which isn’t that, and measure this new harness, whether it’s producing something that’s better.

So, absolutely, that fits into — building harnesses is a little bit like being a Jedi. We get these AI engineers, and they use the standard Claude tool, but eventually they graduate into this Jedi mode where they want to start building their own harness, like a Jedi built a lightsaber. And every lightsaber is a little bit different, but you can see it still has all the hallmarks of being a lightsaber. And so that’s functionally a testable thing. So that’s why I think it becomes a good test for an interview. And they’re also closely held secrets for engineers, that becomes part of their distinguishing factor — how quickly my thing can do something. Is it 70% complete, 80% complete, 90% complete at the end of a task? How did you get that 90% completeness? That’s all measurable.

And so I think having something which is able to measure that will become a tool. We have some harness-testing tools already, and there’s various benches you can go and look at online.

James MacDonald [1:03:19]: And that’s probably the closest thing to an old-school code review, right?

Adam Wintonowski [1:03:22]: Yeah, I think it’s most analogous to a code review. There are still code reviews — does your harness produce something which is good, or red-hot garbage? So there’s still things like cyclomatic complexity and other stuff you can measure.

And so, going through the recruiting process — which was a bizarre thing to enter back into at this point, because I was looking through jobs and applying for things a couple of years ago, and doing it this time round was just so different. I think it’s a functionally broken industry at the moment, in a lot of ways. Because I get an agent to produce a CV particular to a JD, send that through to you. You’ve now got a thousand people doing the same thing. And so your system previously would have ingested that, found some things which match, pattern-matched me to the job description. If a thousand people are doing exactly the same thing, you don’t have 20 bits of signal now, you’ve got a thousand bits of signal.

James MacDonald [1:04:31]: You’ve got AI interviewing AI, essentially.

Adam Wintonowski [1:04:33]: 100%. And so I thought, what would this look — I mean, this is the engineer in me — how could this look different? And so I thought, what if you reverse the paradigm, and engineers put themselves in a marketplace, and then recruiters go and discover them there? They’ve now got the same problem that you’ve got with human-in-the-loop, where you’ve got a context-rich engineer being sourced by a context-poor recruiter. So there needs to be some measures that you can put over that. Exposing their GitHub to a tool which can measure complexity and flaws and errors and run some passes over it, I think, is helpful for the recruiter.

James MacDonald [1:05:17]: One step further than that — are we going to a point where it’s, I’ve got a recruiter agent, you’ve got your own agent, your job application agent, one of your thousand agents. Are we getting to a point where it’s agent versus agent, and maybe a human jumps in at the last point?

Adam Wintonowski [1:05:33]: I mean, there are already some areas where this is becoming real, where my personal assistant agent will call something and they will have an AI agent, and so you’ve got this sort of agent-to-agent interaction. And I think it does need to be formalised a little bit, because there are already protocols to handle agent-to-agent communication.

My set of agents that I built — I had to feed it a huge corpus of information, relatively huge, about me, my work history. It interviewed me three times, built this whole sort of backlog of some canned answers — if they ask this, this is the direction I want you to answer. Built a zero-BS filter, so it couldn’t actually lie to the recruiter. And then all I need to do is give it a JD. It takes that JD, creates the CV, a website, an ATS-ready, ingestible version of it, and then a chat agent that a recruiter could go and have a conversation with, about me, about that particular role. And then of course it’s capturing all of those questions and feeding it back into the system to create better responses for next time.

So that’s one side of that almost solved. From my side — you guys need to catch up. And then, that whole interview process, if we can get a code review across my personal GitHub repos, a benchmarking tool against my harness, and my agent being interviewed by your agent, we could probably cut down an interview process to minutes.

James MacDonald [1:07:28]: I think what you cut it down to is the most important part. And it’s one of the things — probably one of the first questions I always ask is that motivation part. That’s what you mentioned about, if I’m in the company having to sell you on the vision, are you looking for the tooling, are you looking to solve a problem? What are the problems that are important to you to solve? Because when recruitment is done well, it’s more than just a skill set, you can tick these boxes. It’s, are you going to be motivated to be here long-term, solve problems here, add value here? What can I do for you, as much as what you can do for me? And if you get that motivation alignment part right, that’s where most of the success comes from. Even if you don’t have the skills to start with, because you can grow into those skills.

Adam Wintonowski [1:08:05]: And what I want out of a company is very different to what the next guy you interview is going to want. So I may look at your problems and go, geez, they’re boring. I may look at them and go, oh, I love the stability, I love sitting with a problem for 10 years.

James MacDonald [1:08:19]: So if you can take that technical part out of it, and we talk to the human-to-human part — that’s why I’m relatively confident in the future of the recruitment industry for a while.

Adam Wintonowski [1:08:29]: It’s the same problem as engineering, right? It comes down to a taste, or a set of things which are values, which end up being applied, and that becomes intrinsically more human.

James MacDonald [1:08:42]: Yeah, it’s the communication, it’s the empathy, it’s the understanding of motivations. The other things — problem solving, taste — these are the things that will survive, and the people that are good at those things will continue to grow. And I think, as you said, the technical side of things, we’ll get to that point, it’s just the good engineers a couple of steps ahead of the people that are applying that to recruitment testing.

Adam Wintonowski [1:09:05]: Yeah, of course, there’s bazillions being spent on it. So I think that’s a common theme within AI at the moment, this idea of an arms race. You’ve got various arms races going on, and one of them is in recruitment for sure.

James MacDonald [1:09:23]: Yeah, there are more HR tools being invested in globally than I think most other markets, because it’s a broken industry. It’s been a broken industry for a very long time.

Adam Wintonowski [1:09:32]: Yeah. And it’s a high expense too for organisations.

James MacDonald [1:09:34]: Correct. All right, mate. We can both agree on the recruitment process part — what, broken, especially at this senior end of town. And I think recruitment becomes more important at the senior end of town, because we do play to that. You’re in demand, the company needs to attract you to them and then assess your capability.

If you were to give a summary piece — what can a company do? What does good look like in attracting senior-level AI talent at the moment?

Adam Wintonowski [1:10:03]: Yeah. So I think alignment is really important. Not every senior is going to be a fit for every role. So just making sure that they’re culturally aligned, aligned with the vision that you have for the business, and really selling something to them which is meaty enough that they can still learn in — because people at that top end still want to learn.

I think salary is a big thing. Being competitive in salary is increasingly important, particularly as the wages for this kind of role inflate more and more overseas and seem to stagnate here.

I think having a strategy for how you assess what good looks like. And this is probably going to — in the short term, because if you unlock someone senior who’s good, they can unlock the rest of the team, that’s part of the value they bring — but I think in the short to medium term, it’s probably going to come back onto talent agencies more and more, leaning on them to help find good talent. Because often the good talent is sitting in a chair somewhere in some other organisation, and they’re not necessarily hunting. They’re not applying on Seek. And so people like yourself — you have relationships, and you build those relationships over time. So I think utilising organisations like yours, who have a finger on the pulse a little bit, you’re not throwing your hands in the air and saying, where are these people? You know where these people are, and you can help put them into problems that you believe they’re going to be well equipped to solve.

So I think, trust the yous of the world. And have a plan for what good looks like. Be prepared to pay more than you’re comfortable with. And also have a clear vision — or trust that this person can help you build a clear vision for what the future is going to look like. So it may even look like, we bring you into this role — how will you shape it?

James MacDonald [1:12:19]: Which aligns with something you mentioned earlier today about not being put in a box. Give you the freedom to implement your knowledge and solve problems across the business. Don’t put you in a box, solve this one problem, one problem only.

Mate, I think I might title the video, an engineer is an apex predator, a Jedi, a stallion and a unicorn. I think they’re the four we’ve gone with today. Not coming from somebody with no bias, obviously. So I appreciate your time today. Really interested to find out where you end up and the problems that you’re going to solve there.

Adam Wintonowski [1:12:58]: Great, thanks for having me.

James MacDonald [1:13:00]: Here’s my take. Adam said one of the sharpest things I’ve actually heard on the show. Most companies have done a good job of measuring costs — what tokens cost, what their licences cost, they know the salaries of their engineers — but most companies don’t have a clear picture on the value added through AI.

And that one problem explains the AI talent disconnect. Companies say they’re seeing no ROI yet from AI, and Adam’s answer is, actually, you’re not measuring it properly. And that’s a core issue behind the disconnect between senior AI salaries in Australia being too low in comparison to the US. Companies not willing to pay the $300,000-plus that’s needed to get this senior AI talent, because they don’t see the value in it yet. And it’s a measurement problem, not a technology problem.

So I’d encourage most companies, before you’re going to market, talking to a recruiter, going to try to find these senior AI engineers — understand the measurement side of it. How are you going to measure the value that this person’s going to come in and provide? Once you can see that clarity, you can clearly measure the value that this person’s going to provide your organisation, and then you can more clearly map what is worth paying to this individual.

I think if we get that part right, companies will see a lot more alignment between what they’re willing to pay and what the market’s looking for, what the market’s currently being offered if they’re prepared to go to the US. We get that part right, we get more senior AI talent staying in Australia, more companies seeing value from senior AI talent, and everyone’s a winner.

If you’ve got any value out of this episode today, follow the show on any of the channels, subscribe to my newsletter on Substack. Look forward to providing more value next week. Cheers.

This was the Building Tech Teams podcast. We release new episodes every Tuesday. Really keen for feedback on guests you’d like to see, more topics you’d like to see in depth. If you’d love to provide any form of feedback to me, find me on LinkedIn, forward slash James MacDonald AU. You won’t miss me. This episode was produced by Day One.