Salesforce Hiring Edge

AI Cheating in Tech Interviews: Ex-Salesforce Engineer Tells All

• Josh Matthews and Josh LeQuire • Season 3 • Episode 71

Share episode

Send us a text

This episode is WILD.

We sit down with Mike Mroczka—author, ex-Googler, ex-Salesforce employee,  and interview coach—to talk about the growing epidemic of AI-assisted cheating in technical interviews. We're not talking theory—this is what’s actually happening in the trenches.

Mike breaks down:

  • Why AI makes it easier than ever to fake your way into a six-figure job
  • How companies can design smarter, cheat-proof interview systems
  • What red flags recruiters should NEVER ignore
  • Why some coaching is helpful—and when it crosses into gray zones

💡 If you’re hiring in tech or interviewing for a tech role, this conversation is non-negotiable. It’s time to outsmart the system—without compromising integrity.

📕 Get Mike’s book: bctci.co/amazon
 🌐 Learn more: mikemroczka.com

🎧 Thanks for listening to Hiring Edge — formerly the Salesforce Career Show.

For the full visual - check out JoshForce on YT!

Mike Mroczka:

There's a difference between sort of having familiarity with the job and not knowing anything about coding but trying to pass a coding interview with the help of AI. Like there are lots of people now that are just like that's a $500,000 salary yes, I'd like it. Like, might as well try what's the worst that can happen? And some people pass these interviews because employers aren't realizing that you can cheat.

Josh Matthews:

Welcome to Salesforce Hiring Edge the show for leaders who want to hire smarter and scale faster with Salesforce. To Salesforce Hiring Edge the show for leaders who want to hire smarter and scale faster.

Josh LeQuire:

with Salesforce, whether you're building a team or bringing in a consulting partner, we're breaking down what actually works in the real world.

Josh Matthews:

All right, let's get into it. Listeners, a quick note about this recording. Unfortunately, I was not aware that this big, fat, sure microphone that I'm talking into is actually turned off during this recording. That means that these AirPods actually were my microphone and not the best audio. Please, if you can bear the audio on this episode, stick with it. I guarantee you there's very valuable information and expect all future podcasts to make sure that Josh has done a mic check. I really appreciate your understanding. Thank you, okay.

Josh Matthews:

Mike Marochka is author of Beyond Cracking, the Coding Interview. He's a senior engineer. He's a former employee of Google and Salesforce and he's an interview specialist. He offers coaching to candidates and helps companies build cheat-resistant interview processes. His work on system design and interview strategy has been featured on Hacker News, business Insider and Wired. Welcome Mike Marajka, welcome Mike. Thanks for having me. Absolutely.

Josh Matthews:

If you've been following the Salesforce Career Show, you're aware that we are now called Salesforce Hiring Edge. We appreciate you sticking with us through this branding change. And I'm joined also by Josh LaQuire, founder of an SI partner, c Currents. That's a letter C currentscom, if you want to check that out. If you're not familiar with me, my name is Josh Matthews. I run the salesforcerecruitercom, aka Salesforce Staffing. I've been involved in the ecosystem starting since 1999, but really heavily for the last seven years.

Josh Matthews:

We're super excited that Mike's on the show. One, because he's an author. Two, he's an incredibly nice guy and a gifted engineer. But most importantly, he's here to help you, our audience, in really understanding way more than hopefully you ever thought of or ever wanted to know about technical interviewing. And we're going to have two episodes and the first episode is going to be all about cheating with AI in interviews. Now we're going to jump into that in a heavy way here in a second, mike, but maybe you can just tell us a little bit about you and what you do right now and where you got started.

Mike Mroczka:

Yeah, I actually started in physical therapy and decided that wasn't for me, moved to basically computer science. After seeing the Marvel movies and Iron Man's kind of blasting all the people, I was like I need to do that with AI. And then now we're in the age of AI, so it's super cool. It like came right off the page and at this point now I kind of go through the interview process and I help people pass interviews. I end up doing a lot of private coaching to help people pass technical interviews. Engineers specifically have a really hard time with coding interviews that kind of crop up. You're asked a lot of things to be done, sort of on the fly, and people tend to need a lot of practice with that sort of thing. So that's kind of what I do on the side and then, in addition to that, I also help with companies in just this age of AI. There's lots of cheating that's going on, so figuring out ways to kind of combat that is also part of my job, okay.

Josh Matthews:

I got to ask you when someone's getting coaching from you, does that feel like cheating?

Mike Mroczka:

at all, not really. There's usually a lot of work that goes into preparing for interviews. Google, salesforce, even big tech companies like Netflix and Amazon as well, also go out of their way to show you hey, there's an exact thing you need to study for. And even many recruiters will give resources and say, hey, here's some links, here's some YouTube videos, go study. This is something they want you to prep for, because this is something that requires a lot of technical expertise and a lot of demand to know this sort of thing on the fly. So the ability to kind of do this and to take it seriously is a lot of what I kind of provide. Certainly, accountability is a big thing there, too. It's just sort of like, you know, having somebody to kind of whack the stick, so to speak, but it's really a day, just something that's meant to kind of help people pass the thing. You don't need coaching in order to pass it but a lot of people.

Josh Matthews:

Okay, that makes sense. Now there are three. My understanding is there's really a few different styles of technical interviews. There's what you just described, which is sort of that live coding technical interview. I have some clients that offer take-home technical assessments right, it's a mini project. And then there's the technical deep dive, the system design interview. What are you seeing out there in the ecosystem right now? That's being used the most. And of those three and I know that there are for different purposes, right, one might be more for an architect versus a coder, but is there one that stands out as the best, basically like the best type of test to protect your company from making a bad hire?

Mike Mroczka:

It's a great question. Yeah, I think the best test is one that's actually robust and tests different things. So I actually think they're all necessary and I've seen companies even add other ideas and other technical types and variants to it. Every company has slightly different processes and certain rounds are slightly different, but at the end of the day, going just based on one thing makes it almost it's a challenge to an engineer to try and gamify it and see what they can do to sort of maybe not cheat the process. I don't think a lot of people are intentionally well, not as many people are intentionally cheating as some people would be led to believe but at the end of the day, it's really important to test a wide range of skill sets and wide range of skill sets, and that's kind of what any good hiring process should have.

Josh Matthews:

Now, with the age of AI, there's a lot going on. I mean a lot going on, whether I mean there are people who are cheating in these tests and, by the way, I've been involved in interviewing for 26 years and I've seen almost all of it Proxy interviewing. The guy who shows up on site for the job isn't the person that interviewed. There's just some dude who sits there and interviews for everyone else and then they fly you out. You show up to work and they're like I don't think this is the same guy. So there's a lot of. There's been always been grift, but sometimes does it feel like sometimes people need to cheat just to break in.

Mike Mroczka:

Yeah, I mean it kind of depends on what we mean by cheating, cause there's one thing to stretch the truth a little like maybe you were a host and not a waiter in a previous job, and maybe you're you were a security engineer and not a software engineer, but you still kind of wrote code. There's a difference between sort of having familiarity with the job and not knowing anything about coding. But trying to pass a coding interview with the help of AI. Like there are lots of people now that are just like that's a $500,000 salary, yes, I'd like it. Like might as well try. What's the worst that can happen? And some people pass these interviews because employers aren't realizing that you can cheat. You can cheat through the whole process. Now if you're not very carefully trying to guard against it, you'll find a lot of people can pass the process without knowing a single technical thing.

Josh Matthews:

So it sounds like it's a lot easier than ever to cheat in interviews with AI tools and you have a unique job in this space. You've already kind of described a little bit about what you do, but how do you actually help the companies protect themselves from candidates who are leaning on AI to get the job.

Mike Mroczka:

Yeah, I like to describe it a little bit like one of those phishing emails, you know, if you've ever learned about phishing and emails, and I don't mean F-I-S-H, I mean P-H-I-S-H. So if we have a phishing attack in an email, we now kind of know, and it's just general awareness that like hey, we be careful clicking on links and emails, this can lead to malware and other sorts of problems. It's the same thing here. It's the same thing here, like awareness goes a long way to preventing a lot of cheating. Just knowing what exactly are typical signs of things that are problematic.

Mike Mroczka:

If you have a candidate that's like oh no, my camera's not working, like for one interview, sure, probably not that big a deal, and certainly there's sort of like a screening interview to begin with not a not a big thing. But if every technical interview they're off camera, that's a kind of a huge red flag and I think most people intuitively realize that. But this gets even more true as we go through other things. A lot of people are like if it's on a computer then there's a chance for cheating, so let's bring them in. And they think if you bring somebody in person, that also prevents cheating. But that's not true.

Mike Mroczka:

Nowadays we have cameras small enough to go into an ear and we have or I'm sorry microphones small enough to go into an ear and camera is small enough to just fit on a button on a lapel so you can actually get audio and video syncing to somebody outside to use chat GPT to get through this sort of thing.

Mike Mroczka:

There's a couple of big splashes in the news. I'm not sure if any of you are familiar with it, but in North Korea there is a vested interest to try and get large tech jobs to sort of fund other government projects and things like that, and it's kind of been a huge deal because they've uncovered a couple of different spy rings recently that have gone into big tech companies. So people that didn't know anything about it and people going into standup and saying what they did and not having a clue about it, but just sort of you know, being fed exactly what information to say once they've gotten the job, and then they stay for a year or two getting a huge salary to come back, you know, just to sort of you know.

Josh Matthews:

Look, it's corporate, it's corporate espionage, right is what it is, and it's it's grift, and that's been going on for a very, very, very long time. I actually knew someone. Well, I don't know if I could really say that I'm aware of someone that did that professionally for the United States, you know, for the United States government. It's definitely, it definitely happens. I mean, there are cells, sleeper cells, of these types of people. I don't think we have to worry about it at your typical 50-person Salesforce SI practice, right. But if you're at Google, if you're at Amazon, these are things that we need to be definitely aware of. What do you think is the number one thing beyond awareness that hiring managers, hr professionals and recruiters should be paying attention to? Beyond, you know, the camera?

Mike Mroczka:

Besides the camera, I think one thing that a lot of people don't devote enough attention to are questions themselves. So again, we're worried about AI and cheating and stuff like that. I actually consulted with a company not too long ago and they were convinced the whole thing like people were just cheating left and right because everyone was getting answer to the question immediately. Again and again, people would give optimal answers. How on earth could this happen if they weren't cheating? Well, it turned out it was just a disgruntled employee that left a post on a public forum with the six questions that were in the entire database. And especially at smaller companies, this becomes even more important because you're not asking thousands of different questions. You have to be really careful what questions you're asking and making sure that they're not posted somewhere publicly, because all you need is one disgruntled employee to really tank your whole interview process. Yeah Right, discord and Reddit.

Josh Matthews:

Yeah, but watch out. I actually had an interview today with a candidate and I started asking him one question. It sounded very familiar to another question that I've actually, I have a whole blog post about right and he said, well, look, I've done my research on the kinds of questions that you ask and I thought, okay, well, that's awesome, you know, that's great because I'll be able to tell these are not technical questions, this is about your failings, your vulnerability, your self-awareness of the people around you and in your life, this kind of thing. And so there's no, you can sort of prepare a little bit and think about it a little bit more, but you can't cheat it. Or, if you tried, I'd find it out, point it out real quick and I'd eventually get the truth out of you, or I wouldn't hire you One of the two.

Josh Matthews:

But I thought, okay, this is the very first time that anyone has referenced that they'd actually read the blog where I asked this question and that kind of thing and it was really fascinating. So you know, there's this fine line between being prepared and being nefarious in your research, and it seems like that line is you know, we want people to research, we want people to use AI, but we don't want people to lean on AI for all of our solutioning right, but we want them to have an awareness and use it appropriately. So, when it comes to combating AI, is there software, are there tools for protecting you? I'm not asking you to give away the store, because I know that this is part of how you earn your living, but what might be one or two things that hiring managers and people who are interviewing candidates can do to protect themselves?

Mike Mroczka:

Yeah, I think there's maybe three pieces of advice I'd give the first one when it comes to questions like, even if you had questions let's say you had a specific set of questions that you used If you just ask ChatGPT to change them a little bit, you can really twist any question and have it give you an entirely different kind of take, and then every time if you're going in with a new question, that's just vaguely different, just enough to where nobody can just outright prepare for it, especially if it's a technical question. It's very easy to do and it really really kind of prevents that sort of that scenario we just talked about of people knowing exactly what the question is and if you change it just slightly and somebody is really not technical and not a good fit for the job, they're not going to be able to pick up on it. So it's that's a very helpful thing to kind of go through. So that's one just honestly. Chat GBT is a great tool by itself.

Mike Mroczka:

There are publicly available tools. There's some tools that actually have existed for a long time. You know we used to have people that would take the SATs but for mobility reasons couldn't get out of the house and disabilities and things like that. So there are things that will let you basically have them download software on their computer and definitely it's it's an option, but in general I think that's actually overkill when you kind of couple both changing your questions and then sort of rotating, like just being aware of, like all of the other you know, kind of problems that can exist in the interview process. You're probably about 90% of the way there.

Mike Mroczka:

I think this is more about awareness than almost anything else and that last thing I'll say for just number three is that at the end of the day, when that last thing I'll say for just number three is that at the end of the day, when it comes down to this, we're not looking for a perfect interview process that 100% stops all people. We're just looking to not be the main target. So this is exactly like why we you know, if you have cameras on your house and you've got the like beware of dog sign in your yard, like that goes a long way to stopping people from doing it. So it's it's. It's very much about just sort of showcasing, that you're kind of aware of what's going on and people usually drop out just with that in mind.

Josh Matthews:

That makes an awful lot of sense, and I'd like to add that this goes for any interview question where you can tweak it a little bit.

Josh Matthews:

A lot of people I've given advice for years on the career show, like just Google, the top 10 most commonly asked questions. Type up your answers, get it down to 30 seconds, practice it. This is not a bad way to prep. But if someone tweaks those and you don't have it internalized, that's you. You're asking about you, man. You got to know yourself right, so you can't just rattle. The idea is to be able to articulate what you know, and the preparation can help with that, and you should be able to drift and slide with how any of those questions go.

Josh Matthews:

I will bring up one thing that happened the other day. I think it was on Friday, Thursday or Friday last week. I interviewed someone. This person is actually a very good friend of mine. He's an architect, he's a very smart guy, and I interviewed him and it looked to me like he was looking at notes when I was asking him questions. And I asked him and I would have asked anyone this I said do you mind if I ask are you referring to notes right now? And he's like well, oh, no, no, not at all. So his baseline of eye tracking. If anyone knows about eye tracking, you have to know someone's baseline before you can tell if they're deviating from that for lie detection. His baseline was to look down when he's accessing his thoughts and memories, and it looked like he was looking at a different monitor or looking at a pad on the table. So all I did was ask him are you looking at notes? And he was like, oh no, no, not at all, and he might have even tilted the camera to show, or something like that.

Josh Matthews:

So people may often feel nervous about calling people on what they suspect. It's absolutely, really, really important to just call them on it. Are you a proxy? Are you using AI? Did you use chat GPT for that answer? And then and then you know you'll see they'll be like, oh my God, no, that's really different than no. Yeah Right, so you've got to be able to, and if you're just listening to the podcast, I definitely recommend, for if you want to see some of these facial expressions of what we're doing, you can go to Josh Force on YouTube and watch the actual video. Okay, I want Josh to Mr LaQuire, if you could jump in here, because you have hired many people over the years. You've been involved in the Salesforce ecosystem for what? At least 11 years.

Josh LeQuire:

Yeah, since 2011. Yeah, I've been building now since about 2003.

Josh Matthews:

Okay, so a long time. You've conducted a variety of types of interviews. You've participated in some. What you're hearing from Mike, I'm kind of curious, like what's your take on all of this so far?

Josh LeQuire:

Yeah, I think. Well, mike, I'm curious. You know the code test right is a part of the process.

Mike Mroczka:

Yeah.

Josh LeQuire:

I'd imagine there's a series of interviews, a series of questions. You're interviewing with technical people, non-technical people. How important is the code test in your opinion?

Mike Mroczka:

Yeah, I guess it comes down to how much you want the person to code. You know, um, I've definitely met a lot of people that have done computer like masters, phds in computer science and can't code their way out of a cardboard box. It's amazing the number of people that can't do it, and sometimes they've gone to get the advanced degree because they actually struggle to actually do the coding itself and you know, you know academia is easier than the actual code. So it really depends on what you're looking for. If you're not really looking for a coder, it's not that important. But most technical jobs that ask coding questions, you kind of need to know what you're doing. So I'd say pretty important usually.

Josh LeQuire:

Compared to the code test. Also look at problem solving, because to be a good coder, programmer or developer, you have to understand what you're solving for.

Mike Mroczka:

How do you interview for that? Yeah, I think the elephant in the room that nobody likes to talk about, I think, is that these coding tests are a proxy for an IQ test, and that makes it sound like, if you're not able to pass it, you're dumb, and I don't actually think that's the case at all. A lot of people don't realize this, but IQ tests themselves can be studied, for you could, you know, go in with one thing and you can study and get a better score on an IQ test. A lot of people don't realize that. So, again, it comes down to like what's the goal? If the goal is to pass a certain interview, of course you're going to do everything you can to kind of try and again gamify that session. Uh, that that interview as an engineer it's like reverse engineering, of course is why we call it that.

Mike Mroczka:

At the end of the day, I think, though, the problem solving piece comes down to a couple of things. The first and most important one is just like, when it comes to technical concepts, can you communicate those technical concepts to somebody else? Like, let's say, you can't code it, you're not totally sure how to do it. It's kind of there, but you know, you're not entirely sure on the syntax. That's one thing. Then, like you totally don't know what you're doing, can I at least communicate what I would do, even if I don't remember exactly the syntax, so you can engage in this problem solving back and forth with an interviewer and interviewee without necessarily needing to focus on the code itself, that that problem solving piece can still be there regardless. So communication, I think, is hugely important.

Mike Mroczka:

The other thing again comes down to like, again, what's the role for? Is this role a junior engineer? Is this like a senior engineer, if you have some sort of architect style role, like that's going to matter a heck of a lot that they need to be able to communicate technical requirements very clearly and they also need to know things a lot more deeply than if it's a junior role. So yeah, I think those are kind of the two biggest things. Does that kind of answer your question?

Josh LeQuire:

I think it does. I think what I hear you saying is you have to calibrate the depth and breadth of the test of the problem solving capabilities for the role and also for the company and its industry, right, like for somebody who runs a consulting practice. For me, the delivery process is as important the ability to communicate with clients is as important as your technical depth and writing lightning components. You know writing Lightning components or Apex, right so. But if I have you in a job to write Lightning components or Apex, I need to know you can write Lightning components and Apex too, right so? But I would say I'm curious, you know, for companies like Salesforce and Google, where you've worked, you know how is that different than you know, let's say, a services provider, professional services company like Seacurrents, in terms of what they're looking for in the interview process?

Mike Mroczka:

Yeah, I think at these big tech companies you have to realize there's 100,000 software engineers that are all writing code. They get into a very big code base. Google has one of the biggest code bases on the planet multi-billions of lines of code. And it's like I mean, can you imagine that lines and lines of code? It multi billions of lines of code. And it's like I mean, can you imagine that lines and lines of code? It's like how do you actually navigate that? It's overwhelming.

Mike Mroczka:

Especially, junior engineers come in and it's very common for them to have no idea, like, even if they're very good and very technical. It's like where do you start with that? Like, handling ambiguity becomes incredibly important. So these types of big fang I guess big tech companies in general fang style companies ask very specific things that are all around scale, because they scale very at a rapid rate. So if you're going to write code, it needs to be able to last 10 years and it also needs to be able to support a billion people. And those skill sets are sort of very different than if you're working at a startup but you're still building, like you said, lightning components and other things like that. Those are important but it's probably not like 10 billion people are necessarily going to use it. It depends on, of course, the business, I suppose. But at the end of the day, we're kind of judging different things. So from a technical perspective, you can look at things like how well does the code scale? You can look at things like how readable is the code? If I needed to change the code to accommodate, like, something different that the user was asking for, then like, can my code be flexible, as we call it extendability? And at the end of the day, those things are definitely going to be important and they're going to be more important as we're sort of using it for more and more people. But the use case is different, you know. So Google cares about will this exist and still be able to exist 10 years from now, whereas you know if you're writing something, maybe, and still be able to exist 10 years from now, whereas you know if you're writing something maybe you care about it, like, will this work tomorrow, which is when it needs to work for so speed? It becomes a lot more important.

Mike Mroczka:

I think startups and things like that Meta is actually controversial like that because they're always about move fast and break things and they don't really fit the typical archetype of going through it. Mike, what has not been shared with our audience around cheating with AI in interviews that they should know and I mean the candidates as well as the hiring managers, kind of thing I think the thing that AI still hasn't quite figured out is that human connection piece. So whenever you're going to interview somebody, and whether they're an engineer that's going to sit, you know, in a dark room all day like kind of coding, or if they're going to be interacting with people, it's good for them to be able to kind of communicate and kind of have that human connection. I think, at the end of the day, if you ask a question like hey, no-transcript, and it's like, can they connect on an emotional level, ai doesn't do that, at least not yet. So I think people need to be aware of that.

Josh Matthews:

This is funny, like what you just described as someone using AI to cheat. Is like when I'm reading my little that someone's given me or that I've typed up. It's hard. Mike Morachka, author of Beyond Cracking the Coding Interview. Thank you so much for joining us. Please stay tuned, everybody.

Josh Matthews:

We will have Mike back with part two of this conversation, where we'll be discussing hiring the right people. You're listening to Hiring Edge and Josh LaQuire. Thanks so much for joining us again, mike. We do want to make sure that people can connect with you and find you. Now. Beyond Cracking the Coding Interview can be found at bcpcico forward slash Amazon. We will put that up on the video for people to see. If you're watching this on Josh Force on YouTube, they can also check out your website, which is Mike Marochka. I'll spell it it's M-I-K-E for Mike and then M-R-O-C-Z-K-Acom, and on LinkedIn, I promise you it's going to be really easy to find Just type in Mike M-R-O-C and it'll probably pop right up. So you're good to go, mike. Thanks so much for being on the show and everyone else, stay tuned. We'll be back with some more incredible insights from Mike. Bye for now.

People on this episode

Head Shot

Josh Matthews

Host
Head Shot

Vanessa Grant

Co-host