Today we are talking with Shannon Houg, with Karat with a K. She is the global head of solutions engineering. There Karat gets into technical interviewing for companies. But Shannon, just go ahead and tell us a little bit more about what you do. And Karat gives us the quick introduction. Yeah. So like you said, Karat is the world leader in technical interviews. We have created the first technical interviewing cloud which is an always on, scalable and consistent human plus tech solution that is used for conducting engineering interviews on behalf of company 24/7. We conduct technical interviews for more engineering candidates than any company in the world in 2022. And the data from those interviews is helping us develop more strategic insights that companies can use to improve their hiring decision. Okay, great. Now we’re talking to a lot of different people about just what’s going on with technical talents getting more and more hard or harder to hire the right people to find the right people. There’s 500,000 open it roles here in the US. There’s competition for that talent and companies need to do different things to bring them in. And there’s a problem with a lot of companies. How do we qualify who this person is and how we go through technical interviewing is a tough thing and it’s a problem for a lot of companies. You guys have a different answer and this is something that’s very close to me. I have done at least 500 technical interviews. I have a long time, long career as a software developer. I was always the person that the recruiters went to to do the technical interview. I’ve been doing recruiting for myself and I’ve continued to do that. And you guys have done something very different. I’m just going to start off a thing I see in a lot of companies. There are a lot of tests out there that you can have candidates do and a lot of them wind up just being technical trivia or got you questions. You all are different. Proceed. Tell us about it. I think the most important thing to keep in mind right now is that we’re in a candidate first world. I mean, first I can relate. I have been writing software for a quarter of a century and I asked the same tangle questions for ten years in a row. Count the number of lower case as in the null terminated string. Right. But what was I actually testing for? I never been trained to think through what are you looking for in a candidate? So the first question I would have is what are you asking a candidate? Like what are you looking for when you’re trying to got you with some question that may or may not have an answer. Are you trying to get them to do your work for you? Are you just trying to trip them up and see how they handle it? How do you even measure that metric? In a candidate first world, the majority of folks are going to be like, that’s not okay with me because I don’t know what your expectations are. One of the first things that we tell folks is it’s really important to let candidates know exactly what they should anticipate in an interview. Hi, I’m going to ask you these questions, and all of us know there are million ways in which we can answer these questions. But what we’re looking for is the optimality of your solution. Or, hey, we’re looking for you to build tests inside of your solution, but without letting the candidate know what they should be working through and what competencies that you’re looking for. Whether it’s optimality efficiency, debugging and testing, it’s pretty impossible for them to succeed. And candidates need to be set up for success. You can’t afford to have false negatives today. And those got you questions 100% do that. And then, David, we talked about this earlier. What’s really interesting about the test, we often see that as a take home, and we see it as, hey, this may take six to 8 hours. You’re actually building out a product. Well, think about equitable hiring practices. People have lives. They have multiple jobs. Sometimes they’re going to school at night. They have families. You’re really focusing on a core group of developers who have that extra time in their lives where they can spend 6810 hours. Think of all the folks who can’t. And so really, in today’s world, candidate first, there’s not enough talent on the market you cannot afford to be inequitable and not set expectations. And think about the candidate experience first. You’re 100% right on that. And I think I often say if there’s a candidate, you have a decent skill set, you can go to LinkedIn, say open for work, and you’re going to get ten calls this week. And if three of them are saying, hey, we want you to take this six hour assessment, those are three that they’re not going to return. But along these lines, even just with the six hour assessment, there are a lot of people out there. The moment that they hear take home test assessment, they just shut down. I’m not going to apply for this because I’m not going to do that. And then talk to people and find out about the company. Then the way that Karen has its structure, it’s more of a technical interview. Do you want to speak to that a little bit? Yeah, 100%. So what data has told us we’re a data driven organization. That candidate always comes first. And that’s just really important to keep in mind. And what we learned was through our data and through our experiences that candidates with a nontraditional background or minority candidates are more likely to drop out of the process if there’s a take home test. And there are a lot of historical reasons we could talk about the implications of standardized testing for under represented communities and the like. But at the end of the day, the fact is that these folks are going to drop out and you need them to get hired. So what Care did instead of two things. One, it needs to be a human experience. There needs to be somebody there that you’re working with today. About over 40% of the candidates that get jobs out of carrot, especially in senior positions, need some sort of back and forth, some sort of hand holding or communication throughout the interviews. Imagine how many people you would be missing out on without that human experience. So what we did is that we went and we recruited all over the world for engineers with, on average, seven years experience. We actually only take about one to 2% of the folks that want to work with us as interview engineers because it’s really important for the candidate to feel comfortable and at ease so that they can showcase their best selves. We have, I would say 18 or 19 countries in the world now where we have human beings, they go in. They also go through a pretty strict and rigorous certification process. They need to have EQ. They need to make sure that there’s no bias inject in the process. We need to ensure that they’re following the rubric. Fun fact, our interview engineers fill out about a little over 70 inputs throughout a 1 hour interview process, which is insane, but that’s how extensive our roof work is. That’s why we can make data distinct, data driven decisions around the efficacy of the content that we’re actually producing from an interview question point of view. But our interview engineers, they won’t know anything about the candidate coming through. They’ll fill out this extensive rubric. We have teams that will align how to weigh the different competencies for what the client is looking for. We have a second interview engineer to come through and make sure that the first one did their job and didn’t inject that bias. And then we only pass the recommendation that comes from our engine, not from the interview engineer. So really doing everything we possibly can to be completely data driven and not inject any bias in the process, all but still marrying that with that human interaction of setting up that five minutes, making folks feel comfortable, ensuring that they understand what we’re looking for with the questions that we’re asking and really just being there if they have to ask questions or work through a problem together, great. A lot of companies that are still doing all their technical interviewing in house. And I’ve seen this 100 times, hey, development team, you need to go talk to this person. And then someone is immediately Googling questions to ask and.NET interview. And we can do better. Everyone can do better. And tell me a little bit more, because some companies are not looking for the rock star, machine learning data scientist person. Some companies are looking to validate this is an entry level person. They’re going to be doing some SSRS programming inside SQL. You guys can cater a little bit toward different levels. Can you speak to that? Yeah. The secret to that is a few things. One is that we’ve done well over 100 and I think 20,000 interviews now across the industry at different levels, anywhere from University recruiting to L Five, L six, and hands on director and manager level. Essentially what we did is we built a modular interview format. And so we have different components of an interview format that we can move and change out in order to align with two things. One, what are the competencies that folks are looking for? And two, how do they want to weigh those competencies in order to determine level. Right. And then we work with them. We have some more senior content that we can bring in for senior folks. We have some more entry level content for folks that are for interns and University recruiting. And then we can modularize that and switch and swap things out to see what works for each of the companies that we’re working with. So it is a very white glove. Make sure that we’re looking for what that specific client and that specific team would like to focus on. And we can level based on the experience that we have in the market. So it’s really a true partnership between the two. We’re using our data and our experience to help them focus on the competencies that are important to them. And you said it best. A lot of folks get into this problem where they’re going and they’re looking online like, oh, my gosh, I have to get this done. I don’t know what I’m going to ask. Okay. I’m just going to look online, pick three or four questions and go in. Well, companies, we’re starting to see them work with us to think about the process because they’re having trouble, especially in the market today. And so what we like to do there is just encourage your team to get to the other, have a conversation about what competencies you’re actually looking for, and then come up with a question or two and how you would level that question and just start with one question, maybe two per person who’s going to interview. The minute that you start having even a checkbox or some sort of defensible position, you start to think through, oh, okay. This person came in, they answered this question this way. How would I actually level that? I think I love that a little bit more junior than senior. I better add that to my rubric. And it becomes kind of evolution that can translate directly not just to your interviews, but also to performance on site and lovely. So you’re working with a lot of companies all over the place with that as a background, what do you see that companies can do to compete in the global talent marketplace here? What can they do more to get the right people that they want? Yeah, it’s all going to be about flexibility in working with a candidate. So making sure that Carrot is a 24/7 service and this is really appreciated. In fact, over 60% of the interviews are done on nights and weekends. You need to give candidates that flexibility to interview when they want. I guess, like I said, more than 60% elect to do nights and weekends. Speed is another piece that we want to focus on. Interview at speed of your candidates is extremely important. You need to dedicate interviewing time to create an on demand scheduling for candidates. Our data shows that a candidate who interviews in the first 72 hours not only is more likely to perform in the top tiers, but they also have better close rates because you have the opportunity to give them an offer before other companies have. I think the other pieces to keep in mind is expanding your applicant pipeline. You need to diversify your candidate. We talk about that a little bit, and you’ll see it interspersed in our conversation. But essentially, you need to filter in more direct applicants. One tactic that we have recommended to clients is to review the technical interview pass through rates for each of your recruiting sources. So if your direct application pool is close to your proactively recruited candidate pool, experiment by increasing the number of direct applicants that you let through. Right. Like, try loosening your pedigree requirements and send through 15 or 20% of the direct applicants to the interview. There’s a lot more work that needs to be done on the job description. Anyways, we talked a little bit in the beginning about ten X. When you add ten X to a job, most likely you’re having people drop out even before you know. And then the last piece of something that we touched on. Right. You need to make your process a lot more transparent. Once you do start that sourcing and you’re sourcing more direct applicants, it’s really critical that they’re being set up for success. This is why we use human beings. We don’t send home folks with a test where they don’t exactly know what the expectations are. Right. A candidate who has no connection to the big tech world, they’re not going to have the same knowledge of the hiring process or interview questions as a candidate who, say, has a family member that’s working at your company. So really just let’s create an equal place for everybody to get started on the same foot. Make sure folks know what’s expected so that they’re not spending time trying to do something in interview that isn’t going to give you any signal and tell you whether or not they’d be capable of doing the job. Great stuff. So I appreciate it. I’m a big fan of Carrot. Technically, interviewing is just a place where a lot of companies struggle how to get that right. You have to do it. You want qualified people, but you don’t want to limit that candidate pool because they don’t want to look at this got your technical trivia test what a lot of them are, and a lot of good people out there just won’t do it. But if it’s an interview with a third party, hey, I will have a conversation with somebody and talk about it. So I think that what you’re doing is a great thing. A lot of companies need to be made aware of that, and hopefully you can spread the word a little bit. All right. So who else should reach out to? Karen, who are you looking for? Whether companies or people to do interviews? Who would you like to reach out to or find you? Yeah. So first, we would love to reach out to candidates. We’d love to hear about your experience with Carrot and get your feedback. I think the second folks that we would love to reach out to, obviously are going to be engineers who are interested in learning and conducting technical interviews, helping folks get opportunities. We always like to say interviews are opportunities to change people’s lives, and it’s a great and rewarding job. And we have an excellent full of interview engineers with mentorship opportunities and so on. And so I always encourage that. And of course, for companies, anybody who is looking to hire engineers today, I would say reach out, talk to us about some of the trends that we’re seeing in the market. Talk to us about equitable practices and hiring. Talk to us about the impact on your business. If you’re sending out job descriptions that say Ten X engineer and have very traditionally masculine language, these are things that we deal with every single day. And the fact is that engineers a couple of years ago spent on average, about four years at their company before they switched jobs, that in the last couple of years has dropped down to two. So the engineers that are on the market, they’re your engineers. That’s why it’s so important to get this process right and also to be the culture that you’re trying to build. That’s an expansion of Carrot. But let’s say that you hire a bunch of developers and you have these equitable practices, and these diverse candidates are coming through, and they’re getting opportunities to change their family’s lives with your company. And they’re really excited. And then they get inside the job and they’re like, oh, we don’t have equitable practices. We don’t see that same rubric for leveling or growth potential. So these are really important conversations in the interview is just the beginning. So let’s get that conversation started as soon as possible. Let’s get you to talk about equitable hiring practices and learning from all the things that we’ve seen in the market. Right. So I pre appreciate your time today. Thank you very much. And we’ll be in touch. All right. Thank you. All right. Bye.