The GenAI Cheating Shift — How Artificial Intelligence Is Rewriting Exams | Exam Security Summit 2026 [Dan Hughes, Michael Nemarich, Jarret M. Dyer, Radhika Bhatia]

How real-time AI assistance and external orchestration are quietly undermining traditional exam controls

Panel Speakers:

• Dan Hughes (Vice President of Product Development at Talogy | Chartered Psychologist)
• Michael Nemarich (Chief Operating Officer at NAATI)
• Jarret M. Dyer (Vice President of Test Security at Surpass Assessment)
• Radhika Bhatia (Senior Product Manager at Talview)

Transcript

SPEAKER_00 0:00

Welcome to the panel discussion for the Exam Security Summit. I'm Radhika, and today we are getting into one of the most urgent challenges in the assessment world right now, what I'd call the Gen AI cheating shift. Few years ago, cheating was largely an individual act, one person, one device, and one moment of panic. What we are dealing with now is fundamentally different. AI tools that work in real time, coordinated answers shared at scale, and evasion techniques that are genuinely hard to see. The controls the industry spent decades building were not actually designed or prepared for what is happening in the world right now. At Talview, we work with the organizations like the ones represented on this panel. And what we are seeing is that the old model of detecting malpractice in the ACT has fundamentally broken down. The ship we are focused now is moving from isolated event-based detection to pattern intelligence, reading behaviors across sessions. That's the lens I wanted to bring in today's discussion and conversation. I'm joined by three people who are living this problem across language certification, employment assessments, and large-scale awarding programs. Before we dive in, I'd love each of you to introduce yourself in your own words. Jared, let's start with you, followed by Dan and Michael Janet.

SPEAKER_03 1:23

Well, hello everyone and welcome. My name is Jared Dyer. I'm the Vice President of Test Security at Surpass Assessments. I've also worked in the higher ed space for uh over 20 years. And so today I'll be bringing what we've been seeing with Gen AI and assessment forward.

SPEAKER_00 1:40

Thanks, Eric. Dan?

SPEAKER_01 1:43

Hello, everyone. So I'm Dan Hughes. I'm vice president of product development at Talogy and uh chartered psychologist. Um I've I've been working in the field um for around 27 uh years, um, specifically focused really on talent assessment and development. So covering um things like pre-employment hiring and then use of assessments in for for leadership and people development as well.

SPEAKER_00 2:10

Thanks, Nikon.

SPEAKER_02 2:12

Thanks. Uh Michael Nerch, Chief Operating Officer at Nazi, we're the accreditation authority for translators and interpreters in Australia and New Zealand. Uh, we tested in 98 languages last year, uh, and all of them online. So it's been a huge shift, uh, as you've said, Rodrika, in the last couple of years of how people are trying to attempt to get around controls.

SPEAKER_00 2:37

Thanks. Uh, this is great. Uh, we are coming from different worlds, but I think today, in today's conversation, we are going to find out a lot of common ground. We'll start with the big picture. So I want to hear from each of you separately. Michael, we'll start with you. Nati has been running these exams for a long time. Um, when you look at the cheating, uh, how cheating what cheating looked like two years ago versus what it looks like today, what's actually different and what's the shift that you are seeing that you weren't seeing before?

SPEAKER_02 3:09

Sure. Uh I think it's worth pointing out, as you've mentioned, I think already uh people have always tried to cheat. It's just the method in which they're doing it. Uh and even I think in the last two years, what we've seen is more advanced technologies in there, but it's still got a human component. I think that's probably worth pointing out as well. Uh, the most, you know, almost hilarious comment uh is this year we've seen a um a group of people start trying to wear wigs to cheat, which doesn't sound like much uh until you realize they're wearing airpods with live translation functions on it for our interpreting tests. So some of the methods are crude, like wearing very ill-fitting wigs, uh, but that technology that they've got in the background that's supporting them is uh quite challenging for us. I think that um the rapidness of it and the discreetness of it, I think the discreetness of it's really the thing that's standing out to me at the moment. Quite small devices uh and uh you know, text appearing on screen that's sort of trying to get around uh screen share software.

SPEAKER_00 4:18

That's a very interesting take, Michael. Before we deep dive into this, I'd love to hear from Dan and Jarrett as well. Dan, the padding assessment world has always had the challenges of being remotely and largely unmonitored. So, how has the context changed now that everyone has AI on their phones is readily available?

SPEAKER_01 4:37

Yeah, sure. And I'd I'd probably echo some of um Michael's points there that I think um cheating actually is not been uh it has always been there in in certainly in the talent assessment space um as uh since the time that we went online. So it's always been possible, but Gen AI has um provided a a new very sophisticated avenue for people to gain an advantage. So that's what's really changing, I think, is the the the the nature and and and opportunity that's there. So it's it's a form of cheating that someone can do alone, so they don't have to admit to anyone else that they're cheating. Psychologically, that's I think relevant. And um, as as Michael mentioned, that the some of the um the technology that is available is is getting very sophisticated at analysing problems providing solutions um to things. And there's there's a a tool that I find quite interesting on online site called Humanities Last Exam that you might have come across, which is this um set of um very complex, sophisticated questions that um uh experts have said across hundreds of subjects, and it's been really interesting to see they they sort of benchmark all these large language models against it, and just to see the the pace of the improvement is quite striking. So I was looking back at um so chat GPT 4.0 at the end of 2024 was uh was only getting 2.7% correct. Um, now 5.2 at the end of um 2025 was getting about 30% correct. So that it's the the speed at which the the technology is in improving, I think. And and it it does create some particular challenges in the talent assessment space, I think, because that's operated very much on a trust-first policy. There's not been a high level of proctoring, most of it has been remote and unsupervised, and I think this is this is really affecting how organizations are thinking about it.

SPEAKER_00 6:33

Got it. And so similarly for you, Jared, I think you work across a lot of different programs and exam types. Hearing what Michael and Dan have shared, uh, do you see the same patterns across these certification programs? And from your vantage point, what feels genuinely new new about the movement versus cheating challenges that the industry has always faced?

SPEAKER_03 6:57

Yeah, again, I'll say the same exact thing. You know, cheating has been around for thousands of years. I think what is fundamentally different now is we often before were just either, you know, we're kind of looking at like somebody who's bringing in something unauthorized that benefit them on an assessment, or they were trying to remove it for the benefit of somebody else. We're kind of looking at those two different those streams as separate. And now with the technology, with the tools, with the metaglasses that I'm wearing now, right? It's very easy to not only be harvesting that content, saving it in a file, but at the same time sending those picture files out to you know an LLM, having them answer it, come back through a Bluetooth device and give you answers in real time, all independently, and through an app that maybe you vibe coded by yourself for 25 minutes the night before. Um we're seeing now with smart devices more than ever uh pure abilities to buck to lock down some of our lockdown browsers and our other security systems because they're not based on the system, but even those that that are um agentic AI has now allowed us not only, you know, before it was like you'd have a cheat sheet and you looked up the answers, or maybe you got it, or you're really sophisticated and you had some kind of wireless device and you had a proxy feeding you the answers, but you were still engaged in it, right? Now we're seeing literally a student can fall asleep, and amazingly the computer is still populating answers, right? Because on the back end, um, they've frankly some some of the codes are very sophisticated but very simple that allow for bypass, um, eat even right when when the network card and the wi-fi card are are technically turned off. Um we're kind of seeing things uh quite more in-depth. And the other part, I think the assessment industry kind of globally is is also in an interesting space because I would tracking the academic integrity conversations and and prior, I think, to the pandemic, we had a lot of conversations about essay mills. SA MILLs really never hit the US as much as I know they hit colleagues in Europe and in Australia in the higher ed space, right? Where you would find students paying people to write papers for them. Um, the assessment space is a little bit different than that, so we widely hadn't seen the bad actors involved in that space in our space. However, Chat GPT came along, SA Mills pretty much went defumped because you can write your paper by yourself, right? So then where's that new revenue stream? Wow, we now have something that can allow us to proxy test for a whole different market. Uh, and so I find it very interesting. A lot of I call the bargain basement, like the cheaper, more malicious software packages that are being offered out there, are all routing back to the same um general areas where those essay mills had proliferated previously. So it's the same bad actors, but now in a different space. And the assessment space, um, I didn't honestly, you know, we were used to you know some of corporate companies kind of coming in, but weren't really looking at some of these really bad actors. Um, so my my plug with this is what we've then also seen beyond generative AI is those who engage in these cheating services are now reporting that they're also getting extorted, right? So pay $130 to help someone proxy on your test for you or use their AI bot to do it. You pass, you get the job that you want, you go see Dan, you're hired, and all of a sudden they're telling you you need like 750 US, or they're gonna let all the parties know that you cheated on your test.

SPEAKER_00 10:50

That is true. That is very true. Just deep diving into that, uh, Jared, I think what you mentioned in this in similar context, what about like content leakage as well, right? I mean, answer key leakage, coordinated sharing online. It's surprising to see how many telegram groups are running which are selling the answer keys online. And how do you how how do you handle those situations?

SPEAKER_03 11:15

So, I mean, so that's a great question. And in many ways, I would say from that perspective, from a test security perspective, the we're operating the same way that we have before, right? So we're what what I like to tell in these talks is there's so much of a narrow focus just on generative AI. And it can be like kind of scary because it's nebulous, right? But the the way that we approach all of the threats really hasn't changed. It's awareness, it's making sure that in person, that IDs are checked appropriately, that proctors are changed online, that proctors are changed, that technology is still continuously evolving. And I think that's the biggest key right there. Um, it did feel that as for some awarding bodies, it's like we were in the past, we were able to check some boxes and say we're secure. Now, if you don't have someone that's on Telegram, on Reddit like I do pretty much every night, then you're just not staying. It's almost like fashion trends. Like you have to stay in the know with how these things are evolving. Back to Dan's point, 4.0 to 5.2. If you haven't like asked the same questions of Chat GPT, you need to, because you need to know what a curious, cheat, adjacent individual that's on the fence about cheating is able to do. Uh, case in point, I'm not a coder, but I'm code curious, right? I'm an ethical cheater. So what I try to do is try to see what people can do with the basic amount of knowledge and tools. Six months ago, I spent six hours and a lot of foul language with Chat GPT trying to code out a cheating app, and we kept getting it wrong, right? And little things, if you understand how LLM like have really, you know, gotten into them. Like they would give me a bunch of code for Python and then it would say, put it in this folder, for example, cheat my test. And three hours later, we had a problem with the code, and it rewrote that I had to put it in a folder called cheat the test. And if you're a coder, cheat my test and cheat the test, they don't go to the same place. But for an LLM, it makes perfect sense, right? At least it did for 4.0. Um, or actually, I think this was this was later, this was like 5.0. Um fast forward to a week ago, I forgot my cheating laptop for a presentation at home. I had to recreate this code to cheat, to show, to give a demonstration to individuals. It took me 25 minutes to do the same exact thing and it worked flawlessly. So we went from six hours to 25 minutes in a four-month span. That tells me that the candidates that are as curious about doing that and the operators working out of Africa and the Middle East are are right in step in the same way. So we need to be right there with that.

SPEAKER_00 14:09

Got it. I think that is a very interesting take. Uh, but one follow-up question on that, Jared, is what is what is the kind of cheating pattern that is really hard to detect in today's world, given with all the lockdown browsers, proctors available. What would be that one thing that you think is really hard to genuinely hard to detect right now?

SPEAKER_03 14:29

So, yeah, I mean, a lot of these uh AI-assisted apps that are are very sophisticated at um bypassing many of the lockdown, the current standard lockdown browsers on the market. Um, I've I've seen multiple recordings of sessions of three proctors are staring at an individual, and and it's you you know something is happening. You cannot truly detect what it is because the app is running invisibly, uh, it's copying and pasting in a very minute way. Some of these uh software they'll take a writing sample of the student beforehand or you know, a cadence. And so when they're not just copying and pasting an entire essay, right? It literally is going at the speed of which the student had, you know, or the candidate had presented before. Um so that's where Proctor Training comes in. Because honestly, as as humans, we're not machines. It's great every single one of these situations, like the fingers don't align. Like uh, so someone's pinky kept hitting the enter button over and over and over again because this is all they were doing while they were, you know, it was happening in the background. Um, so there are certainly tells, but if you're not trained to do so, and this is why we've been pushing proctor training in person and online so heavily, um, there are some things that you can look out for. The other half is people get bored. We had we had two situations 10 minutes in, they were locked on, man. And then all of a sudden they start like the test is still going. I'm like, there's cute pure tell right there. Um, so you know, in situations like that, flags, and then on the back end, it's the data, right? I think that thank you for bringing that up because that's one of the most important things. If you don't have a fairly robust data stream in person and online, um, some of these things you're gonna miss. And it's it's not like it's the age-old. Why didn't you change in the US? Why didn't you change like your grade from an F to like a C plus? You might have gotten away with that, but everybody goes from the F to the A, the highest score, right? So it's the same. You got you had a 19 on this particular assessment a month ago, and now you got a 99 out of 100. There's no way, like it just doesn't make sense.

SPEAKER_00 16:46

Yeah, that is true. I think uh the the way we need to look at it is more uh more from a signal perspective, how signals match on what you see on screen, what is actually happening behind the scene. We'll talk about uh that a bit more. Michael, a similar question for you. Um, AI is uh actually genuinely capable of translations now. Now, when a candidate's performance could plausibly be AI assisted, um, how do you approach determining what's real? And what's the broader consequence if the credentials get through that shouldn't have gotten through at the first in the first place?

SPEAKER_02 17:24

Uh I'll start with the consequence. Uh, human lives are at risk when interpreters don't know what they're doing. So that's why we're super committed to making sure that people don't get through. I think it's multi-layered though, as well. Um the benefit we have at the moment, uh, which is our advantage is rapidly eroding as these technologies improve. Uh, but it's not good at all languages. That's the start. So 98 languages last year for us. It's excellent at Spanish, it's excellent at Mandarin. It's starting to get worse as you go down the list of languages uh and the numbers of speakers. So we've definitely got an advantage in that. Uh a large amount of our tests are um oral, so interpreting tests. It's not good with accent, it's not good with voice recognition. And how many times does Siri get basic commands wrong? Like I know it's improving, uh, but I think that's for us part of the um trouble, I guess. Uh or the the arms race is you know, we do um the big thing for us is new material, lots of new material all the time. Uh, and the benefit of that is you know, people haven't been able to train it on things. Um and that I think very much as Jared said as well, you're looking for those human signs as well. So again, we've got decades of data of what pass rates look like. Um as soon as you start seeing upticks in certain language groups, you know that there's maybe a tool that's being used out there, and that's something to look at. Uh, we actually just introduced to our terms and conditions last week if we if we have a gut feel something is wrong, we will do a live interview with you. So we can't pin it. We're watching the screen, we've got all these amazing human um examined vigilators and proctors, and they know something's wrong. It's the cadence, the voice, it's the speed of the answer. We don't really know. So we're just gonna do a live interview. And that's that's how we're gonna sort of yeah, it's always an evolution, I guess, but that's something that's not gonna be as easy to trip.

SPEAKER_00 19:39

That is correct. So it sounds like that the signals that we were talking about during exam becomes very important for you as well. Response patterns, consistency across your spoken and written tests, uh, is that the direction that you are leaving to as well, right?

unknown 19:55

Correct.

SPEAKER_00 19:55

Moving on to Dan. Um Dan, there's an argument that I've been hearing a lot in the industry is that using AI uh effectively is itself a valuable workplace skill now, right? And maybe we should be assessing candidates on how well they are able to use it rather than trying to catch them using it. Uh, what do you think about this?

SPEAKER_01 20:17

Yeah, I think it's a it's a really um interesting topic um to explore. I think there's there's some genuine tension um right now in terms of certainly when I look at pre-employment um testing around this issue. So I think there for tests that have been um designed and developed um to be completed without Gen AR use, there's still this sort of need to try and maintain and ensure the security um, I think, of those assessments. But when when you look at how the, you know, as we talked about, how the technology is is advancing so rapidly, it always feels like a bit of an arms race around. I think the you know, this this point about recognising that that many roles will um be uh you know, performance and that role will be drawing on someone's ability to use Gen A, it kind of makes sense to to sort sort of shift thinking to how we can develop assessments that that can effectively um uh incorporate that in in the process. Because it is a it's a view in the certainly in the sort of talent assessment space. I think I think the the fear has been will um will will the wide more wider use of gen AI and cheating lower the validity of of the assessments, the predictiveness in the workplace. But but there is this uh this other side that actually if we can we can capture the assessment of of someone using these skills, those are skills that will be very important um for the job as well. But but I do think you need we we need to be thinking about how we design assessments. That are designed from that principle, rather than maybe saying, let's take a reasoning test, a cognitive ability test that we had already. And okay, if they use Gen AI and do well, that that's a that that tells us something, but it's not telling us necessarily about their own cognitive capabilities. I think designing assessments that specifically are intended to look at Gen AI use. And I think you get into some interesting um design questions there. Like as we've been think about it, you know, do you do you allow people to use whatever gen AI tools are at their disposal? There's such a variety, you know, you pay more, you can get more more um capable, capable tools, or do you try and maybe embed a gen AI tool within an assessment and have them using it so you can actually sort of observe how they use it to support in the answers that they come up with or how they solve the problem. Of course, I wouldn't rule out them using something else, but it would then be a bit more visible if they're not really using the tool directly that's embedded in the assessment to solve the problem. So yeah, it's a really, really interesting topic. And I think it's very much the way it will go. I think they all have to be a sort of area of skill set that is tested for people coming into roles.

SPEAKER_00 23:12

That's correct. That's a great way to look at it. I mean, there are some recent research and reports where uh they were uh interviewers were given some answers given by ChatGPT versus written by human completely themselves, and in a in uh a lot of scenarios, they favored the interviewers, ended up favoring the codes or the uh codes written by ChatGPT rather than the ones written by actual human, which kind of penalizes the honest candidates. So we need to look at the shift and we need to look at how we can incorporate both of the things as well. All right. Um, the this the next question that I want to ask is through to all three of you. So across your respective spaces, um employers, be it regulators, certification boards, where do you feel the awareness of the Gen AI treating challenge is still lagging? And what's the biggest blind spot and how the industry is responding to it right now? Anyone who'd like to answer, Tarat or Dan, Michael, maybe sure.

SPEAKER_03 24:18

I'll go first on that one. Um I think as with as with in so many uh spaces, going back again to what Dan had said, how quickly from last year to this year the technology has evolved. There's always in, especially in the education side, which kind of moves over into those the awarding body side, that there's there's just a natural human uh lag in those areas. And so when I speak to some individuals, they're still saying, like, well, for chemistry and for mathematics, that it's not really that good. So you're absolutely right. Your your headspace is probably a couple of months back, it's much better even this week than it was last week. Um, and so that delays, I think, response patterns as well in in approaching updates to assessments and updates to security, et cetera. And so my sincere wish is that everyone kind of got on the same speed. I'm like, I need everyone to start vibe coding something. It could just be like, how do I navigate my calendar and just see iteratively how fast this the technology really does progress? And I think that would help plan strategically our our next steps in kind of the assessment industry. And so that's why I'm still seeing the gap. Some people just are are not quite there, or it's they're they're still doing the same questions. Oh wow, hey, XYZ, uh, what's the weather outside? Like there's so like that's the seems to be still like the initial question engaging with an LLF. And it's so much deeper and richer. Um, and a lot of areas just haven't gotten beyond those initial, like kind of curious questioning to really understand how well um uh how sophisticated they are. So I would I would say a lot of the key stakeholders need to align themselves with like an AI futurist or somebody that daily is pushing the envelope for AI and getting that feedback.

SPEAKER_00 26:22

That's a great answer, Jared. Dan, what do you like to what's your take on this?

SPEAKER_01 26:27

Yeah, I mean, uh so it's it's again, it's interesting in the from the talent assessment space because um the there's been less concern generally about sort of very sort of widespread large-scale cheating. It's probably more been this concern about the casual cheaters and as well always a recognition that um that there was always sort of acceptance, well, maybe there will be a proportion of people that that will cheat, but um the this the where where um this typically comes to play is these early stage screening assessments, and these people will go through to an interview and to assessment centres and things like that. Um, so there's an opportunity to to sort of pick up on it there. But I I think so so I think organizations when I when I look at the talent assessments, they're probably they're they're very aware around it, but probably um maybe struggling to to navigate how uh how to deal with it when the approach has been very much more a trust-first um policy. So I think it's probably quite different to the the credentialing licensing uh space where but I think the emphasis on test security has always been been been stronger. So I I think organizations are maybe just um they're they're certainly aware of it, they're probably struggling more with how to to deal with it and what the approach is because um while I think some are very um you know they're becoming much more focused on okay, let's shift to more remote proctoring type models, which have which have probably been used much more in a minority in in the talent assessment space. So while some organizations are doing, I think some feel a little bit reluctant to to kind of shift because it's a change in a narrative to like we trust, we trust you and except a few of you might cheat. Um, to when you switch that remote proctoring, it is almost more saying, look, we we have to do this because we think a lot more people are going to cheat. And and therefore the the argument is this about preserving fairness. But that that that I think there's a sort of reluctance, maybe and a concern around how participants will feel um if it switches to to entirely to remote proctoring, but but certainly with the way things are evolving, I can see it, you know, that that certainly the amount of organizations um using remote proctoring um compared to the sort of traditional unsupervised testing we've seen, I think it's gonna grow.

SPEAKER_00 28:55

Thanks, Dan, for this. Uh one follow-up question on that is um a lot of organizations have started reintroducing in-person interviews. Do you think that is the right approach or solution to uh this pro uh this pattern now?

SPEAKER_01 29:11

Yeah, I mean I I think um I mean, generally speaking, there would there were there were typically a face-to-face stage often towards the end or interviews and things like that. I think it it probably means organizations are maybe focusing even more on that and thinking about well, actually, do you reduce down the amount that you do online, unsupervised, and actually spend and invest more effort in um maybe bringing people in or running interviews and the like. Um the the issue obviously that is I think organizations have got very used to the benefits of high volume screening assessments, and it's a much more expensive route when you do get into sort of interviewing and an in-person assessment. So that I think that's the real struggle that the organizations um you know you know have to balance and decide. You know, what I'm seeing is I think that organizations maybe sit differently in their their sort of risk appetite and how they want to approach it, and some really feel no, we need to lock this down. There may be, you know, maybe particular types of assessments, particular roles, particular groups, candidates. So some want to really look it down, some kind of still want to try and take more of a trust-first approach and use um, you know, you there's various sort of tactics and and approaches that can be used to at least mitigate and reduce the risk of cheating. It's never going to eliminate it entirely, but it's about getting it to what we feel is a manageable level, and then at future stages in the recruitment process, you would hope to then start to catch some of these um these these people who've maybe got through unfairly.

SPEAKER_00 30:47

Thanks, Dan, for that um approach.

SPEAKER_02 30:49

Uh Michael, we'd love to hear from you as it Yeah, I think uh starting from zero trust is important for us online. Uh I think one of the key things um for us is that like the initial question of what are people missing, I think you know, I was still talking to people literally last week, it was a where I was saying, Oh, if you use the latest chat JPT or Copilot, and they're like, it's just still good Google, isn't it? Like it's just what you use for search. So I think getting that concept that if you start from a point of zero trust and you think that it can do everything, and then when it fails, that's okay. But if you start with that approach, that it can do everything, and then what do you do to try and mitigate that? Um, I think the other important thing for me is you know, the shift to online, people started with it as this is an amazing cost-saving benefit. Um we can do a million things for a cent each. You know, that's really straightforward. Uh, rather than going, this technology allows us to now test people wherever in the world or interview people anywhere in the world, and it allows us to scale. But it you still need humans and you still need to put effort into it. So stop. I think my big thing for anyone is stop looking at it like a cost-saving mechanism. Look at it as your capability mechanism, this online thing, and put the appropriate resources into it because people still trip up, they they're still humans on the other end. They get bored, they start yawning, the cat walks across and it doesn't, you know, it glitches the wrong way. So I think for me that's really important. Start with zero trust, but also put the proper resources in it, you know, train your train your proctors properly, it's not the same world. That's okay. We just do it in a different way, but we've got the ability, we don't have to rent out hundreds of square meters of floor space to test exams anymore. We can do it anywhere in the world at any time of the day, but you just have to resource it appropriately with a new set of skills with your individuators and proctors.

SPEAKER_00 33:00

Kann it, I think this all of this all of your inverts lead to one one big question for me right now is is anyone actually getting this right or is the old industry uh behind and um catching up with how EI works today.

SPEAKER_02 33:19

That's a learned question. At risk of hubris, I think there are plenty of people that get get it right. Like if I look at our pass rates over the last 40 plus years, they're materially the same. So I I would be uh naive to think that no one has got through um cheating. Absolutely naive. But do I think that it's rampant? No, we you know we ban literally thousands of people a year. Um and that's annoying and it's frustrating, and sometimes it's super hilarious uh when you see someone with an ill-fitting week. But look, I think there is excellent people out there doing things uh on a daily basis. I I I don't think we should go back to handwritten uh exams or you know in-person interviews only in one location. I think I think um there's absolutely people getting it right, the right tools.

SPEAKER_00 34:13

So I have two final questions for all of you. I think one is uh similar to what I asked previously. Uh so what is that one thing in the industry or your organizations that we need to either stop doing or start doing that we aren't doing today? Michael, would you like to go uh for this?

SPEAKER_02 34:35

Uh tricky end questions. What should we stop doing? Um I think if we stop thinking that whatever we've got today is appropriate, like and it doesn't mean that we change it tomorrow, but we should always come at it with the the concept that that we should be willing to change, and that's uh necessary, not just a desire that's necessary. So I think the status quo is what works. I then I reflect a bit on my previous sector of um don't think that you know online is cheap, like you've got to put effort into it and got to put money into it to do it properly. Um but I think stopping is the status quo. I don't think that your assessment today, I mean we've changed our terms and conditions, I reckon, every three months uh the last three years, and we continually update exam instructions, timing of sessions, how we structure um you know voices and the like. If you stay static, I think that's a problem. But uh status quo is the thing that you need to stop, in my opinion.

SPEAKER_00 35:46

That's Jamaica.

SPEAKER_03 35:52

Yeah, I would I would agree. I mean, we're now at such a trajectory that um you know we we had an individual calling for bi-monthly updates, right? I mean, where we used to talk about six-month updates and where where we were going, where test security was going, where uh pre-AI, like what we were doing for things. And so engaging with engaging with someone who's not only just potentially a have to say they have to be an expert in AI, but they have to be AI curious or even an AI futurist, right? So working with somebody that is actively looking at trends, where things are going, and how they can benefit the organization and how they could benefit somebody looking to um engage in some level of misconduct, um then continuing to update process. It's an iterative process now, but the the cycle for that iteration has become much smaller, um, almost to the point that it's pretty much just fluid, right? And so looking at where your organization um really needs to be in the three months, six months, one year, five-year kind of future, not just being reactive to the technology, I think is is probably the most important thing right now. And it goes kind of across the entire spectrum.

SPEAKER_00 37:20

That's a great take, uh Jared. Uh Dan?

SPEAKER_01 37:22

Yeah, I mean, I'd I'd echo that. I mean, obviously, this is this is gonna be a continually changing kind of situation. I mean, I think I think one of the things I think um I don't I don't see every organization doing it from the talent assessments perspective, but they should is to have a really clear and transparent organisational strategy about how they want to approach this challenge in terms of pre-employment uh assessments. So being clear about what they would consider appropriate and inappropriate use of AI in a in the overall hiring process when you look at it holistically, um, making sure there's a very clear policy statement to candidates, asking candidates to sign up to this, and then designing the the hiring process to sort of try and mitigate this impact in doing it. So that there's there's various steps that you can take to reduce the risk, doesn't eliminate the risk, but but more AR-resistant formats using honesty agreements, um, remote proctoring obviously is an option, the threat of random supervisory testing, you know, can can just dissuade casual, um, sort of casual cheaters in the in the the sort of pre-employment space and considering more of these in-person assessment elements. So to me, they have to have a clear, be thinking about a clear strategy, what they want their stance to be, how how much they want to stay on the trust first versus move to a more security-first um approach and and design their process to feel like that they're they're they're taking a number of steps to mitigate um the potential for generating. But then to to to Michael and Jarrett's point, to then recognize you can't define that strategy and then leave it for a year because it's going to be constantly changing and evolving. So it's something that has to be monitored continually and and adapted as necessary.

SPEAKER_00 39:14

That's right, and so based on all of our conversations today, I think the the key takeaway is that we as an industry need to evolve and move faster uh in the same space. What would be your call to action for the people watching this today?

SPEAKER_01 39:36

I think for me, I'd just echo, I suppose, the the point I just made is that for for organizations in the talent who are look looking at their talent assessment, they need to have a clear policy and strategy. And to, you know, they can consult with organizations like you know, like like we do at Telegy to sort of define how how they want to approach it. And and it's okay to maybe choose different stances depending on the nature of the the roles you're recruiting for and the approach. I think from the the test providers side, I think it is that the main one of the main calls to action is about evolving and thinking about some of these new assessment formats where you can actually embrace the use of gen AI, and then it starts to to remove this question about gen AI cheating, sort of part part of that, because then you're actually looking at how can we assess someone's capabilities and in an ideal world to still assess their problem-solving capabilities and that the key skills that you want to measure, but recognizing that a lot of those things in many worlds will be done with Gen AI in future, uh, when when people are actually in the job.

SPEAKER_00 40:39

Thanks, yeah.

SPEAKER_03 40:40

I'll I'll feed you back on that because I really think it's very important right now to be thinking outside the box. And and I think it's perfectly fine to think outside the box with AI, right? And to have a strategy and kind of go forward, there's a lot of conversation right now on the assessment side of you know, one trajectory is perhaps we can just generate on-demand items at all times. And so then we're never really worried about the security aspect if no one ever sees. Um, and that's getting a lot of traction. I can understand some of it. But at the same point, if we're also saying that agentic AI can answer any questions, it doesn't really matter how many questions we generate, if that's way the the they're getting away with it. So I almost I almost envision this is just me, I'll put on my own personal hat. As we go forward, I almost can see individuals engaging in an educational space or working through a recertification, literally being handed their own agent that is constantly going to be assessing what they know, what they don't know, and will be generating in real-time bespoke assessments based on your current knowledge base and what you're lacking from almost uh uh uh like a real-world perspective. This chapter is particularly on organic chemistry, right? How much have you demonstrated over the last week that you truly know about that? How much are you lacking if you're fully going to master this area? And the age on is agents assessing everything you do, and so your actual assessment will be completely unlike anyone else's. Um, and that bespoke aspect uh could be tapped into frank, frankly, now, and I think we'll see larger conversations as psychometricians get involved and as the data scientists get involved about what that could look like in the future.

SPEAKER_00 42:30

Thanks, Aric. Michael?

SPEAKER_02 42:33

I'll keep it short. Uh don't give up on it, uh, but constantly evolve. I think invest and constantly evolve. That's my goal to action.

SPEAKER_00 42:44

That's great. I think I loved hearing from all of you. Uh thank you, uh Jared, Michael, and Dan. I think uh this has been a very important conversation, especially from my standpoint as well, given that this is the problem that we are trying to solve for every day uh as part of our jobs. Uh I think what strikes today uh for me is uh that it's not about only technology, it's also about trust at the end of the day. And uh that's where uh Talview is also continuously working towards uh making sure how we can uh evolve and how we can uh uh ensure that we are ahead of these patterns and are able to identify these scenarios. We have our own seven-layer security framework that doesn't just flag moments, but it's also able to read patterns, it's able to monitor environments and track uh content leakage across the web. These tools exist. It's just now that the question is whether the industry will move fast enough to use them. So that is my uh takeaway as well. Um thank you everyone for taking our time today. Earlier session.

SPEAKER_01 43:54

Thank you.