Shea Brown called it "the last human job" - and it's sitting on a $55 billion market.
The market for "the last human job" is barely $100 million today.
That's a 550x growth opportunity staring us in the face 🤯
So, how would you find out about this job?
Well, you could ask Shea Brown directly...
Or watch our latest episode with Ryan Carrier, FHCA (Executive Director, ForHumanity), and Jimmy Farrell (EU AI Policy Lead, Pour Demain) to find out what this job and its market are.
Watch on YouTube; listen on Apple Podcasts or Spotify.
Episode Summary:
In this conversation, Aashka Patel engages with Ryan Carrier and Jimmy Farrell to explore the critical themes of AI audits, governance, and the implications of AI on job displacement. They discuss the necessity of independent audits for AI systems, the challenges posed by autonomous systems, and the importance of incident reporting as outlined in the EU AI Act. The conversation also touches on the need for international coordination in managing AI incidents, the ethical considerations of AI's role in society, and the future of AI compliance initiatives. The speakers emphasize the urgency of addressing AI-related risks and the potential for significant economic impacts due to job displacement.
Timestamps:
00:00 AIR Bites (Precap)
01:52 AI Audits EXPLAINED ft. Ryan & Jimmy
02:50 Sam Altman: Genius PR or Guilt Trip?
05:13 Will AI Kill Jobs? UBI vs Taxes vs Sovereign Funds
09:30 Auditing AI: The Next $55B Industry?
13:22 Why Auditing Autonomous AI Is 10x Harder
18:05 AI Incidents: From Risk to Reality
20:37 EU AI Act: What MUST Be Reported
23:37 Do We Need a CERN for AI?
25:52 What Happens AFTER an AI Incident?
28:24 Maternal Instincts in AI?! Geoffrey Hinton's Idea
32:44 AI Alignment Is Broken: Here's Why
35:57 India's AI Law & Free Auditor Training
40:11 Australia's AI Stress Test: What They Found
43:10 AI Liability Law Killed by Lobbyists?
45:31 AI Suicides, Deepfakes & Agents Gone Wrong
47:49 If AI Harms You, DO THIS
49:30 Final Thoughts & AI Risk Literacy
49:53 Outro
Transcript:
Aashka Patel (00:04)
hello and welcome to On AIR with Aashka Today we are stepping right into the heart of AI audits and AI governance and joined by Ryan Carrier and Jimmy Farell. Ryan is the executive director of For Humanity, the global nonprofit building independent audit for AI and algorithmic systems. With 25 years in finance, Ryan knows what it takes to make audits credible, enforceable, and essential for trust.
Jimmy Farrell is the EU AI policy lead at Pour Demain where he works on EU AI regulation, incident reporting, and codes of practice. Jimmy’s work directly shapes how we respond when AI causes real-world harm, from financial loss to human rights violations. So welcome to the show, Jimmy and Ryan.
Ryan Carrier (00:52)
Thank you.
Jimmy Farrell (00:53)
Thanks very much for the lecture. Pleasure to be here.
Aashka Patel (00:55)
So let’s dive right into the questions. Sam Altman had spent $60 million on the largest UBI universal basic income study in US history, giving people $1,000 a month for three years So the study found UBI gives people flexibility and autonomy.
but didn’t solve deeper problems like housing or chronic health issues. Meanwhile, Altman just told the Federal Reserve that entire job categories will be totally, totally gone due to AI agents. So the same guy who is building the technology that displaces workers is funding research on how to support them afterward. Is this tech billionaire guilt or brilliant PR? What do you think?
Ryan Carrier (01:42)
From my perspective, neither of the above. It’s the ultimate conflict of interest.
Jimmy Farrell (01:43)
I’m gonna go home.
Aashka Patel (01:45)
Hahaha!
Aha, yeah, yeah, makes sense. And like, should policymakers instead force a job displacement tax on these frontier AI labs to pay for those safety nets? what do you think is working and not working in experiment?
Jimmy Farrell (02:07)
From my perspective, think the data on job displacement is still pretty early and there’s lots of studies going both directions. Some saying that it’s not as bad as some would expect, others saying that a lot of entry-level coding jobs are basically evaporating. And of course, these like just show some years to show in the labor market. So I think it would be good to potentially have more solid data before something like
Aashka Patel (02:25)
Mm-hmm.
Yeah.
data.
Jimmy Farrell (02:38)
a tax could be placed on this, but in general, the fact that AI and big tech companies often don’t pay too many taxes. Also in the EU, there’s been a lot of cases in which huge companies have not paid a lot of taxes. The EU and a particular number states have thought a bit about, for example, just the digital services tax. So this would be more general than a job displacement tax.
The political will is growing for that, especially with European tech sovereignty interests. And that could, for example, morph into a job displacement tax.
Aashka Patel (03:17)
Yeah, yeah, makes sense, makes sense. Ryan, what about you? What are your thoughts?
Ryan Carrier (03:21)
So
I wrote a paper that’s years ago now, essentially advocating the US, UK, most of Europe ought to start their own sovereign wealth funds simply because they will have to pay for what we imagine will be 30 to 40 percent entrenched unemployment. There are already concrete examples, even in the United States, a major US law firm
Aashka Patel (03:23)
Mm-hmm. Mm-hmm.
Mm-hmm.
Ryan Carrier (03:49)
based out of New York City doesn’t even hire junior advocates anymore. Why? Because they don’t need them, which creates an enormous downstream problem for becoming an experienced advocate or experienced lawyer. This shows up not in unemployment because these persons were never employed. It shows up in a lack of hiring.
Aashka Patel (03:53)
Hmm. Yeah.
Mm-hmm.
Smooth.
Ryan Carrier (04:14)
And so what we see is first this becoming a pervasive problem. But the nature of this sovereign wealth solution is the idea of having to pay for the very UBI that Sam Altman was just testing out for. We don’t have a current mechanism for such a thing. And so what we may find is that governments may need to participate with the very companies that are refusing to hire the people in that country moving forward.
Aashka Patel (04:33)
Hmm.
Ryan Carrier (04:44)
Unlike most technology in the past, it’s really a matter of perspective. So if I say to you, what does horse unemployment look like today? If you think about it, horse unemployment has persistently remained at 99.9 % unemployment for about 100 plus years now. And that’s because when we built our technologies, that was what we were able to replace.
Aashka Patel (05:13)
Hmm.
Ryan Carrier (05:13)
And for a
solid 150 years, the industrial revolution, we did not have the tools to fully replace humans. Now we are in a different place. That’s why we call ourselves in the fourth industrial revolution, right? It’s because for the first time we have the ability to fully replace humans in nearly all tasks that we engage in. And the end game for that in capitalism is that those tasks and those jobs will be replaced. Our tax system.
Aashka Patel (05:20)
then
Yeah.
Hmm.
Ryan Carrier (05:42)
incentivizes using machines and automation over employees. These machines can work 24-7. They don’t take vacations. They don’t have to stay home when their child is sick. And so what we will see is an increase in large corporations automating tasks. The results of that may be that we see lots of micro jobs, gig workers. So we may not recognize it as full unemployment.
Aashka Patel (06:02)
Yes.
Ryan Carrier (06:12)
But what it will result in is an income inequality where a lot of the wealth accrues to more and more people and less and less employees. And we’ve already seen that shift for a hundred years. The net result is that we have to think about what the future of work looks like. And it is an enormous challenge. There is no point in advancing artificial intelligence unless it’s replacing jobs.
Aashka Patel (06:23)
Hmm.
Hmm.
Hmm.
Ryan Carrier (06:41)
from the capital owner’s perspective. That is where the cost savings come from. Now they may empower the existing employees to be a lot more successful and leveraged and accomplish more work, but it then also means they’re not hiring more employees. So we may see this technological unemployment manifest in many different ways and creative solutions to meet that challenge are necessary such as
Aashka Patel (06:45)
Hmm.
Okay.
Yeah.
Ryan Carrier (07:10)
starting sovereign wealth funds that maybe participate in the equity capital, the growth of these companies that are allowed to automate because we don’t have any laws that say you can’t automate, you must hire people. We don’t do that, at least in the West.
Aashka Patel (07:25)
Hmm.
Yeah, yeah, yeah. Makes sense.
So Ryan, you have spent 25 years in finance, where independent audits are legally required and auditors make serious money, right?
So now AI regulation is creating similar compliance demands from your finance background. Like do you see independent AI auditing becoming as lucrative and essential as financial auditing? And should entrepreneurs be rushing into this space? And also can we see someone today creating the Deloitte or PWC of AI tomorrow?
Ryan Carrier (07:59)
So I’m going to answer this in a couple of parts. Number one is we believe it is necessary to establish sufficient trust due diligence capabilities by people buying and deploying these tools, that there be auditability or some level of assurance of what compliance steps have been taken inside of all the developers, providers, deployers of these tools. As a result of that, when asked,
The one policy initiative that For Humanity advocates for is mandatory annual audits of all what we call AAA systems. We don’t just say AI, it’s AI, algorithmic and autonomous systems. That’s our scope, but that’s our scope, okay? So with that foundation, if you subscribe to that belief and that future legal obligation, in 2021, the financial audit global revenue,
Aashka Patel (08:37)
Yes.
Yeah. Yeah.
Ryan Carrier (08:56)
was $55 billion per year. I believe that the space of auditing AAA systems will be bigger than that inside of 10 years. Today in 2025, if that revenue is more than $100 million, I would be shocked. Okay, it’s not measured, it’s not well tracked, but I know who’s doing business and what they’re doing in the space. If it’s more than a hundred million, I’d be very surprised. So what that tells you is if you believe
Aashka Patel (08:58)
Okay.
Hmm.
Okay.
Mm. Mm. Mm. Mm.
Okay.
Ryan Carrier (09:25)
what we believe that we’re gonna grow from a hundred million to more than $55 billion a year. The answer is unequivocally, yeah, this is a space people need to be in, especially in light of our previous points about jobs disappearing elsewhere, right? Shea Brown, who runs the algorithmic auditing firm has, exactly, at Babl AI has referred to auditing algorithms as the last human job. And that makes some sense, right?
Aashka Patel (09:33)
Yeah.
Hmm
Beverly. Yeah.
Yeah!
Ryan Carrier (09:55)
And so if that is accurate, then, you know, sort of upskilling or re-skilling, this is a space that would be ripe for opportunity, whether that’s as an advisor, consultant, teacher, auditor, or building technologies to facilitate this whole process.
Aashka Patel (10:14)
Got it, got it, got it. So can you name any big players who are already working into this space like apart from Shea Brown that you mentioned like Babaliya? Yeah.
Ryan Carrier (10:23)
Well, it really depends
on your perspective. have a Credo AI and other technology providers who are facilitating compliance. You have the big four who are providing some advisory services. Let’s be clear, especially from For Humanity’s perspective, no one in the world, no one in the world could meet the audit criteria we have, we have put in place. Okay. So what that tells you is they have to be going through the process of building
Aashka Patel (10:29)
Yeah. Yeah. Yeah.
Hmm. Hmm.
Hmm.
Ryan Carrier (10:53)
audit compliance, just the same as there’s no one right now who’s ready to be compliant with the EU AI Act. It’s the same concept, right? So we’re in the pre audit compliance capacity building phase. And then as that begins to grow, the opportunities will continue to grow. The way I would estimate it is out of a hundred companies, one to two to five of them are, basically spending
Aashka Patel (10:59)
Yeah.
next
Mm.
Ryan Carrier (11:20)
in this space to build capacity now. And that’s another area of growth, obviously, as we move to essentially 100 out of 100 companies.
Aashka Patel (11:23)
Hmm.
Yeah, makes sense, makes sense. So you pointed about the AAA approach that for humanity is taking towards auditing. auditing a model is like grading an exam, like the answers are fixed and you just check them. Auditing an autonomous system is like grading a student who has the tendency of changing their answers while the test is still running.
So in that scenario, like how does independent auditing differ for a model versus an autonomous system?
Ryan Carrier (12:00)
It’s a great question. So model data and concept drift are legitimate risks associated with AAA systems. And the way I would just, well, bound to happen, but then the question becomes whether that drift is legal and ethical, okay? So the way that we describe this is that if you’ve agreed to use a tool that I’m providing,
Aashka Patel (12:06)
Yeah.
Yeah, and they’re bound to happen.
Hmm. Yeah.
Ryan Carrier (12:28)
and I’m using my hands, sort of imagine them shoulder width. I know we’re sort of perpendicular here. So you can’t see that wide on us, okay? But if you and I agree that the purpose looks like this, okay? And if the model, and I’m just moving my hands a little bit, right? If the model just drifts inside of those guard rails, it’s no problem. It’s still consistent with the scope, nature, context, and purpose with which you agreed, okay? And that is both legal and ethical. Here’s the problem. What if the drift,
Aashka Patel (12:33)
Yeah, yeah, I’m gonna like it. Yeah.
Hmm. Hmm.
Hmm. Hmm. Yeah.
Ryan Carrier (12:58)
and I’m way off the screen, right? What if the drift looks like this? We would argue that that is both illegal because it’s not consistent with the purpose with which you agreed contractually and it is unethical. So now what we ask, and my arms are stretched way out wide, right? As I bring them in, what I would ask you is where do those guardrails have to exist to be legal and ethical?
Aashka Patel (13:01)
This, yeah.
Hmm
Mm-hmm. Mm.
Ryan Carrier (13:24)
And the answer is no standards body is even approaching this conversation. No laws have even talked about this. So this therefore becomes what we call an ethical choice. So here’s how we deal with this. We say to all auditable entities, you need to establish your own key performance indicators to identify when you’re staying true to your purpose based on your use case.
Aashka Patel (13:28)
Yeah.
Hmm.
Ryan Carrier (13:51)
And you do this
Aashka Patel (13:52)
Thank
Ryan Carrier (13:52)
with your ethics committee who’s doing this based on their shared moral framework and on the laws and their code of ethics. So now what we ask and what we measure in an audit is number one, have you established your own guardrails? Number two, have you established the ability to measure that model data and concept drift, right? And then number three, have you put in place rules that say if you bump up against one of those guardrails,
Aashka Patel (14:07)
Hmm.
Yeah.
Ryan Carrier (14:20)
What do you do? Do you pause? Do you replace the tool? Do you recall the tool? So that’s year one, right? Now we’re tracking, am I drifting? Okay. So let’s say Jimmy does this, right? He puts it in place and this is what he establishes. And as his auditor, I’m like, fantastic, great. So now I show up next year and I say, how did you do? Now, if Jimmy bumped up against those guard rails a thousand times in the last 12 months,
Aashka Patel (14:25)
Hmm.
Mm.
Ryan Carrier (14:49)
and did nothing? Does he stand any chance of getting his assurance renewed in year two? The answer is no way. Not a chance, right? Because look, it’s his own rules. You establish your own rules and you can’t even follow them. So what that tells you is that you don’t know if your model data concept drift is violating that purpose, right? So therefore it is genuine teeth
Aashka Patel (14:57)
Hmm, no. Yeah.
Hmm.
Anyway.
Hmm.
Ryan Carrier (15:17)
for us to say, Jimmy, when we show up next year, follow your own rules. And if you don’t, you’re not gonna be able to be certified. You’re not gonna conform to the EU AI Act and so on and so forth. It’s legitimate teeth to ensure that the model stays true to what it agreed with its customers. And so this is how we audit something that learns and grows and changes over time.
Aashka Patel (15:24)
and
Hmm.
Hmm.
Ryan Carrier (15:45)
Does that make sense? that a meaningful example?
Aashka Patel (15:46)
Yes, yes.
Yeah, you explained like a teacher. Okay, okay. You have gotten great at your at your artwork then. Yeah, so you mentioned about this model concept drift. So like every AI incident is basically a risk that went from could happen to did happen. So like, since audits are all about managing the risks,
Ryan Carrier (15:51)
I’ve done this once or twice before, I promise.
Aashka Patel (16:13)
Are we considering the actual incidents while doing the AI audits?
Ryan Carrier (16:21)
From my perspective, the answer is yes for two reasons. Number one is all of, at least under the EUAI Act, okay? There has to be a process to evaluate whether an incident is serious or not, reportable or not, okay? So you have to have that process of evaluating, and we have a whole section called incident management to do exactly this. So now when you engage in incident management, you might have to report to your regulators and so on.
Aashka Patel (16:25)
Mm. Mm.
Hmm.
Okay.
Okay.
Ryan Carrier (16:51)
But all we are measuring again is do you have the processes and procedures in place to do this work? That’s year one. Year two, we’re going to say you had, I don’t know, 12 incidents and did you evaluate them for seriousness or reportability? Okay. And then what did you do? Did you execute your process? Again, your named processes and procedures. Did you execute them the right way? Or in the case of, of Jimmy’s work.
you know, was the incident report filed correctly, registered correctly with registration databases and national competent authorities. So we’re basically reviewing, those things happen properly? And if they don’t, the auditor is basically gonna say, I can’t assure you your processes and procedures are insufficient.
Aashka Patel (17:39)
Hmm.
Got it. Got it. So like, does this like enable any kind of friction in the company’s overall working process because of the auditor mentioned that we can’t audit or something like that? Like, are there any kind of strict punishments or penalties that could be given Like because of
Ryan Carrier (18:02)
strongest punishment is, look,
all high risk AI systems have to go through conformity, right? And if they are nonconforming, then they are violating the law. So that’s the strongest teeth that we have here, let alone fines or any other enforcement by regulators. having a nonconforming AI should be the biggest punishment that you have. Why? Because now all your downstream
Aashka Patel (18:05)
The fines? Okay, okay.
Hmm
Mm-hmm.
Mm. Mm-hmm.
Ryan Carrier (18:30)
deployers and so on simply cannot buy your product, which would, that’s how product liability, you know, frameworks, whether they’re law or market driven, that’s how they ought to work.
Aashka Patel (18:33)
Wow.
Hmm
Hmm.
it. it. Got it.
most of your work revolves around the AI incident reporting. So let’s get back to that. So MIT’s AI Risk Repository now classifies incidents across 10 harm vectors, physical damage, financial loss, human rights violations with severity scales from negligible to catastrophic.
and they found 65 % of incidents happen post-deployment and most are unintentional. So does the EU AI act demand this level of granularity in incident reporting? And if yes, can you double click on the exact requirements by the EU AI Act
Jimmy Farrell (19:22)
Yes, so there is some overlap. I would say it’s slightly less granular. Of course, it is a law, so it’s not just a suggestion. Yeah, of course, with AI Act there’s this like, there’s kind of two halves of it which have to be addressed. One side is the system side, AI systems, so the downstream level, and then the other side is the AI models, so the weights themselves.
Aashka Patel (19:29)
Mm-hmm.
Jimmy Farrell (19:49)
comparison would be GPT-5 is the model and ChatGPT is the system. There are slightly different reporting requirements for systems and models. I’ve been mostly working on the model side and in terms of this categorization comparing it to the MIT risk repository, there is some overlap. the recently published code of practice, which is kind of like the details of the general purpose AI model section of the AI Act
Aashka Patel (20:01)
Hmm.
Hmm.
Okay.
Jimmy Farrell (20:20)
and with the categories of harm to a person’s physical health, damage to property or the environment. It also mentions serious cyber security breaches such as like model exfiltration, like somebody stealing a model weight. It also includes infringements of fundamental rights which is very much connecting to EU.
Aashka Patel (20:31)
Hmm.
Yeah.
Jimmy Farrell (20:48)
treaties involving the right to privacy and that sort of thing. One thing that Ryan mentioned before, the threshold at which an incident becomes serious, this is still kind of a gray area. It’s something that’s very difficult to think down because the thresholds are, it’s very hard to compare what a threshold for an infringement of fundamental rights looks like compared to something that’s easily monetizable like damage to property or...
Aashka Patel (20:59)
Mm.
Yeah.
Yeah.
Financial
damage or yeah, infrastructure damage. Yeah. Yes. Yes.
Jimmy Farrell (21:19)
Exactly.
So these are some challenges. so compared to the MIT risk repository, like the code of practice details, like exactly when the reporting has to happen. And there are some like increased urgency on more serious incidents. know, if there’s a death of a person, it’s more urgent than if there’s been some property damage.
Aashka Patel (21:36)
Hmm.
Mm.
Yeah, yeah, makes sense. So there have been talks around creating a CERN for AI and you work on AI incident reporting. So CERN already coordinates cyber incident responses across 42 countries. When one site is breached, threat intelligence is immediately shared and defenses are updated everywhere.
So shouldn’t a CERN for AI work the similar way for AI incidents as well? Like if there is an AI incident happened in New York, shouldn’t the safety protocols for similar systems automatically update in London or Tokyo? what do you think
Jimmy Farrell (22:24)
I definitely think that international coordination will be really important for AI incidents. This is something that is not particularly addressed in the Code of Practice or the EU AI Act because reporting basically goes through the AI office or the relevant member state authorities. Of course, from the provider perspective, there are some...
Aashka Patel (22:39)
Yes, definitely.
Mm-mm.
Jimmy Farrell (22:51)
like serious concerns like, for example, reputational risk, if this is like shared with, you know, countries or governments for which the incident wasn’t necessarily relevant to, then that could be just sort of unnecessary and really like set them back. And of course, certain serious incidents have intellectual property and trade secret concerns. So that’s also something to consider in terms of international coordination.
Aashka Patel (23:03)
Hmm.
Hmm.
Jimmy Farrell (23:21)
course, AI is like inherently cross border. So there is a scenario in which a incident might happen outside the EU, but it’s relevant to the EU because it’s involved in model that is also placed on the market. And this is something that the AI Act accounts for. if it’s the same model, then this is something that
Aashka Patel (23:33)
Yes.
No.
Jimmy Farrell (23:45)
needs to be reported to the authorities. Yeah, think the comparison to like the actual CERN that exists is not quite enough. Yeah, yeah.
Aashka Patel (23:47)
Mm-hmm.
Hmm. Hmm. It will take some time to reach there, maybe. Yeah,
definitely, So like, I was curious about the immediate steps that are to be taken or that are taken currently once an AI incident is reported.
How does it work in the real world?
Jimmy Farrell (24:13)
So before the AI Act, were no requirements specifically for AI incident reporting. Like incident reporting, there’s been a bunch of other fields that exist in cybersecurity, exists in aviation, chemicals, these sorts of things. But it hadn’t existed until the AI Act for AI. Basically what happens is that there’s the...
Aashka Patel (24:21)
Yes. Yeah.
Hmm.
Jimmy Farrell (24:42)
Provider has to submit an initial report with like some quick information so that the authority whether it be the AI office or member state authority is aware of it very early and not necessarily like a very detailed report but just the basic information and then later on there’s can be an intermediate report if it’s an ongoing incident and then eventually as I mentioned there’s different timelines.
Aashka Patel (24:54)
Mm.
Yeah.
Hmm.
Mmm.
Jimmy Farrell (25:11)
different types of incidents, but eventually there will be a final serious incident report. And this is what is really going to be important for authorities to learn from and to prevent them in the future. this sort of thing, this sort of report involves what model it was, what company was involved, the date, of course, the type of damage, the harm. Importantly, it involves a root cause analysis. And this is something like
Aashka Patel (25:13)
Yeah.
Hmm.
Mm-hmm. Mm-hmm.
Yes, yeah,
Jimmy Farrell (25:40)
quite complicated in AI because
Aashka Patel (25:40)
that is important.
Jimmy Farrell (25:43)
establishing causality is quite difficult in inherently uninterpretable systems. So this is something that the code was just released, so we’ll need to see how in practice such a root cause analysis works. But basically the code is quite specific and particular about saying that there needs to be a deeper analysis of not only what
Aashka Patel (25:46)
Yes. Yeah.
Hmm.
Hmm.
Jimmy Farrell (26:09)
features of the model resulted in certain outputs happening or what input in terms of the training data or just like model inputs. So yeah, eventually there’s the final report and from that authorities will be able to move forward exactly.
Aashka Patel (26:14)
Yes.
Take action. Yeah,
yeah, yeah. Got it. Got it. the godfather of AI recently said something instead of forcing AI to submit to humans, we should build maternal instincts into AI models so they really care about people even when they become more powerful than humans. So Ryan to you, could you actually audit AI systems for maternal instincts?
Ryan Carrier (26:55)
First off, can I throw up at the thought? mean, I’m always asking the question why. I don’t even understand the pursuit of general intelligence. Why would we want to take our greatest advantage and subvert ourselves to a machine? We already have controllability problems. And then the quantification and parameterization of maternal instincts.
Aashka Patel (26:58)
Hahaha
Mm-hmm.
Hmm
Yes.
Ryan Carrier (27:24)
These things can happen. See what people confuse is tasks with things that cannot be parameterized. So if I take a care robot, let’s not even go to maternal instincts for a second. If I take a care robot for the elderly, I can absolutely break out tasks like listening. You know, people might say, that’s a maternal instinct to listen to your children, right? No, a recording device can listen to you.
Aashka Patel (27:27)
Hmm. Hmm.
Hmm.
Yeah.
Hmm. Hmm.
Yeah.
Yeah.
Ryan Carrier (27:54)
Okay, so
we need to break out tasks which are very digitizable, they can be parameterized. You can establish metrics, measurements and thresholds. Those are huge words for us, right? Establish metrics, measurements and thresholds around these qualities. And then can you measure them? Absolutely, we can measure many of them. We can’t measure hope. We can’t measure
Aashka Patel (28:08)
Yeah.
Yeah.
Ryan Carrier (28:24)
dreams. We can’t measure love in the true sense of the word love, not I love you. I brought you flowers. I can measure whether I brought you flowers, right? That is a task. That is an act. Okay. But I can’t measure love generically speaking. Right? So getting to maternal instincts, the way Geoffrey Hinton has suggested, you know, no, we can’t get all the way there, but we can certainly get to
Aashka Patel (28:26)
Hmm.
Hmm.
Yeah. Yeah.
Hmm. Hmm.
Ryan Carrier (28:54)
many tasks. Here’s the problem. Humans operate suboptimally many times. So if I’m a tool, okay, I want to optimize on for my benefit. Well, if I choose to take time out of my benefit to care for you, Aashka, that’s a suboptimal choice given the parameterization of me, right? I’m not optimizing for myself.
Aashka Patel (28:57)
Hmm.
Hmm. Hmm. Hmm.
Hmm. Hmm. Hmm.
Ryan Carrier (29:22)
I may be gaining benefit by caring for you. You know, it could be love. It could be warm feelings. I could feel good about myself. I could grow my skills in caring for you, right? But the point is, is it’s a suboptimal choice given a set of parameters that are maximizing my personal optimization. And so we are so far away from being able to do that with our tools that these notions are ludicrous to me.
Aashka Patel (29:25)
Hmm.
Mmm.
Hmm.
Hmm.
Mm.
Ryan Carrier (29:52)
Now,
Aashka Patel (29:51)
Hmm.
Ryan Carrier (29:52)
does that mean in 20, 30, 50 years, we might not be closer to that? Absolutely, we could be. But in today’s place, what we need to do is maintain controllability, intervenability, transparency, explainability, and we need to ensure human oversight.
Aashka Patel (29:59)
Mmm.
Hmm.
Yeah.
Hmm. Hmm. Hmm.
Ryan Carrier (30:16)
and human beneficial ownership. There are people out there who are trying to establish shell companies where autonomous systems can sit in the shell company. Again, it makes me sick to my stomach. Why? Because it’s just a way to avoid liability of humans. And that is dangerous to the nth degree. So these dreams of the future are just that. We have a lot more work to do at the very basic level of robots.
Aashka Patel (30:22)
Mm.
Yes.
Hmm.
Hmm. Hmm.
Basic level,
yeah.
Ryan Carrier (30:44)
governance, oversight, and accountability
of these tools before we can even take much bigger steps such as that.
Aashka Patel (30:53)
Yeah, makes sense, makes sense. Jimmy, what are your thoughts on this?
Jimmy Farrell (30:58)
Sure, yeah, I agree with Ryan’s sort of characterization of the difficulties with aligning AI and even understanding what human preferences would be because those are completely flawed as well. Maybe if we were to take the thought experiment that this was possible, I think the way I would take this conversation is that, even if this was possible and somehow AI had some maternal instincts for us as humans,
Aashka Patel (31:04)
Thank you.
Mm-hmm.
Hmm.
Jimmy Farrell (31:27)
We don’t have the tools to measure this. So we basically evaluate models right now as black boxes. So we basically ask it, are you safe? Can you produce this harmful content? And we see what it’s said. So in that regard, tools like the Code of Practice is a start because the EU has kind of drawn a line in the sand that external evaluations will be mandatory for certain.
Aashka Patel (31:29)
Yes.
Yeah.
Jimmy Farrell (31:55)
models and there’s also some provisions in there to basically have those evaluations be deeper than black box access in the future when such techniques start to come to fruition. There’s multiple techniques, famously one called mechanistic interpretability has now suffered from potentially being a bit more of a dead end than was thought about a year ago where you can sort of look out of the hood and do
Aashka Patel (31:56)
Yeah.
Yeah.
Jimmy Farrell (32:22)
what is akin to like a brain scan of different areas of a deep neural network. And then in terms of, so I guess from the regulatory angle, yeah, these like deeper than black box external evaluations could be a way to test for such tendencies of a model. But then of course the EU is doing all sorts of things, not on the regulatory side, but on the promoting AI research side and investing in AI and that sort of thing. So.
Aashka Patel (32:26)
Yeah, yeah.
Hmm.
Jimmy Farrell (32:52)
such tests, for example, on mechanistic interpretability can be quite compute intensive and researchers don’t have access to this compute. So, building public compute clusters, which they are doing, these so-called gigafactories, making sure that enough is allocated to researchers and academic institutions, universities that are doing this kind of research.
Aashka Patel (33:02)
Yeah.
Jimmy Farrell (33:17)
That would also be something very important to do but to be clear as Ryan said This is also important to happen to risks that we do see definitely in models now like dangerous capabilities like for example tendencies for AIs to be overly emotionally addictive and these sorts of things
Aashka Patel (33:41)
Yeah, the AI psychosis problem and everything is happening. and to be able to reach the maternal instincts stage, we first need to solve the problem of AI sycophancy because I don’t see any mothers being sycophantic to their children. we need to solve the problem of AI sycophancy first and then maybe talk about the maternal instincts.
So Ryan, I read on your LinkedIn. for our viewers in India, especially leaders looking to comply with the DPDPA Act or the Digital Personal Data Protection Act, or anyone curious to learn more, I know For Humanity has been working on something big in this space. You have some session or course coming up.
So can you share what’s launching and how professionals can get involved?
Ryan Carrier (34:31)
Yeah, I’m going to build a little bit of a story here. So we are working with the European Data Protection Board. We are at the last vote of the process to have the world’s first GDPR certification for AI algorithmic and autonomous systems under Article 42. That will be a big deal when that finishes. We’ve taken that process and we’ve replicated it in Bermuda, in California, in Dubai.
Aashka Patel (34:34)
Yes.
Mm.
Hmm. Hmm.
Ryan Carrier (35:01)
and now
Aashka Patel (35:01)
Hmm.
Ryan Carrier (35:01)
in India as well. What we do is we take the local law and we’re sensitive to local law, local terminology. So for example, we don’t deal with controllers and processors. In India, we deal with data fiduciaries and significant data fiduciaries, right? So we adjust our certification scheme to adapt to the local law. At the same time, we are trying to harmonize as much as possible.
Aashka Patel (35:03)
Hmm.
Yes.
Hmm.
Ryan Carrier (35:29)
so that if someone in India is abiding by DPDPA, ideally
Aashka Patel (35:29)
Hmm. Hmm. Hmm.
Ryan Carrier (35:33)
that gets them 70, 80, 90 % of the way to abiding by GDPR and California. Why? Because all for humanity cares about as a nonprofit public charity is facilitating compliance with the law. Why? Because all we care about is mitigating risks to humans, full stop. It’s the only thing we care about. So by creating these audit rules,
Aashka Patel (35:39)
Hmm.
Hmm.
Hmm. Hmm.
Ryan Carrier (35:56)
that are globally harmonized, but jurisdictionally sensitive, we are able to meet the local needs as these laws are produced. So as we’re drafting that law, we spent the last two years doing it, we finished up that work, and now starting on, I think it’s October 27th, we launch our course, Training People, How to Become a For Humanity Certified Auditor on Those Audit Rules Applicable to the DPDPA. And we do that
Aashka Patel (36:14)
Mm-hmm. Mm. Mm.
Okay.
Ryan Carrier (36:26)
with the EU AI Act, we do that with GDPR, we do that with California privacy, children’s code, just on and on and on for our more than 50 plus certification schemes. We have more than 7,000 risk controls, treatments and mitigations for AI algorithmic and autonomous systems. That’s more than any organization in the world by a long way. And all we’re trying to do is to take all of that information and give it to people like Jimmy and anybody else who wants
Aashka Patel (36:32)
Hmm. Hmm.
then.
Hmm.
Hmm. Okay.
Hmm.
Ryan Carrier (36:55)
use it to facilitate compliance for companies with laws,
Aashka Patel (36:57)
Hmm. Hmm. Hmm.
Ryan Carrier (37:00)
regulations, guidelines, standards, and best practices.
Aashka Patel (37:04)
Got it, got it. So, like, is it free of cost
Ryan Carrier (37:09)
all of our courses are online and they are all free for the learning. So if you want to learn, you simply register at For Humanity Dash University on our website. You create a student account and you can go through all the learnings for free. When you finish that, if you’ve completed all of the quizzes that test your knowledge, then you are eligible to sit the certification exams. Certification exams have a cost.
Aashka Patel (37:12)
Okay. Okay. Great. Great.
Mm-hmm.
Mmm. Mmm. Love it. Mmm.
Mm-hmm.
Got it.
Ryan Carrier (37:37)
As a charity, what we want to do is we want to teach as many people as possible. But if you need to be certified, if you need to prove and demonstrate your expertise, then we ask you to pay a fee to sit the certification exam. And for all four humanities certified auditors, we hold you to a code of ethics and professional conduct. So we’re elevating your expertise associated with this knowledge.
Aashka Patel (37:37)
Got it.
as possible.
Got it, got it. Yeah, that’s a great initiative and I’ll make sure that I link up the registration link onto this video when it releases. So most people can take advantage of this. yeah, thank you so much for sharing this.
The Australian AI stress test that you were part of. So it was a collaboration with 64 experts across AI cybersecurity and public policy to assess five AI threats against Australia’s current legal frameworks.
So the findings show that while existing regulators can and should take specific steps to manage AI risks within their domains, significant gaps remain for the national scale risks posed by general purpose AI. So since you were a part of it, can you double click on those gaps specifically?
Jimmy Farrell (38:54)
Yeah, sure. So this is one of my nationalities. that’s the reason why I was involved. I have more expertise on the European side. It was good to dip my toes in the Australian context. Australia has quite a good history of fairly progressive tech laws in general. There’s a great privacy watchdog. They’ve also pioneered some child safety protections. For example, the
Aashka Patel (39:07)
Uh-huh.
Yeah.
Jimmy Farrell (39:22)
They’ve been one of the first to talk about social media bans.
However,
the stress tests really showed that their lack of a comprehensive AI Regulation that the EU now has a lot of other jurisdictions have. The US is starting to have more and more not a comprehensive one, but state-based specific legislations. It identified that Australia really lacks this. And in that sense, it was a very important exercise to undertake and it will certainly
Aashka Patel (39:32)
Mm-hmm.
Yeah.
Mmm.
Jimmy Farrell (39:52)
be a part of ongoing discussions in Australia. In terms of should other countries do the same? Definitely. It’s something Europe has done since well before CHAT GPT, before the AI Act, they’ve been consulting stakeholders since the mid 2010s. And this has clearly resulted in a really democratic German result that is very
Aashka Patel (40:05)
Hmm. Yeah.
Jimmy Farrell (40:22)
foundational in the world as a benchmark for AI safety ethics and protecting fundamental rights.
Aashka Patel (40:31)
Yeah,
yeah. So has any other country done similar kind of stress test or like Australians are the first one to get it done?
Jimmy Farrell (40:40)
To name it a stress test, I couldn’t tell you if anyone else has done what’s called a stress test, but certainly the EU has done lots of research into how this combines with other EU type policy like the GDPR, like the Digital Services Act, Digital Markets Act. There’s still quite some legal research to do to find where these overlaps lie. And this is part of the simplification strategy of the EU moving into their competitiveness
Aashka Patel (40:56)
Hmm.
Yeah.
Jimmy Farrell (41:09)
intentions over the next few years to basically reduce double reporting and that sort of thing which is a no-brainer.
Aashka Patel (41:11)
Mm-hmm.
Yeah, yeah, yeah, makes sense. since we have been talking a lot about the liability thing, so the AI liability directive that was supposed to complement the EU AI Act got withdrawn from the commission’s 2025 work program, right? And MEP Excel was called it a strategic mistake and blamed industry lobbying pressure. So do you agree with him? What are your thoughts on that?
Jimmy Farrell (41:42)
Yes, I would mostly agree with him. think the liability piece is a really important part of the puzzle because the AI act, it’s really like before the facts. you know, are you transparent with your documentation? Are you compliant with copyright law? Do you have safety and security mitigations in place? Do you have serious incident reporting in place? But
Aashka Patel (41:46)
Mm-hmm.
Jimmy Farrell (42:08)
After something happens, actual cost, the liability, it remains to be addressed. One of the justifications of withdrawing it is that certain interpretations of the product liability directive could cover this, but most stakeholders would say that this is not the case and that AI is specific enough to warrant its own. It’s also something that has been heavily looked at in the US, so a lot of California frontier.
Aashka Patel (42:24)
Mm, yeah.
Mmm.
Jimmy Farrell (42:37)
legislations that were in consideration involved liability. Of course, it’s something that the tech industry and especially big tech industry pushes back on strongly. In general, it’s also quite relevant to the EU because the EU doesn’t have giant tech companies and their AI economy ecosystem is likely to be based off more downstream applications and sort of development ecosystem.
Aashka Patel (42:50)
Hmm.
Mm.
Mmm.
Hmm.
Jimmy Farrell (43:07)
And this liability is really important. Liability upstream. So basically if something happens involving a model deployed by a downstream developer, the liability doesn’t fall entirely on them. So this is also something that EU should really prioritize because it fits in with the whole grant strategy on competitiveness and the fact that they have a unique economy which is really bolstered by SMEs and that’s particularly the case
Aashka Patel (43:26)
Hmm.
Hmm.
Jimmy Farrell (43:36)
and
will be the case in
Aashka Patel (43:39)
Yeah, yeah, makes sense, makes sense. So what severe or strange AI incidents have you encountered recently?
Jimmy Farrell (43:48)
So yeah, this is something that has been developing over the last few months. There’s been numerous suicides linked to AI model. Of course, those links are not necessarily causal and it’s still under discussion how much influence AI models had. But there was a case in Belgium. There have been multiple cases in the US.
Aashka Patel (43:59)
Hmm. Yeah.
and
Hmm.
Jimmy Farrell (44:16)
And one is recently, the New York Times reported recently on one in which the chatbot specifically kind of coached the individual on how to carry it out and how to, you know, whether to leave a note, that sort of thing. And no triggers went off to inform the AI company involved or authorities. And this was potentially a preventable incident. Apart from these suicides, there’s also been
Aashka Patel (44:16)
Yes, teen suicide especially, right? Yeah.
Yeah.
Jimmy Farrell (44:46)
Yeah, like indications of psychosis being induced by AI. There was one where an individual was killed due to psychosis that was potentially induced by AI. Other than harm to individuals, you’ve also had financial damage from deep fake scams.
Aashka Patel (44:52)
Yes.
Mm-hmm.
Yeah.
Jimmy Farrell (45:10)
an a company executive come on a Zoom call and have an employee transfer huge sums of money to that account. And also been the first incidents that are potentially related to AI agents. So this is something that seems like quite new, but there was a particular example where an agent kind of hallucinated a task it was given and basically deleted a
Aashka Patel (45:12)
Mm-hmm. Yeah.
Mm-hmm.
Yes.
Hmm.
Jimmy Farrell (45:38)
a large chunk of code and this could be extremely financially damaging to a company
Aashka Patel (45:41)
Mmm.
Yeah.
Yes, yes, yes. And also there have been reports of unauthorized purchases made by the AI agents like OpenAI operator
So for our viewers listening in, if they witness or experience harms from AI systems or these kind of incidents, like what’s the one thing you would want them to do
Jimmy Farrell (45:57)
Yep.
So I would say, yeah, in general, would caveat this by saying that my call would be much stronger towards governments because I think this is their responsibility rather than individuals at this moment. But if it was to be for individuals, I would really encourage them to get the message out and to basically report it to their authorities and also contact the company involved.
Aashka Patel (46:18)
Yeah
See you soon.
Mm.
Thank you.
Jimmy Farrell (46:36)
to
find more information. For example, certainly you always have something called the right to an explanation. So this could be something that could be invoked in the case of an incident. There’s also some incident databases which would benefit from information on incidents from the victims. So the OECD has a really large incident database. there’s a grassroots one called the AI Incidents Database.
Aashka Patel (46:46)
Mm-hmm.
Hmm.
Yes.
Jimmy Farrell (47:06)
And yeah, then it would really be on the responsibility of governments not only to put in better, you know, AI safety legislation or incident reporting, but preventative measures like AI literacy, education programs to make sure that individuals are aware of risks related to AI, whether it be personal relationships or business practices like cyber security breaches.
Aashka Patel (47:23)
Yeah.
Mm hmm.
Jimmy Farrell (47:36)
this was.
Aashka Patel (47:36)
Yeah,
yeah, yeah, yeah, make sense. And that’s why organizations like ours exist for to promote AI risk literacy instead of just AI literacy. So thank you so much for your time. Yeah, edutainment Yeah. Yeah, you remember that.
Jimmy Farrell (47:55)
Yeah. Yeah.
Aashka Patel (47:57)
Thank you so much, Jimmy, for your time. Let me stop the recording.
