Q & AI is an interview series featuring Ivey AI Fellows tackling the biggest questions surrounding artificial intelligence (AI). In its first installment, Sam Ramadori explains why understanding AI's risks is just as important as embracing its potential.
Sam Ramadori, MBA '02, thinks business leaders are drastically underestimating the risks of artificial intelligence.
“I'm an entrepreneur at heart, so I come with a lot of optimism about AI,” said Ramadori, Ivey Business School AI Fellow and former CEO of BrainBox AI, a pioneering firm which leverages AI to improve building efficiency and reduce emissions across the built environment.
But optimism doesn’t mean complacency. Having worked at the forefront of AI, Ramadori observed early signs of concerning behaviours emerging in advanced systems – behaviours that suggest the technology may not always act as intended. What he has seen, coupled with the rapid pace of AI advancement, has left him increasingly concerned that businesses are focusing on AI's opportunities without giving equal attention to its risks.
That concern led him to LawZero, a nonprofit dedicated to developing safe-by-design AI systems. As Co-President and Executive Director, Ramadori works alongside renowned AI researcher Yoshua Bengio, widely regarded as one of the godfathers of modern AI. Together, they are inventing a new path forward for AI systems – one that prioritizes safety, accountability, and the public good.
So, what risks should business leaders be paying attention to? And how can organizations harness AI’s potential without exposing themselves to unintended consequences? Ivey Impact sat down with Ramadori to discuss the future of AI, the challenges ahead, and what leaders need to know now.
Ivey Impact: It seems, alongside a mad rush by firms to adopt and embrace AI, there's also a growing chorus saying: Wait, is this the way we should be approaching AI? Is this how we should be thinking about it as business leaders? Tell us about your stance on AI.
Sam Ramadori (SR): Sure. So, it’s already proven to do amazing things – AI in general, founded with the development of neural networks. In our little world at BrainBox AI, we found an autonomous application to tackle one of the biggest energy-consuming categories of our planet. It’s had a fantastic impact from a climate perspective. Now, AI is starting to hit drug discovery, accelerate material sciences, you name it. There’s a lot we can resolve if we choose to say we're going to dedicate our time, and compensate those doing research in fundamental areas of science and using AI in a fashion that rewards those who focus on areas that will improve our lives.
Ivey Impact: But that’s not the path we’re on?
SR: I mean, it doesn’t take much news reading to see these models are unreliable. Currently, organizations are doing what I call “soft applications.” So, I'm going to have a large language model support my marketing team to create content – great, you can optimize this function. But as we start shifting to the AI models making critical decisions, executing orders, delivering things – that is a whole other level of risk.
Ivey Impact: What risks are you most concerned about here?
SR: These Large Language Models are black boxes. I'm not sure how many inventions in human history were such that we can't go back to the inventor and ask them, I did A and outcome B happened, how come? And the inventor has no choice but to say: I don’t know. Think about the implications of that in terms of ensuring safety and reliability.
Ivey Impact: So, business leaders can use the tools but have no control over the outcomes?
SR: As much as we want to align an AI model to our end objectives, we could never tell if it's creating subgoals inside of it and what those subgoals are. So when an AI model does something dangerous, there’s no inherent evil there. It's just logic. Logic that is unconstrained by all our human morals and the laws we live under.
Ivey Impact: We’ve heard these eerie stories of chatbots blackmailing users – sort of a complete the task at all costs kind of scenario.
SR: Here is an example: when models are being evaluated today to assess their performance, they know they're being evaluated, and they're changing their behaviour in order to score higher on those evaluations. No one asked the models to do that, and we don't want the models gaming the evaluations. So, the implication there is, if you’re increasingly using more models in the form of autonomous agents where they're bouncing around the four walls of your company and making all kinds of decisions, who's tracking those and ensuring they're aligned with what you want? And then, if two different agents start coordinating with each other, we'll have no clue what they're communicating to each other and deciding.
Ivey Impact: Do you feel like business leaders understand these risks?
SR: Based on my conversations with corporations, business service firms, et cetera, the awareness of the risk levels should be higher than it is.
Ivey Impact: How should business leaders be thinking about these risks?
SR: Business leaders need to have that barometer of risk. They need to say: okay, we can use it for less risky applications to be more efficient and save costs. But as they apply AI in more risky applications, they have to have some point where they’re like, okay, we’re not ready to cross this line yet, as models are not yet reliable enough.
Ivey Impact: You’re suggesting some sort of internal structure for calculating that risk based on the business/sector/goals/etc…?
SR: Like any technology adoption, business leaders are grappling with the challenge of the slowness of adoption and resistance to change. So even with a transformational technology like AI, today's adoption is only going at a certain pace. So on the one hand, business leaders should be working towards speeding up adoption to bring this technology in-house. But they should also make sure that they have good sounding boards and advisory teams (internal and external) to help them determine where that “no cross” line is today in terms of applying AI in their internal processes.
Ivey Impact: How do you figure out where the line is?
SR: Set up an advisory board made up of trusted experts who also see applications of AI in other industries. Have the information and tools to keep that line front and centre. Suppose you’re a bank. Maybe you're discussing with your board the possibility of letting AI set the interest rate every week, and decide we’re just not going to go there. We will let it answer individual customer complaints or customer service requests, or work on some of my accounting functions – where meaningful efficiency gains can be achieved at lower risk. But for setting interest rates, the technology is not there today. Have that line set out, knowing that it will slowly shift as time goes on and the technology's reliability improves.
Ivey Impact: How does LawZero fit into all of this?
SR: Today's LLM models take a series of words and develop the ability to predict the next word. That's its core capability. We feed it as much information as we can, and it improves this capability through scaling the amount of information and compute available. You multiply that capability trillions of times over, and an LLM can do everything it does today. But, at its core, it is just one big probability machine. Nowhere in there is this idea of what's true/factual or not. At LawZero, we are working on a new foundational structure for AI that we call “Scientist AI” – AI that provides the same level of intelligence as current LLM's but removes the risk elements that come from sycophancy, deceitfulness, and misalignment of goals. In essence, we want the AI model to behave like a scientist that will answer queries as honestly as possible, without having any ulterior goals.
Ivey Impact: How do you compete with the well-funded models that are already in mass adoption?
SR: It's quite ambitious to say we’re going to reinvent the middle of a large language model when the top four companies have hundreds of billions of capital at their disposal to keep working on the existing LLM infrastructure. But if they don't manage to resolve the fundamental flaws of these models, making them smarter will only increase the risk of a very bad outcome. If we can show up with a model that has a similar level of intelligence, but that can also be far safer and more reliable, we believe we have a chance to change the course of AI innovation toward a safer future.
Ivey Impact: So, the idea is that by grounding these models in weighted truth and facts, we increase the reliability and reduce the risk, creating better outcomes for organizations implementing AI tools?
SR: If we can come forth with a model where its reliability and traceability are dramatically improved, then we can go past the soft applications within companies, such as customer service, and get to a point where we would trust it to run things like our energy grids or transportation systems. But to do that, you have to trust it, and it has to be reliable.
Ivey Impact: Any parting advice about AI adoption?
SR: I believe the world will change substantially in the coming years. I think it is the responsibility of business leaders to have an ongoing dialogue with the government about critical issues that may arise as AI adoption grows. For instance, what happens if unemployment gets into double-digit territory? How do we prepare ourselves for this potential scenario? Let's all be frank: it’s no use getting efficiencies up in your company if your customer base is experiencing mass unemployment.
Interview edited and condensed for clarity.
Sam Ramadori, an Ivey AI Fellow, is Co-President and Executive Director of LawZero, a nonprofit founded by Professor Yoshua Bengio focused on advancing research and technical solutions for safe-by-design AI systems. He is an AI entrepreneur and former CEO of BrainBox AI, where he led efforts to apply AI to decarbonize the built environment. He previously spent 15 years in private equity investing. He holds an MBA from the Ivey Business School and Civil and Common Law degrees from the University of Ottawa.
