Why Trust and Safety in Enterprise AI Is (Relatively) Easy | by Cassie Kozyrkov | May, 2023


Why traditional AI has the reliability advantage over generative AI

In Part 1 of this series, I said something that I’d thought I’d never say: when we’re dealing with typical enterprise-scale AI systems, trust and safety is easy.

What?! Blasphemy!

Hear me out.

Yes, okay, it’s actually pretty hard. But the difficulty pales in comparison to the trust and safety headache that is the new wave of generative AI. Here’s why.

All image rights belong to the author.

Imagine you’re the CEO of an airline without an AI-based ticket pricing system. Your Chief of Staff runs into your office panting that some team of data hotshots in your organization is hours away from a full-scale AI pricing system launch, but they were overheard saying, “I have no idea how good this AI system is. Dunno how much revenue it makes or loses… but it seems useful, so let’s launch it.”

Heads will roll. Such a system’s reach and potential business impact is too massive for this level of sloppiness. You’ll likely fire everyone who had anything to do with this completely unhinged scenario and you’ll be right to do it. After all, as the CEO, ultimate responsibility for the airline’s success falls to you and getting rid of this gaggle of clowns will be a no-brainer given the inappropriate level of risk they almost subjected your enterprise to. The whole situation is criminally stupid. Your company is better off without them.

Say what you will about large organizations, but the one thing they tend to be good at is avoiding anything that frivolously rocks the boat.

Typically, a problem like that is smothered long before it reaches the CEO’s desk. Say what you will about large organizations, but the one thing they tend to be good at is avoiding anything that frivolously rocks the boat. There’s a built-in preference for caution over gambling, which is why an enterprise-scale AI system typically only gets out of the gate if (1) it provably solves a specific problem provably well or (2) it has a provably low potential for harm (because the stakes are low, because errors wouldn’t be very embarrassing/painful, or because the application is of low strategic importance).

The straightforwardness of the AI system’s raison d’être is an extremely powerful simplification tool.

Examples from an airline’s point of view:

(1) An AI pricing system that is carefully launched in a gradual ramp-up and statistically tested to have a positive revenue impact of at least x%.

(2) An AI pronunciation system that allows a gate agent listen to a data-driven best guess about how to announce a passenger’s name to help the agent out if they’re unsure about the pronunciation. A system like this is hardly mission-critical and it comes with the upside of being able to tell the world you do AI without taking on much risk. Also, harmlessness is easier to achieve when trained humans get to approve all the output, so you’d want your gate agents to use their judgment.

“You want me to pronounce what?” (I often get this look when it’s time for people to pronounce my last name, Kozyrkov. I tell them to just say “coffee pot”, no one will notice.) All image rights belong to the author.

The point is that unless the trust and safety issues are already minimized by the very nature of the application, an enterprise-scale AI system isn’t going to see the light of day unless there’s proof that its upside is worth the risk… and getting this kind of proof is impossible by definition when there’s no clarity about the value that the system provides.

Why does this make things easy? Because it means that every mission-critical traditional enterprise-scale AI system (category (1)) tends to have:

  • a relatively straightforward use case statement
  • a vision of what the intended “good behavior” for the system looks like
  • a clear, monolithic objective
  • measurable performance
  • well-defined testing criteria
  • relative clarity about what could go wrong and thus which safety nets are needed

There are plenty of challenges here too, like how to guarantee that a system like this plays nice with all the existing enterprise systems (see my YouTube course for that and more), but the straightforwardness of the system’s raison d’être is an extremely powerful simplification tool.

The key insight here is that the economics for enterprise-grade solutions tend to favor scale. Systems intended for deployment at scale usually have a clear purpose, else a smart leader sends them straight to the garbage compactor. That’s why most enterprise-grade AI systems of the past decade were designed to do one very specific thing really well at scale.

Most enterprise-grade AI systems of the past decade were designed to do one very specific thing really well at scale.

This is a huge advantage for trust and safety. Huge! Sure, there’s plenty of general reliability work to do to ensure that you keep your users safe when your system meets “the long tail” (the unusual users), but it’s still a lot easier to protect a varied group of users from a single-purpose, single-function system than to protect the same group from a multi-purpose, multi-function system. And from most enterprises’ perspective, Generative AI systems are fundamentally multi-purpose and multi-functional.

That’s the key insight, so let’s repeat it:

It’s a lot easier to protect a varied group of users from a single-purpose system than to protect the same group from a multi-purpose system.

If you’d like a better understanding of this insight, continue on to Part 3 of this series.

On the other hand, if this last insight is obvious to you, then feel free to skip Part 3 and head straight to Part 4 where I explain why generative AI doesn’t come with these same simplifying characteristics and what that means for AI regulation.

If you had fun here and you’re looking for an entire applied AI course designed to be fun for beginners and experts alike, here’s the one I made for your amusement:

Enjoy the course on YouTube here.

P.S. Have you ever tried hitting the clap button here on Medium more than once to see what happens? ❤️

Let’s be friends! You can find me on Twitter, YouTube, Substack, and LinkedIn. Interested in having me speak at your event? Use this form to get in touch.





Source link

Leave a Comment