Are you an analyst or data scientist at a large organisation?
Raise your hand if you’ve ever come across these head-scratchers:
- Finding data felt like going on a Sherlock expedition.
- Understanding data lineage was impossibly frustrating.
- Accessing data became a showdown with the red tape monster.
Here’s a common quip I hear from citizen and professional analysts alike:
“Those data governance guys sure know how to make life interesting…”
It’s time to cut them some slack.
Drawing from my experience as an engineer and data scientist at one of Australia’s banking giants for half a decade, I’ve had the privilege of straddling both sides of this heated fence: being a hungry consumer of data while simultaneously standing as a gatekeeper for others.
In this article, I’ll do a three-part dive into…
- Fundamentals: How data flows through organisations. It’s messy!
- Understanding common pain points encountered by data users.
- Enlightenment: Understanding that guardrails support innovation.
The third point is really important.
Organisations worldwide are scrambling to become data-driven firms. There is a constant tension between fostering innovation yet having appropriate controls in place to keep a company’s customers, staff and reputation safe.
As new data use cases arrive — and that’s all the time — data governance structures try to evolve in tandem. And it’s typically a struggle, because unbridled innovation has no natural speed limit.
An all-you-can-eat data buffet sounds good until your firm gets lobbed a multimillion dollar penalty from regulators for having customer data leaked onto the dark web.
Whoops, should’ve had those controls.
Data flows like a downstream river through an organisation.