How to start developing a balanced AI governance strategy

The best defense is a good offense, right? When it comes to AI governance, organizations need both. The reason is that genAI capabilities evolve quickly, hype accelerates investments, and data risks are amplified through AI applications. This article looks at how to shore up your defense while planning and implementing a strong AI governance offense.

Developing your AI governance strategy

Key questions to address in AI governance include what important regulatory compliance is needed, what data can be used in training AI models, what data must not be shared with public LLMs, and what tools can be used to deploy AI agents. Asking these defensive questions can help protect your organization.

A strong offense guides the business objectives, outcomes, and capabilities to focus on when applying AI. Defining your offensive strategy for AI governance helps channel everyone’s efforts to areas where AI can generate business value and drive digital transformation.

“Our thinking about AI governance is often too limited, focusing only on compliance and risk reduction,” says Kurt Muehmel, head of AI strategy at Dataiku. “Governance is a strength that ensures that AI is aligned with business objectives, is produced efficiently, follows internal best practices, is designed for production from the beginning, and promotes reusing components. AI governance thought of this way becomes not an obligation but a competitive differentiator.”

Executing AI governance with defense and offense strategies requires IT and data science teams to address deficiencies in how organizations balanced innovation and governance in the past:

Many devops teams developed applications, then bolted on security until devsecops practices helped drive a shift-left security culture.

Organizations were at first afraid to use public clouds, then aggressively modernized applications for them, only to put finops disciplines in place afterward.

Ask the chief data officer (CDO) about the challenges of assigning data owners and classifying data while departments adopt citizen data science and become proficient in data analytics platforms.

Beelining to AI capabilities without instituting AI governance is a recipe for AI disaster.

The CDO leads AI governance

Many organizations view AI governance as an extension of data governance and assign the CDO to take charge of defining it. CDOs also have many data-related responsibilities that are foundational for safe and secure AI practices.

“The true opportunity with AI governance does not lie in reducing ethical, regulatory, and business risk—AI governance is needed to help organizations drive the trust and adoption necessary to transform the business using AI,” says Kjell Carlsson, head of data science strategy and evangelism at Domino. To achieve this, CDOs must provide visibility, auditability, reproducibility, and control—and implement platforms that orchestrate, streamline, and—where appropriate—automate governance activities so that people can focus on reducing risks versus wasted manual effort.”

Henry Umney, managing director of GRC Strategy at Mitratech, says the key priorities on the CDO roadmap for data and AI governance should include:

Creating a clear definition of AI within the organization and categorizing the new risks AI introduces.

Building an AI model inventory ranked by business impact and regulatory risk like the EU AI Act.

Benchmarking existing governance and risk management structures against frameworks like NIST AI RFM, adding AI-specific controls to existing frameworks across the organization.

The role of the CDO is not just to implement these practices; CDOs must prioritize them effectively and communicate to business stakeholders how governance enables efforts to deliver business value.

“A CDO’s roadmap should balance the adoption of transformative technologies like genAI with the critical need to maintain data sovereignty,” says Jeremy Kelway, VP of engineering for analytics, data, and AI at EDB. “This goes beyond risk reduction as data sovereignty covers governance, observability, and jurisdictional boundaries, laying the groundwork for offensive strategies that drive growth, sharpen competitive capabilities, and enhance customer experiences. By ensuring the appropriate data is secure and shared compliantly with a clear understanding of how the AI models use it, CDOs can confidently leverage their data for real-time insights that spark innovation.”

Bring in business leaders and other stakeholders

Your organization may need a data fabric if many data sources are used in AI models. The chief information security officer should review data security posture management (DSPM) platforms to secure confidential or regulated information stores across multiple clouds, data centers, and edge devices. Then, add observability, data pipeline, data catalog, customer data platform, and master data management capabilities.

It’s a significant enough investment to make a business leader’s head spin. Many of these tools and related practices are important to have in place and upgrade to support AI. However, the CDO and data teams better answer why and why now or risk losing business stakeholder interest.

CDOs should craft an AI vision statement, define a data strategy, and manage a roadmap aligning with a plan to drive AI offense capabilities.

“Build your data and AI strategy and a culture of rapid yet responsible AI from day one, as adding it later is much more challenging and costly,” says Ana-Maria Badulescu, senior director of the AI lab, office of the CTO at Precisely. “The CDO roadmap should go beyond governance by providing a comprehensive, integrated solution that covers data quality, data observability, data catalog, data security and privacy, data enrichment with third-party data, and location intelligence. Break down data silos by creating data governance councils and business glossaries to ensure a shared language across the organization.”

What may be obvious to data teams may be out-of-sight, out-of-mind for business stakeholders. Heather Gentile, director of product management of AI risk and compliance at IBM, suggests reinforcing that the results of a model are only as good as the data on which it is built and trained. “The transparency and explainability of governance also successfully accelerates and scales AI initiatives and business impact,” says Gentile.

Embrace new AI data governance priorities

CDOs, data governance, and data scientists must also consider AI-specific capabilities. For example, modelops is the discipline of monitoring ML models for drift and other conditions necessitating retraining. For genAI, data teams should be explicit about what data was used to train an LLM, RAG, AI agents, and other AI capabilities.

“An AI Data Bill of Materials (AI DBoM) is the foundation for responsible AI at scale and should be a part of the CDO’s governance strategy,” says Kapil Raina, data security evangelist at Bedrock Security. “An AI DBoM tracks all data feeding AI models—training, fine-tuning, and inference—ensuring quick project turnarounds with full transparency into what AI systems access and generate. Without it, CDOs are flying blind—exposed to security gaps, non-adherence to the rapidly evolving regulatory landscape, and stunted innovation.”

CIOs, CDOs, and IT teams should also recognize that what worked as a pre-AI data strategy may need enhancements and overhauls to accelerate AI opportunities. Organizations that distribute significant dataops and data quality work to business teams may want to consider centralization to drive efficiencies and consistent data quality metrics.

Rahul Auradkar, EVP and GM of Unified Data Services at Salesforce, says, “Reducing tech debt caused by different data governance controls, manually classifying and tagging data, and the rise of data-driven decision-making has increased governance priorities for CDOs today.”

Another consideration is for construction, manufacturing, and field services organizations with highly distributed operational teams that use disparate management tools and rely on spreadsheets. Their efforts to consolidate workflow tools can drive efficiencies and pave a faster path to AI capabilities.

“Without solid governance in place, your workforce must scramble to find the information they need to do their jobs because it’s trapped in tools and information silos,” says Jon Kennedy, CTO of Quickbase. Such gray work, he says, “undermines productivity and has a ripple effect on the customer experience. Through a consolidation process, IT can address tech sprawl, centralize information on a work management platform, and eliminate gray work while executing their data and AI governance roadmap.”

Elements of an offense strategy in AI governance

The practices I’ve shared so far form the defensive AI governance strategy. While a good defense is the foundation, your AI governance strategy go one step further.

“A CDO’s data and AI governance strategy should do more than manage risks—it should fuel growth, competitive edge, better customer experiences, and new market opportunities,” says Ed Frederici, CTO of Appfire. “Treating data as a revenue-generating asset and ensuring seamless interoperability can help scale the business.”

Frederici recommends the following ways to enhance your AI governance strategy with a good offense:

Drive efficiencies through AI-driven automation and internal data marketplaces.

Build customer trust and increase engagement through personalization engines and ethical AI.

Improve services by using predictive AI to anticipate needs and reduce churn.

Accelerate product development with AI-powered market insights.

Help businesses stay ahead with cross-industry collaboration and strategic data sharing.

Business stakeholders might treat data as assets when leaders extend data and AI governance to a strategy and practices for developing data products. Data products can simplify how departments reuse data and AI capabilities efficiently and create opportunities for developing customer-facing data products and collaborations.

“By treating governance as a built-in capability of data products rather than a separate control layer, organizations can accelerate innovation and time-to-value while actually improving their risk posture through standardized, reusable patterns,” says Srujan Akula, CEO of The Modern Data Company. “Data products with embedded governance controls become powerful building blocks for growth that help launch new customer solutions faster and expand into new markets more easily.”

Another opportunity is to focus data and AI governance opportunities around sales and other revenue-generating workstreams. Jason Smith, senior principal of strategy and transformation at Conga, suggests, “Data leaders must prioritize AI-driven revenue management tools that eliminate silos between departments and streamline the entire revenue process—from proposal to quote generation and more.”

Some may argue that including offense strategies as part of AI governance bleeds into the organization’s overall business, digital transformation, or AI strategy. Perhaps this is a good thing when getting business leaders excited about the offense, as it ensures they also pay attention to the defense.