Semantic Layers for AI: Start Small, Define Clearly, Build Trust

Almost every financial institution I speak with wants its data to be “AI-ready.” Far fewer can say exactly what that means. In my view, a semantic layer earns its place when it makes business information easier to interpret, use, and govern. The real test isn’t how elegant the semantic model looks. It’s whether the decisions and workflows built on it get better, and whether the organization can explain and maintain the results.

This article picks up threads from a panel discussion in which I recently participated called “Building the intelligence layer — semantic architecture for AI-ready data products,” that took place at the A-Team Data Management Summit NYC on September 17, 2026. I combined what the panelists discussed with my own perspective and supporting research to offer practical guidance for financial institutions.

Dessa Glasser, Independent Board Member at Oppenheimer & Co., moderated the session. I was joined on stage by Sri Bhupatiraju, Director and Principal AI Engineer at BlackRock AI Labs; Rohan Kodialam, Co-Founder of Sphinx; and Matt Katz, Field CTO at Arcesium.

Here is the short version of my advice:

1. start with a business problem worth solving
2. make its meaning explicit
3. grow through measurement and governed change

For executive sponsors, that means funding two things: the technology, and the operating discipline that keeps the resulting information trustworthy.

Before anyone debates platforms or ontologies, someone has to own the problem. Someone needs to decide which question matters, who can approve its definitions, and what a good answer looks like. A new platform or a more elaborate ontology won’t answer these questions.

Supervisors have also highlighted this. The Basel Committee’s January 2026 newsletter on BCBS 239 implementation lists fragmented responsibilities and insufficient senior-management attention among the continuing barriers to effective risk data aggregation, alongside lineage, cross-border issues, and implementation complexity. While the newsletter is informational rather than new supervisory guidance, the message is clear: governance and engineering have to work together.

How does this work in practice? A well-defined first use case makes accountability concrete. For example, take a recurring question such as “How many clients do we have?” First, agree on the population, the reporting date, the exclusions, and the expected answer. Then establish a baseline for today’s accuracy and reconciliation effort before you change anything. This approach turns a vague ambition to “make data AI-ready” into an outcome the business can actually assess.

Of course, starting small is a delivery recommendation, not a promise that you can avoid complex enterprise modeling forever. Reuse the definitions and data assets you already have, agree on just enough shared conventions to support the first domain, and expand when the benefits justify the next increment.

Measuring success has to go further than asking whether an answer sounds plausible.The voluntary AI Risk Management Framework proposed by the National Institute of Standards and Technology (NIST) supports documented test sets, evaluation under deployment-like conditions, and continued production monitoring. It does not prescribe one universal accuracy target, and you shouldn’t expect one. My advice is to build a set of representative questions with reviewed answers, and deliberately include ambiguous terms and incomplete data. Track correctness, whether the system asks for clarification or declines when it should, response time, operating cost, and manual review effort.

With the problem and its measures of success defined, the next step is the foundation beneath them. I find it helpful to picture the surrounding architecture as four cooperating building blocks:

• a storage foundation
• a semantic core of metadata and business context
• a knowledge layer of relationships and expert documentation
• and a distribution layer that handles access and controls

This is a framing device for the architecture around the semantic layer, not a standard definition of the semantic layer itself.

What matters is how those pieces help a person or a system interpret information correctly. A business term should connect to its approved meaning, the data that represents it, and any calculation or qualification needed to use it properly. People need clear explanations and easy ways to review changes. Automated consumers, such as AI/ML models, AI agents, and reporting or analytics applications, need explicit interfaces and rules that can actually be enforced.

This is corroborated by Human-AI interaction research that supports making capabilities and limitations clear and making correction efficient.On its own, though, it doesn’t make an interface suitable for AI agents. A screen that works well for a data steward isn’t the same as an interface an AI Iagent can rely on. AI agents need that same meaning exposed through explicit, machine-readable interfaces and rules they can’t bypass.

Importantly, don’t let debates over modeling formats stall delivery. Pick a representation and validation approach that fits the use case. Keep domain meaning, technical structure, and validation rules explicit, and test that your integrations preserve them. A model being able to read a representation doesn’t prove it will interpret or apply that meaning correctly.

With that foundation in place, the next question is how AI agents should use it. Here’s how I frame it: governed, approved semantic capabilities meet an AI agent’s probabilistic interpretation and planning. The agent interprets the question and organizes the investigation. Business-approved definitions and calculations constrain how agents can use the underlying information. The goal is to make business meaning and permitted operations explicit enough both to guide the AI agent and to assess what it produces.

This distinction deserves care. Calling a semantic layer “deterministic” doesn’t mean it is correct. Incorrect mappings, stale data, or a flawed definition will simply produce the same wrong answer every time. And an AI agent can still pick the wrong approved concept, or wrap a valid result in an explanation the data doesn’t support.

My recommendation is practical. Put definitions, applicable permissions, source context, and known limitations right where the AI agent uses the information. When a question is ambiguous, the system should surface the choices for a person to select from rather than quietly pick one. And every result should carry enough evidence for a person to understand how it was produced. These are design choices you have to make deliberately; no single technology hands them to you.

NIST’s Generative AI Profile identifies confidently stated false content as a risk. It recommends verifying sources and citations, testing capabilities empirically, and avoiding generalizations from anecdotal assessments. A semantic foundation should be evaluated as one control within that broader process. It cannot eliminate every model or workflow failure, and it shouldn’t be positioned as if it could.

Once AI agents work within a business-approved meaning, you have to decide where the underlying data should live. The question of virtualization versus physical consolidation comes up often, and, in my view, the answer should be dictated by regulation, performance, and business outcomes rather than architectural preference. Querying data in place can reduce unnecessary copies and keep you close to current sources of information. On the other hand, consolidating selected data helps when a workload needs repeated processing, reconciliation, or more predictable performance.

The demands of AI agents belong in the second category. Repeated exploratory queries can strain source-system capacity, add network traffic, and raise costs. Look at the actual workload: concurrency, query complexity, and data movement. Where supported, executing filters and aggregations close to the source can reduce how much data has to travel. Queries that combine sources, however, may still require substantial processing and movement elsewhere. Measure the complete execution path against the service levels you need.

Residency requirements may limit where data can be processed, but virtualization doesn’t establish compliance by itself. Results, temporary copies, and downstream AI processing all need to be considered, too. For many use cases, a hybrid approach is worth evaluating: leave some data at its source and materialize the portions where usage and service requirements justify it.

How you organize information for retrieval also impacts cost. Choose an approach that fits the questions users need to answer and the evidence those answers require. Then compare total costs at equivalent answer quality, including preparation, indexing, refresh, and maintenance, and validate the results against representative business questions.

Data placement is only part of the picture, because much of the relevant evidence sits in documents rather than tables. A PDF and a table can describe the same business subject, but they are very different processing problems. Modeling them together pays off at the conceptual level: link documents and records through common business meaning while respecting the kind of evidence each one provides.

Sometimes extracted text is enough. Other times the answer depends on a table’s layout, a chart, or a mark on a form. Research on document understanding identifies structure and visual cues as important to interpreting document images. Use text where it preserves what the task needs, and compare total processing cost and accuracy before you discard other representations.

When the same information keeps getting pulled from documents, consider promoting it into structured form. Preserve the source and its context, validate the extraction, and decide what deserves a maintained representation. Measurement tells you when that investment improves accuracy or reduces recurring work. What you want to avoid is re-extracting a business-critical fact again and again without retaining how it was derived.

Once records and documents share meaning, the challenge becomes keeping that meaning consistent as domains evolve. Shared definitions at the center can coexist with domain ownership and extensions around the edges. That keeps business expertise close to the work while preserving common standards and a clear process for resolving disagreements.

The client-count example described above illustrates why this matters. Funded accounts and legal entities that have completed KYC are different populations. Both are useful measures, but neither should sit under a bare “client count” label. Define the counting unit, population, time basis, owner, and intended use so the difference is visible to everyone who relies on the number.

The European Central Bank’s (ECB) 2024 guide on effective risk data aggregation and risk reporting supports clear ownership, uniform definitions, validation rules, and current attribute-level lineage for relevant indicators and critical data elements within scope. It also permits specific taxonomies by risk type or legal entity when consistent with the applicable scope. In other words, it supports governed specialization rather than one undifferentiated definition for every purpose.

Treating semantic metadata like code is what makes that governance operational. To that end, version definitions and mappings, review changes, test important outcomes, and keep a release history. Before publishing a change, assess its effect on dependent reports, data products, and agentic workflows. The business owner is responsible for approving meaning; the engineering team is responsible for implementing it reliably.

Controlled evolution means changes are explicit. A revised meaning needs an owner, a rationale, an effective date, and an assessment of affected consumers. Unexpected divergence needs investigation. Allowing legitimate change is one thing; letting definitions drift unnoticed is quite another.

Even with strong governance, errors and disagreements will still occur, so you need a reliable way to correct them. Start by separating a legitimate difference in definitions from an actual error in the model, because each calls for a different remedy. A valid domain difference needs clear scope and naming. A wrong relationship, mapping, or calculation needs correction, plus an assessment of its downstream effects.

Lineage lets you trace a result back to the source, transformation, definition, and approval history that produced it. The ECB guide connects lineage with identifying the origins of data-quality issues and supporting validation. That makes lineage part of the corrective process, working alongside ownership and testing.

Keep in mind that fixing something centrally doesn’t automatically fix everything that depends on it. Derived datasets, retrieval indexes, caches, and already-issued reports may each need separate action. A feedback loop is most useful when it captures the disputed answer and routes it to an accountable owner. Close the loop by testing the correction and assessing affected uses. Don’t assume that a new model-generated answer means the underlying issue has been fixed.

Governed meaning and traceable corrections also prepare you for changing external reporting expectations. I see the Financial Data Transparency Act of 2022 as strengthening the case for machine-readable information with explicit meaning and common data standards. The joint data-standards rule was published June 25, 2026, and became effective October 1, 2026. That effective date did not, by itself, change firms’ reporting requirements for financial institutions. Further agency action is required.

The final rule establishes common standards, including the Legal Entity Identifier (LEI), along with properties for searchable, machine-readable data and metadata describing meaning, to the extent practicable. It does not prescribe a particular enterprise ontology or semantic-layer product. The practical step is to maintain governed mappings from internal concepts to the applicable reporting requirements and to track agency-specific rules for each collection. Your architecture supports that work, but it doesn’t establish compliance on its own.

Taken together, these practices form a capability the business has to fund and sustain. The expected benefits are meaningful: clearer definitions, less repeated reconciliation, broader access to institutional knowledge, and faster reuse of trusted information. How large they turn out to be depends on adoption, source quality, and the ongoing work of stewardship and engineering. Measure them against the cost of keeping the capability running.

In financial services, a wrong answer can quickly become a reporting, risk, or operational problem. The consequences depend on how the result is used, not on whether AI happened to be involved. Match validation and human review to that use. And build the first implementation so people can understand it, challenge it, and improve it before you expand its reach.

Speaking as an engineering leader, I have found that semantic initiatives rarely stall because of the technology alone. They stall when meaning stays implicit, ownership is unclear, or no one measures whether the answers are getting better. That’s why my advice comes down to this:

In practice, that means picking one focused outcome, making its meaning clear enough to test, and maintaining it through an accountable operating process.

That is also the philosophy we bring to our work with financial institutions here at InterSystems. The right platform makes this possible, but the real evidence of progress is a business workflow that performs better, with results the institution can explain. If your team is working through these same questions, I’d welcome the conversation.