Originally published by techUK on 16th May 2025.
The recent eruption of AI in UK businesses has been spectacular. The US International Trade Association calculated that the UK’s AI market will be worth more than $21bn by 2035, crediting the country with “one of the strongest AI strategies in the world”.
However, despite widespread adoption, the outcomes have often fallen short of expectations. With the emergence of more advanced techniques such as vector search and retrieval-augmented generation (RAG), the focus is shifting from experimentation to wide-scale integration. The challenge now is whether organisations have the data infrastructure, system interoperability and governance in place to operationalise these models effectively – or risk compounding the limitations that have already hindered AI performance.
Since 2022, when firms began adopting generative AI at scale, public cloud infrastructure has been the default choice for model training and compute-intensive machine learning workloads. But as usage expanded, many organisations found themselves facing six- or even eight-figure cloud bills – costs that not only strained budgets, but also forced a rethink of how sustainable and scalable their AI ambitions really were.
Cloud use has caused costs to balloon
All organisations must think hard about marshalling and preparing data for use in any AI application whether tabular AI or generative AI. Siloed data and legacy systems, both of which present difficulties in the digital transformation process, must still be overcome to get ready for AI. If the public cloud is too costly, organisations need to think about the alternatives.
It is possible, for example, to conduct large language model training using a constrained set of data on Apple MacBooks with M4 chips with abundant memory. Organisations may not want to follow this precise technique but provisioning their own hardware or infrastructure for AI is important for companies that want to maintain control of their data and reduce the chances of its running wild in a public large language model (LLM). For firms that need a 500 or 1,000 GPU cluster for their workloads, however, the cloud still probably makes sense.
Vector search, RAG and the need for clean data
As organisations seek to build a “closed-garden” LLM for their own business purposes, the requirement to normalise their data is just as urgent as for any advanced analytical purpose. The growth of vector search in AI as part of RAG approaches is a major advance in capability. RAG combines the power of vector search with the LLM ability to craft contextually relevant responses in the right tone and format.
Vector search allows an organisation to query data including images and audio, based on meaning and intent rather than the data itself. It is a technique that relies on mathematical vectors to represent data. These vectors will find similar items rather than exact matches as in keyword searching.
This helps overcome some of the limitations of LLMs such as stale data, token limits, and hallucinations. In medical literature researchers can use these advances to find relevant studies even if terminology varies. In patient records, vector search will uncover similarities for personalised treatment.
Most organisations, however, will not create their own LLMs, they will take those that are already available and feed them data from their own vector database, using the models to answer queries. These organisations do not need vast seas of data to start. They can try something out and see what results they obtain before expanding. Not requiring big datasets, this is relatively easy.
Much of the success will depend on wrangling data beforehand. Feedback loops will indicate whether an organisation needs to change how it is vectorising its data to gain improvements. Testing and experimenting will ensure greater benefit from the whole project.
Whichever form of AI application organisations deploy, the data they use needs to be clean, consistent, and free of duplication. Some organisations may find the only way they can wrangle data is in batches overnight after copying over from a database so they can prepare their data. Yet this makes the use of any real-time, or near-real-time AI application (such as AI assistants for customer-facing agents) extremely challenging.
Operationalising AI for rapid and more intelligent decision-making is extremely difficult when faced with these constraints. Some organisations have invested heavily in commercial technologies that prove costly to provision and have significant architectural limitations including a critical inability to scale, severely impairing the benefits of AI implementation.
Many of these commercial technologies are in fact ecosystems of different products that create a high level of complexity which increases as more data is injected. There are engineering processes to manage these challenges but employing them is not easy.
The role of smart data fabric architecture in AI
Organisations need a more effective solution with which to simplify their data infrastructure so they can overcome the drawbacks of legacy systems and disparate data, without introducing overly complex and costly alternatives.
A smart data fabric architecture is proving to be ideally suited to the purpose. It enables organisations to install three vital technologies to power AI – data integration, data management, and vectorisation - in a single solution. This is a key advantage because it does not require a separate vector database. It is able to harmonise and clean up data from multiple sources without duplication or the need to move data around.
Whereas conventional technologies can take many months to deliver any results, a smart data fabric can be operational in a short time.
Nevertheless, organisations still need to do some preparation – primarily understanding what they want to vectorise. When they reach the point of AI implementation and examine their options, many organisations are found to have poor understanding of their data. They do not fully grasp what they want or need to change to give them a more flexible or dynamic model. The belief that transformation comes from sprinkling AI fairy dust suddenly collides with reality.
AI is fast-evolving but has had false starts in organisations with inadequate preparation. As we look forward to vectorisation and RAG, and the expanding use of LLMs, organisations should ensure their data is fit for purpose, deploying a smart data fabric architecture before they embark on more far-reaching AI implementation.
