The Growing Importance of Small Language Models in Business
Image: Why businesses are looking to small language models to create real value
As organizations strive to embrace the power of Generative AI, they face multiple challenges when it comes to effective implementation. From security concerns to the high costs associated with AI projects, businesses often find themselves at a crossroads, uncertain about how to unlock the true potential of AI technology. Despite the promise of enhanced productivity and streamlined processes, the road to AI success is fraught with complexities, including an immature regulatory framework and a rapidly evolving market landscape.
The Challenge of Implementation
The struggle many companies experience with initial AI projects often stems from the difficulty of meeting ever-changing demands. Utilizing AI through cloud APIs can become costly in the long run, and without clearly defined success metrics, proof of concepts (POCs) frequently stagnate without moving forward. Ceri Carlill, Business Value Director EMEA at Red Hat, sheds light on this issue by pointing out that while many employees now leverage AI in their daily tasks, businesses often find it hard to harness that value effectively. This “stealth value,” as he describes it, remains elusive without strategic implementation and planning.
Value in Starting Small
To truly capture value from AI, experts recommend starting small with precise use cases. Robbie Jerrom, Senior Principal Technologist AI at Red Hat, emphasizes the importance of involving key stakeholders—such as IT, legal, HR, and financial teams—early on. Such alignment ensures that the AI initiatives align closely with the business’s overarching goals. For instance, Red Hat successfully focused on its support function, employing generative AI to provide targeted answers to support queries, resulting in significant cost savings and improved documentation.
Small Language Models (SLMs): A Practical Solution
The effectiveness of an AI project significantly hinges on the choice of tools. While large language models (LLMs) possess vast capabilities and knowledge, their breadth often leads to inefficiencies for specific applications. On the other hand, small language models (SLMs) distill the linguistic expertise of their larger counterparts into a more focused, specialized framework. This compactness makes them cost-effective and easier to manage, allowing for deployment even on standard laptops or mobile devices.
As Robbie Jerrom explains, SLMs can be tailored to specific sectors, such as pharmaceuticals or insurance, making them far more agile. They offer a more straightforward approach to optimization and training, which can drastically cut costs and improve implementation timelines.
Agentic Settings: Combining Advantages
The integration of large and small language models presents new opportunities. By employing an agentic approach, organizations can use large planning models to coordinate multiple specialized SLMs, thus leveraging the benefits of both. This setup not only aids privacy and data security but also proves to be more economical than relying solely on a single LLM. As Carlill notes, SLMs can perform specific tasks while the larger models handle overarching structural problem-solving.
Employing both SLMs and LLMs effectively requires a robust platform. Companies like Red Hat are developing their open-source solutions to streamline the deployment of AI models in hybrid cloud environments. This consistency enhances operational efficiency and aligns AI models with the applications they support.
The Advantages of Open Source
The role of open-source technology in AI development cannot be overstated. By utilizing open-source models—whose origins and training data are transparent—businesses can significantly mitigate legal risks associated with proprietary datasets. Carlill advocates for this approach, asserting that open source is not only the best path forward but is already demonstrating its utility in real-world applications.
As the landscape of AI evolves, it’s crucial for organizations to stay informed about the latest advancements and methodologies, including selecting the most suitable tools for their specific needs. By prioritizing open-source approaches, adopting small language models, and promoting collaboration across departments, companies can transition beyond pilot projects and pave the way for meaningful, measurable change in the AI arena.