Navigating Generative AI Deployment: Strategies and frameworks for success

Altman Solon is the largest global strategy consulting firm exclusively serving the Technology, Media, and Telecommunications (TMT) sectors. The advent of generative AI has transformed industries, reshaping how businesses operate and innovate. As organizations increasingly adopt these technologies, understanding the intricacies of generative AI deployment and governance becomes crucial. In this insight, we explore key considerations in the development, deployment, implementation, and governance of generative AI tools through the lens of ASKMe, Altman Solon’s Knowledge Management Explorer.

The age of generative AI 

Generative AI has seen unprecedented growth. Altman Solon’s Putting Generative AI to Work study revealed that 81% of companies surveyed use or are currently implementing generative AI solutions. The question is no longer whether organizations will adopt generative AI but when and how they will use it. 

Over the past 18 months, generative AI has transitioned from a buzzword to a critical business tool across nearly all functional areas. Moving forward, it is imperative for organizations to acknowledge key considerations when implementing these tools. This includes the effective management and operationalization of machine learning models through MLOps practices, as well as adherence to governance and compliance standards to ensure ethical, legal, and secure operations.

ASKMe by Altman Solon 

Altman Solon's Knowledge Management Explorer (ASKMe) is a custom gen AI tool that Altman Solon's Analytics Innovation and Knowledge Management teams developed to harness the firm’s internal knowledge and make it readily accessible to users. Through the organization’s instant messaging application, the tool effectively responds to user queries by searching and synthesizing knowledge management documents and hard-to-reach data. 

ASKMe responds to queries seeking information on industry trends, firm credentials, and subject matter experts across the firm. It also serves as a case study for organizations implementing gen AI solutions with effective governance measures in place to comply with a complex legal landscape. 

A Look Inside: Deployment, implementation, & governance of gen AI 

Deployment

From major cloud providers to up-and-coming startups, companies are introducing new generative AI tools to the market, tailored to almost any imaginable use case. Once a use case has been defined, the next critical decision for an organization is the deployment approach that best aligns with the specific use, organizational structure, resources available, and existing IT tools. Organizations developing gen AI tools internally often face the decision of whether to use a unified platform approach from a major cloud provider or take a hybrid, open-source approach. Each option offers pros and cons. 

• Unified platform deployment from a cloud provider: This method is optimal for organizations with limited developer resources. It offers user-friendly interfaces and requires less coding. Unified platforms offer more streamlined integration with existing IT tools and applications, reducing development resources for building integrations. However, this method can be more expensive and can limit customizations within the tool. 
• Hybrid open-source deployment architecture: A hybrid architecture leverages a combination of open-source and custom components. It commonly utilizes a foundation LLM and fine-tunes or retrains it with proprietary and use case-specific data. This architecture is suited for sensitive use cases that require on-premises operations. Open-source models can bring generative AI to siloed or offline environments. However, this deployment method requires skilled developers to customize and maintain the tool, ultimately requiring more resources. 

ASKMe required a strong generalist model capable of synthesizing large bodies of text. After analyzing the integration with existing firm applications, secure availability of all models, and cost, we opted for a unified platform provider. This was validated through further performance and cost tests between various open- and closed-source models.   

Implementation 

ASKMe leverages the latest in retrieval augmented generation (RAG, also known as in-context learning) to design a robust backend MLOps pipeline that provides informed responses to user queries. RAG takes a user query, searches a vectorized table of internal documents for matching records, and appends it to the user’s original query, with instructions to the large language model (LLM) on how to respond before sending it back to the user. These instructions infuse organization-specific context (e.g. terminology, data sources, points of contact, and more) into the workflow.

RAG architecture has long been the primary method for generating context-aware, use case-specific responses from a generalist LLM. However, the performance of these RAG systems can be substantially enhanced to refine the retrieval process and ensure the most pertinent information is used for response generation. In the case of ASKMe, we developed custom functions specific to our use case that enabled the LLM to intelligently navigate the path for information from query to response.

To further strengthen performance, a multi-agent system was implemented in the backend, optimizing multiple LLMs for specific tasks through calibrated prompt engineering. Every query submitted to ASKMe passes through several LLMs before a consolidated response is formulated, ensuring the user receives a single, comprehensive response.

ASKMe leverages this RAG approach in conjunction with a suite of custom-developed tools, making it a truly intelligent agent capable of combining data in a way that would not have been feasible with traditional approaches. As a result, the tool generates responses from a secure database environment, responding to users on a wide range of topics within 15 seconds.

Governance 

Generative AI governance—or the legal, regulatory, and organizational policies underpinning the development, usage, and application of LLMs—is crucial for responsible model design and deployment. Our governance framework, inclusive of rigorous oversight processes, audit trails, and activity logs, ensures that ASKMe behaves within ethical, legal, and compliant boundaries set by our organization and the jurisdictions in which we operate.

Working with legal experts, our team developed a framework that evaluates risk levels and outlines risk mitigation methods, which are critical to understanding before deploying a gen AI solution. Protecting client data is a top priority. ASKMe does not have access to or incorporate proprietary client data in any way. Its data repository is isolated from our operational systems to provide further protection against unauthorized access. We also incorporate a series of stress tests to identify and reduce potential implicit biases and vulnerabilities, like prompt hacking and inappropriate or out-of-scope responses, to strengthen the efficacy of the tool.

As organizations develop and deploy gen AI solutions, cybersecurity risks will increase. In our recent survey on generative AI adoption, 72% of survey respondents noted security as their primary concern when implementing gen AI in the workplace, which is a 40% increase from 2023. Cybersecurity best practices such as end-to-end encryption, access controls, and network security protocols are implemented to safeguard information.

ASKMe’s impact 

Since the launch of ASKMe in early 2024, the tool has yielded strong operational and economic results for the firm. It has provided seamless access to a vast knowledge base, streamlined onboarding processes for consultants, and enhanced staff efficiency and productivity. ASKMe has consistently responded to hundreds of queries each week, reducing search query time by 75%. In addition, when compared to alternative AI solutions, the development and implementation of ASKMe yielded 85% cost savings for the firm.  

The future of gen AI in TMT 

Strategic investment in generative AI tools can transform business operations and offer a competitive edge in the fast-paced TMT industry. However, successfully deploying these solutions requires a structured approach. Organizations should align the business case with the overall organizational strategy, determine the technical capabilities and personnel necessary to build and implement these tools and define clear governance and security measures.

Altman Solon’s Analytics Innovation practice has developed a structured deployment framework to help clients navigate generative AI, from identifying impactful use cases to solution implementation.  

Submit the form to receive our structured framework for generative AI deployment and governance.

Thank you to Arjun Gheewala and Kimberly Padilla for their contributions to ASKMe and this report.