Portrait of senior woman sitting at table in office and reading few paper work. Business manager working on some documents at office, considering AI compliance

Part 2: Generative AI spawns “AI compliance” assessments at large enterprises

An AI compliance questionnaire – Part 2

 

Last month, we introduced the topic of “AI compliance” – a new process gaining currency as large companies establish protocols to vet the use of generative AI in their business operations.  Legal, compliance, and technology leaders want to comprehend the potential risks and rewards of this increasingly popular, but still poorly understood, technology. (Read Part 1 here.)

When it comes to market and competitive intelligence research, the argument for using generative AI is compelling. Here are more of the questions that corporate “AI compliance” teams are asking (adding to the list we began in June), and how Northern Light is responding with regard to SinglePoint’s new generative AI-driven “question answering” capability.

1. What metrics and mechanisms will be used to evaluate the output of the generative AI application?

A logical starting point is subjective evaluation by a knowledgeable user. For example, a research question such as “What is Pfizer’s strategy in oncology?” is best evaluated by the user to determine the quality of the generated responses.

There are several features of SinglePoint’s question answering capability that support output evaluation.  These include:

  • The document and publisher for every answer are identified and the document and publisher for every insight in the executive summary are identified. This permits the user in each case to weigh the reliability and authority of the generated text.
  • The document every answer was taken from is hot linked, as is the document every insight in the executive summary was taken from, so that the user can click through to the document contributing the insight. This permits the user to “drill down”, vet the quality of the derived insight, and explore the most important insights in depth.

Northern Light has examined many examples and has validated the output by clicking through to the underlying documents to see if the documents’ insights were correctly represented in the answers and in the executive summary.  Users may repeat this process as many times as desired as both an initial and an ongoing validation process.

2. What are the potential risks of using the application?

Organizations are understandably concerned about the risk of AI generating biased or low-quality outputs.  Northern Light SinglePoint addresses this in several ways:

  • Only the vetted, high-quality content sources in a client’s SinglePoint knowledge management platform are used to generate the output. This avoids all the problems reported in the media about ChatGPT hallucinating answers from dodgy internet sources.
  • Every element of SinglePoint’s generative AI answers and executive summaries are footnoted with the contributing documents and publishers, providing users with a means to weigh the authority of each contribution.
  • Click-through to the contributing documents is provided so users can explore the topics in depth using the best source material for the research question at hand.

 

3. What is the expected value associated with solving the target business problem using generative AI?

Northern Light SinglePoint’s generative AI solution delivers value to market and competitive intelligence researchers in multiple ways:

  • It provides a better user experience for searching millions of documents.
  • It positions the user organization on the cutting edge of exploitation of state-of-the-art AI in the eyes of executives and employees, generating stronger executive support for the function and more employee loyalty. (One can imagine after-work conversations by employees visiting with friends that start with, “You should see what my company is doing with AI!”)
  • It greatly reduces staff time spent looking for answers to research questions, increasing efficiency.
  • It creates a more effective research process by providing an executive summary as the first step in the research process.
  • It enables more insightful research outcomes for competitive intelligence and market research sources.
  • It delivers better business research support for crucial business decisions throughout an enterprise, wherever SinglePoint is deployed (e.g., in product management, product marketing, product development, and commercial operations).

Back To Top