Meaning Extraction for Business Strategy
In my previous post, I gave an example of how meaning extraction works in a life sciences research setting. I am pleased to report to you all that the essence of that blog post was expanded into an article that is going to run in Future Pharmaceuticals magazine soon. I will post a link to it when it runs if that is practical.
But since most people work for companies that are not pharmaceuticals or performing life sciences research, I thought I should provide another example in a different context. So let’s say we want to analyze the strategy of a company like Cisco in the arena of voice over Internet protocol (VOIP) and that we have a content repository of a few hundred thousand market research reports from scores of authoritative analyst firms like Gartner, Forrester, and IDC. (Northern Light clients actually have such a database available to them.)
Hope for Amazing Good Luck
If we perform our search ‘Cisco and VOIP’ on a traditional search engine we will get back a search result of thousands of reports. Most search engines then wash their hands of the situation, metaphorically dumping the pile of documents on the user’s desk and saying, “see ya” as the search engine bolts out the door. The user is left to sort through the pile, find some that might be interesting, and then start reading them. The search result itself provides a little guidance in this process, though precious little. For example, the search result will be sorted by some secret formula that will attempt to put more relevant documents nearer to the top of the list. And there will be a little summary provided of each document, perhaps a short paragraph of text, that the user can review. Acting on these scant hints, the user selects a few reports and starts reading. Some are helpful, some are not, and the user preserveres for as long as he or she has time for or for as long as he or she can tolerate this hit or miss process.
Because one cannot know what one did not find, there is no objective way for a user to assess whether the documents that he or she actually took the time to read comprehensively represent the body of knowledge contained in the thousands of returned documents on the search result. What the user is actually doing is desperately wishing that the a documents he or she selects to read contain all the important findings, analysis, and perspective. As even the most determined researcher will read only a very small percentage of the reports on any give search result, Northern Light calls this research strategy: hope for amazing good luck.
The use of hope for amazing good luck as a strategy for dealing with search results is not, of course, the fault of the user in question. It is the fault of a search engine industry that believes lists of documents are the right response to a user whose business purpose for doing the search is to gain intellectual command of a body of knowledge, to answer a profound question, to explore the meaning of events and trends, or, in our example, to analyze the business strategy of a leading global company that can drive the evolution of a new technology and its market.
So how might it work better? Let’s start by review how meaning extraction works:
1. Extract references to important terms and concepts, particular concepts that imply meaning for the business purpose of the search.
2. Apply proximity intelligence to determine which terms and concepts are often in proximity to one another.
3. Identify patterns of proximity-related concepts that imply meaning to a knowledgeable practitioner, and scan all the documents responsive to a query and identify those patterns in all the documents to the user. We like to call these patterns “scenarios” since we cannot really tell if they are significant, only that they are present.
Northern Light MI Analyst, our meaning extraction application, can find all references in search results to terms and concepts such as IT technologies, business issues, product marketing initiatives, corporate strategies, and company names. For example, MI Analyst can find references to product marketing and strategy concepts like price cut, market share gain, new products, clinical trials, acquisition strategy, financial crisis, energy costs, or government bailout.
MI Analyst exposes these terms and concepts to the user at both the document level and the search results level. At the document level, MI Analyst assists a user in gaining an at-a-glance understanding of what is in the document so the user can make a more informed decision about which reports to download and read This helps a user find those reports and articles that are most likely to be of most value.
At the level of search results, MI Analyst assists a user in determining what overall concepts are found in all the documents that are responsive to the query. This helps the user discover knowledge (e.g., what strategy is Cisco following in the VOIP market), as well as drill down into a subset of the search results that will be most helpful to the user.
Northern Light maintains an extensive taxonomy of terms and concepts (with tens of thousands of entries) to facilitate MI Analyst text analytics. We refer to these taxonomies as “meaning taxonomies” because they are designed to organize concepts that when found in proximity to one another and in proximity to company names, imply meaning to the users of the service.
Proximity Analysis on Terms, Concepts, and Words Used In Queries
At indexing time, MI Analyst stores the location within each document in the repository of all the words in the document and all the analytical terms in our meaning taxonomies. This allows the search engine to then perform analysis using proximity as an indicator of relationship. So for example, a searcher could find research reports that have ‘VOIP’ within 20 words of the company name ‘Cisco.’ Proximity analysis permits the user to force very tight associations between concepts so that the resulting document set is highly relevant and more likely to produce an in-depth understanding of the topic.
Automatic Identification of Relationships
MI Analyst trolls all the documents on each search result and automatically identifies relationships between concepts and company names, which we call scenarios, flagging those that it finds for the user to review. These scenarios operate on business issues, corporate strategy concepts, and technologies.
For example, a search using MI Analyst on market research reports discussing the VOIP market produces these actual search results:
Cisco is using a corporate strategy of Acquisitions
Cisco is using a corporate strategy of Strategic Partnerships
Cisco is using a product marketing strategy of Market Segmentation
Cisco is using a product marketing strategy of Target Market
Cisco is using a product marketing strategy of Professional Services
Cisco is using a product marketing strategy of Service and Support
I am willing to bet you a contribution to your favorite charity that you have never, ever, seen search results like the ones above. Cisco’s strategy in the VOIP market jumps right off the page of MI Analyst search results. Specifically, Cisco is targeting specific market segments in the VOIP market and using a combination of high levels of professional services and support and partnerships/acquisitions to penetrate the market. Each of the search results listed above is linked to a list of reports that discuss that strategy concept, sorted by the number of times the strategy concept is in the report so users can rapidly drill into the documents that best elaborate on Cisco’s strategy.
And by the way, when a user is presented with a group of documents that are conceptually related to his or her research interest with a meaningful indication of the concepts the documents contain, the user persists and actually reads more reports than when the user is forced to use the hope for amazing good luck strategy.
Time To Insight
When the above capabilities are taken together, meaning extraction permits strategy analysts, market planners, product managers, and competitive intelligence professionals to understand markets, technologies, and competitors more thoroughly and more rapidly.
The key benefit: meaning extraction significantly reduces the time to insight.