Review: IBM’s Watson Can Learn a lot from Marchex Speech Analytics
Wednesday May 31, 2017. 05:54 PM , from The Apple Blog
Most any caller to a business has heard the phrase “Your call may be monitored for training or quality assurance purposes” uttered as a pre-recorded message. However, perhaps few callers consider what consent to the recording actually means. Many assume, correctly so, that this statement is just a legal requirement that keeps callers informed that others may listen to or record the call. But this phone call also can be considered rich data, available for analysis, reference, auditing or countless other uses. As data, those interactions have tremendous value to any organization, but only if there is a way to mine that data and derive intelligence from it. A conundrum that Marchex is at the forefront of resolving with a product aptly named “Speech Analytics”.
A Closer Look at Marchex Speech Analytics:
The name “Marchex Speech Analytics” is actually a misnomer, since the product does so much more than analyze speech, it actually brings previously unforeseen value to the data that surrounds speech. To better understand what Marchex is offering requires looking at why a product such as this came into existence.
Phone interactions between customers (or potential customers) and businesses have proven to be the life blood of business revenue. Whether it is a call to get a price, order a service, or schedule an appointment, there is an expectation that the call will be successful and result in some type of revenue for the business. However, it only takes a few bad interactions with customers to irreparably harm that business. Those bad interactions can be the results of agents misinforming potential customers, excessive hold times, lack of information, or many other situations that can result in at best a lost opportunity, or at worse, damage to a business’s reputation. Making matters worse, is that the majority of businesses have to rely on manual, costly, and inefficient practices to determine if lost opportunity is indeed a measurable occurrence.
Marchex solves that very problem by combining the power of AI, NLP (Natural Language Processing), speech analytics, and an extensive data lake together to create a system that can derive value from a business’s most critical communications technology, the phone.
Hands on with Marchex Speech Analytics:
It is best to think of Marchex Speech Analytics as a service, one that integrates with a business’s customer call center, and records all of the interactions that happen between agents and customers. Major use cases include national chains that schedule repairs or perform services, especially those where the first interaction with a customer starts with an inbound phone call. Examples of that include automotive repair, appliance services, home improvement services, and so forth.
While recording and organizing that data is an important capability, Marchex brings value to the equation by analyzing that data and presenting it to users with easy to understand dashboards. In other words, Marchex enables a browser to become the avenue of insight into call and communications effectiveness. While the technology that makes that possible is complex, that complexity is completely hidden from the end user. That means leveraging all the service has to offer does not require modifying infrastructure, adding processing power, reconfiguring storage, worrying about scale, hiring data analysts, or even implementing extensive training programs.
Installation and integration is handled by the Marchex team, with the end user receiving access to the analytics platform via a web browser based dashboard. All call data, as well as analytics processing takes place in Marchex’s own secure data center and is enabled by the Marchex Call Analytics (MCA) platform. Onboarding into the system is straightforward, where a hierarchy is setup on the system that breaks down a business’s operational matrix into the most logical elements for analytics. In other words, a master business account is setup, with satellite locations associated with it. That allows data analytics to be sliced and diced based on any criteria needed by the end user. To enhance the value of the metadata, and to improve the analytics provided by the product, there are some initial steps that should be taken.
The majority of businesses have internally defined how voice communications should take place between a customer and an agent. For example, agents are provided with scripts and/or training to educate them on how to interact with a customer. Key elements, such as greetings, introductions, and so forth are introduced to the agent so that they may optimize the interaction with the customer. Those scripts are ingested into the speech analytics system, so that businesses can measure agent performance and track how successful interactions are.
Other preparatory steps may include defining campaigns, where a particular call in number is associated with an advertising campaign. Also, regional information, such as operational hours, geographic locations, keywords, and many other elements can be included to fine tune the analytics to provide the most pertinent information with an interaction.
Behind the scenes, calls are recorded, metadata collected, and speech is analyzed to create graphical representations of cumulative interactions, with the ability to drill down further into specific time frames, individual calls, as well as campaigns. That is exactly where the various dashboards come into play.
Lost Opportunities Dashboard:
Marchex, rightfully so, associates calls with opportunities, which in turn can help businesses determine insights around revenue goals. A better interpretation of that capability comes in the form of understanding that a business may spend a certain amount of money on a campaign, and will associate that campaign with a given agent center, location, or even a dedicated phone number. By analyzing the calls associated with that campaign, a business can determine the return on investment.
For example, a million-dollar advertising campaign may deliver 10 million dollars in new business, however the ability to truly understand the effectiveness of the campaign has been lacking in the past, relying on analytics that’s akin to dead reckoning. With Marchex Speech Analytics, the overall impact of the campaign can be measured and need be, dissected to find out where additional revenue could have been generated.
The lost opportunities dashboard visualizes that information and assigns values to the critical elements associated for deeper understanding. Elements such as the number of calls, the length of calls, failure rates, call abandonment, and so forth are readily displayed on the dashboard, allowing users to drill down into additional data streams. Users can quickly ascertain the types of failed calls that occurred, in other words, did the customer hang up during hold, did no one answer the call, was voice mail involved, and so on. That information is aggregated and presented in such a way that users can quickly determine what is happening. What’s more, that data can be correlated with costs and then translated into the cost of lost opportunities. The dashboard is populated with numerous charts, which give a visual representation to the most critical elements. Users can customize the dashboard, as well as drill down into filtered data, or even individual locations, calls, or agents. In short, the Lost Opportunities dashboard offers exactly what is expected and then a whole lot more.
High Intent Dashboard:
High intent, as a term, is a little more vague than something like the term, lost opportunities. That begs the question, what exactly is high intent? From the Marchex point of view, high intent is a measurement that exemplifies what folks in marketing call a conversion. In other words, high intent is a direct measurement of a successful call.
The High Intent Dashboard provides a graphical representation of call success, where a completed call is identified as something that adds to business, ultimately generating revenue. The information provided proves to be very useful for measuring campaign performance, agent performance, and customer interactions. Marchex presents the information in such as fashion that users can quickly determine call volume, the metrics around when calls happen, and the overall conversion rate of those calls. That information can be directly correlated to revenue, which gives users a dollars and cents view of success.
Agent Script Tracking Dashboard:
Staying “on message” may be one of the most critical aspects that a representative of any company must master. Nowhere is this truer than in the high paced environment of customer to business call interactions. For many businesses, countless hours have gone into perfecting their customer facing messaging, and making sure that messaging is consistent has proven to be a major challenge.
With the Agent Script Tracking Dashboard, Marchex has automated the tedious process of validating messaging and visually demonstrates when and where agents stay on message, as opposed to going off on tangents that may be harmful for the business. For example, a business may have a general rule that every interaction starts with a proper introduction. The speech analytics processing of the Marchex platform can detect if that happens or not. Another example may come in the form of making sure prices are not discussed during an initial call. Here, Marchex is able to detect the words surrounding pricing information and report on it.
Ultimately, script tacking delivers aggregated data that can be filtered to give users insight on how well agents are performing and whether or not a call passes defined criteria. That criteria can be defined by the user and can be representative of keywords, phrases, scripts, and other signaling data, allowing complete customization of what is tracked.
Transcription Search Dashboard:
One of the most impressive features offered by Marchex is the ability to drill down into an individual call and assess the original context of the call. However, when millions of calls are involved, that chore becomes almost impossible. To lessen the burden, Marchex’s speech analytics transcribes and indexes every call, which in turn allows users to search for words or phrases. That proves to be a valuable tool for forensics purposes, as well as identifying certain terms that may relate to business. Users simply access the Transcription Search dashboard and search based upon a key word or phrase. The system, using fuzzy logic, returns a list of calls that match the search term.
Users are then able to drill down further into the calls and listen to the original audio, view the machine transcribed text, or further analyze the meta data associated with the call, such as caller ID, time frames, call length and so forth.
The power of this particular capability is worth noting. For example, a business could search to see if a competitor’s name is mentioned, or if a phrase associated with advertising, a news event, or some other activity is mentioned. Advanced filtering allows users to narrow the search and base results on specific regions, agents, campaigns, and so forth.
Marchex Speech Analytics Schools IBM’s Watson in ASR Comparison:
In a quest to establish dominance in the domain of consumer to business call analytics, Marchex bench marked its Speech Analytics capabilities against other ASR systems, one of which was IBM’s Watson.
As call processing is only one subset of Watson’s capabilities, it is difficult to do an apple to apples comparison for overall customer to business call processing. However, there is still an element that can be compared between Marchex and Watson. One that can be summarized as the accuracy of how voice is processed and ultimately transcribed, which proves to be the key capability of the Marchex solution and what Marchex Speech Analytics was designed for from the outset.
For companies looking to evaluate call data and analyze consumer to business phone calls, Marchex Speech Analytics is the product of choice. Processing accuracy is a viable benchmark and Marchex was able to outperform Watson from that perspective, especially where accuracy was concerned.
Both platforms were tested using the same data, which consisted of 7,000 hand transcribed utterance (or approximately 5 hours of speech) from real consumer to business phone calls that were previously unseen to both systems (i.e neither system trained with the data). The data was fed to both system and the results were measured using two metrics: Word Error Rate (WER) and Perceived WER (where Perceived WER removes any penalties for words with dual spellings like “All Right and Alright or Misses and Ms.)
The results show Marchex has a raw WER of 15.7% compared to 21.7% for IBM Watson. The perceived WER for Marchex was 14.6% compared to 19.4% for IBM Watson. Overall Marchex showed a relative improvement of around 30% over IBM Watson for each metric. For more information on ASR testing, please visit https://arxiv.org/abs/1705.09724
Marchex brings an incredible amount of analytical power to businesses that rely on speech based interactions between customers and company agents. The ability to derive revenue information, as well as better understand business opportunity comes as a welcome capability to businesses that are looking for growth. In the past, businesses could only derive that type of information using manual, tedious processes, that proved to be expensive and unreliable. Marchex effectively turns voice data into a currency that businesses can spend to innovate and grow. This growing company is setting the standard in speech analytics with their purpose-built technology that requires far less customization than other out-of-the-box solutions in their vertical.
May, Sun 20 - 14:03 CEST