Neural network predictions know when companies are being mentioned

Accurately gauging how the public views your company can be a difficult task, at best. But Vainu's is on a mission to "build the most comprehensive database of all the companies in the world." A crucial part of this mission is having the ability to determine when companies are being mentioned in natural language. Named Entity Recognition (NER) is a subset of natural language processing that focuses on understanding when an entity is mentioned in free text.

For example, a sentence like "Apple's logo is beautiful but why has the apple been bitten?" has the named entity (Apple) and the word "apple" in different contexts.

Many companies have names that are also words of their own and therefore searching with text pattern only is not a sufficient way to detect when companies are being mentioned in text.

Neural Network Predictions

Earlier, Vainu used to leverage Google's NER technology, but it soon became evident that instead of only recognizing linguistic patterns, a more advanced solution could be made using company data.

Vainu analyzes approximately 1.5 million news stories about companies on a daily basis; it houses data from about 120 million companies in its database. The raw data to create the training set was always there.

By creating effective tools - using huge amounts of human workers and carefully using cross-validation - in which the same piece of information was evaluated by different workers, the VainuLab team converted the raw data into a massive, structured training set.

"Like with most of the machine learning tasks in the world, the largest task for us has been generating the training set that meets our quality standards and creating the technology to build it," said Tuomas Rasila, CTO and co-founder of Vainu.

What is the value of this technology?

"Beyond the original use case of collecting vast amounts of publicly available information about companies of the world, this technology could potentially be used for a number of tasks like searching companies in unorganized textual databases through corporate databases and emails," said Riko Nyberg, Head of A.I. VainuLabs.

The technology is currently being used as a part of Vainu's company intelligence platform and offered as a part of its technology stack for corporate customers, but it may possibly find its way to wider audiences in upcoming releases, according to Rasila.

As described, recognizing the correct companies is the most crucial element for Vainu's company-centered service. Thus, to serve the overall mission of Vainu, the one measure where Vainu's NER must crush all other services is the recognition of companies. And this is the case in all the languages in which Vainu processes unorganized textual data.

Currenty, the overall F1 score of Vainu NER scored a 94.20% percent accuracy when put to the test.