The accuracy of AI necessary to solve problems

We recently had a conversation with Aaron Edell, Director of Applied AI at Veritone who says data scientists are going about building AI/ML solutions wrong when they try to achieve 100% accuracy. They can spend hours building, testing, and rebuilding until the solution is 100 percent accurate until deploying. When data scientists spend this extra time perfecting a process, they waste time they could be spending delivering and improving results. Consider measuring open space in a restaurant. Businesses don't need to know the exact number of bodies in the room, but rather recognize how a full room differs from an empty one. With this in mind, data scientists only need to build solutions with 90 or even 80 percent accuracy for a similar outcome as 100% accuracy would provide – saving valuable time and money.

ADM: What are the common mistakes data scientists make when building AI solutions?

Edell: Data scientists are often tasked with or become focused on delivering a new algorithm with the highest degree of accuracy. They often are not exposed to the reason why they're building a model in the first place. Without understanding the problem, or given the bandwidth to explore the problem further, they'll just do what they do best which is to maximize metrics.

How can data scientists move through the process faster? The best use of data scientists within an organization is to embed them with cross-functional product teams who are tasked with solving problems. They need to be a part of the solution, not focused on a sole component. Once they are exposed to the holistic approach, they can be a lot more effective in delivering value.

ADM: Considering the accuracy of AI, what level of accuracy do AI solutions really need, and why don't they need 100% accuracy?

Edell: This depends on the problem you are trying to solve. If I told you to focus on building the most accurate people counter model for a specific CCTV camera feed, you might think that you need to reach 100% accuracy in order to succeed. So you waste time and resources achieving that goal. If I told you that we need to use the CCTV camera to determine if a restaurant is busy or not, you might realize that your people counter doesn't need to accurately tell the difference between 110 and 115 people - it just needs to report back that there are a lot of people (or not) in a given viewing area. That is a much easier problem to solve with machine learning that doesn't require 100% accuracy.

You don't need 100% accuracy to solve a real business problem and realize ROI. If I can make a person's job 80% more efficient, then I'm still adding value.

ADM: What should data scientists consider when building out AL/ML solutions?

Edell: The most important thing to think about is the problem you are trying to solve. If you've decided that you've got a model that solves the problem, the next step is how to deploy and scale that model, which can be the hardest part. Data scientists aren't necessarily developers or dev ops engineers. They need to be part of a team that has the expertise to work together to solve a problem.

ADM: What types of business solutions can AI solve? How can businesses recognize what problems AI can solve?

Edell: Great candidates for AI are problems that a human could do in theory, but would take too long to be useful. For example, a human could sort through millions upon millions of photos and crop them around a single person's face. But that would take so long that the process is not worth it. A human could also manually run statistical analysis on credit card transactions to look for anomalies, only to find out that there is a fraud way too late for it to matter.

Machine learning can speed up repetitive or mathematics problems so much that you can start to see massive returns on your time or money investment. It can solve problems that wouldn't have been possible before because of the inability for humans to scale, like automatically creating a profile pic for your social media profile by scanning through millions of photos and cropping the perfect picture of you. It can also load balance cell phone towers to make sure everyone's phone calls are perfectly clear. There really isn't a limit to the size of the problem it can help with, it is a force multiplier much like how factories scaled-up manufacturing. I'm reminded of how humans used to build cars by hand, only to discover one day that they could build hundreds of them with an assembly line. Certainly, technologies enabled that. They weren't means to an end, but they unlocked levels of scale that weren't possible before. This is how I think about machine learning. It is a technology that unlocks levels of scale that weren't possible before because they were traditionally too expensive or too reliant upon unscalable technology.

Aaron Edell Bio

Aaron Edell is a proven technology and business leader who has built products customers love, started and sold businesses, and has put together strong strategic partnerships with some of the largest tech companies in the world. Aaron was CEO and co-founder of AI-company, Machinebox, which was acquired by Veritone just 10 months after starting the business. He's won awards and accolades such as a technical Emmy, JMP Securities Hot 100 startups, Bloomberg's Top 50 startups, and aiShowBiz's best AI startup. Aaron is quoted by Microsoft on their AI landing page, cited or referenced in multiple books, and is a regular industry speaker.