Developers and API standards

For nearly five years, Andy Beier, director of engineering at Domo, has worked to build more than 500 data connectors for the company’s business optimization platform. However, as Andy and his team strove to complete that monumental task, they continually ran into roadblocks because of the lack of standards in API development.
 
Frustration led to action and the Domo team set out on a cross-country “API Standards Tour,” bringing tech thought leaders together to discuss the industry’s API standards issues and develop solutions to improve the situation.
 
We recently caught up with Andy to discuss exactly what difficulties developers face today, the lessons learned from their tour and how we can move toward better APIs across the board.

ADM: What is the current state of the API?

Beier: APIs are everywhere. They are used in apps, wearable technology, smart home devices, cars and more. But the enterprise interacts with APIs in a slew of ways, in order to pull data from the specific technology and programs it uses. The current issue is that there isn’t a development standard for APIs, which leads to a variety of outputs. Imagine if you had to build your own browser for each website you wanted to visit and on some sites you kept losing connection while others loaded at a turtle’s pace. It would make surfing the internet a total nightmare. The way APIs are constructed now, is essentially that. Many are developed well but because the development is not standardized, enterprises must take on additional hassles to make use of APIs. Creators have been far too focused on the tech their APIs are built on and not on the content being delivered through their API. The tech behind HDTV, for example, is not enough to drive its adoption. You also need lots of engaging content. The technical delivery is not why the consumer flocks to it; it is the content.

ADM: What lead to the realization that API standards are lacking in the enterprise?

Beier: At Domo, we’ve created more than 500 connectors for various software to help bring data to the fingertips of our customers in real-time. As I worked on building these connectors I would find that certain APIs were a dream to build on while others sent me into a depression. Getting the data out that people really cared about stood in direct contradiction to how the API was designed. It was a regular occurrence to design pagination strategies in our connectors to work around the fundamental lack of one in an API or to build an entire connector to find out later that data couldn’t be tied out because the API was lacking one small element. This sparked the desire to start an industry-wide conversation around API standards. We launched the “Standards for API Tour,” and traveled to some of the biggest cities in the nation to meet with tech leaders and find out what problems they were facing and how our community could better standardize API development.

ADM: What are the biggest pain points for thought leaders that you meet with?

Beier: We met with a broad representation of groups including channel partners, ISVs and many major enterprise companies during the tour and a number of concerns consistently arose, including:

- An inability to get actionable data in a reasonable amount of time
- Connections continually break and must be repaired
- Data is held in silos rather than being connected across an organization, impairing decision making
- Even today not all products from ISVs have APIs and it is a major barrier to access data.

ADM: If this lack of API standards isn’t addressed, how do you see it impacting businesses long term?

Beier: Real-time data access has become a commanding need for businesses of all sizes. For the internet to work for business, API quality needs to be addressed so that data flow is not hindered. Without access to timely data, everyone from the CEO to the front line worker is left with old information or guesswork to base decisions on. Companies who work in this environment cannot be agile and flexible or make intelligent decisions in marketing, finance, operations or any department where data is required.

ADM: What specific standards need to be put in place?

Beier: Aside from improving connections, there are two major issues that need to be addressed:

- Following the latest OAuth specifications. This by far has proved to be the easiest, most secure and reliable way for people to share their data with another system through APIs. Most of the big companies have done this but it needs to be done ubiquitously.

- Using reporting APIs vs. base data APIs. Base data APIs allow you to pull data but when you want actionable metrics or measures out of it you have to recreate the algorithms on your end, in Excel or elsewhere, to get the numbers you want. Most scenarios call for reporting APIs, which allow you to ask questions and get the number immediately. Imagine that you want to know all of your sales from the last year. A base data API would pull all of the data from that year but then you would manually add the numbers to come up with the metric you’re interested in. A reporting API, on the other hand, would automatically spit out the year’s total.
 
APIs fundamentally need to be structured in a way where you can go to the system, ask a question and then get a metric in a reporting fashion. This will reduce calls on APIs and make a better experience for everyone.

ADM: How would data flow be impacted by API standards?

Beier: APIs are all about providing access to information. The proposed standards would create better connections, allowing for consistent access to data. Reporting APIs would reduce the data analysis load on IT and business intelligence teams resulting in improved speed of data flow to decision makers.

ADM: Who in the industry would you give as an example of doing APIs the right way?

Beier: Google with Google Analytics and YouTube analytics has created an API where you can precisely pull the data you want at the correct grain eliminating wasteful transfer for data that will never get used.

Salesforce is another good example. With SOQL you can query for the exact data you want and their design reduces the need to have an endpoint for every object in their system.

ADM: How would standards benefit API developers?

Beier: Those who successfully execute an API strategy make a better experience, not only for their customers but also for themselves in terms of stability, availability and number of errors. We’ve seen those who have a good API strategy quickly pull away from the pack. Many companies focus on making a great product, but if their core focus doesn’t also include developing a great API experience around the product’s vision (to allow for an ecosystem of apps/services to be built on top of it), it leaves room for competitors with an API-first strategy to come in and conquer. For example, Salesforce could have just made a good service, but by complementing their platform with quality APIs, swarms of people have been able to build their businesses around it and millions now connect to the platform to get information they need. Without quality APIs, customer experience suffers and massive business potential is lost.

ADM: Why have standards not been implemented already?

Beier: During the tour, we found a consensus that a lack of direction and advice on how to standardize APIs persists in the marketplace. Many individuals implement best practices but those ideas have not been adopted by the industry as a whole because there is no central authority bringing the community together and setting the bar for those standards. That void in guidance leaves individuals to their own devices and often results in subpar APIs.

ADM: What needs to happen to make API standards a reality?

Beier: While the conversation we’re having now ultimately aims for a universally-accepted solution, in the short term, thought leaders agree that continued visibility and education around API standards needs to take place. Developers and companies must truly understand the pain points caused by the lack of standardization.

Long term however, a formal governing body overseeing the standardization process may not be needed. Rather, establishing a rating/reference guide for 3rd party APIs would be a useful way to educate and inform companies of the expectations in the enterprise. Then, the bar would be set and companies could begin to demand a new standard.