Why search has become the backbone of mobile apps

ADM: Can you explain how Elasticsearch is used in mobile app development?

Gupta: It helps to start from why Elasticsearch was created in the first place. In 2004, Shay Banon, CTO and Co-Founder of Elastic, began working on a project that become Elasticsearch at a time when AWS didn’t exist, mobile apps were in their infancy, few had even heard the “Big Data” phrase, and search was designed to make money (lots of it) from keywords, not as a tool for developers. What started as a ‘seemingly’ simple problem -- to build a recipe application for his wife attending Cordon Bleu cooking school -- uncovered the many intricate details and challenges behind modern search. For example, how do you collect data from multiple sources? How do you combine both unstructured and structured data? How do you retrieve data in real-time across hundreds to thousands to millions of variables? How do you store and index the results and use the information to constantly refine those results? Shay saw the need to build a next-gen search engine with all the features we expect today—distributed computing, hybrid cloud support, ease of adoption, scalable, and designed with standard APIs on REST/JSON.

Quickly, because of the virility of open source, Shay learned that Elasticsearch could be used for more than “search”. After being downloaded more than 75 million times since 2012, Elasticsearch has become a de facto element in almost any type of application across multiple use cases for mission critical security systems, logging platforms, analytics, and more.

As the company behind the Elasticsearch project, Elastic today provides a set of open source products called the Elastic Stack -- Elasticsearch, Kibana, Beats, and Logstash -- and commercial extensions called X-Pack for security, monitoring, alerting, reporting, and graphing. The Elastic Stack plays a key role in many popular mobile apps and sites we interact with on a daily basis from Dell, eBay, eTrade, Goldman Sachs, Groupon, Guardian, HotelTonight, Mozilla, MSN.com, The New York Times, Spotify, Uber, Verizon, Yelp, Wikipedia, and much more.

ADM: What are some of its common use cases for mobile app developers?

Gupta: Developers use Elasticsearch when they need to integrate real-time data, search, and analytics into their mobile apps.

A common mobile use case is to utilize Elasticsearch in mobile apps so they can react in real-time to user actions, create a more personalized user experience and lead to greater engagement and more effective monetization. Beyond search, another use case is using the Elastic Stack for logging and analytics, using Beats and Logstash to ship data into Elasticsearch and Kibana to visualize and analyze log data in real-time.

ADM: What are some examples of mobile apps that utilize Elasticsearch?

Gupta: Yelp's search and recommendation engine is powered by Elasticsearch to help 23 million monthly mobile app and 69 million monthly mobile web users find just what they want.

The New York Times put all 15 million of its articles published over the last 160 years into Elasticsearch, allowing readers to quickly access relevant information from any device and to see recommendations for other related content.

BlaBlaCar uses Elasticsearch to help 20 million members find ride shares in less than 200ms with its mobile app. Elasticsearch matches drivers and passengers based on any dimension (car type, amenities, favorite drivers, number of passengers, etc.) leading to more rides and revenues. 

ADM: What are the key benefits of using search in the context of mobile app development?

Gupta: Speed. Getting to market and or into production as fast as possible is one of the biggest benefits. Depending on the use case, mobile apps will help drive new revenues or cost savings, and more and more their rollouts have visibility at the highest levels.      

Innovation. Developers should be focused on developing showstopping or value-added features that drive new and expanded usage of mobile apps, not building and maintaining custom plugins and features

Scale to Millions, Billions of Documents. Consider the Wikipedia app. Although its ~5 million docs is not a large number relatively speaking, they cannot all be loaded on a mobile device. You have to rely on servers that expose a limited subset of the data more intelligently, and you have to give them the right subset quickly. Hierarchical navigation, like you use on your desktop, just doesn't work once you have millions or billions of items and thousands of levels of hierarchy. Search allows you to expand the accessible set of items quickly. Moreover, high-quality full-text search, good result-ranking (people often only look at the top 6-10 results of their search), and speedy results (so people don't sit around waiting) are especially important.

Scale to Millions of Users. Good mobile search is scalable both vertically and horizontally so lots of users can hit your search at the same time. It's fast not just with one active user searching, but with hundreds or thousands of concurrent searches.

ADM: How can adding search into the mobile app experience help drive monetization strategies, user adoption and personalization?

Gupta: Users today expect everything to be searchable instantly from a single search box. Great search used to be a differentiator. Now it’s essential. We think search has to do something more to drive real engagement that can unlock new revenue and user adoption opportunities.

Home Depot is doing some interesting things in this area. The company wants its search to understand the different relationships across more than a million products online to trigger cross-sell and up-sell opportunities. It provides suggestions of comparables and alternatives, automatically determining complementary products and limiting choice to reduce returns. It alerts the user of potential issues to avoid, e.g. “If you buy different brands of roofing shingles, you may void the 30-year warranty.” And it provides real-time updates of availability from your local store -- down to the aisle and bin -- or whether products are only available online.

ADM: Context has been one of the key areas of innovation with mobile apps. How can developers improve the context in their search results?  

Gupta: The combination of context-rich search, including personalization, geolocation, filters and more, is natural and a must-have for so many mobile app developers. The same goes for any other data that can drive the optimal consumer user experience or monetization strategies, e.g. recommendations, ads, etc.

For example, context is vital if you're someone like HotelTonight. We want to think of search terms as being signals that provide color to a user’s intentions, so we can bias the results, filter them or both. In terms of geolocation, you don't want to try to offer a hotel in Boston to a user who is in San Francisco, unless you know the person is explicitly looking for it. In terms of personalization, you don't want to push the most expensive items in your inventory to a user who is looking for "low end" and "cheap".

ADM: What kinds of innovations should mobile app developers look forward to seeing in search in the future?

Gupta:  We see three big trends taking place in this area. 

- First, one is the growing importance of “geo-aware” capability, especially in context of the search that's being performed. For instance, if I search for "suspension bridge" while standing next to the Golden Gate Bridge, my app could (or arguably should) use my geolocation context to influence results and bubble a result relating to the Golden Gate Bridge higher.  Some apps like Uber and Yelp are built so much around this concept that you could argue a significant portion of their business model is dependent on it.  

However, many apps haven't even started being geo-aware or are significantly underutilizing this information. We see geo-context-aware search being a really important element in mobile app development in the future.  That puts pressure on search developers to have a system that easily incorporates geo information in the search and result ranking efficiently -- without engaging a separate geo database that may be out of sync. This is something Elastic has been spending a lot of time on, including geo in results and enabling the developer to bias results by distance from a geo point.  We also have added the ability to even index complex shapes (think delivery zones, states, city borders, cellphone signal zones, etc) so you could answer mobile-important questions directly using a single search solution like "is the mobile user currently in my delivery zone?" or "is my trip going to leave me where I won't have cell coverage?" if you had this data available in your app.

- Second, mobile devices are taking on more and more sensors.  We used to think about mobile as laptops and cellphones. Now we have tablets and more exotic wearable devices which have mixed inputs, all of which contribute to the search context. There's a lot of research going on to try to get computers and search to understand multimedia inputs like cameras and microphones. More immediately, we have "exotic" inputs like GPS/speed, heart rate, and temperature light sensors that are tied to our devices either directly or through wireless syncing like bluetooth. All of that data can be taken into consideration with something that might be considered a "search" today, and that creates both opportunities and challenges. The advantages are potentially massive. For example, if my device knows that it's hot outside and that I'm sweating when I search for "coffee," maybe it should bias results toward nearby places known for their cold brew, iced coffees and air conditioning. The obvious challenge is making sure the right information is properly secured and available only to the people you want. As a result, security will play a greater role than has ever existed in even the most locked down of search systems in the past.  

- Third, mobile devices are taking on more and more form factors.  These new form factors are things like smartwatches and activity trackers with tiny screens or screens that are completely absent and running on minimal hardware.  The minimal hardware underscores what we said earlier: if you can't store all of Wikipedia on your smartphone, you're definitely not going to fit it on your smartwatch. Being able to offload that workload out of the device is really important.  Which, in turn, creates a need for good API design in the search layer so that developers can access the result sets in the language of their choosing in the way of their choosing.  Small or nonexistent screens require really good ranking algorithms. The expected result needs to be first or second so the user can take action right away.