Skylight SDK Lets You Create Smart Glasses Apps

Skylight is a software platform from APX Labs that can be used to create smart glasses applications for scenarios where it is desirable for hands-free, in-context, real time information right where work is getting done.

It is a complete system managing the user experience on the smart glasses itself and it is the back end platform which powers the business solution. While most critical functions work out-of-the box, Skylight can be configured, customized and extended to unique needs using the Skylight SDK.

In particular, the platform focuses on ‘deskless workers’ of all types. These users need hands-free, real-time, in-context overlay information for their job, which can be delivered with smart glasses.

Potential users include nurses, doctors, field service and repair technicians, supply-chain and logistics staff, assembly line and QA personnel, security and support staff, as well as public sector workers. 

Skylight works with a variety of devices, including single optic heads-up displays as well as next generation binocular transparent lens glasses. 

The Skylight UI is optimized for see-through, transparent optic displays such as smart glasses. Since they are head-worn for work scenarios, they have to be highly responsive, as would be found in a first-person video game, and they need to smartly keep what is displayed to a minimum, to allow for focus on the task at hand.

On smart glasses, the user can access a hands-free main menu. This allows the user to make video calls, control the camera, review messages/tasks and perform other common functions. On the screen, the user will be able to see multiple info panels which show content of any type continuously updated by the server. Users can see work items, questions, and alerts in the message panel. After a few seconds these messages fade from view, but can be retrieved later from the main menu.

Rich content, like images, text, video and 3D models are delivered to the user in the media panel. Points are augmented reality objects that represent any person, place or thing displayed on the display in their real-world location. Skylight allows users to perform hands-free interactions with points which may open menu options to provide more information or trigger some action on the server, in another application or on a remote device.

Skylight is made of three components. The Skylight client application, the Skylight Server, and Dashboards.

The Skylight client application runs on the smart glasses, is self-contained, and functional out of the box. No additional client development is required to implement a smart glasses solution. The Skylight server is a required back-end for all Skylight clients. It is made up of a set of components that speak to each other through an internal Skylight API. The Skylight Dashboards are web applications that run on the server.

Customizing the system involves creating Skylight add-ins. An add-in is a self-contained component that runs as if it were a native part of Skylight Server. It manages the communication with external systems, provides content and commands to the client, and optionally hosts custom business logic. Skylight enables add-ins built in a variety of programming and scripting languages, and Skylight can interface with external systems through most standard methods and protocols.

See What I See. This is Skylight’s ability to make multimedia calls between users. These calls can be between two people with glasses, or between someone with glasses and a remote supervisor at their PC or tablet. Unlike normal video chat, they provide a first-person video feed of what they are actually seeing as they do their work hands-free. Customers will find that this option enables the unique ability to provide real-time, remote expert help.

A new way to manage messages, questions, tasks and alerts. The system sends and displays messages or questions sent by other users, or alerts and tasks programmatically sent from external applications. These may be one-way messages, or they could request some response back from the user. Unlike phones, PCs and tablets – this information gets to the user in real-time, right in their field of view – enabling the ability to better manage, change and expedite a user’s day to day tasks.

Find and track objects and people. Skylight knows where every user is, as well as the location of important waypoints, objects, markers and devices. Accordingly, it allows the ability to efficiently find and track objects and people and provide navigation advice, optimized routing and visual cues that can help the user do their work faster and safer.

Hands free app access and device control for any networked device Many Skylight scenarios are designed to keep the user’s hands free so they can accomplish their critical work tasks. Using head-tracking, voice and other methods, Skylight allows users to visualize and interact with augmented reality objects that connect to real-world devices, other users and applications.

A real-time system for continuous monitoring. This allows users to see content in their heads-up display provided by any external system, sensor or device. This display could be any combination of text, numbers, icons, images and widgets of various types.

Process data. Skylight takes advantage of the on-board camera to run computer vision algorithms which have been optimized for wearable glasses. This enables Skylight to capture, decode and recognize faces, bar codes, QR codes and markers.

Build “Add-ins” using the Skylight SDK and APIs. While there is considerable out-of-the-box functionality provided by Skylight, in order to deliver a complete solution a developer can build “add-ins” using the Skylight SDK and APIs. These add-ins broker information between legacy systems, control the content and commands sent to the glasses, and execute custom business logic.

Web dashboards. Skylight has a collection of out-of-box web dashboards that can be run on PCs or tablets. They are used to manage system resources and user accounts. In addition, there are customizable Skylight web apps for supervisors and remote users to track real-time status and location of users, send messages or tasks, and to send audio/video calls to the glasses.