How Pebble Converted 135,070 Customized Watchfaces For Pebble OS v2.0

But with the release of the Pebble OS v2.0 software update, things changed for users who had originally created custom watchfaces. Pebble made the difficult decision to break apps that were built for v1.0––primarily for the sake of future expandability of the Pebble platform. This meant that all the 150,000+ watch faces that Pebble users had carefully crafted would break after updating their Pebbles to v2.0. 

To avoid such a situation, Pebble decided to translate all the v1.0-based watchfaces from watchface-generator.de to v2.0-compatible equivalents. You might have noticed that after the installation of v2.0, your creations from watchface-generator.de were automagically converted and still worked on Pebble OS v2.0. How did that happen?

The process of converting all these different watchfaces was technically difficult and challenging. As it happened, neither the source code nor the custom configuration for each watchface had been saved. Thankfully, Paul Rode (Mr. Watchface-Generator.de) helped the Pebble engineering team with the conversion project, providing much needed access to the generated watch faces and keen insight into how the generator works and what the possible variations could be.

In the remainder of this article, we’ll discuss the technical details of how we took on this task and successfully converted 90% of the watch faces to be compatible with Pebble OS v2.0.

 

The watchface-generator.de site makes it easy to create a custom watch face by providing a step-by-step UI that lets you customize different aspects of the watch face.

For users, there are a number of parameters you can configure, including:

Behind the scenes, once you hit the Generate button, the site produces a Pebble watch app project, complete with applicable C code, background image and fonts. 

 

The C code for each watch face is generated based on a template in which value placeholders are substituted with the actual values that the user provided. Code for unused features is left out entirely. The generated project is compiled into Pebble’s app file format. The app only contains the compiled machine code, images and fonts. The resulting app is passed back to the user’s browser and also stored on the server for later retrieval.

The conversion project would have been trivial if the configuration for each watchface had been saved. However, this was not the case. Neatly cleaning up after itself, the watchface generator discarded the configuration that users had entered when creating a custom watchface: only the final products were saved. So, all we had to work with were 150,000 different watchface binaries. From those binaries, we needed to derive the configuration in order to generate a v2.0 compatible version. That was the challenge. 

Initially, we tried deriving the configuration from a binary just by looking at it without actually executing or running it. As a first attempt, we took a number of watch faces and ran them through the disassembler, which translates the raw binary machine code into a form that it is slightly easier to read for humans. 

Looking at the disassembly of a watchface, it's “fairly easy” to find where the watchface calls out to Pebble OS v1.0. This allows us to figure out which Pebble APIs are used and where they are called. Knowing that an API is used by a watchface can reveal that a certain feature of the generator was enabled. For example, the generator’s template uses the gpath_rotate_to API exclusively to rotate and draw analog hands. So if that API is used, the creator of the watchface had enabled the “analog hands” feature. For the hard-core reader, an annotated piece of disassembly of one of the watch faces is provided in Appendix 1, which shows how the API calls work and how they can be found inside a watchface binary.

It turned out that not all features can be detected that way, however. And even if that was the case, it would merely detect a feature’s presence. In order to reconstruct the original template configuration, we also needed to know with what arguments the APIs are called and in what order the calls occur. When passing arguments to a function, the arguments are put into particular registers. Often, multiple instructions need to be executed to put the arguments in the right registers. This is where static analysis stopped being practical, because you basically need to run (pieces of) the code to know which arguments are passed.

To get the answers we needed, we decided to let each watchface run inside a controlled environment, similar to a lab experiment. In other words, we wanted to use a controlled environment, stimulate it in certain ways and observe its behavior. In so doing, its original configuration could be derived from the way the app behaves in response to the stimuli. 

For example: the analog clock hands are rotated using the gpath_rotate_to(GPath *path, int32_t angle). This API takes two arguments: path, which is the polygon shape to rotate, and angle, which is the requested rotation angle. The analog clock hands are drawn using this API. The shape of each hand is represented by a GPath data structure. GPath is basically a sequence of points that make up the shape of the clock hand.

To find which GPath represents which hand, we set the time, for example, to 12:15 AM. Based on the rotation angles with which the watchapp calls the rotation API, we could derive which GPath represents the hour and which the minute hand. Once we figured out which GPath is which hand, we could inspect them and get their shapes, deriving what the creator of the watchface had entered when configuring their watchface.

Internally at Pebble, we had been working on a Pebble emulator, based on the QEMU project. (QEMU, short for Quick EMUlator, is a generic, open source machine emulator and virtualizer.) The Pebble emulator in development enabled us to create that controlled environment that we needed for the task at hand. The emulator, although not quite ready for prime time, could nonetheless run tweaked versions of Pebble OS v2.0 and v1.x, and run watch apps inside it.

This is what the emulator looks like.

The emulator, which is fast and easy to use, also enabled us to take screenshots. For the final watchface conversion process, this proved to be useful: we took “before” and “after” screenshots to verify the correctness of each conversion. If they were identical, the conversion was deemed successful.

One of the most useful features of debuggers are breakpoints, which let you “break” (stop) the code execution at a specific point. When you set a breakpoint, you can specify the location of that point by entering the name of the function (API), the line of code or memory address. When you hit that breakpoint, the execution of the program is stopped and you can then inspect the state of the application, i.e., variables, call stack, and so on. 

Using breakpoints enabled us to derive the configuration of all the watchfaces. We set breakpoints within the implementations of the watchface APIs in Pebble OS and that enabled us to see which APIs each app was using and with what arguments. For example, to get the shape of the hands, we set a breakpoint inside the code that implemented the gpath_rotate_to API. Once we hit that breakpoint, we could ask the debugger to return the points that make up the shape. The debugger was able to do this for the Pebble OS APIs because we had the source code and debugging information to work with.

GDB, the GNU Project debugger that we used at Pebble for the conversion process, can load Python scripts to automate it. The scripts expose bindings to various GDB functionality with its gdb module. Using Python scripts inside GDB, we could do all kinds of cool things, like creating breakpoints and running scripts when the breakpoints get hit to inspect the state and extract the information that we were looking for.

To cover tasks for which the GDB authors didn’t create Python wrapper classes, it also includes a eval()-like function called gdb.execute that lets you execute arbitrary GDB commands (whatever you would normally type into the GDB command prompt) and pass back the result as a string.

Using the Pebble emulator, the GDB debugger and Python scripts, we were able to automate almost the entire conversion process. For each of the 150,000+ watch faces, we wrote conversion scripts that basically followed this sequence of steps:

1. Launch Pebble Emulator with Pebble OS v1.x and the watchface to convert.

2. Launch GDB, attach it to the emulator.

3. Load another Python script into GDB that performs the following tasks:

a. Launches the watchface.

b. Pokes at it by sending date/time and render events, etc.

c. Places breakpoints to capture information and reconstruct the original configuration.

4. Grab a screenshot of the old, v1.x watchface.

5. Based on the reconstructed configuration model, generate the v2.0 compatible equivalent.

6. Launch the Pebble QEMU emulator with Pebble v2.0 and converted watchface.

7. Compare the two screenshots to verify that the conversion was successful.

To illustrate the workings of the DB-Python script at the core of all this, here is a snippet of the code that finds the background color and the pointer to the windo...

The code is slightly modified from the original and annotated with extra comments for the reader's benefit:....

def window_set_background_color(breakpoint):

    # The C function signature of the API is:

    # void window_set_background_color(Window *window, GColor background_color);

    # So, at this breakpoint, the variables named "background_color"

    # and "window" should be in scope. Grab them for later processing:

    background_color = gdb.parse_and_eval("background_color")

    model.background_color = str(background_color) # Store in "model" for later processing

    window = gdb.parse_and_eval("window")

    model.window_addr = Address(window.referenced_value())

    breakpoint.delete()  # Expected to be called once only

# Set breakpoint on symbol "window_set_background_color" and call

# Python callable with the same name, when the breakpoint is hit:

ActionBreakpoint (window_set_background_color)

...

The ActionBreakpoint class is a simple convenience wrapper around gdb.Breakpoint, which implements the `def handle_break(self)` method called when the breakpoint is hit. For a look at the source and other convenience classes that we wrote, see this gist.

Although the conversion scripts worked fairly well, they ended up being relatively slow. It took about 10 seconds for each conversion to complete. There were many avenues for optimization, but for our purposes this rate of conversion was acceptable. 

We deployed the scripts to 15 Amazon EC2 instances that crunched through all ~150,000 watchfaces in a couple of days. Around 10% of the watch faces did not convert, some because particular configurations were impossible to derive correctly, others because the source template appeared to have evolved over time.

As a result of our efforts, the total number of custom watchfaces converted reached 135,070. All were plugged into the v2.0 user migration process. 

So if you have a custom watchface from watchface-generator.de on your watch, the v2.0 Pebble mobile apps will automagically upgrade it to a new, v2.x compatible one. Many thousands of Pebblers have enjoyed this seamless migration of their custom watch faces :)

Based on the success of our conversion process, you may wonder if any arbitrary watchface could be easily converted—albeit with some tweaking—from v1.0 to v2.0. That’s the question that came up on many Pebble developer forums. Ultimately, it’s not really possible in a simple, straightforward way. 

Some APIs were replaced in v2.0 while others were restructured in such ways that they do not “map” directly onto the old ones. Even though many parts of the watchface conversion process were automated, the v2.0 watchface template still needed to be hand-coded with changes and edits in the source. Each change undertaken by the Pebble team was designed to make the code future-proof and a well-behaved citizen on the Pebble platform. 

A big thanks goes to Paul Rode, the creator of watchface-generator.de, who has been instrumental in helping us throughout the conversion process. Paul provided us with the necessary insight into how the site generates a watchface based on the user's configuration and gave us access to the original .pbws. He also added links from the watch face pages on his website to each of the converted watch faces.

The Watchface Generator is now completely up to date for v2.0.0 and supports Pebble Firmware 2.0 only. 

 

Paul notes that the converted watch faces can’t be accessed without having them installed on users’ Pebbles or knowing the URL of the original download-page. Pebble users should know that their personal information on the site always stays personal.

Appendix A: The Inner Workings of the API Shims

This annotated piece of disassembly, which deals with a single watchface, illustrates how the API calls work and how they can be found inside a watchface binary. The code shows how Pebble v1.0 apps make API calls at the lowest level.

     // API Call Shims

     // --------------

     // The Pebble SDK ships with a small libary that contains "shims". These

     // are little piece of glue code between the app and the Pebble OS.

     // This layer of indirection is added so that applications do not need to

     // know the exact addresses where the implementations of the APIs in the

     // are (they change from release to release).

     // A jump table is provided by the app at runtime. Each API has a fixed

     // that does not change from SDK release to release.

     // Based on that index, we can find the address of its implementation in

     // that jump table. The table gets provided by the firmware just before

     // an app starts to run.

     ...

     // Piece of v1.0 watchapp code calling into an API via shim (see below):

 

     // Part of the argument set-up:

     5fc: 4c08           ldr  r4, [pc, #32]  ; (0x620)

     5fe: 447c           add  r4, pc

     600: 4b08           ldr  r3, [pc, #32]  ; (0x624)

     602: 58e3           ldr  r3, [r4, r3]

     604: 4618           mov  r0, r3

     // Jump to the shim at 0x738

     606: f000 f897      bl   0x738

     60a: 4b07           ldr  r3, [pc, #28]  ; (0x628)

     60c: 58e3           ldr  r3, [r4, r3]

     60e: 681b           ldr  r3, [r3, #0]

     610: 4618           mov  r0, r3

     ...

     // The API call shims that are linked into v1.0 apps all look like this:

   

     // Each shim loads the jump table index into r1 and then branches to a

     // simple, shared piece of code (not shown) that restores the stack, loads

     // the implementation address from the table and finally jumps to the API

     // implementation.

     // Shim for bmp_deinit_container (index 18):

     738:   b40f        push    {r0, r1, r2, r3}

     73a:   f8df 17fc   ldr.w   r1, [pc, #2044] ; 0xf38

     73e:   f000 bbd3   b.w 0xee8

     742:   bf00        nop

     // Shim for cos_lookup (index 19):

     744:   b40f        push    {r0, r1, r2, r3}

     746:   f8df 17f4   ldr.w   r1, [pc, #2036] ; 0xf3c

     74a:   f000 bbcd   b.w 0xee8

     74e:   bf00        nop

     // Shim for fonts_get_system_font (index 20):

     750:   b40f        push    {r0, r1, r2, r3}

     752:   f8df 17ec   ldr.w   r1, [pc, #2028] ; 0xf40

     756:   f000 bbc7   b.w 0xee8

     75a:   bf00        nop

     …

     // Shared shim code, that uses the index in r1 to load the address from

     // the jump table into ip (intra-procedure-call scratch register),

     // restores the stack to its original state and finally jumps to ip.

     ee8: a303       add r3, pc, #12 ; (adr r3, 0xef8)

     eea: 6818       ldr r0, [r3, #0]

     eec: 4408       add r0, r1

     eee: 6802       ldr r2, [r0, #0]

     ef0: 4694       mov ip, r2

     ef2: bc0f       pop {r0, r1, r2, r3}

     ef4: 4760       bx ip

     ...

     f38: 0048      // 0x00000048 / sizeof(void *) ==> jump table index 18

     f3a: 0000

     f3c: 004c      // 0x0000004c / sizeof(void *) ==> jump table index 19

     f3e: 0000

     f40: 0050      // 0x00000050 / sizeof(void *) ==> jump table index 20

     f42: 0000

Annotated pieces of watchface disassembly, explaining how v1.0 apps make API calls, at the lowest level.