When Cloud Based IoT Dies

Update November 7, 2016:

It looks like the Lono cloud service is back online. This was not a normal outage however as their domain completely disappeared from DNS. The bigger question of what they will be doing to ensure that the device works locally, even when the cloud is unavailable still deserves an answer (and I filed a support ticket this evening asking both about the outage & about plans for graceful degradation of service should the cloud component fail again).


A while ago now I backed a project on Kickstarter that was creating a more modern sprinkler controller. That actually wasn’t hard to imagine since the user interface of the one our home’s builder attached to wall consisted of a rotating switch, some buttons and an LCD display which could handle numbers & a few other preset things. Like something from the 1980s.

That project was Lono, and, like most Kickstarters, it delivered late & somewhat incomplete. But the hardware looked good, was dead simple to install & seemed to work. The software less so. Over time, things improved a bit though.  I could access the controller, via the iPhone app, from anywhere. Scheduling was added, as was weather and a few other features. I don’t think I saw the truly smart scheduling that was promised, but it was delivering what I needed. Until today.

Cloud Dependency

Today, the Lono died. Well. More specifically, the cloud service behind the Lono died. Now the attractive black & green box on the wall of my garage is essentially useless. Obviously, that is frustrating because I can no longer control my sprinklers, even from home when my phone & the Lono are on the same network. But it frustrates me on another level too. These IoT devices are clearly more powerful when connected to the cloud, but they should not be designed to be dependent on that cloud to do anything. 

There is absolutely no reason why the Lono, discovering it could no longer reach its cloud based control center, couldn’t have dropped back to a LAN only mode. Whether the outage is caused by the company failing (which seems to be the case here), or other things (maybe an ISP failing, or being temporarily offline), there really is no excuse for these things to stop working based on their last known settings & reverting to more local control.

Kickstarter 

I’ve backed a number of different things on Kickstarter & Indiegogo. Typically, while they may have received firmware updates etc, only a couple were really dependent on a cloud based service & only Lono has failed. It does make me think I will be more wary of cloud backed IoT projects in future. Perhaps such projects will need to explain their plans for this scenario. At the very least, it would be good to see they’ve considered this & have some level of “disconnected” functionality baked in.

If they want to truly impress me, they should have hardware design, firmware & app software in an escrow service, with public (or at least customer) release triggered on company failure. Then, maybe, the community could rally around and perhaps continue support for these devices. 

Ongoing Pebble Issues


Update: Check out my more recent update on my Pebble experiences too.

Anybody following along here will know that I have been having intermittent connection issues, as well as bluetooth audio interference issues, with my Pebble smartwatch. But more than that, I have been having issues with their customer support. Not to mention having their Chief Evangelist accuse me of whining, and then block me on Twitter. Great way to treat your customers. Guess she doesn’t want to actually hear from real users. 

On October 17, I was given this answer by one of their support folks:

We well received your logs, and will review them thoroughly and have a reply by Tuesday. Thanks for your patience.

That was a Friday. By the next Friday, October 24, I had still heard nothing other than another canned response on a different case number because I submitted more logs through their app and it generates a new case each time suggesting I upgrade to iOS 8.1 (which I had already done, and which has made no difference).

Continue reading

Pebble: Still Not Ready

Sadly, I have to say the Pebble smart watch is still not ready for general use. There are still too many bugs in the firmware, and too many limitations for it to be acceptable to anyone outside of the early adopter crowd. Even a year after they initially shipped.

In the early days, the regular firmware updates seemed to improve things. Unfortunately, the most recent updates seem to have made things worse. 

Battery Issues

The new stainless steel watches were launched with version 2 firmware and the Pebble App Store all of which seemed great. Except that the battery life of the watch could suddenly drop from the several days normally achieved to just a few hours. And it could go from super efficient to super inefficient at any time. Given that battery life is one of their key advantages, this was a pretty serious regression. Something that should have been caught during testing.

The fixed firmware was released recently, but apparently it is still not really fixed. My watch took around 24 hours to drop from fully charged to 89% (that is pretty much the normal rate I have observed – around 10% a day).

2014-05-18 04:48:27:000 ttery_monitor.c:204 Batt state: 4224mV 99% hardware charging 0 plugged 0
2014-05-19 02:55:26:000 ttery_monitor.c:204 Batt state: 4095mV 90% hardware charging 0 plugged 0 
2014-05-19 03:00:26:000 ttery_monitor.c:204 Batt state: 4086mV 89% hardware charging 0 plugged 0 
2014-05-19 05:41:26:000 ttery_monitor.c:204 Batt state: 3994mV 79% hardware charging 0 plugged 0 
2014-05-19 08:19:26:000 ttery_monitor.c:204 Batt state: 3927mV 69% hardware charging 0 plugged 0 
2014-05-19 09:44:26:000 ttery_monitor.c:204 Batt state: 3869mV 59% hardware charging 0 plugged 0

But then look at what happened. The next 30% drop took less than 7 hours. And for most of that time I was asleep and very few notifications were being delivered (I get far more during the day when all my calendar event reminders are firing off).

Seems the issue with the battery is still not fixed. I have submitted the logs, but at this point I am losing confidence in Pebble’s ability to fix these serious firmware issues.

Audio Interference

For the longest time the audio quality I have experienced when using my car’s hands free telephone system has been terrible. Very occasionally it would be crystal clear, but most of the time it was crackly, sometimes to the point where I would need to hang up and redial in hopes of getting better quality. It never occurred to me that the cause of this noise was the Pebble. 

Last week though I was driving back home after going to pick up some paperwork and I was stuck in traffic listening to music from my phone connected via the car’s A2DP connection. This had always been good quality (further confusing me as to why the telephone audio should be so bad), but now it was experiencing periodic drop outs. Very short times in the music when there was silence, but easily noticeable. Since I was stuck in traffic, often not moving at all for several minutes, I had time to trace the cause.

Remembering that the Pebble had just updated its firmware, that was an obvious place to start. Turning off the Bluetooth on the watch didn’t impact anything immediately but right then the traffic moved, so I turned my attention back to the road; leaving the Pebble’s Bluetooth off. Perhaps 30 seconds or so after I switched it off, the dropouts stopped. The next time I stopped, I turned Bluetooth back on and sure enough the drop outs re-appeared. So now, the Pebble interferes with A2DP music streams (a clear, and serious regression).

Even more interesting, during one of the times I had Bluetooth off I received a call. It was crystal clear. More experimenting with that showed that the interference I had long put down to an incompatibility between my car and iPhone was in fact also being caused by the Pebble. That is not a regression in the latest firmware though; that has always been there.

Some searching online revealed a thread on their support forums describing the hands free audio interference that is happening in lots of cars. And yet the support response I got merely shrugged it off with the advice that I should disable Bluetooth on my watch when in the car & there was no way they could test all cars. Obviously, nobody would expect them to test all cars, but it doesn’t seem hard to find some that show the problem. And there is even a detailed post in that thread stating the problem can be reproduced on Bluetooth audio quality measurement test equipment:

The Voice Quality algorithm used for this test was ITU-T P.862.1 (PESQ). The scale for the PESQ algorithm is 1-5 (5 being perfect). For all tests, the iPhone is on ATT network whereas the far-end is Verizon PSTN. Each test consisted of 3 different calls, each call sending/recordng 4 voice files. After each test i averaged all PESQ scores.
The average score for iPhone5 without Pebble was 2.71. This is average for mobile to PSTN.
The average score for iPhone5 with Pebble was 1.36. This is considered extremely low.
The average score for iPhone4 without Pebble was 2.40. 
The average score for iPhone4 with Pebble was 1.22.

That makes it pretty clear that the Pebble is interfering with the audio quality on iOS devices at least. Again, this should really have been caught during testing.

Recommendation

At this point in time, if you asked me whether you should buy a Pebble I would have to say no. Not unless you are willing to live with pre-alpha quality software, potentially abysmal battery life, poor quality Bluetooth audio connections and relatively little support. When it is working well, the Pebble is a great smart watch, but the ongoing software quality issues are really letting it down right now.

Multitasking & Battery Life

One thing that has annoyed me for the longest time now is this myth that multitasking somehow reduces battery life. The iPhone multitasks today, it just doesn’t allow multiple third party apps to run concurrently.

I’ve written a lot of software, both application and system level (right down to the lowest levels of an RTOS), and believe me, if it is written properly a background app does little or no harm to battery life.

Many of the applications that people would like to see running in the background would spend almost all of that time waiting for a system event. That waiting state doesn’t harm your battery life; only when the application is actually processing something does it really consume power. The push mechanism on the iPhone today might actually be worse since it has to load the app each time, a far more expensive operation (in CPU, and therefore battery) than just switching to one that is already “running.”

Consider the IM app example that is so often used to support the claim that background apps kill battery life. Sure, if you run the IM app (background or foreground) and stream messages at it continually, then it will reduce the battery life. If you just have it sitting there in case somebody tries to start a session though it isn’t doing anything most of the time (occasional presence messages perhaps). I ran an IM app all the time on my Nokia N95, connected over AT&T’s network 24/7. My battery life was unaffected, as expected.

Another example of a well behaved background app is the daemon that we wrote for the jailbroken version of Devicescape’s app (before the SDK and app store existed). It made no difference to battery life because it spent almost all the time blocked waiting for a system event. One that only happened when a new Wi-Fi connection was made. We run in the background on Nokia, Windows Mobile and Android (not to mention Windows XP/Vista/7 and Mac OS X) today without impacting battery life.

So what will affect battery life? Well, an app that continues to do something in the background, rather than waiting for an event, one that polls for an event rather than blocking until the OS tells it about the event, or one that requires a power-hungry piece of the hardware to be on all the time (e.g. GPS). But even those apps have their place. Imagine a background image uploader: it will do something in the background while it is needed, and then exit or wait for a new photo to be taken. Or an app that checks your location every 5 minutes. It is my choice to use the battery that way, so why restrict it? Just make sure it is reasonable for the application, explained to the user, and under my control (can be checked as part of the review process).

These types of apps don’t take any more power than they would if I left them running in the foreground, but letting me push them to the background allows me to choose if I want to watch them work, or read my email etc.

Above all, please stop spreading this myth that multitasking or background processes will harm battery life. Only badly written apps would do that.

Nokia Should Switch to Mac OS X

A little different from my recent posts, but this is something I’ve been thinking about for a few weeks now believe it or not. Nokia should switch to Mac OS X.

OK, I don’t mean they should switch their phones over to running Mac OS X, nor for that matter even their new netbooks. I mean they should switch their application development environment from Windows to Mac OS X.

Why?

Aside from it simply being a much, much better platform to use for development in general, it is also the platform that a large number of mobile application developers already use. Over 125,000 registered mobile app developers out there today are using the Mac platform to write apps for the iPhone platform. The majority of those developers are not going to think of switching to a Windows box to develop on. If Nokia wants to court some of them into developing for its smartphones too, it needs to have a development environment that runs under Mac OS X.

XCode or Eclipse

It doesn’t matter as much as the platform choice, but plugging into XCode as well would certainly make the process more familiar to iPhone developers.

What really matters though is that the tools are simple to install, run smoothly and allow for rapid development (including simple, fully operational device debugging).

Nokia’s tools people need to spend a few days working with the iPhone SDK and getting a feel for how smooth the development process is. (OK, I know the certificate stuff isn’t great, but it is still integrated into the build process, and the newer releases of XCode have made it a little easier to deal with.) Then make sure the Nokia platform is as simple to use, no matter what tools it is based on. This is about a complete system.

S60 or Maemo

While we’re talking about cleaning things up, S60 has gone beyond its useful life. I used an N95 for two years from when they first came out, and believe me that was already stretching S60 beyond breaking point. The newer phones are being seriously let down by S60.

If Nokia could just accept that Symbian is dead, and move their vast momentum behind Maemo, a platform they’ve been developing and proving in the field for several years now, but still don’t have the courage to stand behind 100%, they’d actually have a platform that could compete with Android for sure, and perhaps even Apple.

There are some simple rules for success here though (and something that Android is already failing on):

  • Own your platform. Define it, and keep it consistent. You can mix up the peripherals a little, but keep the screen size the same, and make sure the OS abstracts the interface to things like keyboards so no matter what the hardware supports, the apps don’t need to change.
  • Simple, clean UI. Given where we are now, it is going to be a touch screen interface, so design it as such. Don’t worry about the existing S60 apps – they’re history. Make it clean and simple for all the exciting new apps.
  • Powerful APIs.Let me use things like the network, the location services and the maps without having to jump through hoops, several times, with my hands tied behind my back.
  • Single API.While the APIs need to let me access the full power of the device (and this is the iPhone’s achilles heal), there should also be just one API for each function. KISS matters.

A clean, standard, C++ API based on the Trolltech technologies, and a solid, secure OS like Linux would make a very solid platform.

What About the S60 Apps?

What about them? The folks developing apps for the S60 are going to move on. They’ve probably already moved on – to iPhone or Android. The rest will happily follow.

This idea that you can’t disturb the value chain is nonsense. Even the name implies that: it is a chain, attached to a leader. Where the leader goes, the chain follows. It is how they make money. And realistically what are the alternatives? They’re going to have to change platform regardless, why change more than you need to.

Stimulus

Perhaps this quarter’s massive losses at Nokia will be enough to shock them into activity. The saddest part of all of this is that they have been sitting on the answer to many of the issues with their smartphone platform since before the iPhone and Android were even players in the space. Ironically, they’re also the one company that should feel completely comfortable backing a Linux solution: it is, after all, a Finnish OS.

If, even after all these years with Maemo, FOSS issue is a problem though, how about using NetBSD or licensing a true microkernel like QNX Neutrino? Trolltech’s UI would run on both of those very easily (one of my last demo projects at Wind River was to port the open source version of Trolltech’s code to run under VxWorks AE – it was a simple port, and ran very well).

The Price of Abstraction

Jason Kester has a very insightful post up on his blog this weekend about the use of the terms “magic” and “smart” to describe software tools & frameworks. And more importantly perhaps, about the features themselves, which are often anything but “magic” or “smart.”

The problem is that these features hide the real operations that are going on under the hood. It makes what are potentially complex, time-consuming and performance killing operations seem like a simple thing. They also make it next to impossible to work out when that is the case and when it isn’t. Since I’ve been looking at Ruby on Rails a little at the moment (it seems to be the most popular choice of web development language at the moment), I was interested to see Rails being listed there as well.

In my past I have seen similar problems with C++ (unexpected calls to copy constructors and conversion operators) and had to create coding standards that helped to catch these sorts of problems at compilation time rather than letting them get into executing code where it becomes next to impossible to track them down. That’s harder to do with things like Hibernate where the problem is caused by the abstraction itself, and it not just a symptom of powerful language features.

Apple’s Vista Ad and More…

One of the most annoying things about the early versions of Vista that I was playing with was the continual darkening of the screen and question about whether to allow something I had just asked for. That is what this commercial from Apple is playing off of. Perhaps the worst example of this was having to allow notepad to open each time I wanted to view the source for a web page from IE7. Apart from the fact that by now they should be able to open it inside the browser and display it with syntax highlighting like every other browser, having to approve it every time is really annoying.

Continue reading

Devicescape @ Demo 2007

For those that haven’t already seen it, the 6 minute presentation that Devicescape gave at this year’s Demo conference in Palm Desert is up on YouTube. I’ve embedded it here for your viewing pleasure too.

You can learn a little about Devicescape‘s hotspot login solution, and see some of the website too, but since it is free, why not just go there, download the software and try it?

Roaming San Francisco

Crepe House (Post)I spent Thursday roaming around downtown San Francisco going from one wireless hotspot to the next testing some new features that we’re adding to Devicescape‘s hotspot login service.

I got off the ferry from Alameda just after 8:40am at the ferry building and walked up Market Street (in the rain). First stop was a Starbucks to get my email and do a quick test of a new, faster way to connect to T-Mobile hotspots.

Continue reading

Join The Devicescape Beta

So here is what I’ve been working on for past few months… a service that helps you login at wireless hotspots. What is the big deal there you ask? Well, from your laptop it is only a small inconvenience to jump through a hotspot’s sign-on page, but on a phone like the Linksys WIP300 it is impossible normally since the phone doesn’t include a web browser.

With Devicescape’s firmware in the phone, you can walk into a hotspot (currently a US T-Mobile location, any FON hotspot worldwide or Google’s Mountain View municipal network, with more to follow soon), switch on and start making calls using your Gizmo or other SIP account.

Want to try it? Sign up at http://www.devicescape.com/, download the software for your device (currently supported: Windows XP, Windows Mobile 5, the Linksys WIP300 and the Nokia 770 web tablet), enter your account info and roam around town. Remember, it’s a beta so if you encounter problems be sure to let us know and we’ll try to fix it.