The Downside of Failing Fast

I promised someone that I would start blogging regularly again when I hit 2 million downloads across all of my apps. Well, I passed 2.3 million a little while ago, so I've been dragging my feet long enough.

Just an aside, I don't recommend trying to juggle 40 apps and 2+ million downloads on your own. I handle all the development, support, ops, marketing, partnerships, QA, biz admin stuff for Wombat Mobile by myself and got overwhelmed around 1 million downloads several years go. The products have really suffered since then, and, in hindsight, I should have done things differently around 750k downloads.

But that's a topic for another time. It's been far too long since I've written anything. Let's do this.

First off, I LOVE the concept of Failing Fast. It just fits really well with the current App environment. Put out an MVP, assess the interest, and see if it's worth pursuing further. If not, then move on to the next idea. It's less effective for larger platform/infrastructure plays, but most of the startups I run into these days are aiming pretty low.

There are some downsides, though, to failing fast that I don't see mentioned very often.

Definition of Failure

Some failures are easy to spot. Say after 6 months, you've got less than 5,000 downloads or only 100 monthly active users. It might be time to pronounce the project dead. Is there a good pivot option? What's the opportunity cost of spending more time on this idea versus a whole new concept. At this point I'll usually do a post-mortem and take a cold, hard look at what went wrong.

Was the concept bad? Did people just not care? Was the app buggy? Did people not know about it?

If people didn't know about it, then, looking back, maybe that week that I spent working on some esoteric feature for an edge use case would have been better spent thinking about how I was going to acquire users and make things more viral.

But most of the time, it's less clear if a project is a failure or not. You might get some really dedicated users, some good press, but it's not a runaway success. Also, someone running a lifestyle business might have a different definition of failure than a well-funded team. Or the bar for your first mobile app might be a lot lower than after you've had some successes and are looking for bigger upside plays.

Success & Failure Metrics

One of the metrics I like to look at is Time to 10k. How many days did an app take to get to 10,000 downloads? For evaluating interest in an app, I like to first look at downloads. Are people even willing to download the app in the first place? I put on a different hat and mindset when looking at Retention and Monthly Active Users, but that's a topic for another post.

Another thing I look at is the enthusiasm of initial users. I usually get a lot of feedback on my apps either within the app or they'll email me with suggestions. Sometimes people can be REALLY into an app. They love the concept, have dozens of suggestions and become evangelists for the app. I love users like this, I find one of these power users is more valuable than 50 regular users. If I get a lot of folks like this, then, even if overall downloads are less than normal, I have to think to myself: maybe there's something here.

Depending on the concept, I'll sometimes look at revenue when assessing a new project. I made a new app a little while ago that made way more revenue per user than my other apps. The backstory behind it is that it was in a space that I wasn't very familiar with, so to understand it better, I made an app as a trial balloon to become more familiar with it and to reach early adopters in the area. While the overall download numbers weren't as high as other apps, the absurd revenue per user suggested paying more attention to it.

The key for me is just to have the metrics in place from the beginning to match whatever goals I have for the project whether it's downloads, revenue, #referred users/virality, whatever.

A Trail of Abandonware

Another downside to Failing Fast is what happens to the project when you move on to the next one. Say you've got an app with 100k downloads and a nice percentage of active users. There's good interest in the concept and people are happy with the execution, and it makes a reasonable chunk of change per month. It's a nice app, but the upside is limited.

The app makes enough money that you're still supporting it, but creating new features might not be cost-effective, especially when faced with other projects with potentially massive upside.

I usually just leave the server-side of things up and running so that everything still work since folks may have spent a lot of time in the app and would be upset to see it go away.

But there was one time, many years ago, I had to shutdown a project because I was tired of running separate co-located Linux and Windows servers and wanted to focus on just one platform on the server-side. I had written an Evite alternative for people to invite others to events and informal get-togethers. I left it running untouched for a long time thinking all the events had passed, but after I shut it down someone contacted me because they had used the site to invite guests to their wedding in a few months. I still feel bad about that one.

Won't Somebody Please Think of the Users

This is the biggest downside of Failing Fast. That your failure can affect hundreds, thousands of users that embraced the idea. I still get emails from things I've written a decade ago from users asking me if I know of a replacement/alternative for something I wrote and later abandoned.

When I make an app, I often get to know some of my users very well. Some users will email me really excited about the app with suggestions and get a dialogue started. We'll usually exchange a lot of emails, and I try to understand as much as I can about how they're using the app, what they might want from the app, ideas for other apps based on how they use their phones. Sometimes we even become Facebook friends or connect on LinkedIn.

Since it's just me trying to juggle 40 apps, I often have an app's power users determine what should go in the next release sort of like de facto Product Managers. So declaring a project a failure and moving on is a tough decision on a project you've spent a lot of time building with your users together.

This post is getting longer than I'd like, but even after writing out these downsides, I still think Failing Fast is the best way for me to operate. I know too many folks spending way too much time heads down trying to perfect a concept in isolation when they could find out pretty quickly of a project's potential by getting it out in front of users.

Tags: lean startup, metrics