Using Lean Startup Methods Revamping a Complex Product - Part 1
4 04 2010Lately I've been gaining a lot of interest in the #leanstartup "movement". Eric Ries of StartupLessonsLearned is probably the lead missionary, but there are many out there liking this concept. To give you an idea just search for #leanstartup on twitter.
With this post I'm starting a series where I'll be writing about how to apply lean startup methodologies when revamping a complex product that already has a lot of customers (though not necessarily happy). I'll be expressing my personal opinions on how stuff work and may be completely wrong in many cases. 
My aim here is to work through this problem for my own gain, but with your help, and at the same time share the experience for others to gain from as well. So here it goes...
I'm not going to go in to the specifics of what a lean startup is but here's a crash course.
- Be efficient with your capital. Operate with as little waste (of time) as possible.
- Don't reinvent the wheel - use open source solutions that fits your needs
- Extreme customer centric, rapid iteration, agile develpment process. (see Customer Development by Steve Blank)
These tips seem pretty fair and I got extremely interested in this considering my experience with Tablefinder.com where we were probably as far from customer developed as possible, even though we were agile (in some ways).
A little background
Now I'm part of a much larger company, about 50 people, working in the travel industry. The company is 10 years old, but our CEO still considers it a startup, and I agree in some ways. The way I see it the company is extremely customer oriented and has been since the start - developing a destination packaging system for the first clients. We are now the market leader in Scandinavia and have most destinations as our customers.
Imagine developing a web solution over 10 years where each new customer says "Yeah, I want to buy this... if you make these modifications so it suits our needs". Our customers, Destination Marketing Organisations (DMOs), are all very different in terms of business models and financing and so on. This is why we have had to make our system extremely flexible and dynamic, and also very specific for our customers needs. One problem with this is that some solutions for one customer is directly the opposite of what another customer need.
I guess one could argue that this is Customer Development, but I don't think so. This is Customer driven development which is in some ways completely different. A client says: "I want this feature in the system, and I'll pay you X to do it." Running after all X:s will soon create a chaos in the product and 100% of resources will be busy maintaining the system. Each new improvement of the system will take extremely long to implement and test. And many times, the feature that a client want to pay for may be something that their board has decided that they absolutely need. IE no end user verification what so ever.
We recently acquired a company that delivers free downloadable travel guides for over 400 destinations world wide in up to 22 languages. I'm now the acting CTO and have a mission to develop the new arrivalguides.com site. The main product of the company is a downloadable PDF guide which is free (includes a few local ads for the destination). But the product is much more complex than that. We have a large partner network including partners like Ryanair, SAS, and other big travel related sites, where users can download the guides. Then we have the DMOs, our paying customers, which pay to control (parts of) the content of the guide. And finally we have the local advertisers on each destination, which are paying for each download made.
The problem with both products mentioned here is that due to the complexity of the systems (especially the first, Citybreak DMS) implementing a new customer takes very long time and involves a lot of costs.
So Arrivalguides is also a very complex product that have many different requirements that still need to work after we revamp the product.
How to start?
My goal is to implement the new arrivalguides.com using lean startup and customer development techniques even though this is a complex product already in production.
The problem is... where do I start? I honestly don't know. I'm all for feedback here but I will go down the path that I think is best taking decisions as I go along. People say that I'm bad at planning, and I agree, but it's on purpose :). As in a startup, there is no point writing a 5 year plan, if that delays getting your product out there. I like to call this Reactive Planning. But please do comment on all the things I'll be doing wrong. This is a work in progress.
Analysis
Analysis step 1: Identify problems with existing product
- new partner integration is too complicated
- the system consists of too many parts in different programming languages
- the system is not focused
- the system is used in different ways for different partners
- Many different "customers"
- End users
- Partner sites
- DMO partners
- Guide Advertisers
- (Site advertisers)
- Slow databasemodel
- Slow administration system
- We don't have metrics for everything end users do
Analysis step 2: Identify the stuff that is working
- End users must like something since we reach over 500 million travelers every year. Problem is that we don't know how many we are missing/not converting
- The guides are great, once people have downloaded the PDF they usually love it.
Actions
The first thing do to here must be to evaluate what features people are using, and then set up a couple of different MVPs (Minimum Viable Product) for those and test it out. But how can you test an MVP when the system has so many demands from existing customers?
Action Step 1: Get metrics for how users are using the product
In order to know what metrics to collect I guess I need to know what goals we are trying to accomplish. One revenue stream that is easy to measure is downloads of guides. Of course we already have metrics for the number of guides downloaded per partner site etc. But I want something more where I can track the exact funnel and see where we are loosing downloads.
Action Step 2: Prepare for a continous deployment development environment.
We are going to develop the new site in C# ASP.Net MVC 2, and probably use MongoDB as the database engine. For other projects we use the awesome TeamCity product from JetBrains and will do so for this as well.
Action Step 3: Prepare for a good A/B testing environment
This has been one of my main questions for a while; what are the best practices for setting this up in an MVC solution? I guess that in the end it is just a matter of switching views for a certain number of visitors, and keep the underlaying controller logic the same. Please give me feedback on this if you've done something like it before. I'm also looking forward to a KISSmetrics beta invite to use to be able to track the stuff I want in a way that is more difficult to do with say Google Analytics.
Being lean?
We have all the technical skills we need to execute this project, but one thing we don't have is time. This is mainly due to how other products have been developed in the past, and we are only a couple of guys running this. Isn't this the perfect setting to try and be lean? I mean in a way that we need to absolutely optimize our time and not spend any time developing something that our customers don't want.
One big problem here is the organization. We have very talented sales people that sign new deals everyday selling stuff that we will develop, which is nice on a financial view. But I don't think this is the right way to go. (I hope I don't get fired for writing this though...) We need to have better focus on our existing product and analyze how people are using it. This sounds easy in theory, but how can we do this when we are dependent on the cash flow from paying customers?
I think I'll leave this for part 1 of this series, and it's time to take the first steps. My mission will be to try and force this type of product development strategy in the organization by proving that it works. So again, I need all the help I can get, and if you're a pro in this area, don't hesitate to leave a comment or contact me directly.
Related articles by Zemanta
- Customer Development for Web Startups (steveblank.com)
- Maverick Entrepreneur: Without the Fat (seedcamp.com)
- 2010 03 09 The Lean Startup Gdc (slideshare.net)
- Requirements Gathering Is Not Customer Development - But It Does Define The Problem Statement (meatinthesky.com)
![Reblog this post [with Zemanta]](http://img.zemanta.com/reblog_e.png?x-id=7101eb61-9cdc-4607-b905-0eb236a38252)
[…] Using Lean Startup Methods Revamping a Complex Product - Part 1 | anders.tyckr.com anders.tyckr.com/2010/04/04/using-lean-startup-methods-revamping-a-complex-product-part-1 – view page – cached Lately I’ve been gaining a lot of interest in the #leanstartup movement. Eric Ries of StartupLessonsLearned is probably the lead missionary, but there are Filter tweets […]
Social comments and analytics for this post…
This post was mentioned on Twitter by andefred: Using Lean Startup Methods Revamping a Complex Product - Part 1 http://bit.ly/adVVZN #leanstartup feedback greatly appreciated…
Dear Acting-CTO,
You almost had me when describing your company as a start-up. You sure have several of the issues of a more senior corporation. Or it at least sure sounds you have a maintenance of a legacy application at your hands.
I feel you have two basic strategies to choose from: 1. restarting the development from scratch or 2. wrapping existing blobs of stable & tested functionality behind interfaces and concentrating on new functionality. And maybe, if the solution is still small enough, there might be an option 3. A small task force could refractor the existing solution to something that could be maintained more easily.
It seems you have already selected re-building everything - MVPs and all. In my mind this works, if you have all the expertise from previous products available. It’s quite easy to build the application, if the key members of the dev team have build an application in same problem domain several times before.
These key members of your development team should also be able to tell if there is anything valuable (and stable) enough to use as a black-box component from the current solution.
I just hope you have the core team in place & you don’t have to face the politicking associated in such renewals in larger corporations!
Cheers, Sami
Solving exactly the same problem. Developing an ERP product. Crowded place. Everybody has it. Endless customizations…
But we take time. We use a technology nobody else does. We do only what customers ask but don’t do EVERYTHING our customers ask for. And when designing a feature, I always think like “how this will be used in 10 years?”.
And that thing with sales people - exactly. I don’t want a team of SAP or Microsoft like salesmen. It is exactly the kind of business i DON’T want. So instead we take a time and we are slowly finding ways how to do this better…
Maybe this comment is confusing, but let’s discuss.
Really interesting post! I enjoyed it.
With regard to your actions in the analysis step, I would strongly recommend that you not leave out some qualitative research: interviews with end users, observations of people using the software, usability tests, etc.
You may have “many customers” but most of your customers are not the end users. Gathering qualitative information from those end users is an outstanding way of learning important things like why people aren’t downloading, or what people are doing with the guides, or whether they are returning over and over again. These are things that are generally impossible to determine from pure metrics, but they’re incredibly helpful in figuring out what the actual minimum feature set is for a MVP.
I know it seems like it’s just adding time, but really, interviewing a few customers and potential customers and watching end users with your product can generate great hypotheses about what to include much faster than anything else I’ve seen.
Good luck!
I am not sure if you are approaching it the right way! Re-writing an existing solution without analysis is bad and never a good idea. Your steps should be as following (of course these are my suggestions
a) Identify the metrics that effect your revenue. Are you capturing these? If not why? Can you capture these? If so how?
b) Look at your customer requests and identify the key stories. What I mean from a story is, identify how your customers see your product and use it.
c) Can you capture your user stories and more important track your user usage? If so how?
Once you have these metrics identified, identify your top metric and make changes in your application to track that for this iteration. Every iteration you verify if your key metrics are being tracked if not, identify the top missing one and add them to your code base.
Similarly for your codebase changes, identify your key pain points, the things that are hurting you and re-factor only the current top pain point for each iteration.
One of the key things of the lean process is to do just enough and not more then required at every step. Learn and then redo. Do small, incremental changes at every iteration. Each iterations are small but you do many and fast iterations. Due to the compounding effect the changes will be huge within a small amount of time.
So your first goals should be
- Have an continuous deployment system
- Identify key metrics and capture those
- Small and fast iteration cycles in your work process.
this is like replacing a car engine while the car is zooming down the road at 100 miles per hour.
this is interesting, but I do not think it fits the lean startup model. I don’t think you are a startup at all. you have 50 employees, a bunch of paying customers, and a business model that is bringing in money. it sounds like you are looking for ways to easily scale what you have, because it sounds like it is too costly and involved to sign up a new customer.
so whose problems are you solving, your problems and frustrations, or your customers?
you are not building 1 product for x number of customers. you are custom building unique products for each customer. as long as you have salespeople that sell that way, you will not be able to use a lean startup method. and as long as you have 50 employees you need to pay each week, your CEO will have a hard time agreeing to a change in the way your salespeople sell. once the money starts, it can’t stop.
maybe your product that you build will not really be a product, but instead will be a process for customizing your application for each paying partner that comes along. the faster you can do this customization, the more paying partners you can signup and increase sales.
maybe it’s like ordering from a menu at a restaurant. your meal can be cooked to your liking and you can have a range of dishes that you like, but your choices are limited by the menu.
so what have you learned so far? i’ll be watching this to see what is new.
I want to add to my previous comment now that I’ve had more time to think about it.
Maybe you want to follow the lean startup method concept of interating rapidly through developement cycles to elminate waste to make sure you are doing the right thing.
so you take an idea, like what is the core of our product, what is everyone of our customers doing with that feature, what about that needs to be imporved to make it easier on us and them? that is you hypothesis.
then you make something to do something and figure out how you’ll measure it to see if it meets your hypothesis. in the discovery phase, this could just be a process of reviewing all the custoemrs who’ve signed up to use your product and reviewing the work that went into each setup for them, trying to collect data on what was the most common often repeated process that you did for them. you collect that info and run metrics to see which feature wins.
then you think about it, analyze and learn.
then repeat. now that you know this one thing, what new ideas do you have? how can you build something to test that idea, what do you need to measure to see if your idea was correct? do it and learn. then repeat for something else. maybe you’ll find something that you can improve to speed up the new partner integration process. build upon that success with more success.
i don’t know, I’m not the expert. I’m going to get drunk now because I feel stupid.
Thanks everyone for great comments so far!
Regarding us being a startup or not, I agree that the 50 person parent company is not that much of a startup, but the subsidiary (arrivalguides) which only have 8 employees is. Or I should say that it needs to be. Like Steve said: this is very much like changing the engine of a running car, although I usually say changing the engine of a flying plane.
The thing is that we have to change the engine in order to make this business really scalable. I want to do this by applying my experience with startups and my new found love Lean Startup. 
I completely agree with Satish that rewriting the whole thing at once is probably as far from Lean it can get, and I should approach this with just doing the necessary action at the right time.
I just got my KISSmetrics invite, so that will be one of the first actions I will take, implementing that in the current solution to measure some more stuff.
I’m also working intensly with Google Analytics to try and set up goals in a good way. The problem is that is has been implemented in a way that gives each partner a unique tracker. *sigh* So adding goals will be a bit tough, but I need it anyways.
Laura: I’m not so sure that usability measures are that important in this stage? Isn’t that more important once you have decided on a user story to develop and then test the usability? I know that usability is overlooked by many, often me included, but I’m still not convinced this is the right place for it, please do try and convince me!
And I agree that this isn’t the easiest fit using lean methodologies, but that’s exactly why I want to do it. I won’t know for sure if it works without testing.
Very interesting post. I was advising a company in similar situation few years back (http://www.tetrasim.com/) and we got excellent results.
Anyway, there are lot that I would like to comment, but lets keep it short:
I think you need to clarify the product vision before you go out the building and start customer development. Use the vision as constraint for the customer wishes so that you do not end up developing product that tries to serve everybody and serves nobody.
Use customer development then to validateU/test your vision, be prepare to drop some customers or come up with several products (MVPs even).
If you’d like to have more detailed thoughts or hear more about the war story with similar case just drop me mail at marko dot taipale at huitale dot com.