New York Investment Network



Pitching Help Desk


Testimonials

"Thank you very much for the extra input with my Restaurant/Nightclub proposal. I already have a couple investors who are requesting more info, and that's less than 24hrs after submitting the proposal to you. I am very pleased."
Rodrick Agcaoili

 BLOG >> Recent

Multiple Factor Optimization [Lean Startup
Posted on June 29, 2017 @ 12:04:00 PM by Paul Meagher

Lean Startup Theory advocates the use of ongoing experimentation to find out if customers value your product or not. The use of A/B testing is often used determine if some feature is having a significant positive influence on customers or not. The popularity of A/B testing derives from the fact that it is easy to change some minor feature of a website to see if some success measure is improved or not relative to the control/existing version of the website. The technique is easy and the math is easy (but check this out).

A/B testing is only one experimental technique that might be used and it has some limitations. One limitation is that it is often used to test only one factor or version at a time to see if the factor/version improves success metrics or not. Optimal performance is often a function of the interaction of two or more factors that can not determined by testing one factor at a time. Chemical reactions can occur optimally at a combination of temperature and pressure that is not predicted by studying each factor separately.

Today I want to mention a methodology that is not that well known but which is used in industry to find the factors and the levels of each factor that produces optimal outcomes. That methodology is called Response Surface Methodology (RSM) and is commonly used in chemical industries to find the optimal operating conditions (temp, pressure, catalytic agents, reactants, pH, etc..) for producing a chemical reaction.

Response Surface Methodology begins by listing all the factors that might contribute to the response. It also examine the levels that each of these factors might take on. When you do this you quickly run into a combinatorial explosion of factor levels to test. Where Response Surface Methodology comes in is to help guide you towards a reduced set of factors/levels to interatively test to arrive at an estimate of the optimal factor level settings.

All I can hope to do in this blog is mention a couple of ideas from response surface methods that I found interesting and am still exploring.

If you have, say, 3 factors (temperature, pressure, pH) with 5 levels then to run a full factorial design requires that you measure responses under each of the possible 125 conditions. This is generally not economically feasible so the question becomes whether you can find the optimal condition by studying some subset of conditions, often called a fractional factorial design. One such design that is popular in RSM is called Central Composite Rotable Design. That is an intimidating phrase for a neat idea. Basically you reduce the number of levels by only testing the extreme levels of each factor along with the median level of each factor and interpolating all the values that would fall in between.

So if you wanted to test the growth of a plant as a function of nitrogen and moisture instead of studying all the levels of each factor you would only test the response for the median levels of each factor and the extreme levels of each factor and try to interpolate what the response would be between these levels. Other fractional designs are possible.

The second idea from RSM that is worth thinking about is to plot the levels of your factors on each axis to generate a response surface that depicts our how the variables interact. The combination of moisture and temperature levels will generate a growth response in the plant that can be plotted as a response surface that an educated eye can read to better grasp what the optimal factor levels might be, or what tradeoff might might be best to make. We are all familiar with drawing and interpreting graphs consisting of curves, but not so familiar with drawing and interpreting surfaces and contours so as to understand the interaction of 2 variables. Being able to visualize the interaction of 2 variables as response surfaces can allow our visual system to process the information more thoroughly than a set of numbers would.

I have found that a good starting point for understanding response surfaces is Khan Academy's tutorials on Multivariate Calculus. You don't have to know calculus to learn some useful multivariate skills from the first few tutorials.

I hope this blog has helped to convince you that there are experimental methodologies besides A/B testing that might be applied to discovering the right combination of factors for your product or service. One at a time testing does not provide any insight into the potential interactions of that factor with other factors. For that you need a factorial design and to administer it efficiently you may want to consider response surface methodologies.

The definitive text on RSM is by George Box and Norman Draper with the title Response Surfaces, Mixtures, and Ridge Analyses (2nd, 2007). Not an easy read but worth scanning for ideas and probably very useful if you want to use RSM to optimize your product or service.

Permalink 

 Archive 
 

Archive


 November 2020 [1]
 September 2020 [1]
 June 2020 [4]
 May 2020 [1]
 April 2020 [2]
 March 2020 [1]
 February 2020 [1]
 January 2020 [1]
 December 2019 [1]
 November 2019 [2]
 October 2019 [2]
 September 2019 [1]
 July 2019 [1]
 June 2019 [2]
 May 2019 [2]
 April 2019 [5]
 March 2019 [4]
 February 2019 [3]
 January 2019 [3]
 December 2018 [4]
 November 2018 [2]
 September 2018 [2]
 August 2018 [1]
 July 2018 [1]
 June 2018 [1]
 May 2018 [5]
 April 2018 [4]
 March 2018 [2]
 February 2018 [4]
 January 2018 [4]
 December 2017 [2]
 November 2017 [6]
 October 2017 [6]
 September 2017 [6]
 August 2017 [2]
 July 2017 [2]
 June 2017 [5]
 May 2017 [7]
 April 2017 [6]
 March 2017 [8]
 February 2017 [7]
 January 2017 [9]
 December 2016 [7]
 November 2016 [7]
 October 2016 [5]
 September 2016 [5]
 August 2016 [4]
 July 2016 [6]
 June 2016 [5]
 May 2016 [10]
 April 2016 [12]
 March 2016 [10]
 February 2016 [11]
 January 2016 [12]
 December 2015 [6]
 November 2015 [8]
 October 2015 [12]
 September 2015 [10]
 August 2015 [14]
 July 2015 [9]
 June 2015 [9]
 May 2015 [10]
 April 2015 [10]
 March 2015 [9]
 February 2015 [8]
 January 2015 [5]
 December 2014 [11]
 November 2014 [10]
 October 2014 [10]
 September 2014 [8]
 August 2014 [7]
 July 2014 [6]
 June 2014 [7]
 May 2014 [6]
 April 2014 [3]
 March 2014 [8]
 February 2014 [6]
 January 2014 [5]
 December 2013 [5]
 November 2013 [3]
 October 2013 [4]
 September 2013 [11]
 August 2013 [4]
 July 2013 [8]
 June 2013 [10]
 May 2013 [14]
 April 2013 [12]
 March 2013 [11]
 February 2013 [19]
 January 2013 [20]
 December 2012 [5]
 November 2012 [1]
 October 2012 [3]
 September 2012 [1]
 August 2012 [1]
 July 2012 [1]
 June 2012 [2]


Categories


 Agriculture [72]
 Bayesian Inference [14]
 Books [15]
 Business Models [24]
 Causal Inference [2]
 Creativity [7]
 Decision Making [15]
 Decision Trees [8]
 Design [37]
 Eco-Green [4]
 Economics [12]
 Education [10]
 Energy [0]
 Entrepreneurship [65]
 Events [2]
 Farming [20]
 Finance [25]
 Future [15]
 Growth [18]
 Investing [25]
 Lean Startup [10]
 Leisure [5]
 Lens Model [9]
 Making [1]
 Management [9]
 Motivation [3]
 Nature [22]
 Patents & Trademarks [1]
 Permaculture [36]
 Psychology [2]
 Real Estate [2]
 Robots [1]
 Selling [11]
 Site News [19]
 Startups [12]
 Statistics [3]
 Systems Thinking [3]
 Trends [7]
 Useful Links [3]
 Valuation [1]
 Venture Capital [5]
 Video [2]
 Writing [2]