{"id":1670,"date":"2021-04-27T12:17:00","date_gmt":"2021-04-27T12:17:00","guid":{"rendered":"https:\/\/arielsimpson.com\/?p=1670"},"modified":"2023-03-03T17:33:25","modified_gmt":"2023-03-03T17:33:25","slug":"improve-more-experiment-better","status":"publish","type":"post","link":"https:\/\/arielsimpson.com\/index.php\/2021\/04\/27\/improve-more-experiment-better\/","title":{"rendered":"Improve More, Experiment Better"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">[By Ariel Simpson, originally published October 2019 at <a href=\"https:\/\/engagedagility.com\/engaged-agility-blog\/\">EngagedAgility.com<\/a>]<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Imagine a scientist in a white lab coat, two beakers in hand. You ask what she\u2019s doing, and she says, \u201cI\u2019m conducting an experiment!\u201d When you ask what the experiment entails, she responds, \u201cI don\u2019t know! I\u2019m going to combine these liquids and see what happens!\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><br>Obviously no self-respecting scientist would call this an experiment. However people in almost any other line of business conduct these \u201cexperiments\u201d all the time!<br>\u201cWe\u2019re going to build an open-concept work area and see what happens.\u201d<br>\u201cWe\u2019re buying this new design tool. Let\u2019s see what happens.\u201d<br>\u201cWe\u2019re going to cancel our monthly staff meeting and see what happens.\u201d<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><br>Agility allows teams to pursue continuous improvement through experimentation, but only if we\u2019re actually experimenting. For a real experiment, you need a real hypothesis\u2014a best guess about what will happen. \u201cSee what happens\u201d is not a helpful hypothesis because you can seldom disprove it. Something almost always happens! Keep on generating those ideas, but ask yourself these questions:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>What do I think is going to happen? It\u2019s not an experiment if you don\u2019t have a disprovable hypothesis (at the end, you must know whether your guess was right or not). When you implement this change, what will happen? Will productivity go up? Will the number of bugs discovered decrease? Be specific about the changes you expect.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">2. How and when will I measure my results? Some metrics are easy to track (e.g. number of bugs, call hold time, customer clicks, etc.), but others are very difficult (e.g. employee stress, quality improvement, customer engagement). Make sure you\u2019re choosing metrics that actually prove or disprove your hypothesis, not just the easiest metrics to access.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">3. Who is involved in this experiment? Most of the changes we make impact other people. Choose enough participants so that you are learning from several perspectives, but not so many participants that\u2014if your brilliant idea goes badly\u2014the effects are catastrophic. \u201cNarrow the blast radius,\u201d as my coworker says.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">4. How long am I going to run this experiment? Change is hard, but it\u2019s even harder when you don\u2019t know how long it will last. Choose an appropriate timeline for your experiment and communicate it. This will keep you from monitoring a never-ending experiment, and will build trust with your participants.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What other factors might confuse my results? If you\u2019ve got a new design for your website and you want to measure how many more visitors you are receiving, think about other factors that might affect that result. Are you running your experiment on a weekend when web traffic is higher? Did you also put out a new web ad? Did you change your Google ad words? Limit those \u201cconfounding\u201d variables when you can, and document them when you can\u2019t.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Keep in mind that your experiment is designed to be proven or disproven. If you thought your change was going to boost productivity and, instead, productivity plummeted\u2014that\u2019s a successful experiment! You and your team know more than you did before. You can make more informed decisions, and you never have to waste productivity through this change again. The only failed experiment is the one you don\u2019t learn from.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>[By Ariel Simpson, originally published October 2019 at EngagedAgility.com] Imagine a scientist in a white lab coat, two beakers in hand. You ask what she\u2019s doing, and she says, \u201cI\u2019m conducting an experiment!\u201d When you ask what the experiment entails, she responds, \u201cI don\u2019t know! I\u2019m going to combine these liquids and see what happens!\u201d [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":2009,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_wpcom_ai_launchpad_first_post":false,"_jetpack_hide_featured_image":false,"_jetpack_featured_image_duplicate_dismissed":0,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-1670","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/i0.wp.com\/arielsimpson.com\/wp-content\/uploads\/2021\/04\/louis-reed-JeInkKlI2Po-unsplash-scaled.jpg?fit=2560%2C1707&ssl=1","_links":{"self":[{"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/posts\/1670","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/comments?post=1670"}],"version-history":[{"count":2,"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/posts\/1670\/revisions"}],"predecessor-version":[{"id":2061,"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/posts\/1670\/revisions\/2061"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/media\/2009"}],"wp:attachment":[{"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/media?parent=1670"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/categories?post=1670"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/arielsimpson.com\/index.php\/wp-json\/wp\/v2\/tags?post=1670"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}