Craig Brimhall, Ed.M., Harvard University, Ph.D. Candidate, University of Utah
Some people dismiss experiments as the domain of academics or large, deep-pocketed organizations. You might think that your organization doesn’t have the time or resources to spare for experiments.
Perhaps you’re convinced that experiments might positively affect your organization but worry that your supervisors or employees won’t have the appetite for them. Constraints on time, resources, and human capital can make it difficult to justify running an experiment. Your col-
leagues might even question your decision to divert valuable resources from productivity to experimentation.
These workplace pressures are real. Running experiments requires resources, often at the expense of some other activity. What is missing from such deliberations, however, is the cost of not running experiments. Leaders who dismiss experimentation do so at their own peril. The cemetery of businesses is littered with the gravestones of companies that may have benefitted from experimentation whether it was used to discover a threat or recognize an opportunity. Blockbuster, formerly the largest chain of movie-rental stores in the world, dismissed online movie streaming as a minimal threat to its power as a retail giant. Kodak, an early pioneer of film-based images, thought digital cameras would never replace film. Target was sure it could seamlessly integrate its retail approach into the Canadian market. Yet Block-buster, Kodak, and, to a lesser degree, Target all suffered tremendous losses because they failed to recognize threats or effectively seize opportunities; they were unwilling to change or try something new. Imagine that instead of dismissing movie streaming, Blockbuster had run an experiment, giving some of its customers the opportunity to stream titles from home rather than coming into the store for a physical copy of the movie. What might Blockbuster have learned? We will never know if the company would have discovered a way to access the vast, untapped market of streaming, but perhaps it could have. The cost of not running experiments is potential extinction. In the short term, it may be difficult to justify allocating resources to an experiment, but in the long term, the results might be crucial to the company’s survival.
As important as their potential power is the fact that not all experiments require vast amounts of resources to be effective and significant. Many experiments can be run on a small scale, requiring few initial resources and little ongoing commitment. These three guidelines will help you to start running affordable experiments in your organization: (1) start small and keep it simple; (2) capture and share the learning; and (3) mitigate the risks of the experiments.
- Start Small and Keep It Simple
In 2013, the Los Angeles Unified School District (LAUSD) rolled out an ambitious plan to put an iPad into the hands of every one of its students. The initiative was a partnership between LAUSD, Apple, and Pearson—an educational curriculum company—and it cost the taxpayers of Los Angeles approximately $1.3 billion USD. But as soon as students started using the iPads, it was apparent that the venture was a disaster. Problems included avoidable technical issues that prevented students from accessing the pre-loaded curriculum, a lack of adequate teacher training, and, when the iPads actually worked, a faulty security system that allowed the students to surf the internet without appropriate parameters. Eventually, the iPads ended up stacked in forgotten corners of classrooms, an expensive lesson in the importance of experimentation.
Although each organization shares in the blame for the failure—LAUSD, Apple, and Pearson are still pointing fingers at each other—the massive scale of the failure could have been averted had the organizations worked together to run small, simple experiments in advance, testing the iPads and the proposed curriculum with students and teachers. Such experiments might have included distributing iPads to a handful of randomly selected schools within the district, giving LAUSD the chance both to evaluate if the iPads were working properly and to assess the learning outcomes after the iPads were introduced. It is highly likely that through a small experiment of this kind, LAUSD could have learned enough about the pitfalls and difficulties of the new system to avoid disaster, by adjusting their plan for the district-wide rollout accordingly. Millions of dollars would have been saved, and LAUSD, Apple, and Pearson would all have avoided negative press and a drawn-out litigation process.
Participants in many of CALI’s programs get firsthand experience with experimentation. In one of the Center’s hallmark workshops, participants are immersed in Design Thinking (DT), a process that includes, among other elements, rapid prototyping—where they quickly design and test solutions to solve a particular challenge. Participants are encouraged to test their ideas rapidly, with minimal commitment of resources, and with each test the participants learn more about the problem itself and discover potential solutions for it. They are then encouraged to work on the ideas that produce positive outcomes and to discard those that are less effective.
In January 2020, I facilitated a DT workshop in Singapore with a group of highly intelligent and motivated young individuals from the United Arab Emirates, who had been selected for the program based on their academic achievements and commitment to civic engagement. Throughout the week, the participants brainstormed about the life they wanted and then designed a series of small experiments to learn more about the big personal decisions they would soon face in the future. One participant decided he wanted to open a restaurant that would specialize in sustainable food. To test whether or not he would truly enjoy such a career, he designed a series of small experiments in which he cooked dishes and invited his friends to taste them. The experiment was designed to inform him not about the quality of his dishes but rather about how he might enjoy being a chef, compared to other careers he was considering. In DT, as in other spheres, a small and simple experiment is effective because it allows us to learn something about the world, without needing to expend large amounts of time or resources.
The experiments run in organizations can also be simple and start small. Managers should strive to keep their costs as low as possible and to run the experiment for the shortest duration that will not compromise the integrity of the experiment. In this way, managers can learn things about the world that may identify an opportunity—like if Blockbuster had experimented with streaming—or avert massive and expensive failures like the one experienced by LAUSD.
- Capture and Share the Learning
An experiment without learning is a wasted experiment. Part of designing an effective experiment is thinking through the ways in which the results and interpretations can be captured and shared with your team, your stakeholders, even with your organization as a whole.
After the completion of Disney’s water-saving experiment, the research team utilized the company’s vast resources and many lines of communication to disseminate their findings to people in the company with the power to make overarching policy decisions. Every branch of the Disney company could now use the findings from the experiment to explain and justify a customer-service change in their hotels.
For many of us, our experiments must be on a smaller scale, with our findings localized to our team, because we likely have fewer resources and less power to influence others. The ways in which we capture and share what we learn from our experiments will be different in each workplace and personal situation.
Learn more about the Center for Asia Leadership’s programmes in strategic foresight and adaptive leadership here at bit.ly/lead-asia.
What is your response to the rapid pace of change around us? The best response is to learn and adapt. Here is how we can help your organization adapt to ongoing disruptions and thrive amid challenging times through our Adaptive Leadership Signature Program that addresses the past culture, while building the teams’ capacity to create a new future.
