In November, the Thinkers50 announced its 2017 ranking of the 50 top management thinkers in the world. The ranking was launched in 2001 and is published every two years.
Twenty-three of the top 50 have contributed to MIT SMR, either as authors or interviewees. They include:
Don Tapscott Clayton Christensen W. Chan Kim & Renée Mauborgne Michael Porter Marshall Goldsmith Richard D’Aveni Rita McGrath Erik Brynjolfsson & Andrew McAfee Pankaj Ghemawhat Vijay Govindarajan Nilofer Merchant Hal Gregersen Hermann Simon Zhang Ruimin Anil Gupta Lynda Gratton Gary Hamel Tammy Erickson Morten Hansen Henry Chesbrough Julian Birkinshaw
We congratulate our contributors on this honor and invite readers to explore their work further on our website. Here are just a few examples of their recent work:
These days, scale is a top-of-mind issue among leaders of tech companies. Yet many of these leaders are missing the biggest and most obvious scaling opportunity they’ll ever encounter: old people.
The worldwide population of older adults — people age 65 and up — reached 617 million in 2015 (almost twice the total population of the United States); in the U.S. alone, older adults represent an $ 8 trillion consumer market. Thanks to longer life expectancies, there will be 1 billion people age 65+ by 2030, and 1.6 billion of them by 2050.
The problem — which Joseph F. Coughlin, director of MIT’s AgeLab and the author of The Longevity Economy, lays out in the excerpt below — is that most tech companies simply don’t get this deep-pocketed, growth market. Worse, because of the relative youthfulness of tech companies, their leaders are not only missing the biggest market they’re likely to see before they become old folks themselves, they’re also overlooking the job candidates who could help them tap it as well.
“Goodness to a righteous person, leads to goodness to his neighbor.”
— Babylonian Talmud (300 A.D.)
I spent a gap year after high school living in Jerusalem and studying the Talmud. Throughout this historic work, there are references to the importance of “space” and the influence that a neighbor can have. From a young age, I’ve always focused on the significant impact of “location” in one’s life.
Recently, our next-door neighbors asked if we knew anyone looking for a house prior to putting their house on the market (not a bad strategy to avoid real estate commissions!). My wife and I, appreciating the heads-up, reached out to a few families we knew we wanted as neighbors. It ended up working out.
I have translated my belief in the importance of location — and the people in it — to my professional life as well, first while working in human resources and now as CEO of Investopedia. Specifically, I take a personal interest in seating arrangements. I believe that where people sit has a major impact on managerial and team effectiveness. It influences personal friendships, which lead to business relationships; builds stronger team behavior through collaboration; improves employee retention; and ultimately results in smarter decision-making.
I try to never micromanage, but this is one of the few areas where I do. Here’s how my approach works.
Execs Sit With Execs
Research on the importance of high-functioning executive teams to company performance has been conclusive. Too often, when leaders are asked to name their “team,” they list only their direct reports. Yet the most important team an executive is on is the senior leadership team.
Often, it is the silos between functions that result in a misalignment of company priorities and poor decisions. These silos typically result in a “political” environment where information is shared directly with the CEO rather than between executives. This is why it is critical that your leadership team sits together.
My eight-person executive team sits in an open-space environment. Execs with 20+ years of experience sit beside one another with no preferential seating over a college intern. There are no walls, no barriers. They hear what someone’s weekend plans are and can listen in on business conversations both formally and informally. Indeed, I have long believed that informal conversations are far more important in executive team building than formal (and far less productive) strategy meetings.
Function Over Form
The typical corporate structure calls for people with similar functions to sit together, building relationships with coworkers of similar backgrounds, careers, and interests. As a leader, you need to break this paradigm.
Many job functions have a strong basis for reporting into multiple areas, and it often isn’t clear where people should report. For example, marketing typically reports to a chief marketing officer, but sometimes reports into sales. Product could report into a head of strategy, a head of product, or a chief technology officer.
The best practice here is that individuals should sit near the function that you want them to collaborate with, not the function that they report into. For example, we made the mistake of having our social media team report into and sit next to our marketing department, since the goal of both functions was to drive users to our site. However, we were finding that there was an increasing disconnect between our social media team and our larger editorial staff.
We moved the social media team to sit within editorial. Relationships developed, team members started getting invited to larger editorial strategy meetings, and increasingly, there was a far greater alignment between our content and social media strategies.
Sit people outside their reporting function, and barriers between functions will melt away.
Ask, Don’t Tell
Investopedia has more than tripled its staff over the last couple of years to 140 employees, and we are now in our third (and hopefully last, for a while) home in midtown Manhattan. Before every move, we send a survey to employees that asks who they need to sit with to best achieve their goals, who will they learn most from, and who they do not want to sit near.
Involving employees is extremely important, as they will be spending more time with work neighbors than with their families. We shouldn’t force people to sit next to someone they’re not comfortable with.
When it comes to new hires, I send a personal message sharing my excitement for their joining (every CEO of a company with fewer than 500 employees should do this) and then send a note to HR and the hiring manager recommending where the new hire should sit and why. When there is disagreement, we’ll discuss the pros and cons of different arrangements.
Frequently, I have used seating to provide a development opportunity for a manager. For example, one of my execs is not sufficiently analytical, and I specifically asked that the new analyst sit next to that executive.
Where employees sit directly impacts their personal relationships, teamwork, work satisfaction, ability to learn from colleagues, and executive functioning. It matters. Don’t let your office seating chart be like a game of musical chairs. The music should stop with you.
You can hardly pick up a business publication without reading about the ever-increasing pace of change in technologies and markets and the consequent need for more adaptable organizations. Given the imperative of adaptability, it is not surprising that few words have received more attention in recent conversations about management and leadership than “agile.”1 Organizations ranging from large corporations like General Electric Co. to tiny startups are trying to be both flexible and fast in the ways that they react to new technology and changing market conditions.2
The word “agile” appears to have been first applied to thinking about software by 17 developers in 2001.3 Having experimented with more iterative, less process-laden approaches to developing new applications for several decades, the group codified its experience in an agile manifesto. “We are uncovering better ways of developing software by doing it and helping others do it,” they wrote. In software development, agile now has a variety of manifestations, including scrum, extreme programming, and feature-driven development.4 The results have been significant. A variety of studies show that agile software development methods can generate a significant improvement over their more traditional predecessors.5
But what does this mean outside of software? Can agile methods be successfully applied to other types of work? Many proponents (a number of whom started in the software industry) argue that the answer is yes, and a growing collection of books, papers, and blog posts suggests how it might be done.6 The evidence, however, remains limited to date, and a recent article by two of agile’s founders cautions against applying agile indiscriminately.7 The blogosphere is also replete with discussions of an ongoing agile backlash.
To provide some practical advice to business leaders trying to understand what agile might mean for their organizations, we take a different approach. Our research suggests that in applying agile methods from the software industry to other domains, managers often confuse practices and principles. When agile methods work, they do so because the associated practices manifest key behavioral principles in the context of software development. But, successful as those practices can be when developing software, there is no guarantee that they will work in other contexts. The key to transferring a set of practices from one domain to another is to first understand why they work and then to modify them in ways that both match the new context and preserve the underlying principles.
The goal of this article is to help you understand several key work design principles that undergird not only agile practices in software but also Toyota Motor Corp.’s well-known production system in manufacturing. Once you understand these underlying work design principles — through a framework we call dynamic work design — you can create work processes in your own organization that are both more flexible and more efficient. (See “About the Research.”)
Stability vs. Uncertainty
Academics and managers alike long believed that organizations had to make trade-offs between flexibility and efficiency. A central notion in the academic theory on organizational design is contingency, the idea that organizations and their associated processes need to be designed to match the nature of the work they do. One of the most common variables in contingency theory is the degree of uncertainty in the surrounding environment (often also conceptualized as the need for innovation). When both the competitive environment and the associated work are stable and well understood, contingency theory suggests that organizations will do best with highly structured, mechanistic designs. In contrast, when facing highly uncertain situations that require ongoing adaptation, the theory suggests that organizations will do better with more flexible, organic designs.8
An early advocate of the mechanistic approach to work design was Frederick Winslow Taylor, author of the 1911 book The Principles of Scientific Management.9 Taylor’s essential insight was simply that if work is regularly repeated, it can also be studied and improved. In stable, well-understood environments, it is thus often best to organize work in ways that leverage the efficiency that comes with repetition. For example, in a modern factory, well-defined tasks are specified, and the work proceeds serially, moving from one carefully constructed and defined set of activities to the next. There is little need for collaboration in these settings, and the organizational structure that surrounds stable and repeatable work tends to be hierarchical to ensure that everybody follows the prescribed work design. The cost of such efficiency is adaptability. Due to the high degree of routinization and formalization, mechanistic process designs are difficult to change in response to new requirements. Though efficient, a mechanistic design is not agile.
When, however, the environment is unstable and uncertain, discrete tasks are harder to define, and therefore organizations cannot rely on a sequence of clearly defined steps. For example, product development teams often face challenges for which there is little precedent. Contingency theory holds that in unpredictable environments like new product development, organizations rely more on things like training and collaboration and less on routinization and careful specification. Developing a breakthrough product or service usually can’t be organized like a factory assembly line. Marketing experts may develop a set of initial requirements, which are then passed on to designers and engineers, but the requirements often evolve through multiple iterations as designers and engineers determine what is technically feasible. Consequently, effective development processes often require ongoing real-time collaboration, rather than rote adherence to a set of sequentially organized steps.
Though the contingency theory was first developed more than 50 years ago, its basic insights reappear frequently in contemporary management thinking. Many flavors of process-focused improvement, such as total quality management, Six Sigma, and business process reengineering, are extensions to Taylor’s fundamental insight that work that is repeated can also be improved. Recently, the increasingly popular design thinking approach can be thought of as a charge to tackle ambiguous, uncertain tasks with a more collaborative, less hierarchical work design.10 In general, contingency theory gives managers a straightforward approach to designing work: Assess the stability of the competitive environment and the resulting work, and then pick the best mix of defined tasks and collaboration to fit the challenge at hand. (See “A Traditional Approach to Work Design.”) If the work being designed consists of well-defined tasks (for example, assembling components), then it is best to organize it serially, or, as we label the cell on the bottom left, using the “factory” mode. Conversely, if the work is highly ambiguous and requires ongoing interaction (for example, designing new products), then the work is best organized collaboratively, or, as we label the cell on the top right, in “studio” mode.
Though powerful, this approach to work design is not entirely satisfying for two reasons. First, it describes an unpalatable trade-off: Work done using the serial factory design isn’t very flexible, making it hard to adapt to changes in external conditions, and work done using the collaborative studio approach often isn’t very efficient. Second, few types of work perfectly fit the archetype of well-defined or ambiguous work. Even the most routine work has the occasional moment of surprise, and conversely, even the most novel work, such as designing a new product or service, often requires executing routine analysis and testing activities that support each creative iteration. Academic theory notwithstanding, real work is a constantly evolving mix of routine and uncertainty.
At first glance, agile methods appear to fall more toward the collaborative side of the work spectrum. However, our research suggests a different interpretation. The conventional approach to process and organizational design is almost entirely static, implicitly presuming that once a piece of work has been designed, everything will go as planned. In contrast, a dynamic approach to work design suggests viewing work as an ever-evolving response to the hiccups and shortfalls that are inevitable in real organizations. As we will describe later in this article, agile methods actually transcend the traditional serial vs. collaborative work framework by creating better mechanisms for moving between the two basic ways of organizing work. By identifying mechanisms to cycle back and forth between well-defined factory-style tasks and collaborative studio modes when appropriate, an agile approach can considerably reduce the trade-off between efficiency and adaptability.
Dynamic Work Design at Toyota
What does this look like in practice? Consider a well-known example of work and organizational design, Toyota’s Andon cord. Work on Toyota assembly lines is the epitome of the serial, mechanistic design. Tasks are precisely specified, often detailing specific arm and hand movements and the time that each action should take. In a plant we visited recently, training for a specific role began with the trainee learning to pick up four bolts at a time — not three and not five. Only when the trainee could pick up four bolts regularly was she allowed to learn the next motion. But, despite an attention to detail that would have made Taylor proud, sometimes things go awry. In the Toyota scheme, a worker noticing such an issue is supposed to pull what’s known as the Andon cord (or push a button) to stop the production line and fix the problem.
While the management literature has correctly highlighted the importance of allowing employees to stop the line,11 what happens after the cord is pulled might be more important. During a recent visit we took to a Toyota supplier in Toyota City, Japan, we observed that one operator on the factory floor was struggling to complete her task in the allotted time, and so she hit a yellow button, causing an alarm to sound and a light to flash. (This factory has replaced the Andon cord with a yellow button at each operator’s station.) Within seconds, the line’s supervisor arrived and assisted the operator in resolving the issue that was preventing her from following the prescribed process. In less than a minute, the operator, now able to hit her target, returned to her normal routine, and the supervisor went back to other activities.
What, from a work design perspective, happened in this short episode? Initially, the operator was working in the “factory” mode, executing well-defined work to a clearly specified time target. (See the box on the lower left in the exhibit “Dynamic Work Design at a Toyota Supplier.”) But when something in that careful design broke down, the operator couldn’t complete her task in the allotted time. Once the problem occurred, the operator had two options for responding. She could have found an ad hoc adjustment, a workaround or shortcut that would allow her to keep working. But this choice often leads to highly dysfunctional outcomes.12 Alternatively, as we observed, she could push the button, stop the work, and ask for help. By summoning the supervisor to help, pushing the button temporarily changed the work design. The system briefly left the mechanistic, serial mode in favor of a more organic, collaborative approach focused on problem resolution. Once the problem was resolved, the operator returned to her normal task and to the serial work design.
The Toyota production system might at first appear to be the ultimate in mechanistic design, but a closer look suggests something far more dynamic. When a worker pulls the Andon cord, the system actually moves between two modes based on the state of the work. Though the nature of the work couldn’t be more different, such movement between the two modes is also the key to understanding the success of agile software development.
Agile as Dynamic Work Design
As we discussed earlier, the last two decades have witnessed a significant change in the conduct of software development. Whereas software was once largely developed using what is known as the waterfall approach, agile methods have become increasingly popular. From a dynamic work design perspective, the waterfall and agile approaches differ significantly.
In the waterfall approach, the software development cycle is typically divided into a few major phases. A project might include a requirements phase, an architecture development phase, a detailed coding phase, and a testing and installation phase. A waterfall project typically cycles between three basic modes of work. First, the bulk of the time is spent by software architects and engineers working individually or in small groups, completing whatever the specific phase requires. Second, typically on a weekly basis, those people leave their individual work to come together for a project meeting, where they report on their progress, check to ensure mutual compatibility, and adapt to any changes in direction provided by leadership. Third, at the end of each phase, there is a more significant review, often known as a “phase-gate review,” in which senior leaders do a detailed check to determine whether the project is ready to exit that phase and move to the next. Development cycles for other types of non-software projects often work similarly.13
Agile development processes organize the work differently. For example, in the scrum approach14 (one version of agile), the work is not divided into a few major phases but rather into multiple short “sprints” (often one to two weeks in length) focused on completing all of the work necessary to deliver a small but working piece of software. At the end of each sprint, the end user tests the new functionality to determine whether or not it meets the specified need.
Like the waterfall method, the agile approach to software development also has three basic work modes — individual work, team meetings, and customer reviews — but it cycles among them very differently. First, proponents of agile suggest meeting daily — thus moving from individual work to teamwork and back every day — in the form of a stand-up or scrum meeting, where team members report on the day’s progress, their plans for the next day, and perceived impediments to progress. Second, agile recommends that at the end of each sprint, the team lets the customer test the newly added functionality. Finally, in something akin to the Andon cord, some versions of agile also include an immediate escalation to the entire team when a piece of code does not pass the appropriate automated testing, effectively again moving the system from individual work to the team collaboration mode.
Viewed from a dynamic work design perspective, agile offers two potential benefits over waterfall. First, in waterfall development, the frequency of collaborative episodes is usually too low, both among the team members and between the team and its customers. A developer working for a week or two without a check-in could waste considerable effort before it’s clear that he or she has made a mistake or gone off course. In practice, developers often do not wait this long and informally check in with supervisors or teammates. While seemingly functional, these check-ins can lead to a situation in which the entire team is not working from a common base of information about the state of the project. In such cases, the operating mode starts to migrate from the box on the lower left, the “factory” mode, to the one on the lower right, where ambiguous work is organized serially. This results in costly and slow iteration, which we call ineffective iteration. (See “Dysfunctional Dynamics.”) Research suggests that in R&D processes, this mode can be highly inefficient.15 Similarly, checking in with more senior leadership only in the form of periodic phase-gate reviews means that the entire team could work for months before realizing that it is not meeting management’s expectations, thus also potentially causing rework.
The agile approach to software development also improves the quality of the time that developers spend working alone. The focus on developing pieces of functionality means that both the team and the customer are never more than a few weeks away from a piece of software that can be used, making it far easier to assess whether it meets the customer’s need. In contrast, in waterfall, the early phases are characterized by long lists of requirements and features, but there is nothing to try or test. It’s not surprising that waterfall methods often lead to projects in which major defects and other shortfalls are discovered very late in the development cycle and require costly rework.16
Applying Dynamic Work Design
Both the Toyota production system and agile-based software methods are thus examples of what we call good dynamic work design. In contrast to traditional static approaches, dynamic work design recognizes the inevitability of change and builds in mechanisms to respond to that. Once managers recognize the necessity of moving between more individual and more collaborative modes of work, they can build on four principles to create shifting mechanisms that are well matched to the work of their organization.
1. Separate well-defined and ambiguous work. Begin by clearly separating well-defined and ambiguous tasks. Trying to handle both types of work in the same process often leads to trouble. (See “Dysfunctional Dynamics.”) Often, the two types can be separated by inspection, but if not, then look for the signature element of ambiguous work, iteration. When work is well defined, it can be moved to the next stage like the baton a relay runner hands off. When done correctly, it doesn’t need to come back. In contrast, when work is ambiguous, even the best effort often needs to be revisited. If you find that a particular task often requires multiple iterations through the same set of steps, that’s a good sign that you are confronting ambiguity inefficiently.
2. Break processes into smaller units of work that are more frequently checked. If you strip away all the hype, the agility of any work process — meaning its ability to both adjust the work due to changing external conditions and resolve defects — boils down to the frequency and effectiveness with which the output is assessed. In both traditional, pre-Toyota manufacturing and waterfall software development, the assessments are infrequent and not particularly effective. Consequently, both approaches tend to be slow to adjust to changes in the external environment, and quality will be achieved only through slow and costly rework cycles. In contrast, when assessments are frequent and effective, the process will be highly adaptable and quality will improve rapidly. The fundamental recipe for improved process agility is this: smaller units of work, more frequently checked.
3. Identify the chain of individuals who support those doing the work. It is also important to identify the help chain — the sequence of people who support those doing the work. In manufacturing, the help chain starts with a machine operator and extends from foremen to supervisors all the way up to the plant manager. In software, the help chain often begins with an engineer and moves through the team leader to more senior managers, ultimately ending with the customer. It is critical, in our experience, that you identify the chains of individuals who do and support the work, not their roles, departments, or functions. Increasing agility requires knowing whom to call when there is a problem or feedback is needed.
4. Introduce triggers and checks that move work into a collaborative mode. Once you understand the help chain, you have two basic mechanisms for activating it: triggers and checks. A trigger is a test that reveals defects or misalignment and then moves the work from a factory mode to a more collaborative mode. In our opening example, the Toyota operator’s inability to complete the assembly task on time triggered her pushing a button and then receiving help from a supervisor. A check involves a prescheduled point when the work is moved to a more collaborative environment for assessment. In agile software development, this shift happens daily in stand-up meetings where the team quickly assesses the current state of the project. Completing a sprint creates a second opportunity, this time to check in with the customer.
Improving Procurement Performance
Using this dynamic work design framework within a company can lead to significant improvements in both efficiency and adaptability. Consider the case of a company we’ll call “RefineCo,” which owns several oil refineries and distribution terminals in the United States. The company had a procurement organization that was uncompetitive by almost any benchmark. RefineCo paid more for similar parts and services than its competitors, and the procurement group’s overhead costs were higher than the industry average. Even more troubling, when critical parts were not delivered to a refinery, it often turned out that the location was on “credit hold” due to an inability to pay the supplier in a timely fashion. Every participant in the system, from senior management down to the shipping and receiving clerks, was frustrated.
The procurement system at each of RefineCo’s sites worked roughly as follows. To purchase an item or service from an outside vendor, an employee would enter the requirements into the electronic procurement system, which would then appear as a request to the central procurement function. The staff in the procurement office would then review the request and issue a purchase order. That order would go to the supplier. When the product arrived at the refinery or the service was completed, a packing slip or service order verification slip would be generated, which would also be entered into the procurement system. Later, the supplier would generate an invoice that was also entered into the system. The electronic system would then perform a three-way match to verify that everything was done correctly: The purchase order should match the verification receipt, which, in turn, should match the invoice. If there was not a three-way match, the invoice would be “kicked out” of the system and the supplier would not get paid until the discrepancy was resolved.
The job of resolving those discrepancies fell to the staff in the refinery’s purchasing office. Unfortunately, the products and services procured frequently failed the three-way match, leading to both an overburdened purchasing department and frustrated suppliers. Though the refinery was part of a large and successful company, it was frequently on credit hold with its suppliers for failure to pay invoices on time, making it difficult for the staff to do their jobs and run the plant safely. The dedicated procurement staff worked 10-plus hours per day and had hired temporary workers to help manage the backlog, but they were still falling behind.
Most of the members of the procurement team complained bitterly about being “overworked” and how “screwed up the system was.” Nobody saw any opportunity for improvement beyond adding what appeared to be much-needed staff. For us, the critical moment in our work with the procurement staff came when one of the longtime team members explained that a good purchase request contained “all the information I need” and could be turned into an official purchase order in “five to 10 minutes.” A difficult one, however, lacked key pieces of information and might require one to two hours to process as the purchasing staff traded emails with both the requesting unit and the supplier. Despite this effort, difficult purchase orders were usually the ones that failed the three-way matching process and got kicked out of the system. Further investigation revealed that the purchase order system was completely gridlocked with the kicked-out orders, and the team spent much of its time trying to clear the backlog. The system had descended into the classic “expediting” or “firefighting” trap: There were so many purchase orders in process that the turnaround time for any given one was very long. But long turnaround times created unhappy customers and suppliers who constantly called to complain and ask about their particular order or payment. Consequently, the procurement team was constantly reprioritizing its work and reacting to whichever customer or supplier was most unhappy.
Our first insight came in recognizing that the procurement team was engaged in two different types of work that corresponded to what we call serial “factory” work and collaborative “studio” work. When the requested item was standard and all the needed information was provided, a single person could easily process the request without collaboration; then, once the purchase order was entered, it would easily flow through the system, just like an item on an assembly line. However, standard requests flowed easily through the system only if the request came with the correct information. If it did not, then it could require several rounds of iteration, usually via email, to issue the purchase order. So the purchasing function created a simple checklist that described a good purchase request. The idea was to ensure that standard orders would always arrive with the correct information. To give the various departments an incentive to use the checklist, the purchasing function promised that any request received by 7 a.m. with the proper information would result in a purchase order being issued by 2 p.m. that day. At that time, a one-day turnaround was unheard of because every order simply went into the “to do” pile. The purchasing department also created a simple trigger to improve productivity: Purchase orders that were missing items on the checklist would be immediately returned to the requesting unit.
The second part of the intervention came in recognizing that not every request could be supported in factory mode. In the existing system, neither the requesters nor the purchasing staff distinguished between a standard request and a novel one. Thus, when a request for a new product or service showed up, the agent would do his or her best to process it, typically requiring multiple emails with the requester, often over several days, to nail down all the relevant information. In many cases, when the agents couldn’t get the information they needed, they would make their best guess and then submit an incomplete or incorrect purchase order. This, too, created additional iteration, as the supplier, unsure of what was being requested, would call or email the agent. The purchasing process was thus living in the lower right-hand box of our matrix, attempting to accomplish ambiguous work in a serial fashion and thereby creating slow and expensive iteration.
Creating an effective collaborative studio mode to handle the complex purchase orders required two changes in work design. First, the team created a clear trigger: If a request was nonstandard, then it was moved into a separate pile and not dealt with immediately. Second, each day at 2 p.m., the team would work together to process the more complex cases. By working collaboratively (in studio mode), they were able to resolve many of the more complex cases without additional intervention — somebody on the team might have seen a similar order before. Also, having a face-to-face meeting was far more efficient than the endless chain of email that it replaced. And, if additional information was needed, the team could schedule a phone call in the time window after 2 p.m., rather than send an email, again reducing the number of expensive iterations.
The results of these two changes were significant. Creating a factory mode for the standard orders allowed the team to make good on its “in by 7, out by 2” promise almost immediately, generating an immense amount of goodwill with the requesters. Spending the afternoon in studio mode also sped the processing of the complex orders. The two changes created enough space that the team was able to use studio time to not only process the more complex requests but also work through the backlog of unresolved older orders. In the end, due to the efficiency improvements, the procurement team reduced its staff by the equivalent of two full-time staff members, while providing far faster and more reliable service. These process improvement insights were then applied to the company’s other U.S. sites and, as of this writing, RefineCo pays more than 90% of its invoices on time, resulting in a far happier collection of suppliers.
Look for Best Principles
Managers and consultants are often obsessed with the search for best practices — those activities that appear to separate leading organizations from the rest of the pack. The idea behind this search is that once identified, best practices can be adopted by other organizations, which will then experience similar gains in performance. While there is certainly some truth to this idea, the supporting evidence is decidedly mixed. Organizations frequently struggle to implement new tools and practices and rarely experience similar gains in performance. In many industries, the performance gap between the top and middle performers remains stubbornly difficult to close. A key reason for these failures is simply that organizations are complex configurations of people and technology, and a set of tools or practices that works well in one context might not be equally effective for a major competitor — even if that competitor is located just down the street.
Best practices are “best” when they manifest an underlying behavior principle in a way that is well matched to the organization that uses them. Toyota’s famed Andon cord and the localized problem-solving it catalyzes work by capitalizing on the efficiency that comes from individual repetition and the innovation that comes with collaborative problem-solving. Conversely, agile development methods work by channeling the creativity of software engineers through frequent team meetings and customer interactions. More generally, organizations become more adaptable when they find defects and misalignments sooner. A dynamic approach to contingency, supported by triggers and checks, can open the path to creating practices that support increased agility in the work of your organization.
Experts, academics, and experienced innovators frequently espouse the virtues of eliminating hierarchies to make sure every idea is heard and to unlock innovation.1 As intuitively appealing as this view is, it does not stand up to scrutiny. In fact, a growing body of research, including studies by one of this article’s authors, shows that the right hierarchy can help teams become better innovators and learners.2 We have also seen what happens when teams insist upon being flat. They often become unfocused, tumultuous, and inefficient because their pursuit of perfect equality prevents the more expert team members from resolving conflicts and playing leadership roles in group learning and innovation.
Debunking the Myths
Research on social species ranging from ants to zebras shows that hierarchies are important for group functioning.3 When a group has a chain of command, disagreements can be more easily resolved so that the group can take coordinated action. Coordinated action improves the odds of survival. Human beings also have a tendency to think and act hierarchically.4 In fact, hierarchies — distinct differences in group members’ power and status — can be found in virtually every human group, from children on the playground to executives in the boardroom. Depending on the circumstances, hierarchies can be formally designated or emerge naturally. And while the idea of hierarchies may go against democratic instincts and beliefs, they can and do play useful roles.
IDEO, the product design and consulting firm, offers a useful example. In 1999, ABC News’ “Nightline” chronicled the efforts of an interdisciplinary IDEO team to redesign the supermarket shopping cart. Since airing, the video has become a classic example of how innovation works. Initially, IDEO founder David Kelley expresses strongly negative views about hierarchy, saying, “In a very innovative culture, you can’t have a kind of hierarchy.”5 But as the story unfolds, a small group of senior IDEO people step in to direct how the product development team allocates its time. When asked why the intervention was necessary, one senior person explains that the process of finding creative solutions sometimes needs to be “very autocratic for a very short period.”
In reviewing the empirical research on the role of hierarchies in learning and the innovation that results from learning, and through our own studies, we have found that a properly deployed hierarchy is an essential ingredient for helping a team engage in and get the most out of its efforts to learn and innovate.6 (See “About the Research.”)
The Role of Hierarchy
Hierarchies help teams of people innovate much the same way they help animals survive in the wild — they keep teams moving in the same direction even when strong disagreements threaten to keep the teams from progressing or even tear them apart. Specifically, we found that hierarchies help teams generate, identify, and select new ideas by performing three critical functions (and then getting out of the way): bounding solutions, converging ideas, and structuring processes.
During idea generation, hierarchies set the parameters and goals of innovation. A paradox of creativity is that people are more innovative when they have clear constraints (such as time, budget, customer requirements, etc.) within which their solutions must fit.7 But teams aren’t very good at establishing constraints on their own. Team members with influence can accelerate the learning process by clearly setting the bounds for innovation and then giving the team wide latitude to explore within those bounds.
In the early stages of innovation, teams come up with a large assortment of ideas and possibilities. Ultimately, however, some ideas are more promising than others, in part because they better line up with the company’s capabilities and resources.8 Hierarchies can assist here by helping teams decide which ideas have promise and should be pursued, which ideas should be put on the back burner, and which ideas go on the waste pile. As IDEO’s Kelley noted, innovation in its early phases is “a messy process.”9 To transition from generating to refining and implementing ideas, teams need to develop mechanisms for deciding which ideas to hone in on. This can be easier said than done — different team members may be emotionally attached to different ideas. By helping teams converge on a direction, hierarchies keep teams from getting lost in aimless exploration.
Finally, effectively going through the learning process requires members to use their specialized knowledge to propose potentially wild ideas and challenge potentially sacred beliefs. These behaviors are interpersonally risky in that they open up members to ridicule and social sanctions. As a result, teams must have norms and processes in place that lower those risks so that team members are able to engage in the learning process.10
Hierarchies can actually help here, too, by creating ground rules that enable and encourage members to speak up. Research has shown that brainstorming groups struggle without a hierarchy to provide structure to what can be a haphazard process.11 To that point, one of us surveyed and interviewed teams at a Fortune 100 high-tech company and demonstrated that teams with clear hierarchies did a better job of creating an environment and establishing norms that encouraged each member to speak up and share what they know.12 In short, clarity about who is in charge and how each member contributes can help everyone on a team feel like they could — and should — engage in the learning process, and it can give them a structured way to do so.
When those with more power in a group aren’t needed to help with bounding, converging, or structuring, they need to get out of the way so that the team can do what teams do best — share, discuss, and integrate diverse perspectives and knowledge to come up with new ways of solving problems. In other words, the best hierarchies are invisible most of the time, operating in the background and only coming out of the shadows when power differences are needed to keep things moving along. Even well-meaning hierarchies become problematic when people at the top are too heavy-handed and interfere when their interference isn’t needed.
Making Hierarchies Work
People are suspicious of hierarchies for a reason — they sometimes stifle good ideas and the learning process that leads to good ideas. For example, dysfunctional hierarchies have been blamed for long periods of stagnation that companies such as General Motors Co. experienced.13
So, how can organizations foster learning and innovation? Here are three things leaders can do to leverage the power of hierarchy on teams yet avoid its pitfalls.
Have a clear chain of command. With other researchers, one of us recently surveyed teams at more than 50 organizations in order to understand how the shape of a team’s hierarchy affects conflicts and performance.14 The study found that hierarchies work best when there is no confusion about who defers to whom. Teams with a clear chain of command (clarity and agreement about who defers to whom) were less likely to get bogged down in conflicts and stalemates than teams where influence was more cyclical. Indeed, a study of 62 pharmaceutical research and development teams found that teams with a clear hierarchy were more involved in the learning process that is central to innovating.15
Create a performance-based culture. A clear chain of command means that some team members will defer to others who are “higher up” in terms of status or respect. Many of us know what it is like to be in a situation where incompetent people are running the show. So, how can teams improve the chance that members with the most relevant knowledge are higher up in the hierarchy?
The key is getting teams to identify the members who possess real knowledge. This is often easier said than done, in part because we tend to have implicit biases about the characteristics or backgrounds that signal expertise. For example, a study at a high-technology Fortune 100 company found that, not surprisingly, teams perform better when their more expert members rank higher in the team’s hierarchy. That study also found, however, that teams often pay attention to the wrong things as they sort out who will have more or less influence — things like gender or ethnicity rather than training, expertise, and experience.16 Because these biases operate below our conscious awareness, they are difficult to correct unless more accurate expertise signals are emphasized.
One way to counter the biases is to create a performance-based culture, where performance is measured, publicized, and celebrated. Studies suggest that when people have opportunities to demonstrate what they can do, experts tend to rise to the top.17 Moreover, when credible evidence shows that less vocal members are better qualified to make informed decisions, groups will limit the influence of bombastic pseudo-experts.18 In other words, groups listen to experts when they can identify who the experts are. Group hierarchies in performance-based cultures are more likely to be based on expertise and less likely to be based on physical characteristics.
Use team feedback. Another way to improve the way hierarchies function is to encourage those at the top to act in ways that support the group rather than acting in their own best interest. How do you make sure this happens? One of the authors worked with a group of researchers from the Netherlands to examine this question in 46 teams from a wide range of industries including banking, medicine, software development, and management consulting.19 The study found that, contrary to what previous research had shown,20 power differences within teams did not necessarily hurt team learning. In fact, hierarchies promoted learning and performance when goals and feedback were group-oriented, but they stifled learning and performance when goals and feedback were individually oriented.
Group goals and feedback encourage higher-ups to use their advantaged position to encourage members to collaborate through information sharing, experimentation, and reflection. Individual goals and feedback keep people focused on their own tasks and outcomes.