Summary: Developing Team Collective Intelligence

Summary: Developing Team Collective Intelligence

This is a summary based on:

• Team Collective Intelligence: Developing and testing a knowledge integration model
• Team Collective Intelligence: Developing and testing a digital team intervention for knowledge integration

Introduction

Human history is built on people working together. Our story tells of the free-roaming herd animals, that learned to cultivate, then to industrialize and finally to connect the entire world into a complex web for prosper and survival. Yet, our understanding of the modern world is built on the idea of a society, comprised of organizations, that in turn are collections of individual jobs. Amiss in the equation is the notion that almost everything is either done, or controlled by, small groups of people working together. More or less everything in our modern society is therefore at some point dependent on the quality of how people cooperate. Countries are not governed by a person, nor by institutions. They are governed by people and depend on the way they cooperate. Health care, schools, private enterprises, all are a continuous result of how well small groups of people manage to cooperate. It may seem like a paradox, but as complexity in society grows, our interest in how we cooperate at this level of small groups will increase. We will become more dependent on distributed decision making, and because of that we need intelligence rather than efficiency in our organizations. And the source of this intelligence must be the small teams, where the knowledge distributed on different experts in practice must be assembled. So, our interest in understanding both intelligence and people cooperating at group level will increase. What we here call – collective intelligence. Our study is based on this assumption. We argue that this development generates two fundamental questions. First, what is team intelligence and how does it work? Second, can team collective intelligence be systematically developed? These are the questions this report tries to answer.  

TEAMS – THE KEY TO COORDINATING INTELLIGENCE

The reason teams will play an increasingly critical role in organizations is continuously higher levels of specialization and knowledge distribution. Those are the means with which we develop our organizations and society. Though it is neither intended nor wanted, this development increases complexity, and the more complex our organizations and society are, the more intelligence we need. Due to knowledge distribution, this intelligence is less often individual. We want the intelligence people can create together. It is in teams we need the ability to interpret situations, define problems, creatively identify potential solutions, align them with purpose, make decisions, form efficient division of work, and then ongoingly coordinate, reflect, reevaluate, change and make adjustments.

It should go without saying that this cannot be easy for a team. Especially since different competencies give different perspectives and priorities, and any complex situation is full of alternative interpretations. We should think of teamwork as something difficult. As something we need to work on – hard – to achieve. But for some reason this is not the case. Instead, in most cases we expect people to do advanced teamwork together without any particular preparation or support. We either assume that all people can work together, or that the teamwork we get is what we can get. Yet, it has long been known that there is a large variation in how teams perform. Teams can do far better, but also far worse, than single individuals. Overall, the average level of team performance seems to be just slightly above what the average individual can do on her own (Kerr & Tindale, 2004; Allen & Hecht, 2004; Witte & Engelhardt, 2004). A result that probably surprise many readers.

So, what could we do instead? A large part of this variation in team performance is explained by factors within the teams themselves, such as: the patterns of communication (Mohammed & Dumville, 2001), psychological safety (Edmondson, 1999), defensive behaviors (Argyris & Schön, 1996), implicit coordination (Rico et al, 2008), heedful interrelating (Weick & Roberts, 1993) etc. Phenomena that are a result of the interaction and relations between team members. This means that the process of integrating knowledge in teams is under no ones’ control.  Not the organization, nor the manager or any other single team member, but it is something created by all the involved individuals. This means that it is something that the team itself must work on.

Against this background, a key question for organizations will be how they can support their teams and their members to develop their ability to integrate knowledge.

WHAT IS TEAM INTELLIGENCE AND HOW DOES IT WORK?

DEFINING COLLECTIVE INTELLIGENCE

To figure this out, we need to start by understanding what intelligence is, and in particular, collective intelligence at team level. Intelligence is often understood as an individual’s capability of abstract reasoning. So defined, it has been measured since the early 1900s based on the discovery of a statistical factor by Charles Spearman, the so-called g-factor. This is a measure of a general ability of cognitive reasoning, but what is interesting is how it has turned out to be amongst the most accurate (in terms reliability and validity) of all psychological measures and assessments (Gottfredson, 1997). The g-factor (measured as IQ) has repeatedly been strongly related to educational, occupational, economic, and social outcomes. Its relation to the welfare and performance of individuals in education and work life is very strong. It is moderate but robustly related to social competence, and modestly but consistently related to law-abidingness. A high IQ is an advantage in life because living requires constant reasoning and decision-making. The advantage increases as life become more complex, as in novel, ambiguous, changing, unpredictable, multifaceted and disorganized. A high IQ is no guarantee for success in life, but life seems to favor individuals with high IQ (Gottfredson, 1997, Malone & Bernstein, 2015). We can therefore expect this to be true for teams in complex organizations as well.  

However, IQ is not measuring whether actions are intelligent, but only a capability of drawing conclusions from abstract reasoning. If we are interested in what acting intelligently in a certain situation is, we need to look at other definitions of intelligence. A generally accepted definition does not exist, but by using a number of different definitions we have compiled the following definition of intelligence for this study.

Acting intelligently is using a cognitive capability to interpret the environment in a certain situation, by which values, priorities, goals and preferred outcomes are defined, and used to plan and govern mindful actions.

As with intelligence, the definition of collective intelligence has so far eluded mankind. Some examples are: “A collective decision capability at least as good as or better than any single member of the group” (Hiltz & Turoff, 1978); “A form of universally distributed intelligence, constantly enhanced, coordinated in real time, and resulting in the effective mobilization of skills” (Levy, 1994); “The ability of a group to find more or better solutions than … would be found by its members working individually” (Heylighen, 1999)

If we use our definition of intelligent action above and look at this as a dynamic process enacted by intelligent individuals, we can identify two steps that would need to be collective. The first step would be integrating individual cognitive processes in the interpretation of the situation and definition of desirable outcomes, i.e. thinking together. Second, in the definition of intelligence we use here, it is required that action follows upon, and is governed by, this thinking. In the case of collective intelligence, this must be collective action. Since only individuals can act, what would be collective in a system of coordinated individual actions is the shared understanding, mental model or representation, from which their individual actions are coordinated. It follows logically that the potential to coordinate both thinking and actions can be improved, the more of the situation, priorities, work division etc. that are shared, and the broader and deeper the understanding of the situation and task is. However, it is only a potential. If only a small part of the team members mental models is shared, a lot of individual thinking may not be understood of the other members, and few individual actions can be efficiently coordinated. Even with a larger and more developed representation, there is still only a potential. A large and shared representation will not be any more useful than the individuals chose to engage in it through their actions. However, in this process of coordinating mindful actions, what can be collective is not the action, but the shared representation or mental model (Mohammed & Dumville, 2001).

Based on our definition of intelligent action above, and what would be collective in that process, we define collective intelligence at team-level as:

A process in which a group of given individuals integrate their individual knowledge resources in a learning process, to interpret their environment in a certain situation, defining values, priorities, goals, preferred outcomes and plans into a shared representation, used to govern and coordinate their mindful individual actions.

DEVELOPING A MODEL OF KNOWLEDGE INTEGRATION

Donald Schön (1983) has described how individual practitioners practice knowledge as they deal with complex and ambiguous problems. The knowledge worker engages in interpreting situations, but since there are alternative perspectives on everything, any problem solving needs to start with a problem setting. A framing of how to view the situation and problem at hand. It is always possible to define the same situation and problem differently. Think of the well-used example of having a half full or half empty glass of water. The solution to every problem is contingent on the way the problem is defined. If the problem setting changes, the possible solutions change. So, the practitioner uses his experience to try out different problem settings, and within them, different solutions. S/he experiments in his/her mind, having the situation “talk back”, as Schön calls it. If the outcome is not satisfying, the practitioner may try other solutions, but may also “go back” to the problem setting and try another one by using new basic assumptions in his/her view of the problem. This process of reflection-in-action, as Schön calls it, is practicing knowledge in the situation or moment and alternating between thinking and doing, as well as taking things for granted or opening them up for questioning. It is this process of reflection-in-action that we want to understand how it works when we are a group of people working together, having different skills.

When a team face a problem, the collective process of reflection-in-action will be much more difficult. Instead of starting to mix thinking and actions, the team has to start communicating, to learn from each other. In a team we need to think but must also channel our thoughts into words for others to understand, which is difficult and time-consuming. At the same time, we need to try to understand what other people are saying to us. Maybe we don’t like what we hear and get upset. Maybe we misunderstand. Regardless, we need to agree and make decisions. And again, we need to turn the agreements into words that we can agree upon, to share a representation. Words that however much time we spend formulating them, in details can be interpreted in many different ways. As a team, we now need to divide work between us, and then act in a coordinated manner. Maybe call for a collective pause, if something happens that we need to consider together. Having reached this far, we have just completed the first iteration of thinking and acting. As Schön points out, the reflective practitioner may repeat this iteration many times to find the wanted solution. S/he may go back and alter the assumptions totally, to find a new problem setting to work from, and then go back again to the old one, just “testing”. Imagine this process of altering a basic assumption back and forth in a team. It is easy to realize why this calls for strong and sound relations between the team members. Diversity is the blessing and the curse of teams. The paradox is that the entire value of thinking together lays in thinking differently, but differences also make it hard to be a team.

Describing collective reflection-in-action is describing the conditions and components of knowledge integration in a human system. A system being human also means that it is subject to the challenges of people interrelating. A model of knowledge integration needs to consider that. The first challenge of people interrelating is that individuals are not necessarily prepared to learn from each other, depending on their relations to each other. On the contrary, the standard behavior of individuals under uncertainty is to be defensive, and a way of staying in control is by maintaining your own view of things (Argyris & Schön, 1996). This protects individuals from doubts about their own ability, as well as having to care about being doubted by others, thereby upholding their ideas of authority and legitimacy. Being unwilling to think aloud in front of each other help us avoid exposure, but by holding back thoughts and not speaking up also follows collective risks. As in group think for example, when the urge to avoid embarrassment and public disagreements leads to collective decisions that no one believes in or wants. Such behavior can even become routine, as in “skilled incompetence” (Argyris, 1990) or “functional stupidity” (Alvesson & Spicer, 2012). People practice routine behavior (which is a skill) to produce organizational results that they do not intend (incompetence), by agreeing on descriptions and decisions that everyone knows are wrong, or at least overly simplified (stupidity), because it allows them to continue working together smoothly (functional), even if this is at the cost of acting unintelligently.

The second challenge is that teamwork is never better than the team members make it, or how well they integrate into a team. Collective capacity is the continuous result of team members’ shared understanding of the situation, but also of their effort to coordinate and interrelate. To act and integrate heedfully, creating high levels of knowledge integration, is no more an obvious behavior of teams than learning. Such behavior also depends on social factors such as commitment, motivation, relations to other members etc. (Weick & Roberts, 1993, Meyer & Herschovitch, 2001). This means that knowledge integration relies on team relations, as in the social context and the climate of the team, but also on the team members’ self-management to integrate into a team. This represents two different perspectives on the conditions of successful collective efforts. One perspective is that the qualities and properties of the team govern how people act. For example, if the team represents a psychologically safe environment, people will be prepared to learn. The other perspective draws attention to how the character of individual actions forms the collective. For example, the heed with which the team members interrelate creates the quality in which the collective system is enacted. With that view, psychological safety of a team is the result of how the individuals decide to treat each other. Both perspectives are relevant, but they tend to lead to opposite ideas of how the different variables are related. Must the team be psychologically safe for team members to heedfully reflect or must team members heedfully reflect to make the team psychologically safe? The answer is that both must be true. Therefore, a model of knowledge integration should acknowledge both orders of cause and effect.

One way of describing this is by the concepts of processes and emergent states (Kozlowski & Ilgen, 2006). Processes capture how team members combine their individual resources, coordinate knowledge, skills, and efforts to resolve task demands. A team capability, like effectiveness, is an emergent result of these processes that unfolds over time. However, repeated interactions among individuals that constitute processes, tend to regularize and crystallize into shared structures and emergent states, that then serve to guide subsequent process interactions. Process begets structure, which in turn guides process. Hence, cognitive structures and emergent states can be echoes of repeated process interactions and, therefore, indicative of the nature and quality of dynamic team processes.

With our definition of collectively intelligent acting, and the concepts of processes and emergent states, we have the components to formulate a model of knowledge integration at team level. In the first dimension of the model, we use the two fundamental forms of knowledge integration, as in:

1. thinking together or the analytical process of dealing with the environment. We call this a learning process, as the individuals exchange their individual thinking with the purpose of altering and integrating each other’s knowledge, creating new knowledge. It is dependent on an emergent state of the team which make individuals prepared to change their current way of thinking, relations. Learning is also the result of integrating individual reflections.
2. acting together as in enacting a human system. Collective actions are individual actions being coordinated. The emergent state enabling the coordination of knowledge is a shared representation. The actual coordination of actions we call integration behavior. The second dimension of the model is what constitutes a collective phenomenon, which gives us team knowledge integration as the result of:
3. an ongoing process that represents the reflections and other integration actions as the team members enacts the team. It varies with the care and attention individuals devote to these interrelating actions.
4. emergent states, that are the repeated and crystalized patterns of team members’ behavior into relations and representations. Patterns that make it possible to assign attributes to teams, based on the assumption that the behavior will be repeated, and even inherited if we change team members.

DOES THE MODEL WORK?

To test the model, we developed a team survey. To complement the team assessments, we also used the input of an external observer, a direct or indirect manager with knowledge about the team’s performance. Data were collected during the fall of 2018 and spring 2019. The teams in the study were selected from 22 different public and private organizations operating in Sweden. The organizations were selected based on contacts trough the researchers or the research sponsor (Influence AB). Selection criteria for the teams were e.g. a complex environment, a task that could be ambiguous, the main resources to manage this should be the team members and that the knowledge needed for the task is distributed amongst the team members so that they have different expertise, but are all needed in the task. Examples of teams were work groups, project teams and management teams. The response rate was high, on average 98% for the teams and 88% of the observers.

The study confirms the model of knowledge integration and its effect on team performance. Teams that had high levels of representation , relation, reflection and integration also had higher levels of performance.

Looking more closely at how the model works, the study shows that most important for performance are the Representations and Relations of the team. The processes of learning and integration are important, but to build representation and relation. It is through increasing the psychological safety of the team, and the shared understanding of purpose and meaning of the task, that the teams increase their performance. Reflection that is not turned into shared understandings of purpose or meaning, or integration behavior that do not lead to developed psychological safety, does not drive performance.

It is of interest that we here get the opposite relation between Reflection (learning behavior) and Relation (Psychological safety)than Amy Edmondson, who in 1999 first looked at the relation of these to concepts. In our result, team reflection contributes to team members feeling of safety in the team, instead of the reversed order. Yet, we use the same measures as Edmondson developed. Our interpretation is that representation and relation are quality indicators of the team processes. When the processes are of high quality, they crystalize into high quality emergent states. The emergent states in turn, support higher levels of the processes in a reciprocal effect. Reflections become more advanced as 1) relations support more thoughts being brought forward to the other team members, and 2) because the representation contains more information, more perspectives, more ideas on cause and effect, more possible scenarios and alternatives for actions etc. Integration behavior increase when 1) the whole picture of the task is better understood, and 2) initiatives for team development and resolving conflicts are increased and eased by more developed relations.

Our result also means that we can have reflection and integration that are not good enough to increase team performance. Developing the representation and relation in teams should therefore be an end in itself. Teams needs to continue reflection and integration until the result is a more developed and shared representation, and a psychologically safer environment. If they do, it will release more of already existing capacity of teams, and it will make them more ready to act intelligently as their task unfolds.

In the study we also tested some well-established antecedents of successful teamwork: interdependence, member stability, team boundaries, team members level of emotional intelligence, and team members trust in each other’s competence. These antecedents turned out to have surprisingly little influence on the knowledge integration variables. This result may suggest that teams of today are less sensitive of e.g. team boundaries and stability. This, in turn, may be because individuals are used to working in many teams in parallel, as well as changing teams often. Under such circumstances, understanding the essence of quickly establishing high quality knowledge integration will be even more important.

CAN TEAM COLLECTIVE INTELLIGENCE BE DEVELOPED?

We think this is a critical question for most organizations. There are two reasons for that. First, what organizations will want from teams – more for every day of increasing complexity – is their ability to enable rapid, flexible, and adaptive responses to the unexpected. They want the teams’ intelligence. However, there is no guarantee that teams will integrate their knowledge and act intelligently. On the contrary, social processes and defensive behaviors create unintelligent behavior, like for example group think (Janis, 1991), shared information bias (Stasser & Titus, 1985), skilled incompetence (Argyris, 1990) and functional stupidity (Alvesson & Spicer, 2012). Studies show us that teams rarely reach what would be an expected level of performance, i.e. outperforming the most capable individual of the team or a comparable nominal team (Kerr & Tinsdale, 2004). In brainstorming, teams are normally outperformed by nominal teams, as the latter produce more ideas, of higher quality, in the same time period. Teams working on well-structured problems perform on average at the same level as the second-best individual of the same team does on her own (Hollingshead, 1996; cited in Witte & Engelhardt, 2004). If problems are complex and unstructured, the average team doesn’t show better results than randomized interventions (Witte & Engelhardt, 2004). Therefore, we can expect the variation of teams’ knowledge integration to be high, and that organizations will benefit it there is a way to increase it.

Second, even today few organizations work systematically with team effectiveness. Most organizations remain in the concept of work in large organizations that developed in the late 19^th and early 20^th century, focusing on designing organizations through their structures (like charts and systems), and to “fill” roles with individuals “matching” the demanded skills. This is not necessarily wrong, but it is lacking the awareness that average team performance level is an organizational performance factor. All organizations could develop their performance if they systematically increased knowledge integration at team level.

However, teamwork can be supported or ruined by any individual in the team. This implies that the means of working with a capability of a specific team should lays with the team itself, and the individuals of the team. What is missing in most organizations is therefore the systematic procedures, at organizational level, to support development teams.

THE CASE FOR DEVELOPING A DIGITAL TEAM INTERVENTION

Teams need to be nurtured and supported by their organizations (Klein et al, 2009). Doing so should pay off. Macy and Izumi (1993) analyzed 131 studies of organizational change and found that interventions with the largest effects on financial measures of organizational performance were team-development interventions. Research has also consistently confirmed that teams can be worked with and developed by different kinds of interventions, e.g. team training (Salas et al, 2008), team building (Klein et al, 2009) and team debriefs (Tannenbaum & Cerasoli, 2013). Team training and team building have positive relationships with team process and performance outcomes, but are often expensive and time-consuming interventions, both to develop and conduct effectively (Klein et al, 2009; Salas et al., 2008). Furthermore, research finds that less than 10% of competency acquisition in organizational settings occurs in such forms of formal training (Tannenbaum, 1997). So ideally, organizations should instead find ways where their employees can learn from experience, and during ongoing operations (Eddy, Tannenbaum & Mathieu, 2013).

Against this background, team debrief seems to be a powerful intervention. It is based on teams learning and developing from their own work and experience. Research confirms that teams that conduct debriefs outperform those that do not. On average, debriefs improve effectiveness over control groups by approximately 25% (Tannenbaum & Cerasoli, 2013). Villado & Arthur (2013) showed that teams using debriefs had higher levels of team performance, team efficacy, openness of communication, and cohesion. Debriefs improve teams by helping the team members to collectively make sense of their environment, to develop shared visions and to decide on how to proceed in the future. For this capacity, there should be four defining elements of debriefs:

• The learning should be active, as opposed to passive. Debriefs are fundamentally a form of emergent self-learning in which individuals and teams use an iterative process of reflection and planning to improve performance.
• The intent should be developmental, as opposed to administrative: Debriefs are intended primarily to serve developmental purposes rather than evaluative or judgmental purposes.
• They analyze should be on specific, as opposed to general, events: Part of what defines a debrief is a focus on specific activities, episodes, and events rather than on general performance or competencies. Reflecting on specific past events provides a different degree of focus and allows for a deeper examination of particular actions, cue-strategy associations, underlying cognitions, and so on, than does a general discussion of overall performance.
• They should use multiple, as opposed to single, information sources: Multiple sources increase the coverage and yield more diverse and complete accounts of a recent performance episode.

However, simply providing teams with an opportunity to debrief does not necessarily facilitate shared team cognition and developed team processes (Smith-Jentsch et al, 2008; Eddy, Tannenbaum & Mathieu, 2013). Instead, guided team self-correction is a debriefing strategy where members use a structure, or model, to guide them in what topics do discuss and how to do so constructively. Smith-Jentsch et al (2008) investigated the effects of using an empirically derived expert model of teamwork as the organizing framework. In two studies this expert model was used to structure the process of guided team self-correction. Results indicate that teams which debriefed using the expert model developed more accurate mental models of teamwork, demonstrated better teamwork processes and more effective outcomes.

Finally, prior research has confirmed that facilitated team debriefs are effective, but few have examined how to enable teams to conduct their own debriefs. Eddy, Tannenbaum & Mathieu (2013) compared two team-led debriefing techniques: an unguided debrief and a guided debrief designed to incorporate lessons learned from prior debriefs. Results suggest that the team-led guided debrief intervention resulted in superior team processes as compared to the unguided debriefing method. Team processes, in turn, related significantly to greater team performance and increased individual readiness for teamwork and enthusiasm for teaming.

The objective of this study is therefore to develop and test a form of organizational support tool that should be possible to use throughout the entire organization. It is aimed for teams and other kinds of micro-systems, with the purpose of them developing higher levels of knowledge integration. For this reason, the support will be developed as a team intervention, that is generally applicable, team-led and self-guiding. We therefore chose to base it on a digital application, consisting of two parts:

1. A training module introducing a model of knowledge integration in knowledge intensive environments, that can be used to support guided team self-correction;
2. A debrief module that supports a team-led debrief process during a period of time.

DEVELOPING A DIGITAL APPLICATION FOR TEAM COLLECTIVE INTELLIGENCE

Based on the key debriefing functions and known limitations in team information processing, Eddy, Tannenbaum & Mathieu (2013) suggest that an effective team-led debrief approach should include the following five features:

• (a) allow team members to reflect independently and anonymously, for psychological safety and to avoid being influenced by the most vocal team member;
• (b) ensure all team members provide input to enhance their sense of ownership and capture all perspectives;
• c) focus attention on teamwork and not just taskwork, because teamwork also drives team effectiveness and groups tend not to discuss it;
• (d) guide the team to discuss divergent or high priority needs early in the debrief and not simply areas of agreement or comfortable topics; and
• (e) lead to the formation of future-looking action plans and agreements.

We decided to develop an application that consists of two integrated parts. The first part is a digital training module in which team members can learn the knowledge integration model, its theory and references. The second part is a digitally guided process of debriefing, designed for approximately two months of use, with a frequency of one debrief per week, giving 8 debrief sessions in total. The application uses the model of knowledge integration presented in figure 2 as the expert model. The model was also used to structure the design of the intervention and thus the teams’ discussions, feedback and common planning.

The purpose of the training module is to give the teams an understanding of the organizational background and why increasing complexity will need more of teamwork and knowledge integration. It also teaches the specific background and theories behind the model. The assumption is that understanding the basic motives and theories of knowledge integration will help the teams perform, but also adjust the debrief processes, according to their unique conditions and preferences. The training module was developed based on 10 video lessons ranging from 3 to 10 minutes. The same content was available in text as well as in presentation slides, within the application. Some videos were also integrated in the debrief processes, while the others were to be viewed when preferred by the team members. The debrief process was designed for eight debrief session of approximately one hour each (15 min pre-meeting, 45 min meeting) over a period of two months. This was deemed sufficient for changes in processes and performance to be detectable by the measures used. A pilot test of a debrief prototype in two multinational tech companies in Stockholm showed that the content of the debriefs needed to vary and develop over time in order to attract sufficient attention. The design of eight debrief sessions was only due to limiting time for the research study. The concept of debrief sessions could be continuous. In the last of the debrief sessions the teams were asked to make their own plans for continuing with future sessions.

The team members were instructed to use a pre-meeting module before each debrief, where they individually were asked to answer questions or fill out shorter surveys. This information was compiled for the debrief meetings, where the individual answers were presented, but anonymized. Debrief meetings could be face to face, but also include on-line participants. During the meetings the participants were instructed by the application to perform different actions. For example: to enter individual answers to questions, watch videos or have open discussions and enter common answers. During the meeting all comments entered in the application were anonymous as well. The eight debrief sessions varied in focus and format, following the dimensions of the knowledge integration model. For an overview of debrief content, see Table 1.

We tested the application on 50 work- and management teams in 22 knowledge intensive organizations. We chose a research design where we compare the processes and results of experiment teams with unmanipulated control teams. The teams have the same type of task and are acting in the same type of environments and organizations. For this we asked the participating organizations to provide several teams from which the research team could randomly assigned the teams to experiment or control treatment. Control groups were provided with and used the application after their period as control groups. To evaluate the use of the application and the debrief process a survey was developed . The survey was designed for self-assessment by the teams before and after intervention. In addition, each team had an external observer, a direct or indirect manager with knowledge about the team’s performance, that also answered a survey before and after intervention. Response rate per team was generally high, ranging from 76-98% between the three occasions at which the survey was distributed .

DOES THE APPLICATION WORK?

Yes, the study shows that teams using the application significantly improved compared to teams that don’t. Regarding performance the result is clear, both when measured by the teams and by the external observers. If we use the average of their assessments, Average performance , the relative increase was 14,83% for experiment teams, compared to the control groups. If we look at just the external observers’ assessment, Observer team performance, the relative increase was 21,54%, which is on a similar level as previous studies of facilitated team debriefs, where the average improvement was 20 to 25% (Tannenbaum & Cerasoli, 2013). The observers rated, on average, a higher increase in development than the teams themselves.

A surprising result from the study was that the control teams decreased in performance during the period, -4,61% in Average performance and -6,46% in observer assessed performance. Even the self-assessed team-performance decrease by -2,10%. This is a somewhat surprising result. The traditional expectation is that teams working together over time improve, both in teamwork and performance. However, group development theories (Tuckman, 1965; Wheelan, 2005) do not say that teams automatically develop their performance positively over time. The different phases of team development tend to follow from the needs of the team members rather than the need of the task. Only at the later stages (higher levels) of group development can we expect team performance to increase. The early stages are characterized by the teams’ attention being spent on establishing necessary structures, and at the same time trying to develop relations. If left to themselves, these are often inefficient processes in teams. Wheelan (2005:97) found that in the early stages (1 and 2 in Wheelan’s model), teams spent on average 38% of communication on task related statements, while approximately 55% had to be spent on relation-driven fight-flight-dependency statements . For comparison, teams at stage four, the highest level of Wheelan’s model, spent 76% of their discussions on the task. Inefficient communication processes, especially at the second stage (called “storming” by Tuckman and “Counterdependency and Fight” by Wheelan) may result in team performance developing negatively. An explanation to why this happens to the control teams could therefore be that, on average, they are at the early stages (lower levels) of group development. In complex environments with high levels of knowledge distribution, we could expect that the average team member is part of many teams at the same time and that team membership often changes. Such conditions could result in many teams not having the prerequisites (time and focus) of reaching higher levels of group development, without support. Instead, they get stuck in the earlier stages of team development.

Regarding knowledge integration, the application seems to develop only part of these processes and emergent states. All parts of the model increase more in the experimental teams than in the control teams, but not all changes are significant. The debrief-application give the teams a significant increase in Relations and Reflection. It also gives them a significant increase in Representation as in who knows what on the team (Expertise location). It does not significantly develop a clear purpose and objectives or a whole and meaningful task, more than the control teams. The application also increases the team members individual initiatives of team development, but there is a mixed result in increasing initiatives for conflict management, and it does not significantly increase the team members initiatives to share their knowledge.

In general, it seems as if the intervention is more effective for team-oriented development, rather than task-oriented development. This may be because the need for task-oriented development is less. Most teams already have a high value on Representation as meaning and Integration as the team members sharing their knowledge. These two variables have the highest mean values of the study before the intervention. It could also be that this particular type of debrief, developed for this study, is not doing good job in supporting the processes of task development. Instead, the value of this debrief stem from the team members getting to know each other better, developing relations and trust, and increasing listening and learning. If so, developing a more task supportive intervention could lead to even higher increase in knowledge integration and performance.

To summarize, the hypothesis that teams can be developed using an application is supported. There is a clear improvement in performance. There is mixed support for the application developing knowledge integration. All variables of the model increase more in the experiment teams than in the control teams, but not all changes are significant. Our conclusion is that the performance increase caused by this application is mainly due to it developing Reflection, Relations and Integration. The latter in the form of team members increasing their individual initiatives for team development.

WHICH TEAMS DEVELOP – AND WHICH DON’T?

In addition to confirming the effects of the debrief-application, we analyzed the data to see if we could understand more of which teams that benefit from the intervention (and which don’t), and in what way. To get an overview, we started by sorting the teams into three groups, showing high, medium and low development of their Average performance. In table 2a-c we ranked and grouped the teams based on: a) how performance developed during the intervention (i.e. to see what characterize the teams that developed the most/least) b) performance before intervention (i.e. to see what happened to high vs. low performers due to the intervention and c) performance after intervention (i.e. to understand the high vs. low performers after intervention). To compare these groups we use the mean value of Average performance of each group, before and after intervention.

In table 2a, we see that a third of the sample, those with the highest level of development, increase their performance with almost a full unit on average (+0,97). At the same time, another third, the lowest developed didn’t develop at all (+0,02). This indicates a rather large spread in development due to the intervention.

In table 2b, we see that the teams developing the most are the low performers before the intervention. This third develops on average with +0,75 units. The high performers before intervention develops the least, but still has a positive development of +0,25 units. This means that the intervention evens out performance among the teams. The difference between these groups of high and low performers before intervention was on average 1,25 units. After the intervention, it is 0,74 units.

That the intervention has a positive effect on both high and low performing teams is confirmed in table 2c, sorting the teams on performance after the intervention. This allow us to use a more organizational perspective. We can now compare, for example, the low performers after the intervention with the low performers before intervention, although this does not have to be the same teams. The lowest performance level of teams in the sample increase on average with +0,62. We can also see that using this perspective, the intervention is efficient for the high performing third as well. They have an increase of +0,45. This means that from an organizational point of view, we have an increase at all performance levels.

A puzzling result is that the lowest developing third of the sample didn’t develop at all on average (+0,02). A third of the sample had no development, on average. And yet we see that teams at all levels have a positive effect of the intervention. This makes us suspect that some teams developed negatively.  

How do the teams develop? If we look at table 5a, where the groups are sorted after how much they developed, we can see that the application has rather a dramatic effect. After the intervention, the third most developing teams catch up in performance with the third lowest developing teams. Before the intervention, the difference between these groups were +0,81 units in favor of the third lowest developing teams. After the intervention, the difference is +0,13 in favor of the teams that developed the most. This development is true for all knowledge integration variables as well, where the high developed teams either catch up, or pass, the low developing teams.

THE MYSTERY OF THE “SECOND BEST”

The analysis above indicated that some teams may have had a negative development due to the intervention. When we look into this, it turns out that seven teams amongst the experiment teams develop negatively in performance.  We therefore conducted a more detailed analysis to find out more about these teams. In Figure 4, all teams are ranked using performance before and after the intervention. Between the two ranking tables we first plotted the development of the teams developing negatively. Then we plotted the seven teams that developed the most. In this case the seven team that developed more than 1 unit. From this analysis we found a surprise. Of the teams developing the most, all but one came from the low performing third. This is not so surprising since they also have more potential for development. What was surprising was that five out of seven of the teams developing negatively belonged to the top performing third. If there are teams that do not benefit from the intervention, we would expect to find them to be either randomly distributed in the sample (if there is no relation between negative development and performance before intervention) or amongst the low performers, where we would find the lowest levels of psychological safety, learning behavior etc. The group of negatively developing teams seemed to be clustered in the top performing half, but just below the top performers. We therefore call them the “second best”.

Does this mean that high performing teams have problems with the intervention? In table 5a we saw that the lowest developing teams were high performers before intervention, and on average, this third of the sample did not have any performance development at all. To test this hypothesis, we added to our analysis the development of the seven highest performing teams before intervention, the top seven in the ranking. Here we found a second surprise. All of these teams remain amongst the top ten after the intervention. They have an average development of almost 0,4 units in performance. It means that they are on par with the average development of the whole sample, 0,49 units. This tells us that high performance in itself is not a barrier to additional development and to benefit from the self-guiding team debrief.

So, why do the particular cluster “the second best” develop negatively? To see what sets them apart we compared them with the other teams immediately before and after them in the ranking. First with the seven top performing teams before intervention (“the best”). Then we also add the group of seven teams that follow closest to the second best (“the rest”). Together they form the top performing teams before the intervention , and we can compare how these teams differed from the beginning, before intervention, and how they developed integration variables compared to the “second best”, but we also find that the “rest” have higher values than the “second best”. However, looking at the ranking of teams based on performance level before intervention, most of the “rest” teams follow below the “second best teams” in the list. We had therefore expected “the rest” to have lower levels of knowledge integration variables.

Looking at development, the seven best teams develop their performance with 0,40 units on average. The seven “rest” teams develop on average a bit more, approximately 0,45 units. Surrounded by these groups of teams, the “second best” had a negative development of -0,27 units. The development of knowledge integration variables follows the same pattern. We can see that the “second best” actually develop knowledge integration variables negatively or not at all. The team members initiatives to share their knowledge goes down with 0,48 units. The perceived wholeness and meaningfulness of the task goes down with 0,30 units.

Therefore, being able to identify these “second best” teams seems important. It is clear that the intervention is not just ineffective for them, it seems to affect them negatively. To search for explanation and possible ways of identifying the “second best” syndrome, we did an additional analysis of the teams’ assessment of themselves. We compared the self-assessments of performance made by the teams with those of their observers, see Table 8.

From this we find that the “best” performing teams continuously rate themselves lower than their observers, (-0,16 before the intervention and -0,34 after the intervention), which indicates that they can look critically at their ability. If we look at the “rest” teams, they rate themselves better than their observer before intervention (0,25 units) but end up at the same level (0,04 units) after intervention. This can be interpreted as some form of adjustment or calibration in relation to the observers. Their ability to assess themselves is aligned with their observers. The “second best” teams, however, rate themselves continuously higher than their observers, 0,36 units higher both before and after intervention.

They seem to be less critical of their own performance and show no alignment in their assessments in relation to their observers. The sample is small so these results should be interpreted with care.

DISCUSSION FROM COMPARING GROUPS OF TEAMS

That the teams developing the most are those with the lowest performance level may not be surprising, considering that this gives them the most potential for development. Though we were not sure of this result beforehand. We expected these of being low on psychological safety, and therefore be low on learning behavior, which they also were. But instead, that seems to be were the application does its job the best. It develops learning behavior and trustful relations in teams, but what it develops the most is team members individual initiatives to support and develop their team.

The most important conclusion from comparing the groups of teams are that all performance levels seem to benefit from the intervention. But not all teams do. Instead, there is a special cluster of teams that develop negatively, the “second best”. All teams, except these, follow some form of linear improvement due to the intervention. They all have a positive development, but inversely proportional to their initial performance level. So, something is setting the “second best” apart from this pattern. It is the level of their knowledge integration that is lower than expected from their performance level. In particular, they have less understanding of their task, less individual initiatives by team members to exchange knowledge and feel less safe with the team for personal risk taking, compared to other teams at the same performance level. Their initiatives for knowledge sharing and psychological safety are actually below the average of the entire sample. In addition, they are also less prone to question their own performance, both before and after the intervention.

These teams show the indicators of the internal dynamics that Argyris & Schön (1996) call defensive behaviors. That is individuals, when lacking trust in the relations of a team, will tend to uphold their individual interpretations of what is going on. That is their way of staying in as much “unilateral control” as possible. They avoid discussions on key assumptions of their view on things. At the same time, they uphold a polite behavior, avoiding anything that could lead to “unpleasantness”. But the consequence of this is that they will avoid any deeper analysis of things. They will stay at saying and discussing only what is expected and acceptable. The effect will be low levels of learning and the team members will stay in different representations, as they keep their individual interpretations intact. In a study of consulting teams, Argyris (1990, 1991) found that this lack of ability to create a learning climate was a consequence of individuals being used to being high performers. They were not accustomed to failing and had therefore develop a fear for looking at their own mistakes. They were prepared to question just about anything but their weaknesses and faults. If they ever did, proven beyond doubt of a failure, they instead exaggerate everything negative in the incident (especially themselves). They “zoomed into a doom loop”. This was “smart people, unable to learn”, as Argyris called them. If this hypothesis holds, the intervention may actually be negative for these teams, since it will force them to try to discuss things that they want to avoid. They will sense their own reluctance and doing so will bring forth a feeling of uncertainty. The defensive behavior will become apparent, but without the team having the capacity to talk openly about it. Instead, if they follow the patterns of the teams in Argyris study (1990), they will turn against the debrief-process and the application.

But if this is true of the “second best”, why doesn’t the “best” and the “rest” top performing teams suffer from the same defensive behaviors? The difference could be explained by them having reached higher levels in the knowledge integration model. They have developed a deeper understanding of the purpose of their task (Representation) and developed the relations within the team. This makes them more prepared to look at their own mistakes and weaknesses. For the “second best” teams, the combination of having high performance, but being “underdeveloped” from a knowledge integration point of view, creates a “catch 22” situation. They overrate their performance in relation to the observers, which makes them see less need to develop. As the top performing team show, they could also learn more from their situation, but they do not have the psychological safety needed. The intervention does not help them break this negative loop. On the contrary, we suspect that it even worsens the situation by forcing them to look at and discuss things they are uncomfortable with. We find some support for this interpretation when we look at the evaluations of the “second best” teams.  Amongst them we found a number of comments indicating that the team members didn’t feel they learned anything from the intervention. Some examples of responses to the question “What is your strongest impression after being through this eight-week process” were:

• “I can’t see that we have developed 1 mm.”
• “How little this has contributed to our team development. But CIP (the name of the application, Author’s comment) has created some discontent and irritation during the boost-sessions.”.
• ”My strongest impression is that we as a team already are very good at many of the things this study wants to demonstrate, which is positive!”

A lot of team members in the “second best” teams argued like above, that they already were a good team, how much they already agreed on things, that they have a good understanding of their purpose, that they trust in each other etc. The debrief was not only deemed as “of no use”. It also seemed to create negative feelings and irritation, for example by “taking time from other tasks”.

Against this background, it should be important to find indicators of teams like the “second best”, so that they can be identified and given other types of support than the self-guiding team debrief. Ideas on how to identify these teams could be looking at the combination of performance and knowledge integration levels, for the combination:

1. High performing teams,
2. that rate themselves relatively low on knowledge integration, and
3. where team performance assessment is higher than observers

This combination could be a working indicator of the “second best” phenomena.

Summary and discussion

The background to this study has been the increasing complexity of organizations due to specialization, and the consequential challenge to develop new and more advanced forms of coordination on par with this. As this coordination is often performed in different forms of team level collaborations, it will be essential for organizations to find ways of supporting such. We argue that this support must be local, and in ongoing operations. The concept of debrief is therefore promising. It means that in addition to working with their task, teams should be expected, and prompted, to consistently work on their ability to act and develop as a collective unit.

The current study confirms that one form of support for this could be a digital tool for self-guiding team debriefs. The self-guiding tool of this study developed performance at level with previous studies of team debriefs, a 20-25% increase in effectiveness (Tannenbaum & Cerasoli, 2013). The results show us that teams at all performance levels can develop by the intervention. But there seem to be an exception. There is a well assembled group of teams that negative development due to the intervention. For organizations it will be critical do be able to identify these teams, or for them to identify themselves, and to develop suitable alternative interventions for their needs.

Author

Philip Runsten

philip.runsten@hhs.se

Researcher
PhD, Stockholm School of Economics

Senior Consultant, Founder
Influence