Individual and Collective Learning in Collaborative Knowledge Building

Highlights

• Informal learning involves processes of collaborative knowledge building, meaning that a group of people collaboratively create new knowledge using shared artefacts.

• Three learning tools were developed that support this type of learning (KnowBrain, Confer and Bits & Pieces), two of which received positive reactions of healthcare professionals and there is some evidence for changed learning practices for group work in these settings.

• In further experiments, we found individual learning is tightly coupled with how the collectively created knowledge evolves. Individuals learn more if a shared understanding is created in the group.

• When people actively engage with others’ contributions (“uptake”), this contributes to a shared understanding

• Recommendation mechanisms implemented in the shared environment that display others tags contribute to shared understanding in the group, but do not restrict individual divergent thinking. These recommendations are more readily accepted if they use a model of search of human memory that looks for resonating semantic categories.

Research Challenge

Informal learning often happens in open-ended tasks and requires creative problem solving. In practice, this is mostly done by creating groups or teams to tackle these tasks. Learning Layers has delivered a number of learning tools (e.g., KnowBrain, Confer and Bits & Pieces) that allow such groups to collaborate in varying collocated and distributed arrangements. For these groups to be successful requires people to establish a shared understanding, and this is more difficult in online settings when collaboration happens over distance. At the same time, there also needs to be a balance in creating convergence in the group (that contributes to shared understanding) without limiting the creative divergence in the group.

Theoretical Contribution

When a group of professionals working as a team aims at delivering an innovative solution (e.g., a new product, a shared service or a project proposal) it might decide searching the problem space collaboratively and make use of joint artifacts (e.g., annotated resources in a shared Web environment). During this activity collaborative knowledge building (CKB) takes place, where individual, self-directed learning activities (e.g., searching resources, reflecting on and learning from them) blend with social phenomena that emerge out of the collective information search (e.g., the shared collection of resources).

The role of artifacts is to mediate between individual learning and collective search: For example, tags assigned to collected bookmarks preserve the interpretations (individual understanding) of already found sources and allow people to reflect on others’ thoughts and corresponding search results. The cognitive act of reflection provides a learning opportunity (e.g., [1]) for it can involve search of memory to assimilate familiar aspects (e.g., by retrieving and strengthening semantic associations) and the accommodative moment of forging new, episodic associations to come to terms with novel aspects (e.g., [2]). The outcome of this reflection-based learning process (e.g., altered problem understanding and adjusted information goal) affects future search and contributions of oneself and others. Hence, an iterative process comes into being that dynamically couples individual learning and (malleable) artifactual structures of collective search ((missing reference)).

Over time, this coupling circulates and strengthens patterns of interpreting and labeling objects, thereby giving rise to a shared understanding in CKB (e.g., [3]), becoming manifest e.g. in a shared vocabulary. A socio-cognitive phenomenon, which becomes possible through shared understanding and drives individual learning, is resonance [4] that denotes the experience of a subject of being and acting in a world that appears to feedback and respond coherently. In case of CKB, a resonate subject-world relationship can be thought of as an individual who experiences self-efficacy in the ongoing process of learning from and contributing to search and reflection: Based on a shared understanding and vocabulary, the individual can process others’ reflections smoothly by exploiting already learned associations. This familiarity-based retrieval of associations (e.g., [5]) spares mental resources for the more effortful endeavor of dealing with novel contributions to explore and forge new associations. Summarizing, a coupling of individual and collective processes that results in a shared understanding in the sense of a resonate subject-world relationship can be expected to have positive effects on individual learning.

Conceptualizing CKB that way, we have raised a series of research questions (RQs) described next and discussed in the subsequent sections from an empirical viewpoint. The first question has been whether CKB can be observed in informal workplace learning and whether it can be supported by collaborative learning technology (RQ 1). To examine RQ1, three tools have been developed, namely KnowBrain, Confer and Bits & Pieces, to support different aspects of CKB. KnowBrain is a tool for digital curation and allows for sharing bookmarks of Web resources and reflecting on them by means of a shared tag cloud. To deepen reflection and further develop shared understanding, Confer facilitates progressive inquiry in a group. Bits & Pieces on the other hand, puts focus on collaborative sensemaking and reflection.

We also used these tools to test more specific research questions. The second question was whether an increased shared understanding is indeed conducive to individual learning (RQ 2). Directly linked to this question, we were interested in the mechanisms that drive shared understanding. Referring to research on grounding ([6]), the experience of resonance between each other’s reflections (i.e., “conceptual coordination”) becomes manifest and is enhanced when particpants adopt each other’s word choices. The third research question therefore has been whether shared understanding (i.e., similar associations to domain concepts) is larger among people who frequently interact with collaborators’ interpretations by taking up each other’s word choices (i.e., tags) than among people who exhibit comparatively lower uptake rates (RQ 3).

While in RQs 2 and 3 we focussed on convergent processes in the group, we also did not want to lose sight of the effect of shared understanding on creative thought that also needs divergent processes. Referring to studies on networked search (e.g., [7]), convergence on a collective level facilitates exploiting each other’s search results. But this should not paralyze individual divergent thinking, which supports the collective in exploring different aspects of the problem space. Interacting with shared artifacts (e.g., tag clouds), which blend different ideas, can in fact be expected to stimulate divergent thinking, in particular, ideational fluency, which is the ability to come up with different ideas fluently when thinking about a given problem. The fourth research question therefore was whether individuals who take part in a collective search mediated by shared artifacts exhibit higher levels of ideational fluency than individuals performing an information search alone (RQ 4).

To support convergent and divergent processes in the group independent of the tool, we have worked on the design of a tag recommendation service pushing the uptake of already existing tags – a process expected to promote both shared understanding (RQ 3) and ideational fluency (RQ 4). The underlying algorithm is taken from a computational theory of search of human memory (denoted SoMe, sharing principles with 3Layers, see here) and suggests tags from the collective vocabulary that resonate with an individual’s mental associations. The fifth research question finally was whether SoMe achieves higher tag acceptance rates (i.e., is more likely recommending tags the user actually adopts) than a conventional most popular tags (MPT) recommender only considering tag reuse frequencies (RQ 5).

Methods

Several empirical studies were employed, including pilot studies with collaborative knowledge building tools in the healthcare domain, namely Confer and Bits & Pieces. In these pilots, co-design was undertaken with healthcare professionals to design the tools to support their informal learning. The goal was to further our understanding of the healthcare professionals’ knowledge building practices and assess how the tools supported them (RQ 1). In particular, these were applied in one group of practice managers (N=19) and two groups of healthcare professionals (N=5, N=6) aiming to develop shared services, communication plans and training programs, respectively (see Report of Summative Evaluation in the Healthcare Pilots, [8]]) as well as a team of healthcare professionals within a workshop setting (N=6), see here. Reactions of healthcare professionals (to how the tools fit into these CKB activities) were collected through interviews and focus groups that repeatedly took place (at beginning, middle, end) along a period of about five months each.

In order to do some more fine grained analyses on the cognitive mechanisms and to be able to introduce experimental manipulations, two field experiments were conducted in realistic contexts, in which groups of learners solved open-ended tasks (e.g., coming up with a conceptualization for a project proposal) by performing a collective search (e.g., sharing bookmarks) and an artifact-mediated reflection (e.g., through tag clouds). To observe the dependent variables, we both analyzed participants’ learning activities (captured by log files) and their performance in tasks administered outside the learning environments (e.g., free associations tasks, FAT, using search topics or social tags as stimuli). These studies were of an experimental nature as the participants were assigned randomly to different conditions realizing different levels of our independent variables.

Field experiment 1 addressed RQ 2 (impact of shared understanding on individual learning) and involved 24 students of a university course sharing a social bookmarking system for eight weeks (for details see [9]). The independent variable “Semantic Stabilization” differentiated between two conditions that gave rise to either a high or low level of shared understanding (that was operationalized by the degree of stabilization in the shared tag vocabulary). Individual learning was defined as the increase in the number of associations produced to specific domain concepts (specific tags) of the search task.

Field experiment 2 [10]; [11] focused on RQs 3-5 and involved 18 university employees sharing KnowBrain for four weeks . The first independent variable (addressing RQ 4) was “Social Setting” that assigned half of the participants to a collaborative condition (shared environment with social tags) and the other half to an individual search condition (searching alone and without tag-based interactions). The dependent variable was “ideational fluency”, which is reflected in a FAT by a slow rate of decline (along consecutive seconds) in producing associations to predefined topics. To address RQ 3 ([12]), we followed a correlational approach and, in a first step, distinguished between participants exhibiting low vs. high tag uptake rates. Then, we compared the two groups (median split) with respect to their shared understanding, which we defined as the group members’ overlap in their associations (determined through a network-based analysis).

Finally, to address RQ 5, participants’ tagging behavior was supported by the recommendation of already used tags, which were filtered automatically either by MPT or SoMe (second independent variable; within-subjects). Based on the log files, we could then evaluate the accuracy of each of the two tag recommendation mechanisms (TRMs) by calculating the average rate at which recommended tags had been adopted for the actual tag assignments.

Findings

Pilot studies on the relevance of CKB in the healthcare sector (RQ 1)

The pilot studies revealed that the professionals perceived Confer as a convenient tool helping to collect different perspectives on a problem at hand and to mutually stimulate the generation of ideas for a first recommendation of how to approach the problem see the quotes below), see also here.) Additionally, the professionals valued the possibility for adding items outside the meeting which led to more people contributing. Overall, it helped exploring and becoming aware of central topics in the endeavor of sharing and implementing practices in communication and training.

We let the people look [in Confer] and see where we got up to with our thoughts <…> they put in some suggestions which meant that when we had the meeting I was already presenting what we got in the light of people, what they wanted, so I think we were already slightly changing it and then changed it much more in the light of that [discussion].

User of Confer in Pilot B, Final workshop, July 2016

I think it forced us to have that sort of focussed, yeah, that focussed discussion

User of Confer in Pilot C, Final workshop, July 2016

They loved it in the meeting <…> It facilitated the discussion <…> when you’re having your meeting you have it on there and type some things in whilst you’re there and then it feels as though you’re adding

User of Confer in Pilot C, Final workshop, July 2016

Group feels as though they all want to contribute so… <…> it’s a nice statement of where that group’s got to I think.

User of Confer in Pilot C, Final workshop, July 2016

I got this up on the screen and went through with them all the things that we’d gone with. The outcome of that was very interesting, [what] they said, <…> so it has helped with that and one of the guys also had a new novel idea.

User of B&P, Final workshop, July 2016

Bits & Pieces on the other hand was perceived as a tool that could help developing and deepening a shared understanding of these topics. Members of shared episodes (digital curations of Web resources and Evernote snippets) valued the opportunity to visualize and align their understanding of categories in form of circles and tags that group and index similar items (e.g., resources, snippets), respectively. Both tools also seem to support both divergence in an earlier phase of developing a first draft of recommendations for a problem at hand as well as convergence in a subsequent phase of developing shared understanding: Confer, for example, helps bring in various opinions from a diverse set of participants early on and then supports convergence and refinement over time. Bits & Pieces, on the other hand, focusses on individual collection and sensemaking, but then supports the sharing and aligning of understanding in a group.

I just feel that it [B&P] does like bring a group of people together and allow them to have ideas and comments on it because it’s nice and visual

Use of B&P, Final workshop, July 2016

If you organise the circles well you can get people’s understanding of why you’ve done that, then get the people’s comments on why you’ve done that, how better you could have done it, so I think it does help formulate ideas and discussions.

Use of B&P, Final workshop, July 2016

We could use the Learning Layers tool [B&P] to be able to do that and be able to share responsibilities, share ideas, and at the end create something that we needed to create like a document, a policy, a protocol or information that we share externally

Use of B&P, Final workshop, July 2016

Field experiments on the impact of shared understanding on learning and divergent thinking (RQs 2-4)

Results of the first field experiment (see also [9]) supported the assumption of a positive impact of shared understanding (SU; stabilization of tag vocabulary) on individual learning of domain concepts (social tags) during a collective search for Web resources (RQ 2). As Figure 1 shows, students who participated in groups that succeeded in developing a shared tag vocabulary (high SU) gained more knowledge about specific concepts, i.e., could produce more associations to specific tags, than students of groups reaching lower levels of shared understanding (low SU).

1) Impact of shared understanding (SU) on individual learning (number of associations produced to general, medium and specific tags; [9] )

Figure 1: Results on RQ 2: Participants of high SU groups reach a more refined knowledge about medium and specific tags than Students of low SU groups.

The second field experiment ([12]) further revealed that a key mechanism behind the development of shared understanding is the rate at which members of the collective search interact with each other’s artifacts (i.e., tag uptake rate; RQ 3). The results of the network analysis performed on the FAT (Figure 2) revealed the amount of overlap in associations to particular search topics (i.e., shared topic understanding) to be larger among employees with high than among employees with low tag uptake rates (left and right network, respectively). These results indicate that frequently shared artifacts (social tags) evoke phenomena of resonance (coherent subject-world experiences) by arousing overlapping patterns of mental activation and hence, convergent interpretations among members of collective search.

2) Shared understanding (amount of overlap in mental topic associations) for high uptake subgroup (left network) and low uptake subgroup (right network; [12])

Figure 2: Results on RQ 3: Larger overlap in associations among participants with high uptake rates (left network) than among participants with low tag uptake rates (right network).

With respect to RQ 4 ([10]), the results further demonstrate that these convergent effects of shared artifacts do not compromise individual processes of divergent thinking. In fact, Figure 3 discloses that ideational fluency, i.e., the ability to produce associations to search topics at a high rate, was more strongly pronounced under the collaborative condition (shared environment with social tags; see unfilled triangles) than under the individual condition (searching alone and without tag-based interactions; see unfilled squares). We therefore conclude that the opportunity to take up each other’s artifacts does not only bring about intersubjectively mediating phenomena of resonance but also help the individual in thinking about search topics in a divergent and thus, creative way.

3) Ideational fluency before the study start (baseline) as well as after collaborative and individual search conditions (with and without uptake opportunities, respectively; Seitlinger et al., under review)

]

Figure 3: Results on RQ 4: Relative to the study start (Baseline), the decline in producing new associations is smaller (i.e., ideational fluency is higher) after the collaborative than individual condition.

Evaluation of a tag recommendation mechanism searching for resonate tags to increase uptake rates (RQ 5)

A natural strategy to increase the probability of tag uptake is to support tagging behavior by means of TRMs, which suggest and therefore, push the reuse (uptake) of existing tags. Results on RQ 5 ([10]; [11]) demonstrate that an effective TRM can indeed be designed by searching for tags of the collective vocabulary, which – according to an employee-specific search of memory (SoMe) model – can be expected to resonate strongly with already formed semantic structures (associations). Figure 4 clearly shows that the tag acceptance rate achieved by SoMe is higher than that achieved by MPT, which is a conventional baseline recommender only considering reuse frequencies.

4) Performance of tag recommenders MPT (baseline) and SoMe under the collaborative search condition (Seitlinger et al., under review)

Figure 4: Results on RQ 5: Tag acceptance rate (measured by the F-score) is higher for SoMe than the baseline recommender MPT.

Links to other sections

Informal Learning Tools: Bits & Pieces, Confer, KnowBrain

Infrastructure: Social Semantic Server, Tag Recommendation Services

Cases & Impact: Supporting Distributed Teams in Healthcare and Healthcare Teamwork in and between General Practices

Learning Scenarios: Sensemaking and Meaning Making, Progressive Inquiry

Further Reading

This article is based on the following publications:

• Take up my Tags : Exploring Benefits of Meaning Making in a Collaborative Learning Task at the Workplace, EC-TEL 2016, [12]

• Dynamics of Human Categorization in a Collaborative Tagging System: How Social Processes of Semantic Stabilization Shape Individual Sensemaking, Computers in Human Behavior, [9], Open Access Fulltext

• Patterns of Meaning in a Cognitive Ecosystem: Modeling Stabilization and Enculturation in Social Tagging Systems. In Mass Collaboration and Education, Springer, [3]

• Visualizing Workplace Learning Data with the SSS Dashboard, Workshop on Learning Analytics in Workplace and Professional Learning, LAK 2016, [11]

• Reconceptualizing imitation in social tagging: a reflective search model of human web interaction, WebScience 2016, [13]

• Balancing the Fluency-Consistency Tradeoff in Collaborative Information Search, Journal of Human Computer Interaction, [10], Open Access Fulltext

Contributing Authors

Tobias Ley, Paul Seitlinger, Sebastian Dennerlein, Tamsin Treasure-Jones, Patricia Santos, Elisabeth Lex, Dominik Kowald

References

1. B. Renner, M. Prilla, U. Cress, and J. Kimmerle, “Effects of prompting in reflective learning tools: Findings from experimental field, lab, and online studies,” Frontiers in Psychology, vol. 7, p. 820, 2016.
2. W.-T. Fu and W. Dong, “Collaborative indexing and knowledge exploration: A social learning model,” IEEE Intelligent Systems, vol. 27, no. 1, pp. 39–46, 2012.
3. T. Ley, P. Seitlinger, and K. Pata, “Patterns of Meaning in a Cognitive Ecosystem: Modeling Stabilization and Enculturation in Social Tagging Systems,” in Mass Collaboration and Education, Springer, 2016, pp. 143–163. DOI: 10.1007/978-3-319-13536-6_8
4. H. Rosa, Resonanz: Eine Soziologie der Weltbeziehung. Suhrkamp Verlag, 2016.
5. A. P. Yonelinas, “The nature of recollection and familiarity: A review of 30 years of research,” Journal of memory and language, vol. 46, no. 3, pp. 441–517, 2002.
6. S. E. Brennan and H. H. Clark, “Conceptual pacts and lexical choice in conversation.,” Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 22, no. 6, p. 1482, 1996.
7. D. Lazer and E. S. Bernstein, “Problem Solving and Search in Networks,” Cognitive Search: Evolution, Algorithms, and the Brain. The MIT Press, Cambridge, MA, 2012.
8. R. Dewey, M. Geiger, M. Kerr, R. Maier, M. Manhart, P. Santos Rodriguez, C. Sarigianni, and T. Treasure-Jones, “Report of Summative Evaluation in the Healthcare Pilots,” pp. under review, 2016 [Online]. Available at: Link
9. T. Ley and P. Seitlinger, “Dynamics of human categorization in a collaborative tagging system: How social processes of semantic stabilization shape individual sensemaking,” Computers in human behavior, vol. 51, pp. 140–151, 2015 [Online]. Available at: Open Access DOI: 10.1016/j.chb.2015.04.053
10. P. Seitlinger, T. Ley, D. Kowald, D. Theiler, I. Hasani-Mavriqi, S. Dennerlein, E. Lex, and D. Albert, “Balancing the Fluency-Consistency Tradeoff in Collaborative Information Search with a Recommender Approach,” International Journal of Human–Computer Interaction, vol. 34, no. 6, pp. 557–575, Jun. 2018. DOI: 10.1080/10447318.2017.1379240
11. A. Ruiz-Calleja, S. Dennerlein, T. Ley, and E. Lex, “Visualizing Workplace Learning Data with the SSS Dashboard,” in Proceedings of the First International Workshop on Learning Analytics Across Physical and Digital Spaces co-located with 6th International Conference on Learning Analytics & Knowledge (LAK 2016), Edinburgh, UK, 2016, pp. 79–86 [Online]. Available at: Fulltext
12. S. Dennerlein, P. Seitlinger, E. Lex, and T. Ley, “Take up My Tags: Exploring Benefits of Meaning Making in a Collaborative Learning Task at the Workplace,” in European Conference on Technology Enhanced Learning, 2016, pp. 377–383. DOI: 10.1007/978-3-319-45153-4_30
13. P. Seitlinger and T. Ley, “Reconceptualizing imitation in social tagging: a reflective search model of human web interaction,” in Proceedings of the 8th ACM Conference on Web Science - WebSci ’16, New York, New York, USA, 2016, pp. 146–155 [Online]. Available at: Link DOI: 10.1145/2908131.2908157