The SSS as an Infrastructure for Learning Analytics

Description

Workplace learning is a multi-episodic activity that is tightly coupled with the needs of the work. This type of learning happens across different contexts, some of which are more formal, such as a professional training course, and some other more informal, such as learning from a question asked to a colleague. Workers typically learn by using several software tools in a spontaneous and difficult-to-foresee way.

These characteristics imply additional challenges for workplace Learning Analytics (LA). In order to support LA in a wide range of workplace learning scenarios it is needed to integrate data from different, and very diverse, tools that can potentially be used for learning during the normal activity in real work environments. The data from these tools should be coherently processed and offered back to the participants to enhance the learning activity. A special focus on learning theories is required to collect, process and and offer the data as the individual activities of workplace-based learning are not and cannot be guided by a design developed beforehand.

Current workplace LA solutions do not take all these characteristics into account, which hinder their support to understand and enhance workplace learning processes. On the one hand, most workplace LA solutions are unable to gather data from more than one application; this drawback hinders their applicability in different domains and their support in realistic scenarios where multiple tools are used. Finally, it is commonly the case that pedagogical theories are not taken into account when designing and developing LA solutions [1]. This aspect is especially relevant in workplace learning settings, as the lack of pre-defined learning designs implies a deeper understanding of learning processes when collecting and processing data.

Description

We considered three scenarios in order to illustrate how workplace LA can be improved by Learning Layers.

• A professional teacher training course where a group of teachers use a bookmarking tool to share information on how to introduce new technologies and pedagogical techniques into their classes, and a blog to write individual reports related to this topic. It would not be possible to support the understanding of this learning process unless a LA solution collects data from multiple applications and is able to represent how the community introduces, shares and evolves learning artifacts.

• A collaborative digital curation scenario where several researchers coordinate themselves to use a collaborative bookmark application to share topic-relevant resources (i.e. web pages or text documents). The shared resources are organized in a set of predefined categories and then freely tagged by the researchers. The spread of innovative concepts inside the community of researchers can be supported by a recommender system that takes into account the context (category) where the tag is added.

• A sense making scenario where 6 healthcare professionals freely use a set of four applications to collaborative make sense of their learning experiences at workplace. The professionals can tag the resources collected to organize them. Again, the community can benefit from a recommender system to support the tagging system, but in this case it would require the collection and integration of data from four applications used during common work activities.

Learning Layers proposes to exploit the Social Semantic Server (SSS) as an infrastructure for workplace LA inspired by the knowledge creation metaphor [2]. The SSS collects data from different tools used for workplace learning and offers it back for LA applications to exploit it in order to provide services for workers or trainers.

Two characteristics of the SSS play a major role in this support. First, its software architecture and data model enable the integration of many different learning tools and LA applications. Second, it promotes LA applications to follow the knowledge creation metaphor, thus exploiting the knowledge that emerges inside a community of learners. The flexibility of the SSS is key for the support provided: its software architecture needs to be flexible so as to integrate many different data-consuming and -publishing applications; and its data model flexibility enables the introduction of unexpected concepts that emerge in the learning process.

We assessed the suitability of the SSS as a LA infrastructure and its potential impact in three workplace learning situations that represent the situations previously described:

• The professional teacher training course was enhanced by the SSS. The role of the infrastructure was to collect and coherently integrate the log data from the bookmarking tool and the blog editor. The data collected was offered back to the trainer and students using the SSS Dashboard. Using the SSS Dashboard they visualized the data collected by the SSS to understand how learning artifacts (either documents or topics) evolved over time and how learners interact with those artifacts using different applications.

• The collaborative digital curation scenario was supported by the SSS. The infrastructured collected the tags introduced by the learners and used them to support the tagging process by mean of tags recommendations. Two different recommender systems were used: 3Layers and MostPopular. A formal comparison between these recommender systems showed that the F1 score, calculated based on the accepted tag recommendations in comparison to the amount of tags recommended, was higher in 3Layers (0.34) than in MostPopular (0.27). Additionally, we noticed the capability of 3Layers to raise awareness of topics once they are introduced into the community, thus promoting the new topics to be uptaken by other learners.

• The sense making scenario was enhanced by the SSS to support the tagging process by means of a recommender system. The SSS could satisfactorily integrate the data from different applications. However, as the healthcare professionals did not make extensive use of the tagging functionality, the impact obtained in this pilot study was very limited.

These pilot studies showed the potential impact of the SSS as a LA infrastructure: it is able to collect, and coherently integrate, data from different workplace learning tools (e.g. Bits & Pieces, KnowBrain, etc.) and offer it back to some LA application (e.g. SSS Dashboard or the resource recommender services / tag recommender services. In this regard, three aspects of the SSS should be underlined: First, the SSS offers a flexible software architecture that facilitates the integration of a wide range of data-publishing and -consuming applications. Second, its data model allows to explicitly define semantic relationships between actors and artifacts. Third, the pilot studies show that the SSS can be used in real workplace learning settings.

Description

The SSS is currently offered for industry and academic developers as an open source infrastructure. It can well be exploited to integrate data from different learning applications and coherently integrate and process it for LA purposes. More information about the technical aspects of the SSS can be found here.

Due to its flexibility, the SSS can be adapted to support LA in a wide range of learning scenarios, which may differ based on the given domain, the applications used and their level of formality, as seen in our pilot studies. Additionally, we showed how to exploit the SSS for LA purposes with theoretical basis, which paves the way its future use.

It is very important to note that the SSS can be adapted to different learning scenarios and not the other way round. Hence, the SSS does not require learners or trainers to make an additional effort to take advantage of its LA support. Instead, the SSS collects data from the applications they are using and offers this data for learning applications to feed learners and trainers back. In this sense, we say that the SSS does not replace workplace learning activities, but further supports them.

To date, the SSS data has been exploited by learning dashboards and tag recommender systems. We show how these applications promote the awareness of the learning process and the uptake of learning artifacts. However, it is still to be researched whether it can be exploited for other purposes, such as evidence-based portfolios, expertise detection or managing support.