Tag Recommendation Services

Purpose

Tag recommendation services support learners in finding descriptive tags for their resources (e.g., Web links or documents). Thus, these services suggest a set of tags for a given user and resource. Although there is already a large body of research available in the area of tag recommendation, current approaches do not sufficiently account for the particular expertise of a user and the context of knowledge creation, which is very important for informal workplace learning.

Our main contributions in the research area of tag recommendations are as follows:

• We came up with four novel tag recommendation approaches based on models of human cognition: 3Layers [1], BLL+C [2], 3LT+MPr [3] and BLLAC+MPr [4]

• We have evaluated our approaches in various settings and found that our cognitive-inspired approaches outperform other state-of-the-art algorithms [5]

• We provide tag recommendations via the TagRec framework [6], which is fully integrated in the Social Semantic Server (SSS)

• We generalize our approaches for providing hashtag recommendations in Twitter [7] [8]

Description

Social tagging is an important instrument for users to arrive at a shared understanding via collaboratively adding metadata to content [9]. Hence, all kind of objects in the SSS can be tagged, for example physical locations, persons and digital artefacts. In order to assist users in providing meaningful tags, we are designing, implementing and evaluating algorithms for personalized tag recommendations. Moreover, tag recommenders have an influence on the overall tag distribution in folksonomies and consequently on semantic stabilization. Thus, we also regard tag recommendations as one form of gardening collective knowledge structures in the SSS.

However, in specialist domains like workplace learning, where the particular expertise as well as the context of knowledge creation is very important, recommending most popular tags may not be the most appropriate strategy. We therefore have pursued four alternative strategies to improve the current state-of-the-art in tag recommenders.

• (i), and to take into account the expertise of a person, we have created a tag recommender that is built upon an understanding of the human categorization process: 3Layers (3L) [1]

• (ii), we have implemented a tag recommendation algorithm that aims to mimic how people access information in their memory using time information and the semantic context of past tag assignments: BLL+C [2]

• (iii) and (iv), we have combined both approaches into two hybrid approaches, 3LT+MPr [3] and BLLAC+MPr [4]

Our tag recommendation algorithms are implemented in our open-source TagRec framework [6] visualized in Figure 1 [10] [11].

Figure 1: TagRec framework [10].

This figure shows, that any kind of folksonomy data can be pre-processed in order to create a common data model (i.e., user-resource-tag relations). This data model is used by various recommendation algorithms (which can also be gathered from external libraries) to create tag recommendations. These tag recommendations are then either evaluated to measure various recommendation quality metrics or passed to a client application (e.g., Bits and Pieces).

Use Cases and Tool Support

Our tag recommendations are fully integrated in the SSS (via our TagRec framework) and thus, can be easily accessed by various Layers tools.

For example, in Bits and Pieces, tag recommendations are shown to suggest the user relevant tags both for annotating resources with metadata and to support the categorization process in the sense making phase (i.e., when users assign labels to created arrangements). This is illustrated in Figure 2.

Figure 2: Tag recommendations in Bits and Pieces.

Tag recommendations are also used in KnowBrain, where tags to be assigned to uploaded resources are recommended based on chosen categories. This is illustrated in Figure 3. Furthermore, we evaluated our algorithms using KnowBrain in form of an online study [12].

Figure 3: Tag recommendations in KnowBrain.

Research Results and Impact

In this section, we present our research results in the area of tag recommendations. In Layers, it was our aim to understand the underlying cognitive processes of social tagging in order to model these processes in form of novel tag recommendation algorithms. These novel algorithms were then evaluated in order to show the usefulness in real-world scenarios. Please also have a look at the impact section describing how learners are guided and supported via tag recommendations.

Understanding Social Tagging Processes

We conducted two lines of research in Layers in order to better understand individual and social processes in tagging resources. In the first one, we analyzed the factors that play a role when people reuse tags [13]. In this study, we utilized six social tagging datasets and found that past usage frequency, recency (i.e., the time component) and the current semantic context greatly influence the reuse of tags.

In the second strand of tagging research, we studied uptake events of tags (i.e., reusing the tag of a peer) in order to explore the benefits of meaning making in a collaborative learning task at the workplace [14]. Therefore, we have constructed a collaborative, digital curation scenario, in which 18 university employees have explored a given topic (namely, “designing workplaces that move people”) during a work-integrated Web search for a period of four weeks. Each participant was assigned the task to continuously collect topic-relevant Web resources (approximately three links/documents per week) with the help of KnowBrain and to annotate them by means of a tagging interface subsequently. Our study showed that high uptake of others’ tags had a twofold effect: (i) increase of shared understanding indicated by higher overlaps in collaborator’s conceptual knowledge, and (ii) narrower and more elaborative search indicated by a slower theme exploration with more considered resources.

Modeling Cognitive Processes in Form of Tag Recommendation Algorithms

In order to account for the individual expertise and the current context of tag recommendations, we investigated the relation of cognitive processes and social tagging [15]. In this respect, we identified three types of cognitive processes that play a role when people assign tags to digital resources: (i) categorizing resources, (ii) temporal dynamics, and (iii) imitating tags.

In order to account for categorization processes, we build on the 3Layers (3L) approach [1] developed in the first year of the project. 3Layers implements methods of the human category learning theory MINERVA2 [16] with the help of Latent Dirichlet Allocation (LDA) topic modelling [17]. In this sense, MINERVA2 is a simple pattern learning algorithm that abstracts common patterns from instances over time. It is therefore in line with our ideas on how patterns of meaning emerge in social informal learning environments, and corresponds to the knowledge creation metaphor of learning.

Even though the 3Layers algorithm takes into account the individual expertise of a user, it ignores the context of time. Thus, to model these temporal dynamics, we made use of the BLL+C approach [2] developed in the second year of the project, which adapts the Base-Level Learning (BLL) equation of the cognitive architecture ACT-R [18]. Since BLL+C also makes use of the most popular tags used by other users, it also accounts for tag imitation processes.

We combined 3Layers and BLL+C in a hybrid recommender termed 3LT+MPr [3] in the third year of the project. We evaluated this novel approach on various real-world folksonomy datasets in terms of recommender accuracy. The results clearly show that our novel hybrid approach 3LT+MPr not only outperforms 3Layers (3L) and BLL+C but also state-of-the-art algorithms like FolkRank (FR) [19]. Thus, we assume that all three types of cognitive processes, (i) categorizing resources, (ii) temporal dynamics, and (iii) imitating tags, are highly relevant for the task of recommending suitable tags to users in specific contexts.

Apart from that, in the third and fourth year of the project, we worked on an extension of BLL+C that implements the full activation equation of the cognitive architecture ACT-R (not only the base-level learning equation) and thus, gives a higher weight to tags that are more suitable in the current semantic context. This resulted in our BLLAC+MPr approach [20] [4], which integrates the information of the current semantic context via tag co-occurrences and thus, does not need any category information. Additionally, we further analyzed and evaluated the individual components of this algorithm [13] and found that the efficacy of a tag recommender algorithm greatly depends on the folksonomy type of the given system (i.e., narrow, mixed or broad folksonomies).

Evaluating Tag Recommender Algorithms in Real-World Scenarios

Most tag recommender algorithms are evaluated in simulated environments in order to guarantee a minimum amount of training data for the algorithms. Thus, so-called p-cores are generated where only users, resources and tags are taken into account that occur a minimum of p times (e.g., 5). However, this procedure is clearly at odds with real-world social tagging systems where most resources are only tagged by the uploading user. This is also the case for the tagging behavior used in most Layers tools. Furthermore, most studies only look at a limited set of metrics (i.e., only accuracy) and algorithms (i.e., only classic ones like FolkRank).

Thus, we conducted an extensive tag recommender study in order to validate the usefulness of various algorithms in real-world settings in the third year of the project [5]. Another objective of this study was to compare the two tag recommender strategies developed in the project (i.e., BLLAC+MPr and 3LT+MPr) in order to decide which algorithm should be implemented as a service in the SSS.

In this respect, we provided results in terms of accuracy (both in narrow and broad settings), diversity, novelty, runtime and memory consumption of the algorithms. Theses findings are shown in Table 1. Apart from that, we studied the imbalance between current tag recommendation algorithms and real-world folksonomy structures [21].

Table 1: Tag recommendation evaluation results [5].

It can be found that both of our algorithm provide overall better results than other state-of-the-art algorithms. This clearly indicates the advantage of our cognitive-inspired strategies. Furthermore, although both of our approaches generate similar results in terms of recommender accuracy, BLLAC+MPr provides lower computational complexity, which is an important factor for scalability. Thus, BLLAC+MPr is used in the SSS via our TagRec framework to provide tag recommendations.

In the fourth year of the project, we evaluated BLLAC+MPr in various TEL environments including European Schoolnet’s TravelWell portal and the MACE portal (see [22]). Here we found, that our approach outperforms related state-of-the-art methods not only in large social bookmarking environments but also in these small TEL environments. Thus, we conclude that our cognitive-inspired approaches can support and guide users in informal learning settings. Apart from that, we evaluated our approaches also via a case study in the Healthcare sector [23] and via an online evaluation using KnowBrain [24].

Recommending Hashtags in Twitter

After the project, we generalized our tag recommendation approach for providing hashtag recommendations in Twitter. These research results were presented at the World Wide Web Conference [7] and the European Computation Social Sciences Symposium [8] and are used in H2020 project AFEL [25].

Material

Links

• Further readings: [1] [2] [3] [4] [5] [22] [24] [7] [8]

• TagRec framework: Link

• Learning scenario: Individual and Collective Learning in Collaborative Knowledge Building

• Impact section describing how learners are guided and supported via tag recommendations: Link

• Slides about the BLL algorithm presented at WWW’2014 in Seoul, Korea:

WWW2014: Long Time No See: The Probability of Reusing Tags as a Function of Frequency and Recency by Dominik Kowald

• Slides about the influential factors of tag recommendations presented at Hypertext’2015 in Halifax, Canada:

HT2016: Influence of Frequency, Recency and Semantic Context on Tag Reuse by Dominik Kowald

Developers and Contributors

Dominik Kowald, Paul Seitlinger, Tobias Ley, Elisabeth Lex

References

1. P. Seitlinger, D. Kowald, C. Trattner, and T. Ley, “Recommending tags with a model of human categorization,” in Proceedings of the 22nd ACM international conference on Conference on information & knowledge management, 2013, pp. 2381–2386 [Online]. Available at: Fulltext Online DOI: 10.1145/2505515.2505625
2. D. Kowald, P. Seitlinger, C. Trattner, and T. Ley, “Long time no see: The probability of reusing tags as a function of frequency and recency,” in Proceedings of the 23rd International Conference on World Wide Web, 2014, pp. 463–468.
3. D. Kowald, P. Seitlinger, S. Kopeinik, T. Ley, and C. Trattner, “Forgetting the words but remembering the meaning: Modeling forgetting in a verbal and semantic tag recommender,” in Mining, Modeling, and Recommending’Things’ in Social Media, Springer, 2015, pp. 75–95.
4. C. Trattner, D. Kowald, P. Seitlinger, S. Kopeinik, and T. Ley, “Modeling Activation Processes in Human Memory to Predict the Use of Tags in Social Bookmarking Systems,” The Journal of Web Science, vol. 2, no. 1, pp. 1–16, 2015.
5. D. Kowald and E. Lex, “Evaluating tag recommender algorithms in real-world folksonomies: A comparative study,” in Proceedings of the 9th ACM Conference on Recommender Systems, 2015, pp. 265–268.
6. D. Kowald, S. Kopeinik, and E. Lex, “The tagrec framework as a toolkit for the development of tag-based recommender systems,” in Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization, 2017, pp. 23–28.
7. D. Kowald, S. C. Pujari, and E. Lex, “Temporal effects on hashtag reuse in twitter: A cognitive-inspired hashtag recommendation approach,” in Proceedings of the 26th International Conference on World Wide Web, 2017, pp. 1401–1410.
8. D. Kowald and E. Lex, “Studying Confirmation Bias in Hashtag Usage on Twitter,” arXiv preprint arXiv:1809.03203 - presented at European Computational Social Science Symposium, 2018.
9. U. Cress, C. Held, and J. Kimmerle, “The collective knowledge of social tags: Direct and indirect influences on navigation, learning, and information processing,” Computers & Education, vol. 60, no. 1, pp. 59–73, 2013.
10. D. Kowald, E. Lacic, and C. Trattner, “Tagrec: Towards a standardized tag recommender benchmarking framework,” in Proceedings of the 25th ACM conference on Hypertext and social media, 2014, pp. 305–307.
11. C. Trattner, D. Kowald, and E. Lacic, “Tagrec: towards a toolkit for reproducible evaluation and development of tag-based recommender algorithms,” ACM SIGWEB Newsletter, no. Winter, p. 3, 2015.
12. D. Kowald, P. Seitlinger, T. Ley, and E. Lex, “The Impact of Semantic Context Cues on the User Acceptance of Tag Recommendations: An Online Study,” in Companion of the The Web Conference 2018 on The Web Conference 2018, 2018, pp. 1–2.
13. K. Dominik and L. Elisabeth, “The Influence of Frequency, Recency and Semantic Context on the Reuse of Tags in Social Tagging Systems,” in Proceedings of the 27th ACM conference on Hypertext and social media, 2016.
14. S. Dennerlein, P. Seitlinger, E. Lex, and T. Ley, “Take up My Tags: Exploring Benefits of Meaning Making in a Collaborative Learning Task at the Workplace,” in European Conference on Technology Enhanced Learning, 2016, pp. 377–383. DOI: 10.1007/978-3-319-45153-4_30
15. D. Kowald, “Modeling Cognitive Processes in Social Tagging to Improve Tag Recommendations,” in Proceedings of the 24th International Conference on World Wide Web, 2015, pp. 505–509.
16. D. L. Hintzman, “MINERVA 2: A simulation model of human memory,” Behavior Research Methods, Instruments, & Computers, vol. 16, no. 2, pp. 96–101, 1984.
17. D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” Journal of machine Learning research, vol. 3, no. Jan, pp. 993–1022, 2003.
18. J. R. Anderson, D. Bothell, M. D. Byrne, S. Douglass, C. Lebiere, and Y. Qin, “An integrated theory of the mind.,” Psychological review, vol. 111, no. 4, p. 1036, 2004.
19. A. Hotho, R. Jäschke, C. Schmitz, and G. Stumme, “Folkrank: A ranking algorithm for folksonomies,” in LWA, 2006, vol. 1, pp. 111–114.
20. D. Kowald, S. Kopeinik, P. Seitlinger, T. Ley, D. Albert, and C. Trattner, “Refining frequency-based tag reuse predictions by means of time and semantic context,” in Mining, Modeling, and Recommending’Things’ in Social Media, Springer, 2015, pp. 55–74.
21. D. Kowald and E. Lex, “Overcoming the Imbalance Between Tag Recommendation Approaches and Real-World Folksonomy Structures with Cognitive-Inspired Algorithms,” arXiv preprint arXiv:1805.03067 - presented at European Computational Social Science Symposium, 2018.
22. S. Kopeinik, D. Kowald, and E. Lex, “Which Algorithms Suit Which Learning Environments? A Comparative Study of Recommender Systems in TEL,” in European Conference on Technology Enhanced Learning, 2016, pp. 124–138.
23. P. Santos, S. Dennerlein, D. Theiler, J. Cook, T. Treasure-Jones, D. Holley, M. Kerr, G. Attwell, D. Kowald, and E. Lex, “Going beyond your Personal Learning Network, using Recommendations and Trust through a Multimedia Question-Answering Service for Decision-support: a Case Study in the Healthcare.,” Journal of Universal Computer Science, 2016.
24. P. Seitlinger, T. Ley, D. Kowald, D. Theiler, I. Hasani-Mavriqi, S. Dennerlein, E. Lex, and D. Albert, “Balancing the Fluency-Consistency Tradeoff in Collaborative Information Search with a Recommender Approach,” International Journal of Human–Computer Interaction, vol. 34, no. 6, pp. 557–575, Jun. 2018. DOI: 10.1080/10447318.2017.1379240
25. M. d’Aquin, D. Kowald, A. Fessl, E. Lex, and S. Thalmann, “AFEL-Analytics for Everyday Learning,” in Companion of the The Web Conference 2018 on The Web Conference 2018, 2018, pp. 439–440.