Tải bản đầy đủ - 0 (trang)
Protus 2.1: Applying Collaborative Tagging for Providing Recommendation in Programming Tutoring System

Protus 2.1: Applying Collaborative Tagging for Providing Recommendation in Programming Tutoring System

Tải bản đầy đủ - 0trang

Protus 2.1: Applying Collaborative Tagging for Providing Recommendation


tags in e-learning environments can be used by integrated recommender systems to

suggest useful and most appropriate resources to learners.

Social tagging is the activity of annotating and classifying digital resources with

keywords (tags as metadata). It is used by most of web-based information systems for

the collaborative indexing of massive amount of information [5]. Tagging represents an

action of reflection, where the tagger sums up a series of words into one or more summary

tags, each of which stands on its own to describe some aspect of the resource based on

the tagger’s experiences and beliefs [6].

The creators of metadata need no longer be experts from the field or authors of the

educational resources [3]. Instead, educational metadata is generated by the users of the

educational resources, who can describe teaching material with tags that are meaningful

to them and that can enable search options and retrieval among of learning resources.

In this paper, we discuss issues related to collaborative tagging as a means for

describing resources within tutoring system. Protus 2.1, a tutoring system that is used

for learning programing basics is presented. We will concentrate on approach for

providing personalized recommendation using collaborative tagging in Protus 2.1. This

paper aims to analyse capabilities of integration recommender systems based on collab‐

orative tagging techniques into adaptive and intelligent web-based programming

tutoring system – Protus 2.1 that takes into account pedagogical characteristics of the


The paper is structured as follows. Section 2 summarize the recent research on the

effects of the use of collaborative tagging for knowledge building and learning.

Section 3 provides an overview of Protus 2.1 system, which was used for applying tag

based recommender system. Evaluation of the proposed approach will be presented in

Sect. 4. In Sect. 5, conclusions and ideas for future work are given.


Related Work

Recommender systems have been used in e-learning environments to recommend useful

resources to users [4]. Several studies have investigated the anticipated benefits of tag

based recommendation within e-learning systems. These studies provided evidence

about potential of social tagging of resources to enlarge their metadata descriptions, and

for improving search and recommendation of educational resources.

Within the literature covering collaborative tagging in e-learning, there are existing

works that have investigated the expected benefits of tagging when applied in describing

educational resources: PLEM [7], CROKODIL [8], SOAF [9], TaCS [5] and TAK

system [10].

PLEM is a social bookmarking service for learning purposes. It is developed to act

as an aggregator and filter by supporting learners in retrieving, reusing and sharing

learning resources. These resources, called learning entities, can be bookmarked,

managed in collections and can be classified as learning resource collections [7].

CROKODIL is a platform that offers collaborative semantic tagging, recommenda‐

tions and social network functionality to support the learning community. The concept


B. Vesin et al.

presented in the paper proposes to use contextual information in folksonomies to rank

learning resources in a personalized recommender system for e-learning [8].

In [9], authors proposed a system architecture called SOAF for the semantic indexing

of learning objects from a repository. This architecture combines automatic techniques

of information retrieval with collaborative tagging of documents made by users. In [5],

authors investigate how social tagging could be used in education as a support for

learning processes with use of Tag-based Collaborative System (TaCS), meant for

supporting social and collaborative learning.

An online collaborative reading platform based on Web 2.0 social tagging techniques

that allow learners to annotate various online resources (materials) with freely chosen

tags has been proposed in [10]. Authors presented the TAK (tag-based prior knowledge

recommendation) system, which is comprised of two key components: a tag-based

article reading interface and a tag-based prior knowledge recommendation tool. These

components help learners retrieve and apply their knowledge effectively and efficiently,

and improve their learning performance [10].

Previously mentioned studies provided evidence that collaborative tagging has the

potential to enlarge metadata descriptions of learning resources. Other examples of using

social tagging in metadata creation in the context of e-learning repositories have been

examined in [11–13]. All this systems are more collection of learning resources from

different sources rather than provide original material through specific courses.

More in depth studies are needed regarding the use of tagging within intelligent

tutoring systems, specialized for specific courses. We will present Protus 2.1, tutoring

system that is used in learning programming that shows how recommendation of lessons,

learning material and resources is achieved through tagging.


Protus 2.1

Protus 2.1 is a tutoring system designed to provide learners with personalized courses

from various domains. It is an interactive system that allows learners to use teaching

material prepared for appropriate courses and also includes parts for testing acquired

knowledge. The first completely implemented and tested version of the system was for

introductory Java programming course [14]. The course is designed for learning

programming basics for learners with no previous object-oriented programming expe‐


The implemented system presented in the paper, can deploy an unlimited number of

personalized courses from different domains as also implements various forms of

personalized learning materials to each individual learner.

3.1 System’s Architecture of Protus 2.1

Model and architecture of Protus 2.1 tutoring system consists of clearly defined adapt‐

able, expandable and separated components [15]. System enables easy modification of

adaptation options and personalization of learning materials that are offered to learners

(Fig. 1).

Protus 2.1: Applying Collaborative Tagging for Providing Recommendation




Collaborative tagging


1th lesson (concept)

Resource 1 Resource 2

... Resource m


2nd lesson (concept)

Resource 1 Resource 2

Learnerís actions

Learner model

Learner model



... Resource m



nth. lesson (concept)

Resource 1 Resource 2



... Resource m


Fig. 1. Protus 2.1 architecture

The learning content is divided into units, each of which consists of several lessons

(Concepts). Every lesson contains several resources (presented in different tabs – Fig. 2):

Introduction, Basic info, Theory, Explanation, Examples, Syntax rules, Activity, etc. To

every lesson an arbitrary number of resources and tests can be attached. Their number

can be increased by teachers using an appropriate authoring tool. Protus 2.1 adminis‐

trator’s module contains additional functionalities for adding new learning material:

lessons, resources and tests.

Fig. 2. Course options within Protus 2.1

One or more appropriate tests of learners’ knowledge are attached to each lesson.

Based on the results of tests, system determines the level of learner’s progress, update

the learner’s model and generate further personalization options.


B. Vesin et al.

When a learner is logged in, a session is initiated based on learner’s specific data and

sequence of lessons is recommended to him/her (Fig. 2). A learner has possibility to

change order he/she will attend lessons by choosing the options from lesson/course

sidebar. After selecting a lesson, from the collection of lessons available in Protus 2.1,

system chooses presentation method of lesson based on the learner’s preferred style.

During sessions, learners visit certain resources and solve various tasks. When the

learner completes the sequence of learning materials, the Protus 2.1 system evaluates

the learner’s acquired knowledge. Tests, designed for every lesson, contain several

multiple-choice questions. Protus 2.1 provides feedback on learners’ answers and gives

the correct solutions after every question. The learners’ ratings are interpreted according

to the percentage of correct answers.

Application of tag-based recommendation is used in Protus 2.1 in order to person‐

alize the system. This process will be explained in more detail in Sect. 3.2.

3.2 Tag-Based Recommendation in Protus 2.1

Collaborative tagging activities in e-commerce caused the appearance of tag-based

user’s profiling approaches, which assume that users expose their preferences for certain

contents through tag assignments [16]. Thus, the tags could be interesting and useful

information to enhance recommender system’s algorithms. The innovation with respect

to the e-learning system lies in their ability to support learners in their own learning path

by recommending tags and learning items, and also their ability to promote the learning

performance of individual learners.

A tag is a keyword assigned by a user to represent the subject content, format, utility

or affective characteristics of a bookmark, photograph, video, audio, post, wiki, blog or

other online resources. The goal of tagging is to make a collection of resources easier

to search, to discover, to share and to navigate [17]. Using tags for characterizing digital

educational resources is commonly referred to as collaborative tagging, whereas the

collection of tags created by the different users individually is referred to as folk‐

sonomy [3].

Learners could benefit from writing tags in several important ways. Tagging is

proven to be a meta-cognitive strategy that involves learners in active learning and

engages them more effectively in the learning process. As summarized by [18], tags

could help learners to remember better by highlighting the most significant part of a text,

could encourage learners to think when they add more ideas to what they are reading,

and could help learners to clarify and make sense of the learning content while they try

to reshape the information.

Tagging interface in Protus 2.1 (Fig. 3) provides possible solutions for learners’

engagement in a number of different annotation activities - add comments, corrections,

links, or shared discussion.

Learning resources in Protus 2.1 have been created using the authoring tools by

instructors and they have been stored in a resource repository. Along presentation of

resources to learner, user interface also contains options for creating and reviewing tags

for every resource. Therefore, learners are able to view and rate the resources.

Protus 2.1: Applying Collaborative Tagging for Providing Recommendation


Fig. 3. Tags menu

To create a tag in Protus the learner simply starts by clicking on active learning object

in the content and enter arbitrary keywords in the appropriate textfield (Fig. 3). The

system allows participants to enter as many tags as they wish, separated by commas.

Whenever the learner returns to that particular learning object, the list of tags he/she

has previously made will re-appear. When particular tag is selected, two options are

presented: Edit/Remove, which give learner option to modify or delete this tag.

The most popular tags added by other learners, appear under Others’ Tags. Learner

has ability to add any tags from the Others’ tags to My tags list.

According to the comparative analysis of tag-based recommender algorithms that

we performed, the recommended tags list is generated according to the learners’ and

experts’ tags based on Ranking with Tensor Factorization model which produced more

accurate recommendations then existing state-of-the-art algorithms [19].

3.3 Tag Browsing

Tag browsing, in terms of an individual learner, is an interface to automatically cate‐

gorize information based on tags. This functionality can be accessed through the tag

menu, and provides three options: My Tags, Others’ tags, and Community Tags.

Information provided by the individual learner is located under My Tags in the inter‐

face (Fig. 4). My Tags list presents all the tags the learner has been used, which are

ordered from the most to least frequently used tag. When individual tag is selected, an

option Visit selected lesson is presented, which links the learner to the lesson that was

tagged. Textfield named Name filter adds functionalities for search among tags.

By expanding the Others’ Tags section, the list of active learners in community and

the tags they entered.

By expanding the Community Tags section, the most popular tags are shown in

descending order of number of times used according to the overall use of tags, inde‐

pendent of individual learner who specifies tags. This gives the learner an idea, at a much

higher level, the overall view of all the content. The learner can get a sense of what are

the most important terms and/or ideas at a course level. By choosing one of the tags, the

learner can visit lesson that was tagged. If a learner chooses to search for any specific

tag, corresponding lesson and learning object will be displayed.


B. Vesin et al.

Fig. 4. My tags section



In this section, we will present and discuss the results of evaluation of tagging process

in Protus 2.1, with focus on tagging trends and categories of collected tags. We will

describe the dynamics of tag usage over time, and present observations regarding the

evolution of individual learners’ tagging activities.

In order to test the tagging interface and activities of the learners, we conducted an

experiment with a group of students at Centre for young talents, Novi Sad. The group

of 33 students were chosen for this experiment and were asked to use and test the system.

They were encouraged to use the tagging interface of Protus 2.1 and to express their

thoughts, opinions and plans by tagging learning resources. They had been using Protus

2.1 for six weeks and were encouraged to use implemented tagging options.

We collected quantitative data by gathering information from the tags repositories

in Protus 2.1. We extracted tagging data (tag values and their timestamps) and identified

major categories of tags that emerged during evaluation process. During study, total of

314 different tags have been entered. Figure 5 presents how the number of entered tags

evolved over time. Gross number of tags increases until 5th week, then dropped in the

last week. This suggests that students accommodated over time on tags and tagging

process in Protus and they find it more and more useful as learning continues (and tags

repository increases).

Protus 2.1: Applying Collaborative Tagging for Providing Recommendation


Fig. 5. Tagging in Protus

Gathered data also indicated that tags use increased over a testing period, and that

students found them more and more helpful. Increased number of tags could be the

consequence of their increased attention on tagging interface. The fact that we only

counted number of newly entered tags (and not entering of already existing tags) explains

the drop that has occurred in the last week.

Further, focus in the experiment was to identify the most popular tags and their

categories. Based on some previous findings on the categories of tags in social tagging

systems, we set an initial set of tags categories by their role and goals (General, Technical

and Self-referring) [20–22]. The experiment revealed that there was a higher use of tags

concerning the usability and usefulness of learning resources (useful, difficult, not

understandable, recommend, etc.) rather than more technical ones (loop use, syntax,

task. etc.) or self-referring (return to, cover later, important, for the test, etc.). Figure 5

presents the distribution among different categories of tags during experimental period

of six weeks.

The popularity of 3 categories of tags used by the students in the given period varied,

although some trends were clearly visible. The distribution shows that, use of self-refer‐

ring tags increase the fastest over testing period. This category of tags had the slowest

start as students probably realize later in the process that they can use tagging reference

as a way to remind themselves of previous thoughts about the subjects, lessons or


For the future work, we will try to identify patterns and categories of users based on

tagging practices. By analysis at tagging practices, we could identified different cate‐

gories of tags, their popularity and how it reflects interests of users. We will also analyse

who are the most active users in tagging and identify possible clusters of users according

to their tag usage. The goal is to gather details on tag usage that could benefit the subse‐

quent recommending process.



B. Vesin et al.


This study provides a new learning method for learners, based on gathering tags in order

to better explain learning resources. This way of characterizing digital educational

resources is referred to as collaborative tagging and is defined as the process of adding

keywords, also known as tags, to any type of digital resource by the users rather than

the creators of the resources. Our approach extends the application of social tagging by

designing a tag-based interface for recommendation of learning resources that provides

opportunities for learners to interpret contents of learning resources and find knowledge


We presented approach for providing personalized recommendations using collab‐

orative tagging in Protus 2.1. It is a tutoring system that performs personalized courses

from various domains. We discuss issues related to collaborative tagging as a means for

describing resources within tutoring system. These techniques allow learners to annotate

various online resources with freely chosen tags.

The Protus 2.1 architecture is comprised of two key components, that is, a tag-based

learning resource interface and a tag-based recommendation tool. These components

help learners retrieve and apply their knowledge efficiently, and improve their learning


Tags were used to combine the concept of tutoring system with collaborative tagging

methods to assist the learners studying basics of Java programming language. Tagging

certain activities can help learners summarize new ideas and quickly grasp the structure

and concepts of programming basics. System identifies suitable supplementary material

for learners based on the collected tag repository.

The main goal was to demonstrate that the proposed approach benefits the learners

by embedding the additional information in learning resources. Collaborative tagging

of educational resources is an important issue to study since educational resources are

not meant to be used only by their creators, but ideally to be re-used in different context

and different purposes. One issue to investigate further is the possible influence of

learners’ tagging motivation to the resulted enlarged metadata descriptions and how

student’s background and learning style influence the resource tagging.


1. Kim, H.H.: A personalized recommendation method using a tagging ontology for a social elearning system. In: Nguyen, N.T., Kim, C.-G., Janiak, A. (eds.) ACIIDS 2011, Part I. LNCS,

vol. 6591, pp. 357–366. Springer, Heidelberg (2011)

2. Ocepek, U., et al.: Exploring the relation between learning style models and preferred

multimedia types. Comput. Educ. 69, 343–355 (2013)

3. Zervas, P., Sampson, D.G.: The effect of users’ tagging motivation on the enlargement of

digital educational resources metadata. Comput. Hum. Behav. 32, 292–300 (2014)

4. Salehi, M.: Application of implicit and explicit attribute based collaborative filtering and

BIDE for learning resource recommendation. Data Knowl. Eng. 87, 130–145 (2013)

Protus 2.1: Applying Collaborative Tagging for Providing Recommendation


5. Lavoué, É.: Social tagging to enhance collaborative learning. In: Leung, H., Popescu, E., Cao,

Y., Lau, R.W.H., Nejdl, W. (eds.) ICWL 2011. LNCS, vol. 7048, pp. 92–101. Springer,

Heidelberg (2011)

6. Bateman, S., et al.: Applying collaborative tagging to e-learning. In: Proceedings of the 16th

International World Wide Web Conference (WWW 2007) (2007)

7. Verpoorten, D., et al.: Self-reported learning effects of a tagging activity carried out in a

personal learning environment (PLE) by secondary-school pupils (2011)

8. Anjorin, M., Rensing, C., Steinmetz, R.: Towards ranking in folksonomies for personalized

recommender systems in e-learning. In: SPIM (2011)

9. Cernea, D.A., Del Moral, E., Gayo, J.E.L.: SOAF: Semantic indexing system based on

collaborative tagging. Interdisc. J. E-Learn. Learn. Objects 4, 137–150 (2008)

10. Chen, J.-M., Chen, M.-C., Sun, Y.S.: A tag based learning approach to knowledge acquisition

for constructing prior knowledge and enhancing student reading comprehension. Comput.

Educ. 70, 256–268 (2014)

11. Zervas, P., Sampson, D.G., Aristeidou, M.: Investigating the effect of users’ tagging

motivation on the digital educational resources metadata descriptions. International

Association for Development of the Information Society (2012)

12. Pirolli, P., Kairam, S.: A knowledge-tracing model of learning from a social tagging system.

User Model. User-Adap. Inter. 23(2–3), 139–168 (2013)

13. Kohlhase, A., Reichel, M.: Embodied, constructionist learning: Social tagging and

folksonomies in e-learning environments. In: Conference Proceedings, mICTE 2006 (2006)

14. Klašnja-Milićević, A., et al.: E-learning personalization based on hybrid recommendation

strategy and learning style identification. Comput. Educ. 56(3), 885–899 (2011)

15. Vesin, B., et al.: Protus 2.0: ontology-based semantic recommendation in programming

tutoring system. Expert Syst. Appl. 39(15), 12229–12246 (2012)

16. Niemann, K., Wolpers, M.: Usage context-boosted filtering for recommender systems in TEL.

In: Hernández-Leo, D., Ley, T., Klamma, R., Harrer, A. (eds.) EC-TEL 2013. LNCS, vol.

8095, pp. 246–259. Springer, Heidelberg (2013)

17. Ding, Y., et al.: Integrating social tagging data: upper tag ontology (UTO). In: 2008 IEEE

International Conference on Systems, Man and Cybernetics, SMC 2008. IEEE (2008)

18. Bonifazi, F., et al.: A web-based annotation tool supporting e-learning. In: Proceedings of the

Working Conference on Advanced Visual Interfaces. ACM (2002)

19. Milicevic, A.K., Nanopoulos, A., Ivanovic, M.: Social tagging in recommender systems: a

survey of the state-of-the-art and possible extensions. Artif. Intell. Rev. 33(3), 187–209 (2010)

20. Kim, H.-L., et al.: Evolution of social networks based on tagging practices. IEEE Trans. Serv.

Comput. 6(2), 252–261 (2013)

21. Bagheri, E., Ensan, F.: Semantic tagging and linking of software engineering social content.

Autom. Softw. Eng. 23(2), 1–44 (2014)

22. Treude, C., Storey, M.-A.: Work item tagging: communicating concerns in collaborative

software development. IEEE Trans. Softw. Eng. 38(1), 19–34 (2012)

Domain-Specific Recommendation by Matching

Real Authors to Social Media Users

Jun Wang1(B) , Junfu Xiang2 , and Kanji Uchino1



Fujitsu Laboratories of America, Sunnyvale, CA 94085, USA


Nanjing Fujitsu Nanda Software Tech. Co., Ltd., Nanjing, China


Abstract. It is important to discover informative users disseminating

fresh and high-quality domain-specific contents over social media in order

to keep up-to-date with and learn cutting-edge knowledge, but that is not

easy, especially for new learners due to information abundance or even

overload. We propose an efficient approach to discover potential informative users by matching real-world authors extracted from the latest

domain-specific publications to corresponding social media user accounts.

Mutually reinforcing methods are further applied to identify informative users and recommend domain-specific contents in social media. Our

experiments on real data from arxiv and twitter are used to verify feasibility and effectiveness of the proposed methods.

Keywords: Domain-specific recommendation

ing · Mutually reinforcing relationship



Social media




Social media allow students, faculty, scholars, and the public at-large to communicate and collaborate in ways that disregard institutional boundaries, and are

used for keeping up-to-date with topics, following other researchers work, discovering new ideas or publications, promoting current work/research and making

new research contacts. Many researchers and scholars deem social media, such

as twitter, to be one of the most informative resources available for learning

the latest domain-specific knowledge [1]. However, given the information abundance or even overload over social media, it is not so easy for new learners of

a specific domain to promptly collect these informative users, and actually it

often needs significant time and efforts, especially in cutting-edge technology

domains. Generally, social media users with domain-specific expertise are more

likely to create or share high-quality contents. The latest publications reflect

the state of art of a specific domain, so they are valuable sources for discovering

active domain experts [5]. If we can match real-world authors extracted from the

latest publications to corresponding social media user accounts, potentially we

can find an efficient way to discover candidates of informative users. As social

c Springer International Publishing AG 2016

D.K.W. Chiu et al. (Eds.): ICWL 2016, LNCS 10013, pp. 246–252, 2016.

DOI: 10.1007/978-3-319-47440-3 27

Domain-Specific Recommendation


networks have evolved, they have shifted focus from directly connecting users

to connecting users through contents. A informative user promotes fresh and

valuable contents, and a valuable piece of content is promoted by informative

users [6]. We can leverage the network structure linking social media users and

contents and calculate ranking of users and contents by modeling a mutually

reinforcing relationship (MRR) between them, and further identify informative

users and recommend useful domain-specific contents.

In this paper, we first propose an efficient approach to discover potential informative users by matching real-world authors extracted from the latest domainspecific publications to corresponding social media user accounts. Second, mutually reinforcing methods are proposed to identify informative users and recommend domain-specific contents in social media. Third, the proposed methods are

demonstrated and examined with real data from arxiv and twitter.

We organize the remainder of the paper as follows. Section 2 illustrates the

general framework we propose. Section 3 presents our experimental results on

real-world data. Section 4 introduces some related work. Finally, Sect. 5 discusses

the future work.



Domain-Specific Recommendation


When fetching publications from domain-specific online literature databases,

we can extract author names and other meta-data including titles, abstracts,

references and corresponding author institutions if available. For better engagement and communication, researchers and scholars are highly likely to provide

their real names and professional profiles on social media. For each author name

extracted from the latest domain-specific publications, we can search on social

media sites and check if there is any user with the same name. An author name

often matches multiple users with the same name, so we need to further analyze

these returned users and identify which user really matches the author.

Social media user profiles often provide very informative signals or features

for identifying correctly matched accounts. Many profiles have brief descriptions of research interests containing some domain-specific terms. For instances,

a machine learning researcher’s profile often contains terms such as “machine

learning”, “deep learning”, “statistics” and “optimization”. Personal home page

URLs and professional occupation terms such as “professor”, “researcher” and

“PhD student” are often found in profiles as well. The contents posted by a

social media user reflect her interests, and are also helpful signals or features for

matching. We also noticed that, even if a user has no profile information and

her posts are too noisy to clearly identify topics contained, sometimes we can

still identify and match the account with high reliability by checking who are

followed by the account.

If we have enough verified and labelled training data of correctly matched

author-account pairs, we can use machine learning methods such as the decision

tree to automatically learn the matching rules. If verified and labelled data are

Tài liệu bạn tìm kiếm đã sẵn sàng tải về

Protus 2.1: Applying Collaborative Tagging for Providing Recommendation in Programming Tutoring System

Tải bản đầy đủ ngay(0 tr)