Personalized Adaptive Web Systems (PAWS) group

PAWS meeting - Nov 10, 2009

2009-11-17T13:27:00.006-05:00

In today's meeting, there were two presentations. In the first one Rosta presented an introduction to the theory behind people's participation in organizations, which comes mainly from the organizational domain. Following, she presented a study, conducted in two steps, about socialization tactics on several WikiPage projects. The study was focused on the participation of contributors (not on viewers). In the first part of the study it was observed the behavior of the users. in the second part was measured the impact of socialization techniques specially on newcomers. One interesting result was that personalized messages produced more participation than standardized messages on newcomers.

In the second turn, Denis presented the paper "The Effect of Correlation Coefficients on Communities of Recommenders". This paper was written by Lathia, Hailes and Capra from the University College London. They compare different measures of similarity showing their distribution (using MovieLens as dataset) and comparing their accuracy (MAE) and coverage results.

They show an interesting result on their study: that the similarity coefficients don't have a significant impact on the accuracy metrics compared to the neighborhood size. The experiments show that in some cases, using a random similarity measure between users can result in better accuracy of item prediction whether a large number of users have been used in the neighborhood. At the end of the paper there's a discussion about these non-expected results. Between 3 of them (criticism to accuracy metrics, data sparsity and the use of user-based similarity) they highlight that their results show a lack of support to the user-based similarity as a measure to capture an important factor on providing recommendations. So far, different similarity metrics show different rankings and distributions, but none of them extensively outperform any other.

One open question is: Should this result be extrapolated to any other dataset? Other questions: Which kind of similarity measure could help to capture better the concept of "word-of-mouth"? Do we really know what the ratings mean and how to use them to provide better recommendations?

PAWS meeting - Oct 20, 2009

2009-10-20T12:30:00.002-04:00

Summer project at Telefonica: Expert-based recommendation system

Jae-wook introduced his project at Telefonica summer internship. The project is called Wisdom of the Few, which is an expert-based recommendation system. The original idea of the expert-recommendation was presented at SIGIR 2009. This time the study was extended aiming at the following objectives.

Re-examine the idea using the music dataset collected from Metacritic.
Build a client-based UM and recommendation, so that it can solve several issues such as privacy and scalability.
Make a user interface using Adobe AIR, which can be ported to different platform.

We will do the evaluation soon, in the winter 2009.

PAWS meeting - Sep 8, 2009

2009-09-08T11:57:00.004-04:00

The second meeting of the Fall 2009 semester is devoted to discussing selected papers from the UMAP 2009 conference. The two papers presented are focusing on mobile recommender systems.

Paper 1. PBohnert, F. and Zukerman, I. (2009). Non-intrusive personalisation of the museum experience. In Houben, G.-J., McCalla, G. I., Pianesi, F., and Zancanaro, M., (Eds.), 17th International Conference on User Modeling, Adaptation, and Personalization (UMAP 2009), pp. 307–318, Trento, Italy.

Authors utilize a special hand-operated tool - Geckotracker to obtain tracking data from visitors of Melbourne Museum. In particular exhibits of interest and viewing times are collected. The data is then analyzed in order to build a prediction model, capable of recommending new unvisited exhibits to see. Actual log viewing times are used as a primary measure of user interest. Several competing models are built. Leave one out method is used to estimate models' performance.

Paper 2. Partridge, K. and Price, B. (2009). Enhancing mobile recommender systems with activity inference. In Houben, G.-J., McCalla, G. I., Pianesi, F., and Zancanaro, M., (Eds.), 17th International Conference on User Modeling, Adaptation, and Personalization (UMAP 2009), pp. 307–318, Trento, Italy.

Paper focuses on mobile activity recommender system Magitti. Magitti recommends 5 classes of activities: eat, shop, do, see, and read. The data used for building several alternative recommender models was provided by the Japan Statistics Bureau and has data from 10 000 people reporting their activity every 15 min during one whole day. Recommender models take into account several factors, including location, surrounding venues, time of the day, personal calendar, etc. Magitti has gone though a small scale evaluation by 11 researchers and administrative staff users. Results show that a combination of location-based and personal-activity-pattern models works best.

Also discussed:
- UMUAI Special Issue on Educational Data Mining
- UMAP 2009 Proceedings

PAWS meeting - Apr 15, 2009

2009-04-15T14:35:00.003-04:00

Jae-wook presenting paper Combining document representations for known item search by Paul Ogilvie and Jamie Callan

The paper investigates the pre-conditions for successful combination of document representations formed from structural markup for the task of known-item search. As this task is very similar to work in meta-search and data fusion, we adapt several hypotheses from those research areas and investigate them in this context. To investigate these hypotheses, we present a mixture-based language model and also examine many of the current meta-search algorithms. We find that compatible output from systems is important for successful combination of document representations. We also demonstrate that combining low performing document representations can improve performance, but not consistently. We find that the techniques best suited for this task are robust to the inclusion of poorly performing document representations. We also explore the role of variance of results across systems and its impact on the performance of fusion, with the surprising result that the correct documents have higher variance across document representations than highly ranking incorrect documents.

Dhruba Baishya presenting a set of innovative visualization techniques, including:

- eigen factor score
- dewey circles
- ny times api
- flickr ecosystem
- ted sphere
- radial social network
- knowledge network
- author co-citation
- euro2004
- web trend map
- los ojos del mundo
- botanical tree
- tagging behavior in nicovideo
- flickr group

Related links
- http://www.visualcomplexity.com/vc/
- http://infosthetics.com/
- http://developer.nytimes.com/visualizations_app/

PAWS Meeting (25 March, 2009)

2009-03-25T14:25:00.003-04:00

1. Denis discussed his experiment on Recommendation for CiteULike.

2. Danielle presented a paper about “Tagsplanation” - best paper at IUI’09.
Authors investigated the use of tags for generating and explaining recommendation.

3. Tomek presented a paper that describes Document Summarization based on Eye-Tracking with Web-cam. The paper raised some concerns.

4. Katrina (from Switzerland ;-) introduced her research on culturally-adaptive user interfaces. The system is modeling users’ culture along several dimensions (nationality, religion, education, etc.) and tries to adjust its interface.

PAWS Meeting - Mar 18, 2009

2009-03-18T12:56:00.007-04:00

1. Jeniffer presented an article entitled "Accuracy in rating and recommending item features." The paper discusses work aimed at comparing item- and feature-based ratings of images of artwork.

2. Denis presented an article entitled "Clustering the tagged Web." The paper investigates the question of the impact of user-generated tags on improving Web document clustering. Denis also talked about the progress he made on his project and suggested that having Chillean wine while not sleeping enough may be a good idea.

PAWS meeting - Feb 25, 2009

2009-02-27T17:20:00.002-05:00

Comments on the first presentation here.

In the second part of the meeting Zhen presented her work about collaborative information behavior (CIB). She studied aspects of CIB by simulating e-discovery tasks and obtained some insights such as: Communication is frequent and an essential component of CIB; the division of labor is common in the collaborative task of e-discovery; and it is important for collaborators to keep “awareness” of each other’s activities to make sure the collaboration goes well. Based on these insights, some functions for retrieval systems that support CIB were proposed: Collaborative information retrieval technologies should support collaborative information behaviors including verbal communication and text exchanging. A well designed CIR system should support both synchronous collaboration and asynchronous collaboration.

[week 5] Muddiest Points

2009-02-10T10:51:00.002-05:00

Talking about the Probability Ranking Principle, it was mentioned the case of retrieval costs but we didn't deepen into that concept. In the case that Costs are taking into account in a retrieval model:

Which are commonly the values for these "C" costs?

Which variables or factors are taking into account to set these costs values (hardware, size of collection, mean size of documents, etc)?

Another question I have is that several models take constants into account and after some experiments they suggest a range to set the values of those constants when defining a model. I am not absolutely sure, but these ranges must come using metrics such as precision and recall, and the documents collections must come from a programme like TREC. Is this enough to establish a model? It seems that all the theory for these probabilistic and language models is, in practice, oversimplified by smoothing factors and other constants added to the models. How can we be sure that the theory still states when adding these factors and constants in practice?

PAWS meeting - Feb 04, 2009

2009-02-04T13:02:00.003-05:00

housekeeping things:
1. use tags to represent projects and systems in CiteULike. to keep our publications up-to-date. alternatively, update them on wiki.

Project 1: Personalized Exploratorium for Database Courses
Tag#1: DBExp
Tag#2: QuizGuide
Tag#3: WebEx
Tag#4: SQLKnoT
Tag#5: SQLTutor(If the paper is related to the system or Tanja is involved in the system)

Project 2: Adaptive Explanatory Visualization for Learning Programming Concepts
Tag#1: Adapvisual
Tag#2: cWADEIn
Tag#3: jWADEIn
Tag#4: Problets

Project 3: Supporting Learning from Examples in a Programming Course
Tag#1: ExampleSupport
Tag#2: NavEx
Tag#3: WebEx

Project 4: Map-Based Access to Open Corpus Information
Tag#1: MapAccess
Tag#2: KnowledgeSea2
Tag#3: KnowledgeSea

Project 5: Educational Software for Teaching and Learning Information Retrieval
Tag#1: EduSW

Project 6: Project 6: Individualized Exercises for Assessment and Self-Assessment of Programming Knowledge
Tag#1: IndExe
Tag#2: QuizPack
Tag#3: QuizGuide

Projects which are not listed in neither on Taler nor on Paws.
- QuizJet
- Proactive
- Pittcult (Even this is my personal project, you can decide whether this could be listed on Paws or not)

2.
produce slide presentation which includes good system screenshots.

3. Dhruba presented the workshop paper for IUI conference.
CiteAware: Visual Group Awareness for a Reference Sharing System
Awareness thru visualization removes the hierarchies.
Demo the system, CiteAware.

PAWS meeting - Jan 28, 2009

2009-01-28T12:24:00.010-05:00

1. Integrating Conceptual and Procedural Instruction for Middle-school Math – A Cognitive Tutoring Approach (Gustavo Santos)

Gustavo presented his course project on integrating conceptual instruction into cognitive tutors which are only based on procedural instruction. They have designed think aloud task analysis to figure out the common errors and common strategies to design the conceptual part of the cognitive model. The task analysis was done with university students. He is going to continue on the project by evaluating the approach with middle school students and adding adaptation into the approach.

His presentation was followed by some interesting discussion on how to add personalization, how to model student's knowledge (conceptual vs procedural) and how to evaluate the approach.

2. Modeling Problem Solving in CUMULATE (Michael Yudelson)
Mike presented his idea on improving knowledge modeling by blending user knowledge from trying examples with knowledge from problem solving. He presented some result on effect of his blending approach on prediction of user knowledge by both CUMULATE and Knowledge Tracing. His results suggest some careful consideration of blending :-)

His presentation was extended with lots of discussion about CUMULATE versus Knowledge Tracing user modeling.
blending examples and problems

PAWS meeting 2009/01/14

2009-01-14T13:14:00.001-05:00

Web-based parameterized questions for object-oriented programming

Sharon presented her study about the parameterized questions for the Java OOP language. QuizJET and JavaGuide were implemented and tested in classrooms by the undergraduate student at the School of Information Sciences, University of Pittsburgh.

The research questions are:
(1) whether the parameterized questions will work in OOP domain
(2) are en encourage students to work with the questions more
(3) how does the question complexity affect students' assessment

According to the analysis results, the adaptive navigation support implemented in JavaGuide promoted students' participation, increased the success rate, helped strong/weak students to access appropriate questions, encouraged students to make more attempts on easy quizzes, and found strong students had higher success rate for hard questions.

S-vLab virtual lab for teaching and learning in higher education

Anna Riccioni, a visiting scholar from Italy, presented her work. The project is a virtual laboratory for supporting information security course. It is aimed to help heterogeneous students (background, expertise, preferred learning procedures) achieve conceptual understanding and practical skills. The solution to this problem is the virtual lab, called S-vLab in blended learning context. It is used for teaching information security by verifying security principles, model, test, develop secure systems in diverse learning phases and paths. Important S-vLab features are support tools (mentor, expert, judge), modeling (graphical editor), testing, implementation, simulation & evaluation.

She plans to expend the current work to embrace the personalization and the distributed architecture and tracking capabilities including learner profiles and social mechanisms.

System url: http://lia.deis.unibo.it/Courses/TecnologieSicurezzaAK/S-vLab.html

PAWS meeting 2008-12-02

2008-12-02T12:39:00.001-05:00

Tailoring of feedback in online assessment: lessons learned [presented by Sharon]
- importance of system feedback to students, summary of 4 studies
- check feedback influence against learning styles (Kolbert)

PeerPigeon: a web applcation to support generalized peer review [presented by Sharon]
- peer review cycle (stages of review), peer review transforms (review items)
- several peer review patterns (conference style, journal style, etc.)
- RESTful design, ruby on rails

HeyStaks (presented by JaeWook)
- FireFox extension, overlay for Google search (tags, comments)
- Organize search activity as "staks"
- Share staks with peer users, receive staks, search items, tags from peers

PAWS Meeting 11/25/2008

2008-12-01T20:07:00.000-05:00

In this session, Danielle and Jae-wook presented some papers from RecSys 2008, VAST 2008 and VIS 2008.

Danielle presented the paper “The long tail of recommender systems and how to leverage it”, in which the author’s aim is to improve recommendations using many items that are commonly in the long tail, because they have only few ratings (Danielle shows an example graph from MovieLens). The authors present and analyze three approaches: Each item method, Total Clustering method and Clustered Tail method.

Danielle also recommended reading the following papers:

Integrating tags in a semantic content-based recommender

Personalized recommendation in social tagging systems using hierarchical clustering

Jaek presented 3 papers:

Entity-Based Collaboration Tools for Intelligence Analysis (VAST 2008): This paper presents a system called CorpusView. This system is based on entities and is intended for information analysis. Among different functionalities, it provides an entity workspace with a tool for collaboration, note taking, note sharing, collapsing information and recommendations through spreading activation.

Visual Cluster Analysis of Trajectory Data with Interactive Kohonen Maps (VAST 2008): In this paper Jaek showed us a system aimed to make cluster analysis. The system combines clustering and specialization (Kohonen maps) showing trajectory data. It also integrates the work of experts in the automatic process to monitor and control by domain knowledge.

Vectorized Radviz and Its Application to Multiple Datasets (VIS 2008): Radviz is a radial visualization with dimensions assigned to points called dimensional anchors (DAs) placed on the circumference of a circle. Records are assigned locations within the circle as a function of its relative attraction to each of the Das [taken from the abstract of the paper].

Blogger: Denis (Dalpasa)

PAWS Meeting 11/18/2008

2008-11-21T12:44:00.004-05:00

Rosta and Dhruba presented some papers from RecSys and InfoVis this year.

RecSys (Rosta)

Who predicts better? - talks about whether impersonal recommender systems perform as human recommenders
Crafting initial user experience to achieve community goals - talks about the new user activities of recommender systems, including how many new users they lose, how much users will users do, how can they shape ongoing user behavior, etc.
Recommending topics for self descriptions in online user profiles - talks about the user profiles ("about you" entries) in social network sites.

InfoVis (Dhruba)

VisGets - a temporal + geographic + tag cloud visualization system for blog and news on the Web.
Rolling the dice - the best paper of InfoVis '08. Showed high dimensional data visualization.
Who votes for what - an interactive canvas formulating queries (SQL). Adopts a radial visualization that users can do activities like adding, removing variables, and specifying values visually.

PAWS group meeting - Nov 11, 2008

2008-11-11T19:05:00.001-05:00

Meeting included two presentation by Sharon and Sergey.

First Sharon presented a summary of ICCE 2008 (International Conference on Computer Science Education) that she had attended recently. She focused on papers from the following sub-conferences
AIED/ITS & Adaptive Learning
CSCL & Learning Sciences

She briefly presented 5 papers titled:

"Peer Explanation in Primary Mathematics Learning"
"The design of scaffolding in online peer assessment system for learning programming adaptive peer assessment design"
"Formative Evaluation of the SIETTE collaborative Testing Environment"
"Extension of the educational system that can generate explanation of programs - For handling programs in which pointers are used"
"Web based learning material development with less experineced staffs' participation"

You can check more details about these papers on Sharon's blog.

Second, Sergey presented a paper from recent ICKM'08 conference titled "Peer Production of Structured Knowledge - an Empirical Study of Ratings and Incentive Mechanisms".

The paper discusses a rating based incentive mechanism to build large scale ontology trying to address the following problems:

Quality of peer-production
Motivation to contribute
Quality of ratings

paws meeting 11.04.2008

2008-11-11T11:31:00.000-05:00

1. guest presenter: Dr.Amir Pourabdollah
Title: Ternary relations model of information management
TRM is a formal description of a technique that establishes bi-directional and dynamic node-link structures in which each link is an ordered triple of some three nodes.
TRM may be used as a tool for the analysis of information models, to elucidate connections and parallels, and as a construction kit to build new paradigms and/or applications in information management.

2. Gustavo Santos
Title: e-Fer - A Web Based Learning Environment for Chronic Wounds

general objective: to develop and train skills for diagnosis and treatment of chronic wounds

so far, focusing on inner loop adaptation. however, it still requires more data(cases) for the adaptation.

could reference to Dr. Crowley's work.

PAWS Meeting 10-21-2008

2008-10-25T11:59:00.001-04:00

The presenter of this week was Tomek. He talked about "TEPR" based on the paper "Measuring the Task-Evoked Pupillary Response with a Remote Eye Tracker". Couple experiments of TEPR were discussed:

Mental multiplication
Short-term memory
Aural vigilance

Some issues of remote ETs were proposed by Sergey and Michael. Michael brought up a question "why not applying remote ET study by using webcam?" But for pupillary dialation, webcam is not good for that.

Reminder : no meeting next week (10/28/2008)

PAWS Meeting 09-23-2008

2008-10-04T13:44:00.003-04:00

The speaker of this meeting was Rosta. The main theme was tagging and some of the subtopics were:

Ranking in folksonomies
Tag recommendation
Resource recommendation

There was a brief comparison between Google adapted page rank algorithm and the folkrank algorithm in the beginning of the presentation. Seems that folkrank is able to do a better job. Daniele raised the question about the applicability of TF/IDF in large amounts of text.

Rosta presented the Dogear game, a game that uses human intelligence to acquire some metadata about pictures and finished the presentation making some reference to the tag cloud tool Wordle.

PAWS Meeting 09-30-2008

2008-09-30T13:39:00.001-04:00

We started one hour later this week. Danielle's presentated on "Why people tag". She based it on the two following two papers:

Ames, M and Naanan, M. Why We Tag: Motivations for Annotation in Mobile Online Media. CHI 2007.
Nov, O and Naanan, M and Ye, C. What Drives Content Tagging: The Case of Photos on Flickr. CHI 2008.

Rosta raised a question: "why people don't tag".

Peter suggested discussing Conference Navigator some time soon.

Reminder: No meeting next week.

PAWS Meeting 2008-9-16

2008-09-29T02:13:00.001-04:00

Dhruba talked about his work on promoting social awareness.
Three systems have been presented, that follow the same approach - providing holistic visualization of activity going on in a small group (2 to 20 people).
The contributions of single users are represented based along with the novelty of a contribution.
The systems are:
1. Vlaag: supports a small group concentrating on a certain task such as planning the event or discussing a small topic. The system is used as an discussion aggregator. Instead of mailing her/his contribution to everyone in a group a user posts it to Vlaag. S/he can also see there the other users' posts, see who has been active and react on the most recent contributions.

2. Peercev.com provides visualization for twitter.com. The system retrieves twits that the user is following and creates a map, where s/he can observe who and when has been added new information.

3. my.like.groups is a similar client for citulike.com. The system creates two kinds of visualization:
- user centric, showing the recent activity of the communities that the user is part of, and
- community centric, showing the contributions of all users in a community.

Several ways of developing the systems have been suggested:
- so far systems do not model/take into account tags, which could help a user to do meaningful information filtering
- the created maps are not clickable. By clicking on a user, the system could open its communities, by clicking a community a system could unfold all its users, etc. Supporting navigation over a social network seems like a nice idea.

PAWS Meeting 2008-9-9

2008-09-09T12:49:00.003-04:00

Topic: Peer Review
Speaker: Sharon
1) PeerWise

multiple-choice question
Auckland, UBC
Large class
Be able to rate difficulty each question: 0 - 5
Be able to identify question into category
Active use of the system may contribute to deep learning

2) Open community authoring of targeted worked example problems

A prototype tutor authoring system using community volunteers to create personalized solution
Four phase: Generation, Evaluation, Use, and Improvement
Elementary Math Domain
Question quality: Math teacher > Amateur > Other teacher
Solution quality: Amateur > Math teacher > Other teacher
Educational content system can benefit from open contribution

PAWS Meeting 2008-09-02

2008-09-02T14:09:00.000-04:00

Inaugural meeting for the Fall 2008 series

Jaewook presenting 4 of his systems:
- YourNews
- YourSports
- TaskSieve
- NameSieve

YourNews. News aggregator. Open user modeling. Short/Long-term model based on sessions and time. Allows individual feeds and tabs of feeds to be added, keyword-based user model allows addition/removal of keywords for user model fine tuning.

YourSports. A clone of YourNews for news related to 3 major US sports: baseball, football, and hockey.

TaskSieve. Task related news exploration. Based on TDT4 data set. No personalization. Note-taking for specific tasks.
NameSieve. Based on TDT4 data set with respect to 4W(who, what, where, when). Utilizes external name entity extractors.

2008-04-10 Dr. Lillian Cassel talk. DL series

2008-04-10T16:24:00.001-04:00

Summary in names and links
Dr. Lillian Cassel http://www.csc.villanova.edu/~cassel/
NSDL: National Sciences Digital Library http://nsdl.org/
CITIDEL Computing and Information Technology Interactive Digital Educational http://www.citidel.org/
5S Model http://www.dlib.vt.edu/projects/5S-Model/
SMS Strand Map Services http://strandmaps.nsdl.org/cms1-2/docs/index.jsp

04-02-2008 Paws Meeting

2008-04-02T13:34:00.001-04:00

presenter: Jennifer Lin
Topic: Moving digital libraries into the student learning space (the GetSmart Experience) 2003

6 steps of information search process.
there are some limitation in existing e-learning tools; visualization tools should be embedded in the tools.

Concept Mapping tools:
1. CMap
2. WebMap

research questions:
1. will it help improve information search by integrating CMS, digital libraries?
2. will concept map help collaboration?
3. will it increase academic performance?

Questions:
how do they use concept map to improve search?
integrate the CMS with external search from digital libraries,(not very clear) how does it help in info search?

experiment results:
1. search increases
2. after deploying GetSmart, the average quizzes assessment is lower. performance is higher.
students do more studies after GetSmart introduced.
3. increase the collaboration opportunities

we don't have the tool allows students to manipulate their own content! visualize the content is suggested, may ease the complicated text structure. but we do have the self-assessment tools, which they don't. but in some sense, HCI course, higher level concepts are not easy to covered in the current design,.
--some kind of tool allows students to self organize the content is important. new tech seems much more attractive. will concept mapping be replaced by tagging?
--in their system, maybe miss some social technologies.

PAWS Meeting 3-19-2008

2008-03-19T13:38:00.001-04:00

1.
Javier presented his work on data mining. It looked exactly like below:

var p = new Presentation();
p.setSpeaker("Javier");
p.setTitle("How can help Data Mining tool to the instructors?");

p.hello();
p.doYouRememberNovemberPresentation();

while (p.hasMoreSlides()) {
var s = p.getCurrSlide();
s.present();
}

p.questions();

Very briefly, Javier talked about:

His view on the structure of the course
Anomalies in the log files
His choice of visualizing them for the instructor (classification trees)
SimuLog -- a student activity log generator
His plans for his stay here

2.
Jae-Wook showed a new version of YourNews system featuring three customizable tabs.

PAWS Meeting 2-20-2008

2008-03-03T20:19:00.000-05:00

Mike presented the project he participated in while working with ITS group at UPMC. The project resulted in the UMUA paper.

SliceTutor is tutoring system helping medical doctors to learn making a diagnosis on the basis of a tissue samples.
The main goal of the project was to evaluate multiple cognitive models and test corresponding approaches of user modeling data aggregation. The predictive validity of resulting user models was compared using several metrics. The metrics themselves were also analyzed.
The main contributions of the project are:
- the argumentation in favor of curve-based vs point-based metrics for evaluating user models;
- recommendations for the improvement of the classical model tracing algorithm, and
- a good practice of exhaustive analysis of the diversity of user models that could be derived from a single initial data set of raw usage transactions.

PAWS Meeting 2-20-2008

2008-02-25T13:35:00.001-05:00

Danielle presented 2 papers: "Adaptive Faceted Browsing in Job Offers" and "Database Courseware from Kennesaw State University".

The former, it was about we have "too large" cyberspace. We need adaptive tools to help browse this corpus. Adaptive faceted browsing is a good approach. It is not just normal faceted browsing. It can scope down and show related topics in what we are looking for. Its engine comprised 2 parts; user similarity model and global relevance.
From the user study, a number of facets does matter on time user spending on the system along with adaptation feature. In others way, time is not a factor on systems without adaptation or system with recommendation. The user behaviors did the same as time spending. One thing that was not clear was the paper can't tell how much more time spending on the page was good or bad.

The latter, the paper presented database courseware from Kennesaw State University. Their coursewares have both online and offline applications. They presented in animation. It is a good oppurtunity for us to integrate your social navigation toward their applications.

PAWS Meeting 2-13-08

2008-02-20T14:45:00.001-05:00

Maria Harrington presented her recent pilot study on her virtual reality project. This project aims to simulate a part of Pittsburgh surburban terrain where lots of plants and animals are residing. She has been trying to examine the validity of this system especially in terms of supporting elementary school students when they were going out field trips to the area which the system simulates.
Today's focus was especially the comparison between virutal reality and real environment in which sequence they were presented to the students. She conducted a study comparing VR-to-Real (VR) and Real-to-VR (RV) and which sequencing is more stimulating the subjects with some elementry students aged from 8 to 11. In VR sequence, students were presented with the VR system in their classrooms and then they went out to the real field trip, and in RV was in vice versa. In order to measure which sequence is more effective, she asked the subjects to annotate a map (real) about the things they found out during the study. By this measure two grups (VR and RV) were compared.
What she found was as follows,
• Environment effect: VR can match the real for in-situ performance especially for in curriculum data.
• Order effect : in both total and plant only map annotations, practice makes perfect.
• Interaction : Salamander effect - critical design feature, seen in the total map annotations, not the plant only annotations.
• Transfer from RV and VR, with RV showing stronger effects and stronger for in curriculum data.
• Main contribution : use real and virtual together for maximized learning, prime before real, and reinforce post real.

After Maria's presentation, group members talked about how to promote the update of the current group wiki system and maintainers for each section were assigned.

PAWS Meeting 12-07-07

2007-12-07T14:41:00.000-05:00

RecSys'07 Overview
RF gave An overview of the RecSys'07 conference. The sessions were as follows:

Privacy and Trust
Collaborative Filtering
Learning
User Issues
Short Papers (poster format)
Indutry Panels

As part of the overview RF summarized two papers. Group members were often initiating interaction with RF due to a mix of confusion and interest in the presentation content. The papers chosen were:

A Recursive Prediction Algorithm for Collaborative Filtering Recommender Systems
Incorporating user control into recommender systems based on Naive Bayesian classification

Kudos to RF for good presentation!

For more details refer to the slides.

Agenda Items
1. PB expressed his concern about group member not using blogs. We should start unless we want to switch to CoPE.

2. TL suggested making both meeting minutes and presentations private. He will create an additional account on WordPress and take care of the rest.

3. Cookies were delicious.

PAWS Meeting 11-9-07

2007-11-12T18:07:00.000-05:00

Two visitors from OPen Adaptive Hypermeida group, Universidad Autonoma de Madrid, Javier Bravo and Manuel Freire introduced the research projects and discussed about how OPAH and PAWS can collaborate.

Brief descriptions of the projects introduced among a lot of them are as follows.

* Wotan - an adaptive hypermeida course system
* WotEd - Authoring tool for Wotan, based on a graph visualization framwork, CLOVER
* SimuLog - supports authors to test the evaluation tool and helps detect anomalous patterns
* ASquare - Author Assistant, provides high level abstraction using mining tools and helps instructors.
* Willow - mining tool using concept map and histograms, makes use of free text questions and answers from instructors.

In terms of PAWS collaborations, following issues were discussed.

1. Platforms
- Wotan as an activity for the KT ecosystem (or as a portal to enable acess to KT activities)
- Future UM testing/datamining
- Sharing user model between CUMULATE and Wotan UM with name space sharing/mapping

2. Technology
- Graph visualization, Plagiarism detection (novelty detection), Similarity visualization, Free-text question grading

PAWS Meeting 10-12-07

2007-10-16T15:02:00.000-04:00

"Developing Recommendation Systems Using IC-Service" presented by Aliaksandr Birukou

Aliaksandr is a visiting scholar from University of Trento, department of information and technology, part of a research group doing the research in intelligent information processing, and software engineering. Aliaksandr's work focuses on intelligent information processing.

This talk was focused on his project on "Implicit Culture". The main idea is helping human or artificial agents to grasp group's culture of the community since the knowledge of the communities is often implicit.

Implicit culture framework is built to encourage desired behavior of the members of communities by observing what the members of the community do, extracting their actions and suggesting appropriate action to others.

They have developed a system called IC-Service that he discussed in some details in the presentation. The system provides recommendation service in any domain. Every new domain only needs to be defined in terms of agents, actions, objects, attributes and the community culture. The community culture is defined as a set of rules. IC-Service The system is open source.

Aliaksandr provided an example of IC-service in "web service" recommendation in this talk. This is a very different domain than usual recommendation application which is somehow recommending services to application plus humans. The domain brings some challenges for evaluation. The main goal here is recommending services that have been selected by others for similar requests. The evaluation of the system is done through simulation.

We had some discussion about the evaluation methodology that the real community behavior cannot be simulated and the simulation assumes perfect behavior of the community. We also had some discussion about the difference of "implicit culture" and "social navigation" which implicit culture can be much more general focusing on all kinds of behavior of a community while social navigation focuses on navigational behaviors. However, examples given by Aliaksandr are more in the overlap part of implicit culture and social navigation.

PAWS Meeting 10-05-07

2007-10-12T13:21:00.000-04:00

The IUI '07 paper "From social bookmarking to social summarization: an experiment in community-based summary generation" by Oisin Boydell and Barry Smyth
http://doi.acm.org/10.1145/1216295.1216311
presented by Sharon.

Nice paper. Several interesting ideas.
Authors try to generate web-page summaries based on the social tags from del.icio.us
Tags and web-pages themselves are used as input queries for the Lucene snipet generator. The resulting snipets are analyzed and combined to generate a summary of a page.
Four experiment have been done:
1st Experiment compared the closeness of socially-generated summaries to the summaries provided by people on del.icio.us.
2nd experiment explored the relation between the quality and the length of social summaries
3d experiment analyzed how the quality of a summary would depend on the number of tags available for the page
finally, in the fourth experiment authors tried to generate the community-focused social summaries.

PAWS Meeting 2007-09-28

2007-09-28T16:50:00.000-04:00

Paper presentation by Danielle
Khan, L., McLeod, D., and Hovy, E. 2004. Retrieval effectiveness of an ontology-based model for information selection. The VLDB Journal 13, 1 (Jan. 2004), 71-85. DOI= http://dx.doi.org/10.1007/s00778-003-0105-1

Technology in the field of digital media generates huge amounts of nontextual information, audio, video, and images, along with more familiar textual information. The potential for exchange and retrieval of information is vast and daunting. The key problem in achieving efficient and user-friendly retrieval is the development of a search mechanism to guarantee delivery of minimal irrelevant information (high precision) while insuring relevant information is not overlooked (high recall). The traditional solution employs keyword-based search. The only documents retrieved are those containing user-specified keywords. But many documents convey desired semantic information without containing these keywords. This limitation is frequently addressed through query expansion mechanisms based on the statistical co-occurrence of terms. Recall is increased, but at the expense of deteriorating precision. Focusing on audio data, we have constructed a demonstration prototype. We have experimentally and analytically shown that our model, compared to keyword search, achieves
a significantly higher degree of precision and recall. The techniques employed can be applied to the problem of information selection in all media types.

Discussion
Decided to keep reading logs in private www.wordpress.com blogs only visible to group members

PAWS Meeting 2007-9-21

2007-09-21T14:53:00.000-04:00

Chirayu presented a paper titled as "The folksonomy tag cloud: when is it useful?" In this paper, the authors intended to see the utility of the famous "tag clouds." Even if it's already being used in many commercial Web services like Flickr, its utility has not be clearly identified so far. Even Tomas Vder Wal, the one who coined the term "folksonomy", refered TC as cute but with little value.
They performed an experiment and compared two information seeking options: searching vs. TC and find out if TC was conceived as really useful by subjects. The subjects were given 10 information seeking tasks for 10 articles (per task). The system showed top 70 user tags and search box they can use.
The experimental results shows that TC was favoured by the subjects in the expriment sessions: 48.0% vs. 41.2%. People relied on TC made mor queries (higher efficiency) even when relevant keywords were in TC. Their conclusions include that TC was useful for browsing and non-specific information discovery, it provided visual summaries, and it required less cognitive load.
What was clear from this study was the strength of browsing compared with the ad-hoc searching (as in our study with KnowledgeSea). TC provided a new browsing option but the study did not take into account other browsing options than just simple searching. It might be needed to see exactly how TC is accepted differently than other browsing techniques because the results of this paper might have been made due to the browsing scheme itself, not from just the TC.

20070914 PAWS meeting: Ontology-based Annotation

2007-09-14T14:37:00.000-04:00

Ontology-based Annotation by Sergey Sosnovsky

provide semantic annotations
knowledge sharing
indexing by human

motivation: using ontology on applications
definition: creating a markup of web documents using a pre-existing ontology and/or populating knowledge bases by marked up documents.

important characters: automation;format; languages; etc.

SMORE by UMaryland:
it does extract data from documents, but it still replies on human to edit and index the concepts.
but there are problems of manual Annotation, eg. time, expensive, storage, trust
solution: search engine like annotation service

O-based Annotation:
supervised: eg. MnM, human user can accept or reject the annotation.
unsupervised: eg. Amilcare;Annie; T-REX; Pankow: use the templates and query them from google, and collect the hits
SemTag(by IBM): it extracts a huge amount of webpages, automatically get a huge amount of disambiguated semantics tags
Conclusion:
it's a necessary thing
manual is bad, automatic is good

Questions/Comments:
-2006 AAAI paper, there's a work, codes&errors, the value of mining the database? what's the relation btw this and the C parser? (C parser is not the annotation, it's the formal structured grammar)
-common sense is certainly more than grammar

-
Adaptation Hypermedia Technology <--------------------> simple key words
(meaningful concepts)
we can consider to use meaningful parsers and other technology mentioned earlier in this talk to bridge these two.

2007-09-14T14:18:00.000-04:00

Personal Services - Summer Visiting Researchmanship and Implications for Adapt2.
A group member talked about his one-and-a-half month long stay in Hannover, Deutschland. While there they worked on personalization component of the Personal Reader application. The presentation is available at PAWS Website. That work (as far as the minutes sindicator understood) focused (at least partially) on bringing a common communication layer to all services responsible for generating the content of an adaptive information access system. The choice the designers made was RDF and RSS. More details in the presentation slides. The presenter also showed a few nice pictures from Hanover.

Paper Summaries Repository
A discussion on paper summarizing and sharing was initiated. Given the choices, majority of people wanted to use CiteULike. Member would be storing summaries in a separate instance, perhaps a web log. No one objected avoiding Google as a web log provider in order to screen them from yet another piece of sensitive personal information they collecting every second.

Eva Millan Presenting at SIS.Pitt

2007-05-30T10:00:00.000-04:00

Today Eva Millan, professof from Malaga visiting SIS presented her work on:
Intelligent web-based tools to support e-learning

Abstract
In this talk I intend to present some of the work being developed by the (ia)2 group at Malaga University, Spain. To this end, I will present two of the systems I have been working on lately. One of them is TAPLI, which is an adaptive web-based learning environment for Linear Programming, composed of a set of adaptive tools: a) an adaptive hypermedia component, that is responsible of presenting contents; b) a testing component, and c) a drill-and-practice component, which automatically generates problems and coaches students while solving them. The other tool is a log analyzer that has been developed and integrated into Moodle by means of web-services. This log analyzer allows to infer the the behavior of a learning community. The inference is supported by a Bayesian Network Model. The indicators are automatically computed from the logs, and used as evidence for the Bayesian Network, which computes the posterior probabilities and uses them to generate a natural language representation for the teacher to learn whether the students divided work, coordinated, cooperated or collaborated during the course.

2007-05-18T16:43:00.000-04:00

Rosta Farzan and Jill Freyne, ASSIST: Adaptive Social Support for Information Space Traversal

Jill and Rosta presented their effort to integrate multiple social technologies, called ASSIST. The social technologies included in this approach is like social search and social navigation which are to support ultimately social information access.

The ASSIST engine works in two domains, ACM scholarly articles and YouTube videos. The engine interact with the users and builds matrices for storing interaction information with users for searching and browsing modes and make recommendations based on them. The process is achieved solely social based, not content based.

Evaluations are planned with 20 graduate students in real settings for 2 months in order to test two hypotheses, 1) Interaction will be affected by social recommendations 2) Social searching and browsing will add values to the system.

Michael Yudelson, Brainstorming: A "Perfect" Academic Paper System

The group talked briefly about the polling for the new features to be added to CoPE system. Suggested features from the members so far are arranged in http://www.sis.pitt.edu/~paws/perfect_paper_sharing.html. We will think more about it and do the preliminary polling next week.

2007-05-04T15:06:00.000-04:00

Jae-Wook Ahn, "Open User Profiles for Adaptive News Systems: Help or Harm?"

System YourNews, RSS feed aggregator, aggregates 60 feeds. Traces user model. 6 news categories. UI lists news items and open user model (keywords). Open UM is editable, users can enter their own keywords.
Personalization - filtering, tf-idf, importance of the keyword weighting.
Focus of the study - open user model. Viewing and manipulation of the keywords of interest. Does editable user model help or harms users.
Study design: task oriented, 10 subjects, 2 news topics, 2 usage sessions, find news relevant to topics. User activity has been logged. Results of news search were compared to results collected by experts.
User feedback. System performance depends on news topic. Experimental system was better received. Keyword adding dropped from session 1 to session 2.
Conclusions. Open user models could harm. Experimental system showed higher trust.

Apr 27th Meeting

2007-04-27T00:02:00.000-04:00

First Presenter: Michael Yudelson
Topic: Comparison of 3 Paper Sharing Systems (CoPE, CUL, Comtella)

Michael compared three similar system in the purpose to share papers - CoPE, CiteULike, and Comtella. He compared them from various aspects such as support of community, navigation and interactivity, along with demonstration of each system. After his presentation, he collected questionnaires asking opinions about paper sharing system and explained the results. Privacy is important and quality and easy of use is also necessary to users. The discussion about what makes a certain system superior to others was followed. In order to attract world, the new functionality to meet users' needs like the CiteUlike case and usability should be considered as well.

Second Presenter: Rosta Farzan
Topic: 'Social' Systems: Designing Digital Systems that Support Social Intelligence

Rosta made a presentation about a paper regarding how to design a system to support social intelligence. Using social proxy, the authors provided several visualizations to show activities happened in online chat rooms. The information about the presence and activities can help create coherent behavior comprising social intelligence. In our system, Dr. Brusilovsky pointed that it is necessary to share the social activities happening inside of our system and the information about who contribute what in the system. Dr. Vassileva suggested that using Comtella it was motivational to users to see the effect of their actions.

PAWS Meeting 04/20/2007

2007-04-20T17:59:00.000-04:00

It was a first meeting of the "Social" Month. Though Jill's presentation was not social, but an interesting one.

1. Sharon was presenting Dogear system - IBM-developed system for social bookmarking on the enterprise. Nothing too novel. They took everything good from del.icio.us and implemented it in nice and fancy way. The main question they have - can the social bookmarking system be beneficial in the enterprise settings. They use REST as a communication protocol. Tags and bookmarks have a representation in form of RSS feeds, hence it is possible to subscribe, for example on some user's bookmarks. They also plan to implement recommendation based on collaborative filtering.
They describe an experiment with 686 participants. Only 1/4 of them created tags and bookmarks. Half of them navigated bookmarked resources. The question then, - what did the rest of "participants"?
They did some quantitative analysis of numbers of tags per bookmark, number of people using a bookmark etc., they also applied some social network analysis to identify the connections between people through the resources they tag, bookmark and navigate.

2. Jill talked about a system Volant, that implements a rather novel methodology for Web search. The main motivation behind the system is following: Modern Search engines do not take into account that the querying is only a first step in the search process; usually after people query an engine they start navigating through linked pages. Hence the idea is that the search engine should not try to generate the pages which are the best answers to a query, instead they should direct users to the pages that are the best entry points, starting from which a user can easily navigate to the page meeting his/her information goal.
Then there was a lot of math formulating the work of a search engine exploiting navigational patterns existing on the web and retrieving the best entry points. And at the end it some evaluation was described, a very thin, though.
Interesting project.

PAWS Meeting April 13, 2007

2007-04-16T22:09:00.000-04:00

First Paper: Design and Evaluation of an Adaptive Incentive Mechanism for Sustained Educational Communities
Presenter: Rosta Farzan

Online communities have faced insufficient participation. Even though providing incentives is such a good way to solve the problem, its affect gives too much, low quality participation. The authors proposed a new incentive mechanism to generate quality and quantity contribution from online users. From a related work, collaborative quality evaluation mechanism, not only need user participation but also quality control. In academia, we measure quality of papers or journals by counting times they are cited as well as in online communities, counting on numbers of clicks. But not every click means a positive attitude. In an explicit way, measuring on user rating has a drawback, "rich get richer".

The authors came up with the Comtella, online community system. Comtella provides an ability to share class-related web-resources. C0mtella also has a mechanism encouraging users to rate resources by providing a reward to user contribution. Users can contribute on the system by two ways: one is to post a new URL web resource or, two is to rate others' contribution. Each user receives a certain number of rating points to use. The number of rating points varies on a level of membership. Calculating adaptive rewards is done by two models, community model and individual model. Community model controls a number of contributions and usefulness of news resources. Individual model monitors quality of rating, based on the assumption that the average of all ratings the resource gets should be equal. From their experiment, users in the test group were more active in rating articles and the articles with higher ratings were more likely to be chosen to summarize. Limitations of Comtella are a scale of rating is too rigid, 1 and -1. The system should provide more on the aspect of rating such as originality, interestingness, relevancy etc. Another thing is the system should not reveal the absolute c-point. It should provide only relative ordering of the article.

Discussion
The features Comtella has should be integrated to CoPE. CoPE would get benefit from an ability for users to rate articles.

Second Paper: Classifying Learning Engagement Through Integration of Multiple Data Sources.
Presenter: Tomek Loboda

The paper is about combination of self reports from high school students and teacher reports. The system classifies action sequences into five finite state machines. The motivation of self reports is related to cluster of behaviors associated with learning success such as effective monitoring, study behaviors, etc. The motivation of learners and self-regulation can be a stronger predictor of learning performance than prior academic achievement. The web-based tutoring system for high school mathematics was set to prepare students for high-quality exams such as SAT.

In summary, sources can be integrated and used to classify students' beliefs into learning situation. Self reports from learners can be identified as disengaged but like other systems that students use ITS in a manner eliciting learning pattern. Results indicated that there is possibility that if we seed this models in advance with learner profile data which is timely and inexpensive to elicit and with strategies quite predictive will be used once when student begin studying with the ITS.

Group Meeting March 30, 2007

2007-03-30T14:09:00.000-04:00

presenter: Sergey Sosnovsky
topic: Ontological Technologies for User Modeling

once upon the time....while ontologies meet user modelling

firstly, Sergey briefly described the definition/design/history of user modelling and introduce the most frequent used technologies in user modelling.
he mentioned that ontoligies may specifically help in representation, elicitation, application technologies.
secondly, Sergey explained ontologies in semantic web, light-weight/heavy-weight ontologies and multi domain ontologies.

so, how ontologies can help user modelling???
a. ontology overlay UM
b. perosonal ontology views, including subset of concepts from the domain ontology, part of the domain ontology and individual network of concepts
he also brought up the discussion regarding to share ontologies and gave examples
1. OWL-OLM system visualizes the personal ontology view
2. Foxtrot system visualize the sets of concepts
c. ontologies for UM mining

Experiment Design:
a student has some c knowledge and then shift to learn java. there are some share concepts between these two. the suggestion is to use ontologies mapping. use GLUE algorism to calculate the relevancy between C and java. (due to time limit, Sergey didn't drill into this section, please refer to ppt file from paws website)
the point is to find out is it better in the transition from C to java manually than automatically?
discussion:
1. why not try pairwised comparison??
2. this is the case of C and java, they have similar case, but how to make it more generalizable?
3. if the domain is the same, how to make the ontology mapping much more accurately?

Group Meeting March 23, 2007

2007-03-26T21:39:00.001-04:00

First Presenter: Michael Yudelson
Topic: Digital Object Repositories centered on FEDORA

Michael presented digital object repositories centered on FEDORA. The digital object can have data stream and data stream has its own disseminator that operates on contents type. There was a debate about defining digital object. In our systems, specific quiz can be a digital object and the function of disseminator is to create and display contents. He also introduced FEDORA and it makes possible to store and discover by indexing digital objects.

In terms of collaborating and managing learning contents, FEDORA can be helpful.
Beyond Fedora how we can all objects with navigation support and handle digital object user-adaptively is our problem to solve.

Second Presenter: Jaewook Ahn
Topic: Personalized search and visualization for YourNews

Jaewook presented some newly added functions in his YourNews system. The system has short-term and long-term user profile and generates user profile by observing user’s actual view. It adds top five keywords in entire user profile to every query and Rosta pointed that there is some possibility to lose user’s intention by disrupting their point of view. Dr. Brusilovsky added that query expansion is good technique but we should be careful when apply it to system. In long-term model, the range of focus can be sparse than short-term model, hence it can be meaningful to cluster data in long-term model into meaningful group.

Group Meeting March 16, 2007

2007-03-18T19:48:00.000-04:00

Presentation 1: Danielle

Title: Learning SQL with a computerized Tutor

Danielle presented projects related to SQL tutorial which is a new project for several members of the group. The main goal of the project is developing a personalized SQL tutorial using different techniques from adaptive hypermedia.

Danielle presented an existing SQL tutor which is developed by University of Canterbury which is designed to address students’ problems regarding to learning SQL such as problem to memorize database schemas, or misconceptions in students' understanding of the elements of SQL. The tutor tries to provide useful hints and helps when students make any error regarding a specific SQL command instead of general messages that are hard for students to understand.

Danielle provided a demo of the SQL tutorial from University of Canterbury in which students get a SQL question and have to fill out empty boxes. While working on the problem, students can ask for different kind of hints and helps.

We had a discussion about the difference of “bug model” and model of students’ evaluation. Basically, bug model is the model of students’ misconceptions while evaluation model deals with students’ evaluations of their responses to a problem.

Peter also described briefly about the new SQL project which will be similar to our C project but for Databases. It means the system will have different components such as quizzes, examples, and script based evaluation. It also includes components developed in different places and the main challenge of the project is a meaningful integration of the components and a design of a centralized user model that communicates with all the components. So in this SQL system, the teacher will be able to structure the course and assigns activities to each lecture. Every link in the system will provide guidance and the guidance can be social or knowledge based.

Presentation 2: Sharon

Sharon presented her example authoring system. The system is a community based example authoring in a way that the teacher puts the example into the system but then every line of the example will get annotated by the students. Students are doing the annotations as an educational activity. They are also able to criticize annotations of other students. In the current design, once a student start annotating an example, it will be locked for that student and other student can only rate the annotations. This might be changed in later design.

Sharon is planning to finish the implementation as soon as possible to run the first study which will be done in a classroom with 9 students. The study will allow half of the examples to go through two rounds of annotation and ratings and the other half through only one round of annotations and ratings. At the end an expert will evaluate all the annotations and ratings. This part is only for the purpose of the study and evaluation.

In longer term the project will let use to provide guidance on open core examples since the content of the examples are not enough for providing the guidance but added annotation can help to provide guidance or even provide better search for the examples.

PAWS Meeting 2007-03-02

2007-03-02T14:09:00.000-05:00

1. English for Dummies: Searching for Rules versus Exploiting Examples

Presented by Angela Brunstein

In our recent study on the Chemnitz Internet Grammar, we replicated and extended our earlier results on that system by presenting one and the same version of the grammar system to all students and by unifying tests and instructions. Altogether 200 students processed two chapters of the grammar on Present Continuous and Present Perfect for 30 minutes. They either searched for specific contents in the grammar or explored the grammar on their own. Before and after processing the grammar, they answered factual questions and mastered application tests. Corresponding with literature and our earlier results, students with the specific search task answered questions related to the contents they had searched for more in detail. In contrast, students with the unspecific exploration task answered more questions but less in detail. More interestingly, both groups differed remarkably when applying their knowledge. When learning by examples, students who explored the grammar on their own improved their skills more than students who had searched for specific contents. This effect can also be found in tendency when learning by rules and examples, but not for learning by rules only. In that case, a guiding instruction highlighting most relevant facts in the grammar seems to enhance learning.

2. PEBL: Web Page Classification without Negative Examples

Presented by Chirayu Wongchokprasitti

Web page classification is one of the essential techniques for Web mining because classifying Web pages of an interesting class is often the first step of mining the Web. However, constructing a classifier for an interesting class requires laborious pre- processing such as collecting positive and negative training examples. For instance, in order to construct a “homepage” classifier, one needs to collect a sample of homepages (positive examples) and a sample of non homepages (negative examples). In particular, collecting negative training examples requires arduous work and caution to avoid bias. This paper presents a framework, called Positive Example Based Learning (PEBL), for Web page classification which eliminates the need for manually collecting negative training examples in preprocessing. The PEBL framework applies an algorithm, called Mapping-Convergence (M-C), to achieve high classification accuracy (with positive and unlabeled data) as high as that of a traditional SVM (with positive and negative data). M-C runs in two stages: the mapping stage and convergence stage. In the mapping stage, the algorithm uses a weak classifier that draws an initial approximation of “strong” negative data. Based on the initial approximation, the convergence stage iteratively runs an internal classifier (e.g., SVM) which maximizes margins to progressively improve the approximation of negative data. Thus, the class boundary eventually converges to the true boundary of the positive class in the feature space. We present the M-C algorithm with supporting theoretical and experimental justifications. Our experiments show that, given the same set of positive examples, the M-C algorithm outperforms one-class SVMs, and it is almost as accurate as the traditional SVMs.

3. Discussion of job-recommendation system Proactve

presented by Daniela Hyunsook Lee

PAWS Meeting 2007-02-23

2007-02-23T11:55:00.000-05:00

1. Personalized Course Navigation Based on Grey Relational Analysis

Paper presentation by Sharon

This paper introduced a study on a personalized course navigation based on a measure called WGRA (Weighted Grey Relational Analysis). It aims to let users browse open corpus educational Web resources and recommend them according to user interests. The task is done by the WGRA measure and implicit feedback information. It can provide easily maintained user models, low complexity for the computation required, and ease to add knowledge.

Even though this study seems rather simple, but it has a strong relevance to some of the projects done within our group such as KnowledgeSea, so it was recommended to the persons to check the paper.

2. Unsupervised and Supervised Machine Learning in User Modeling for Intelligent Learning Environment

Paper presentation by Tomek

The motivation here is to reduce the effort for constructing user models and improve transferability across applications. The proposed framework takes two learning steps, unsupervised and unsupervised in order to model students' learning during interaction with an education support systems (exploratory learning environment). The first step is done by clustering, where users' interaction behaviors are fed in. After the experts examined the clusterings, they label each of them according to the students' achievements and use them in the next step, online classification (k-means).
The ideas for applying the proposed approach to relevant projects were discussed and this paper was also recommended.

PAWS Meeting 2007-02-16

2007-02-16T17:43:00.000-05:00

1. First Paper
Rosta presented a paper "Accurately Interpreting Clickthrough Data as Implicit Feedback". The authors investigate users' behavior given a Web search task. The data collected involved eye-tracking data. Questions that the paper addresses are:

- Which links do users click (as opposed to only look at)?
- What does each click mean (does it mean?
- Do users scan top to bottom?
- Which links do they evaluate before clicking?

2. Second Paper
Danielle presented a peper "ResultMaps: Search Result Visualization for Hierarchincal Information Spaces". This visualization is based on a Treemap. It presents search results in the context of full information space. A "global" ResultMap is showed next to regular textual list of search results. Each textual result can be expanded to see its contribution to the "global" ResultMap (a "local" ResultMap is presented). Each ResultMap presents resources of different type (e.g. article, lecture, video) with different color. The sizes of map cells does not carry any information. User performance evaluation and enhencements of the representation is on the authors future research agenda.

3. Housekeeping
We discussed reviving our Wiki. The proposed structure looks as follows:

Journals
Conferences/Workshops
Relevant Groups
Systems
PhD Theses
People
Opprtunities (Companies, Jobs etc.)
Community Services (Tools etc.)

Jae-wook will create the structure. All PAWS members are responsible for adding content relevant to their area of focus. We may assign people responsible to different categories later.

PAWS Meeting 2007-02-09

2007-02-09T14:08:00.000-05:00

1st talk: Michael Yudelson
Topic: Recommenders for Information Seeking Tasks: Lessons Learned.
Summary:

Goal of the recommender is to meet user specific needs with respect to:Correctness ,Saliency,Trust,Expectations and Usefulness

General Advice :

o1.Support multiple information seeking tasks

2.User-centered design nShift focus from system and algorithm to potentially repeated interactions of a user with a system
3.Recommend nNot what is “relevant”, But what is “relevant for info seeking task X”
4.Choice of the recommender algorithm: Saliency (the emotional reaction a user has to a recommendation) , Spread (the diversity of items) , Adaptability (how a recommender changes as a user changes) , Risk (recommending items based on confidence)

experiment results found out:

Bayes - generated similar recommendations for all users and PLSI - generated random, “illogical” recommendation

Discussion:
very good papers for everyone.

2nd talk: Jae-wook Ahn
Topic: A Novel Visualization Model for Web Search Results.
Summary:
-what's missing with current approaches? semantic relations/views, degree of relevance in terms of subjects of interest
-speed and subject of interests are new in this research.
-the system list of keywords for user to manipulate the their interests. weights are adjustable.
-WebSearchViz system, will release the opensource soon. so far, not so much detail demostration on their web.

Discussion:
how does the rotation really work?
profile the keywords has been done long time ago, it's been moved to profile the concepts

universal portal for centralized login discussion:
benefit from each system's user model.
Chirayu talked about the RBAC backend role based server as an option.
how to use existing technology to integrate all these?

PAWS Meeting 2007-01-26

2007-01-26T14:23:00.000-05:00

Talk by Sergey Sosnovsky
Flexi-OLM: Flexible Open Learner Modeling

Summary. 3 papers by Susan Bull presented at ITS conferences (2004, 2006). OLM - visualize UM and let negotiate and/or change it. Students learn by "reflecting" on the observable content of UM. Not all OLMs are for students, there are some for teachers. FlexiOLM - different views on the UM. Domain - C programming. Color-coding user knowledge level. Domain structure - topics with child concepts. Piece of interactive content is indexed with a single concept. Misconceptions are also modeled. Evidence propagation - aggregation of the means. Users can edit their models when: 1) new user comes, 2) learning occurs outside the system, or 3) system estimates knowledge incorrectly. Persuading LM - user expresses disagreement, system explains it's beliefs, and possibly allows user to prove his point. Negotiating LM - chatting with the system trying to align user's beliefs and system's beliefs. Chat-bot is controlled by a person. 350 negotiation pieces were pre-authored.
LM visualizations - hierarchy, lectures, concepts map, index ranked, summary.
Experiment 1. (2004) Is having different visualizations beneficial? Is there a preference for a particular visualization? 23 subjects. Users in general are positive about the system.
Experiment 2. (2006) What are student preferences regarding persuading, negotiating ad editing? 8 subjects (3rd year grad students). No significant patterns found.
Experiment 3. (2006) Chat-bot study. Liked the bot less, when it was not agreeing.

Discussion. There's no attempt to investigate gaming behaviors. 5-value likert scale is prone to over-inflating category "N/A". 4-scale is better sometimes. The study design is rather weak. No implicit hypotheses tested, only explicit (questionnaires). Choice of subjects is questionable (knowledgeable C-programmers and not currently learning students).

PAWS Meeting 2007-01-19

2007-01-19T14:23:00.000-05:00

1st Talk by Joerg Brunstein

Topic: Eye movements as a window to the mind.

Summary:
the paper is mainly explaining the experiment on eye tracking.
they test on people in light, in dark, retell and describe the story after it's being told. Then measure their eye movements by looking at a whiteboard, contrast to the case of looking at the picture itself.

Discussion:
how does this experiment fit into our visualization interfaces design?
how do we do it differently in our studies?
possible solution: measure the eye movements from context and annotation, use the eye tracking result as a confirmation checking.

2nd Talk by Chirayu Wongchokprasitti

Tpoic: NewsMe: A Case Study for Adaptive News Systems with Open User Model (Preliminary Examination Rehearsal)

Summary:
NewsMe is a Personalized News Access System.
It allows users to provide feedbacks about theirs interests in news. Feedbacks are further used to construct the user model and recommend relevant news articles to users. It currently retrieves news from 82 RSS news feeds from 21 sources.

Discussion/Suggestion:

the blacklist tracking graphs are odd. Because only certain people were really using the function. The specific case could be raised separately. It might due to that users were not aware of the blacklist function. however, telling users the capabilities of the functions should be done before the experiments.
the precision on business topic is bit higher than other topics, why? why topic based?
if user have better knowledge on the topic, did they do better? (it wasn't mentioned/analyzed in the study)
implicit data is not reliable. the data is noisy. cos the question was asking for Japanese Car industry. people tend to recognize news title and may consider that news as relavant.
describe the problem and technology>describe the study,what do you want to explore more>demo the system>explain the experiment

PAWS Meeting 2007-01-12

2007-01-16T11:56:00.000-05:00

1st talk by Sharon:

Ko-Kang Chu, Maiga Chang and Yen-The Hsia Designing a Course Recommendation System on Web based on the Students’ course Selection Records

A system recommends non-mandatory courses to students based on the model of their topic preferences. The model of preferences is populated using the history of non-mandatory courses a student took earlier.

Every course belongs to several topics (relations are weighted). The system classifies student interests and recommends her new courses similar to the interesting ones.

The results of the systems evaluation do not support the prior hypothesis that the accuracy of recommendation would go up with the time.

Discussion:

Main criticism to the paper was concerning the idea of topic-based modeling of interests for courses recommendation. If a student took 3 courses on AI the system should think that she must have a strong interest in AI, but the student would not need any more AI courses – she already knows everything she needs by that time.

2nd talk by Tomek:

Nikolaus Bee, Helmut Prendinger, Arturo Nakasone, Elisabeth Andre, and Mitsuru IshizukaAutoSelect: What You Want Is What You Get: Real-Time Processing of Visual Attention and Affect

The paper presents a model and an experiment on using eye-tracking and physiological data (galvanic skin effect, blood rate) for prediction of user’s preferences. They try to build a system that predicts when a subject makes up his mind towards one of two options on the basis of changes in the unconscious behavior caused by building up the preferences (relation between the attention and emotions).

In the experiment a user chooses between two actions (prefer one tie over another).

Discussion:

Criticism:

- The cognitive model reported in the paper is questionable. What if users were presented not ties but shoes or laptops? What if the choice should be among 5 or 100 alternative? What if the alternatives are not similar (not only ties)?

- The experiment is affecting subjects. The system makes a guess, what tie a user would like to buy – it influences the results. What if a user was not sure about her choice? What if a user tries to be nice? Or vice versa – tries to contradict the system?

There were more comments during the discussion. If anyone remembers, please add them here.

How we can apply models like this in our work?

Two directions:

1. Using Eye-tracking as a research tool for:

- Systems (Interface) evaluation

- Modeling information:

a. As a supplementary source if modeling info, for example to prove that the user is working on what we think he is working -> we model them correctly (Conati)

b. As one of the main sources of modeling (as in the discussed paper)

2. Using eye-tracking as a novel interface in real applications.

PAWS Meeting 2006-12-4

2006-12-11T02:37:00.000-05:00

1. Encouraging contribution in learning networks using incentive mechanisms presented by Rosta
- Social Exchange Theory
- Learning Network is a network of resources and people

2. Folksonomy-based Course Authoring for Flexible Student Modeling Presented by Sergey

- Folksonomy-base authoring is the way people author courses together but during the meeting we have a discussion on what exactly is folksonomy on course design because course is pretty much stable and solid. It means that it does not require a group of people to collaborate on course design authoring.

PAWS Meeting 2006-11-27

2006-11-27T14:16:00.000-05:00

1. Ion Juvina's presentation

Title: "Model-based Highlighting to Support Selective Reading on the Web"

Ion Juvina, a post-doctoral fellow at CMU, Psychology Department, ACT-R Group, gave a talk about his dissertation work. He talked about a series of studies on Web goal-oriented behavior he and his colleagues run. In each of them the task users faced was to find goal-relevant information on a series of real Web sites.

In the first study they identified spatial abilities as the best predictor of performance. They controlled for many confounding factors, e.g. working memory capacity, text comrehension and processing speed. They also found spatial abilities to negatively correlate with amount of navigation needed (revisits, back button usage, etc.). They proposed some changes to CoLiDeS model. They found the new model (CoLiDeS+) to perform better, but not significantly.

The second study involved presenting voice suggestions to users (generated by the model), e.g. "Click link X". They found it significantly improving performance of non-spatial users.

The third study involved visual annotation of links the model recognized as goal-relevant. They found that these suggestions helped to increase performance.

The fourth study attempted to test the above ideas on visually impaired users. Again, they found spatial abilities to be a significant predictor of performance. They didn't find any effect of suggestions.

2. Michael's CoPE functionalities update

Additionally to all absolutely stunning features the system had before, now everyone can:

- get system's help with adding authors of a paper
- search papers

PAWS Meeting 2006-11-20

2006-11-27T11:03:00.000-05:00

Sharon presented the following papers from ELeran 2006 related to student advising. Both papers were from the same group at California State University, Dominguez Hills.
1. Intelligent On-line Advising with Expert System Shell
  This paper discusses the reason for having an automated advising system and what the system needs to offer. Partly it describes the requirement related to specific design of the study program at Cal State. The system uses an expert system shell to provide advice to the students. It allows conversation with students leading to an advice. The system has been weakly evaluated over one semester by only asking the students to play with it.
2. Student Advising SystemThis paper describes the process which is required to provide an advice to a student for taking a course. It describes what factors needs to be considered such as pre-requisites, co-requisites, indirect-requisite, importance of the course, ... . The current version of the system described in this papers requires manual work of the students.
  We discussed implementing some of the ideas discussed in this paper in CourseAgent. Specially since currently CourseAgent lacks the consideration of the requirements. Peter mentioned the idea of providing personalized requirements for the students based on their preferences and for example planned courses
Danielle presented a paper from IADIS International Conference on WWW/Internet 2005 titled An Integrative Model on Web User Experience.
The paper describes a methodology for measuring the effectiveness of interactive systems such as adaptive systems based on "Technology Acceptance Model". It also provides an example of evaluation of an interactive system.
Sergey suggested that given the data collected over the last semesters through the questionnaires, Danielle can try some of the ideas suggested in this paper.

PAWS Meeting 2006-11-06

2006-11-14T13:18:00.000-05:00

InfoVis '06

Peter introduced InfoVis '06 conference. While we consider using the toolkits and resources announced there, we can think about,

1) How to make visualization adaptive and use in our context?
2) We have lots of approaches but same set of users. How can we apply visualization in this environment?

Bayesian Nets

Tomek presented two papers on Bayesian networks.

1. "Distributed Bayesian Networks for User Modeling". Roberto Tedesco,
Peter Dolog, Wolfgang Nejdl, and Heidrun Allert

2. "Intelligent Bayesian agent as a facilitator in e-Learning". Maomi
Ueno and Toshio Okamoto

Basically BN can be used as modeling tools in that it is transparent and easier to be generalized. The first article talked about BN based user models, which exist over several distributed Web based adaptive eLearning applications and can be merged into a global user profile.
The second paper introduced an approach which provides students with motivational messages, so that the drop out rate of them greatly decreased. It also adopted BN and the performance was better than other similar ones such as decision trees. This paper was quite controversial in that there was too high advance in drop out rate (what if giving random messages?) A possibility of confirmation bias was pointed out and the variables, learning time/access time might have been weak as evidences to support the conclusion.

Social Navigation for ELearn Schedule

Rosta introduced a system for community based searching, navigation, and scheduling. It was built upon the corpus from CACM and the accessible URL is as follows.

http://ir.exp.sis.pitt.edu/ks2/DL/AnnotatEd

PAWS Meeting 2006-10-30

2006-10-30T16:16:00.000-05:00

Discussion of Central Login for PAWS Adaptive applications

Overview
Centralized login is an analog of .Net Passport. There exists a unified storage of identities (here user identities) that applications refer to and rely on when it comes to registration of new users and authentication o existing.

Motivation for the centralization of a login
- a lot of users use multiple applications but have separate logins for them
- maintaining user identities at the site of each application is costly and ineffective

Note
User identity is not user model/profile, user application specific data about user is not a part of identity

User Identity fields

Mandatory fields

Name: Last, Given

Password

Recommended

gender

Optional

Organization/Affiliation

City + State (Region outside U.S.)

ZIP (Postal code)

Country

Comments (How did you hear about us, etc)

Incorporating group and group membership information has been postponed till further more thorough discussion.

Planned course of action
- developers should provide their user identity data for future merger
- identity data should be retrieved from Central User Identity Store and not from a local application
- conflicts in user names should be resolved in an organized fashion when needed
- interface of interaction with User Identity store to be presented later

PAWS Meeting 2006-10-23

2006-10-24T18:04:00.000-04:00

I.
1) Peter told great news: two PAWS papers won the outstanding ELearn2006 paper awards:
- "AnnotatEd: A Social Navigation and Annotation Service for Web-based Educational
Resources" by Rosta Farzan and Peter Brusilovsky
- "Time and Structure Based Navigation in Web Lectures: Bridging a Dual Media Gap" by Robert Mertens, Peter Brusilovsky, Sergey Ishchenko and Oliver Vornberger.
Congratulations to the authors!
2) More about ELearn2006: turned out it was an earthquake there and a pretty strong one. Seems, like bad things can happen even on Havaii.
3) Finally, Peter showed the system for community-based navigation of ELearn2006 schedule. The paper about this one should win ELearn2007 outstanding Paper award ;-)

(*)
We again had problems with Java5 on Mac. Need to order PC for our future meeting or push Steve Jobs (or who is responsible?) to release Java5 for Apple.

II. Rosta's presentation about the SWEL@AH06 FOAF paper:
Authors analyzed several schemas for User(learner) characteristics representation:
- FOAF (5 categories),
PAPI (6 categories: personal, relation, sequrity, preferences, ...)
IMS LIP (11 categories)
eduPerson By Internet2 and Educause (43 elements and 2 categories)
P. Dolog's framework

They built a taxonomy of user characteristics taking FOAF as a basis and extending it with elements borrowed from other models.

Discussion (How we can use it?): probably publish FOAF profiles of CourseAgent users, then it will open CourseAgent user models and interconnect them w/ the rest of FOAF network, which allows the system to reason beyond its own knowledge.

III. Vikrant's demo. Visualization for CourseAgent.

Multidimensional visualization of a student model.
Courses are represented as dots in the space, where carrier goals are dimensions. VIBE-similar interface. Course color reflects the status: planned, not-planned, taken.

New Visiting Scholar from University of Joensuu, Finland

2005-09-24T11:05:00.000-04:00

Starting today and for three weeks a PhD student from Computer Science Department, University of Joensuu, Finland - Roman Bednarik - will be visiting PAWS to help with setting an experiment that involves eye-tracking.
Welcome, Roman! Vítám! Tervetuloa!

Congratulations ir. Ewald Wladimir Alexander Ramp

2005-07-03T16:36:00.000-04:00

On June 14, 2005 PAWS member Ewald Wladimir Alexander Ramp was awarded an MS in Computing Sciences of Technische Universiteit Eindhoven.
Paul de Bra, Marc Voorhoeve (both TU/e) and Peter Brusilovsky (University of Pittsburgh) gave him 9/10 points for his dissertation with condidtion (or "moral obligation" as Prof Brusilovsky referred to it) that he (Ewald) will write one more paper about research results presented in the dissertation.
Congratulations, Ewald. Goed gedaan!

E-Learn 2005 accepts 2 papers [co-]authored by PAWS members

2005-05-31T08:44:00.000-04:00

E-Learn 2005 - World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education has accepted 2 full papers [co-]authored by PAWS members:
1. Authoring and Delivery of Adaptive Electronic Textbooks made Easy (by Ewald Ramp, Paul de Bra, Peter Brusilovsky), and
2. Collaborative Paper Exchange (by Peter Brusilovsky, Michael Yudelson, Sergey Sosnovsky)

Dr. Peter Brusilovsky (PAWS Director) wins NSF career award

2005-05-03T13:13:00.000-04:00

Dr. Brusilovsky - PAWS Director - won the prestigious five-year, $440,000 award to fund his work in personalized information access. Congratulations!
"The inspiration for Brusilovsky’s NSF proposal was his award-winning research on QuizGuide*, an adaptive system that he and his colleagues developed to help students in a programming course select the self-assessment quizzes most relevant to them."
*QuizGuide is one of the PAWS projects

PAWS member leaves Pittsburgh yet still with us

2005-04-29T09:51:00.000-04:00

Our good friend Ewald leaves PAWS main quarters at University of Pittsburgh soon.
Despite this fact he is still a PAWS man. Now PAWS are on both sides of Atlantic.

Dr. Peter Brusilovsky (PAWS Director) gets tenure @ PITT

2005-04-27T13:20:00.000-04:00

Dean Larsen (School of Information Sciences, University of Pittsburgh) invited the SIS faculty and staff to a reception to celebrate the accomplishments of faculty and staff who have recently received awards of promotion, tenure and recognition of contributions to the community.
In particular Peter Brusilovsky was recognized to the rank of associate professor with tenure.
(from SIS NEWS)

PAWS hits double at AIEd 2005

2005-04-07T09:40:00.000-04:00

The PAWS group members have got two full papers accepted at the 12th International Conference on Artificial Intelligence in Education:
1. Peter Brusilovsky, Sergey Sosnovsky, Michael Yudelson, Girish Chavan. Interactive Authoring Support for Adaptive Educational Systems
2. Michael Yudelson, Peter Brusilovsky. NavEx: Providing Navigation Support for Adaptive Browsing of Annotated Code Examples
Needless to say this fact is a good news and will surely be celebrated by the group.