Today I am at the e-Learning Professionals & Practitioners (eLPP – an internal University of Edinburgh group) brown-bag lunch session on Wikis. The current chair of eLPP is Wilma Alexander and she’s kicking off the introductions to today’s speakers just now.
How wikisite characteristics affect users’ editing anxiety and usability – Benjamin R. Cowan, Research Fellow at the HCI Centre, University of Birmingham
Ben will be talking about his PhD research, conducted here at Edinburgh University. He’s now based at the University of Edinburgh at the HCI Centre and they are a new group keen to work with others and look at possibilities for collaboration.
IT in Higher Education is pervasive and, when Ben began his PhD, the use of Web 2. and the drive towards collaboration and interactivity was starting to become apparent with a rise in popularity for wikis. A wiki is a fully editable web-space, users can collaborate, co-create content, structure and navigation, it’s totally modifiable and flexible, the user in in control. From an HCI point of view there are Read interactions and Edit interactions. So we are seeing an example (Confluence hosted) wiki on screen. The Read interaction is the site we see on the web. The Edit interaction is about the wiki markup interface and/or the Rich Text interface (with more limited features than Wiki Markup on the whole).
So, what’s the problem? Well literature shows a low edit rate, sometimes down to how wikis are or are not integrated into the curriculum. However students also report issues around usability and s sense of anxiety and lack of comfort with wikis amongst students, around editing. So Ben’s work looked at what might explain and change that uptake rate.
Wikis commonly use Sandboxes (exploratory technique) and/or WML Tutorials (instruction technique) as training tools but there was little literature on the effectiveness or response to these. So Ben wanted to look at In built training spaces – how do they effect ysability and emoptions towards wiki editing? How does a poor first experience link to a negative ongoing perception.
– editing wiki for Psychology course
personality background page
told other students have been adding content to the course wiki (done by Ben and colleages).
They gave them an introduction to wikis and then one of four training conditions – Direct Edit; Sandbox (without guidance); Tutorial; Tutorial with support.
They devised a Wiki Anxiety Inventory (WAI-E) – a 24 item, 5 pt Likert scale for before interaction, during interaction and about future interaction. This is based on computer anxiety inventory and they are trying to develop this measure presently so do contact Ben if you would be interested in using this in your own work/research.
They also created a Wiki Usability Inventory (WUI), a 22 item, 5 pt Likert scale, again based on computing usability scales. And interviews took place in addition to these questionnaires.
Summary of Findings
Tutorials significantly improved usabiliyt compared with other training spaces – we think this is because they create useful mental models for users.
Tutorials lead to less anxiety during interaction than non tutorial spaces.
They did not influence future related anxiety.
Sandboxes left students feeling left alone to find their own path and unsure what to do.
A second experiment took things further and shifted focus from novice to users with experince and from the interface to the wider wiki system.
The social side of wikis brings in lots of different actors who can potentially see the page, and others edits. This may bring in concerns about judgement and performing for others.
Existing Wiki research has been mostly qualitative, has found users fear jusdgement by other users, lack of confidence in the quality of contributions, and users are reluctant to edit “others” content – which is critical in making wikis succeed.
Can aspects of the system help?
Perhaps anonymity leads to increased satisfaction in chat scenario, Offers protective cloak from evaluation from peers, anonymity attracts lurkers to contribute. BUT anonimity leads to no increase in reputation or social capital for good contribution. And it’s very difficult for marking student contributions and identifying students. Perhaps the solution is a pseudonum so that contributions can be identiies, status can be built, and anonimity from the real world is still in place.
So the research aimed to investigate the effect of user identity in user anxiety around wikis. Students were given editing tasks – they were asked to contribute to content on an existing wikipage, given extract from journal paper (BBS journal) and asked to contribute that in their own words, using the Rich Text Editor. They experienced in 3 identities, named, anonimised, and pseudonym. And under two conditions: adding content and editing others’ content.
They found that identity during editing impacted on anxiety but not usability. But the type of edit had no effect on anxiety – perhaps because of how that was administered. Did a correlation analysis to see how the number of edits effected usability – the idea is often that more use = less anxiety – but they found no such correlation in this experiment.
So the protective cloak seems to be in effect in wiki editing. But Wiki differ from other Computer Mediated Communication and vistural knowledge communities- there are no real time communications, no specific reference to user in viewable text, edits can be tracked in page history. There is a sense of negative emotions associated with social spaces even though this is only a partially social space.
Anonimity isn’t practical in a HE scenario. Pseudonum can be as anxiety inducing as names – people will be in the same student cohort for years and student numbers may not be anonymous. Something around role and transactions may be a better pseudonym.
Athough the experiment found no difference between types of editing this is very hard to test and we are still looking at this at the moment.
At the moment Ben is looking at Public and Private Wikis – there is some concern about these but public wikis can lead to more accurate content. Wikipedia is a far more social issue than a private wiki. BUT a private wiki is a small group that you communicate with a lot vs. the larger more distant Wikipedia community.
We are looking at the social dynamic of a wikispace – incorporating social psychological concepts into this work on how the social structures of wikis work.
And we are looking at the behavioural implications of characteristics – we have lots of data to look through.
Q1) Matric Number usage – surely students will feel tracked with that?
A1) That is definitely something we found in the interviews – fellow students, staf etc. could all trace back via student ids. They felt happier when anonymous but cared less about what they put up in this role.
Q2) How was the editing framed?
A2) They were told that they should edit work by another student. But they were told that that content was wrong…
Q2 again) But that’s very different. You give them permission to edit that content by doing that…
A2 again) Yes, it is a problem. Improving the page would be a nice way of doing that. We are in the process of developing that process for testing editing right now.
Q3) Wikis can be quite brutal social spaces. We’ve found people meeting offline before editing, or using comments to make contributions more tentative/less confrontational
A3) Before this study I was tutoring on Psychology and they were using Wikis. They had tiny pages with HUGE comments sections. Editing the page felt like making your mark, comments allowed social validation of comments before editing. It becomes a bulletin board not a wiki when that happened.
Q4) Maybe we use Wikis with students where we actually should be using a different tool. For comments on a paper or similar you want a different sort of aggregation. Maybe we are searching for other types of aggregation tools – something like Tweetdeck or Paper.li. A way to make a different sort of collaborative document. And I’ve seen wikis used where actually you want a blog with comments for that activity really.
A4) Yup, going around organisations using Wikis there is some of that going on – enthusiasm over fully thinking his through. Wikis have to be embedded in the curriculum for a good reason. Sometimes blogs, bulletin boards, etc. are a far better tool for the task.
Expectation, experiences and afterthoughts: student perspectives on wiki work – Clara O’Shea
Clara will be talking about non-experimental use of wikis with students in courses here at Edinburgh. Clara will be talking about expectations, experiences and afterthoughts and will be using their words as much as possible as as a tutor you do start to see things differently to the students.
I will be talking about wikis in two modules on the MSc in eLearning programme, an online distance programme with about 150 students across 35 countries. We use a range of technologies: webCT, Skype, Social media etc. So use of the wiki is sort of pre-scaffolded. Students can be studying for from one to five years and that brings a different level of emotion and investment in the programme.
We have been running a student writing: innovative online strategies for assessment and feedback project, where students observed and blogged on their courses.
Psychological.. & eLearning run by Hamish McCloud (PsychSocial) and this is an open course. The first few weeks of the module are fairly stuctured but then the students choose what they look at, this all runs through a wiki which they are graded on. In this instance we had about 9 students on the course.
The other course I will be talking about is the Online Assessment module and in this instance we had about 12 students on the course. This also uses wiki and they are also collectively graded based upon that wiki.
Both courses have discussion boards. They are only really used in the first few weeks of PsychSocial but are used more over the course of the programme for Online Assessment.
I was originally thinking I’d address expectations, experiences and afterthoughts in a linear way but of course that is not what happens. People don’t work that way. Instead you have students reflecting on themselves, changing, reflecting on each other. Almost a shared mythology that everyone buys into. I am beginning to think that is an important part of online learning experiences – we have to pretend we share that space even when we are so diverse in terms of location and temporality.
We were looking at feedback cultures; emotion conflict and investment – because students are post grads, they study to develop their career in some way, because there is a sense of the programme as a small space, they are so involved and investment and work so hard but that can increase the risk of editing someone else’s work, you put yourself out there for judgement by peers (and higher!); and the tensions of absence and presence, isolation and community.
Isolation & Community – this comes through particularly in the newbie vs elder dynamic – a newbie being someone starting on the course, few contacts, just starting the introductory module in parallel perhaps, versus students who have done multiple modules, have been through the introductory modules, have good support in the programme. For instance a newbie feels “incompetant” when working with students who have been on the programme for a while whilst an elder talks about the group as being like a friendly tutorial room, to have a sense of them, their interests, their schedules even so a real sense of support. And there are technical understanding differences as well. Students who had been through the introductory modules knew how to navigate and manage the space more confidently.
Particularly for the PsychSocial wiki there is the issue of individually vs. convergence. The goal of that course was to build a connection across the course (similar in Online Assessment). A student comments that they are preparing around their theme offline even though they know they can go back and edit, others report exploring other topics as a fun aspect of combining different contributions. And there is a potential sense of disconnect voiced about the first few weeks due to emails wih tutors.
Of course online if you don’t post on discussion boards, you don’t edit pages, you don’t seem to be present, in fact you don’t seem part of the course. We found students on Online Assesment – a bigger group with a collective task – complained far more often about those not posting. On PsychSocial – a smaller group with some individual responsibilities – there seemed to a group dynamic. And students reported others not pulling their weight, holding them back etc. whilst those students reported reading and writing offline instead.
And finally I want to talk about “placeholders” – is a placeholder for something to be developed a seed or a fence? People posted a vague start – a seed – it was being read as demarcating a territory for developing that idea. We don’t think that is the motivation for creating those stub pages but it can be read that way by other students. There are interesting issues of arguementation – do you correct people or argue your point, how can that be appropriately handled in wikis.
These tensions are what we work with. We need to take the risks that cause those tensions. We have to make them creative and productive, we need to scaffold those situations. The course should work bcause of the tension.
Questions (for Clara or Ben)
Q1) I think the seed and the fence idea is really interesting. It happens in so many scenarios – like funding proposals on a wiki. Colleagues don’t want to offend one another.
A1) Because this was the experience of 2010 we decided to scaffold more for 2011 to encourage truly collaborative spaces by scaffolding more. Some technical stuff in week 1 and then in weeks 2 and 3 we asked small groups to look at papers and to co-author a critique. It created mass anxiety and I found it quite strange. We had had moments of anxiety before but nothing like this. Perhaps there was a cohort effect but making the co-authoring so obvious so early it may have been a step too far. A critique is also quite value laden.
Comment) Sometimes people can use difficulty with the technology (though perhaps not the right tool for the job) but as an excuse to avoid a complex intellectual task
A1) I almost put “procrastination” rather than “destructive” down around the tension.
Comment) People can find ruthless editing of text – style not content – very difficult to deal with, even in, academic publishing scenario.
Q2 – to both Clara and Ben) Social and task orientated interaction
A2 – Clara) Context can be so different. A Psychology cohort is enormous but our modules are tiny groups where you personally know everyone
A2 – Ben) Perhaps there is comfort from a known group, in a larger cohort the social anxiety is different.
Q2 again) Do you think length of task has an impact here – would anonymity
A2 – Clara) We try to let practice develop organically but we assess so of course we have an important role in that space. And 12 weeks is not long – asking something to organically grow in that space is tough, maybe more nuturing flowering plants. Also worth noting that where groups are more academically aligned in terms of skills, experiences, etc. there is less tension there, you can have competing cultures when distinct levels of experiences in the group.
Q3) Have you tried using Prezi or Prezi meeting for placeholder development collaboratively before going to the wiki?
A3 – Clara) We have tried to give each student spaces to experiment with ideas but actually we do want to encourage collaboration and sharing of comments. Having quite open discussions can be far more supportive and productive. So we have thought about having some “Opinion” pages like a newspaper to separate content from arguement and make it more clear for students
A3 – Ben) Yes, that clarity is very important. You want to develop people to gather opinion so using it that way would be very interesting.
Q4) Would you see one way of developing this kind of research to look at differences between different subject areas or different levels of student?
A4 – Clara) There is a lecturer from New South Wales (video of YouTube) who puts up his notes from his lecture on a wiki as a skeleton and lets students fill in lots of content and comment there.
Q5) Do Wikis promise more than they deliver
A5 – Clara) Well there is so much you *can* do but it’s getting students to do that. In this Online Assessment group there was a sort of expectation of equal contribution but we’ve had really interesting discussions about whether that contribution does need to be equal and actually perhaps it doesn’t
Comment) The thing is this is just a tool, we can’t ask wikis to do our job for us – they are just a way to facilitate classroom discussion. I would like to see sharing of experience here.
Q6) Clara mentioned that about half of the MSc in eLearning students were not UK students.
A6 – Ben) Many of our students were UK or American students but there was a mix representative of most of these sorts of course.
Q6 – follow up) I was wondering whether loss of face could be more of an issue for students from another culture or nationality or language background versus (say) a UK student.
A6 – Clara) It can sometimes be the opposite for us. The usual classroom dynamics don’t neccassarily reply. As a tutor I hear the same issues from all students, no matter what location: students doubt their contribution is valued, or what they want to say has already been said.
Comment) In community education there is research that you are given a role in the group then you may stay in that role, not sure that research is well understood with regards to wikis etc.
A6 – Ben) It can be hard to set up within a course that contributions can be different in size and value and type.
A6 – Clara) The problem online is that abence can be interpreted in very different ways – our subconscious fills in those blanks very differently and that can create anxiety and tension.
Q7) Did either Ben or Clara encounter students who had already edited Wikipedia before and did they bring that culture to private wikis with them?
A7 – Ben) We had very few people who were using wikis and there are so few people editing Wikipedia so I don’t think any of our students had done that. Wikipedia can also be quite a small cliquey space
A7 – Clara) We have had a couple of people who have edited wikipedia, more interestingly we have students who had used wikis in other course. That latter group definitely brought a culture and aesthetic into latter wikis. The practice developed over a few years, beyond just one iteration.