Having hotfooted it over from George Square where I’ve given my Innovative Learning Week session on Social Media – more on that in a future blog post – I’m now at the School of Education for a seminar from Dr Melissa Terras, Reader in Electronic Communication at the UCL Department of Information Studies and Co-director of the UCL Centre for the Digital Humanities. She’ll be talking on “The Virtual Visitor: What Do We Know About Users of Digitised Cultural Heritage?”.
You can find out more about the event on the Digital HSS website: http://www.digital.hss.ed.ac.uk/?page_id=493
As usual this is a liveblog so may have spellings typos etc. Just let me know if you have corrections and comments.
Jen Ross is introducing Melissa and mentioning her fantastic talk yesterday. And now over to Melissa:
I’m originally from near her, from Kilcaldy, so delighted to be up here. So… I do a lot of analysis but over the last few years we have been getting increasingly requests from cultural heritage organisations around where we are, in Bloomsbury, who want to learn about technology and work with us. So I’ll be talking about how users use the online collection database and a bit on log analysis. So I’ll talk about what we know, what we can’t know and what technologies are available to us at the moment.
So the project I’ll be talking about is the British Museum (BM) Collection Database Online (COLL?). And particularly the research area. That area of the website was launched in 2007, by the end of 2009 there were 2 million objects online, 600+k images. By the end of 2011, 800k images. But the BM wanted to know how they were being used, what impact this was happening.
So we’ve been doing user studies at UCLDH for some time
Log analysis of Internet Resouces in the Arts and Humanities – we got permission from the servers to see how many people were using AHRC resources (not all sites had very many at the time but it was a while ago). We had an idea of if we build it they will come but we have a better idea now, we know that it isn’t that simple
User Centrerd Interactive Search with Digital Libraries – led by my colleague Claire Warwick this looked at the comparison of in person and online library experiences – functionality and information seeking environment features such
Virtual Environments for Research in Archeaology – we took a huge long term dig site used by the University of Reading. We set up wifi and we provided technology to try to speed up note-taking. Huge fun… technology isn’t great in the rain… and struggles in the sunshine!
QRator – this is with KCL and an exhibit of biological specimens. We have iPads around the room and visitors can answer questions there, or on the website, or on Twitter – like “is this an ethical thing to display”. And we can analyse those comments. And the vast majority are intelligent answers – my PhD student Claire Ross is working on that. And she has a NESTA grant to do the same thing will all 5 sites of the Imperial War Museum.
Linksphere (Claire Ross was research assistant here) – the intent here was to build a huge integrated system with the University of Reading. But we did have this highly engaged student who wanted stuff to do. She wanted to do user studies but the original project wasn’t ready for that so she asked someone at the BM to let her see their log data… and she worked on that!
We also have workplacement students and one of them, Vera Motyckova, was engaged on this specific project.
We have various PHd students working with the British Library, the British Museum, the Science Museum, Grant Museum. We are very lucky to work with amazing partners here.
So I was going to talk about how all museums and cultural heritage are putting materials online but we don’t know why people come to these sites, what they search for. So we wanted to try and find out with the BM website.
So, Methods here…
Quantitative – log analysis, link analysis, analytics (eg. Google Analytics), surveys
Qualitative – Open ended survey tasks, interviews, focus groups
Toolkit for the Impact of Digitised Scholarly Resources – fantastic tool outlining these methods.
So Google Analytics or web log analysis tells us where visitors come from, which pages they have visited, how long they stayed, where they came from, search terms captured from the search within the site. Online survey (survey monkey( allows an in-depth suvvey. In-depth interviews – we didn’t have time for one to one interviews but these would have been very valuable.
So some up to date contect. The BM had 10.5 visits/60m pageviews in 2011. 1.2m included a visi to the Research section containing the collection online – that’s about 11% of the visitors. That’s the serious section on research. By comparison 5.8m physical visits to the museum (a rise of 4.9%).
The study I’m talking about was a year and a bit ago. So we looked at stats from 20o9 – 2010 when there were almost 9 millions visits, averaging 6 pages per visitors. 30% of returning visitors from 230 countries. almost 2 million searchs on the site.
The most common search terms are about famous artefacts at the BM. But we are interested in the database and what people are looking and why.
So when you look at log analysis and Google Analytics the problem is it’s a bit dry… So some 21% people that come to the collection do so directly. About 15% come in through other sites – mainly the BM shopping sites. Many visits also came from other sites that indicate good amounts of links in.
So this graph (on screen showing a major jump) shows access of the BM site via mobile. There is an instant boost in mid October. And that was 1 week – the sim card switching on period – after O2 launched the iPhone. It’s that obvious in the stats. But whilst you can see usage you don’t know about intent or usage behaviour.
We put a pop up survey request on the site – it hit 1 in 5 people – and ran from 3rd June 201 to 2 July 2010. We got 2.5K respondants. We were delighted by that. We did 30 main questions – multiple choice, Likert scale and defined tests.
So if we look at the age of respondants – most were 21-30, a good amount between 31 and 50 years old. Not in use withe under 20 youths of course. Some 29% of users were in UK, 91% in England; 6% from Scotland, 3% Wales – non in Nothern Iereland. And also a large group from the US.
When we asked people how they had come to know about BM COL most said they had heard through a friend or a colleague, many from an academic websie. But relatively few had come in via search enine.
And when we asked users what best describes their reason for COL – 5o% were doing academic research. 12% using it for professional research, only 18% ish for personal interest/fun. And when we looked at academic role it was clear that PG Researchers were the major group, also professors and other students.
We asked users how they wanted to search. Most wanted free text. They also wanted to search by type of object which is interestin though th emetadata doesn’t support that yet. Some did want the data organised by museum gallery but most were not passionate about replicating the offline experience online.
On the whole people were looking for specific ojects, some on a group of objects. Unsurprising that latest research not as well used – that’s what they were doing.
And next… a lesson in phrasing questions well here.. we asked what type of object poeple were coming to the website to see. Some of the print materials are hard to acces in the museum so of particular interest.
In terms of how often the collection database was used something like 30% was using the COL for the dirst time. Another third occassionally. We really needed to understand that – it’s the response rate rather than who is using the site.
When we asked the users what improvements they would want – most wanted more images, many wanted improved search facilities, and some wanted more collections. Some wanted zoomable and 360 degree images. Great stuff but is it realistic to do on this scale. Notably most users wanted to have COL be easy to find and access and improvements have been made to address that.
Users wanted images in higher res, image ordering, less steps for retrieving images once logged in. Option to return to search results after ordering an item.
Users wanted to see more objects – better information on physical presence/loan included.
And we asked which social media appliations people wanted in COL. 65% said no. Some said Facebook. Some said tagging. This was a big Meh! The users were not that interested, there for serious research.
We asked users if they reuse image based content elsewhere – some said no, some said yes… we didn’t but should have asked for more info here.
We asked people are you on the BM;s website as a result of a visit to the museum – we asked twice and people were clear that visist and website visits were differnt. COL use it for serious research vs. fun of in-person visits.
And finally we had a task at the end of the survey. In computer science you give people a sample task to find out how they would run a search on your website. So we used the example on Greek Vase. I knew that specific item and couldn’t find it… so “you are searching for a greek case which you know in the BM as you have seen it in a print catalogue. It is an Attic black-figured lekythos from the myth of psychostasia. And it has the catalogue number B 639.”
So, if you look for B 639, or psychostasia, or lekytos you get nothing . The 6 people out of 174 users who found the vase took the space out of “B 639”. The British Museum were surprised at this…
We had a similar task for a painting. And in both cases we found that users try to search in Google type ways – people learn information strategies in these places and we can’t be the same but we have to be aware of that.
One of the interesting analytical things you can do is compare where people say they are with where the logs show them. These cross checkings help you see if you have a representative sample. We didn’t offer the survey in other languages though – perhaps we should have. But little cross correlation we can do between logs and the survey. And in between there is a space where you can’t quite touch about motivation and behaviour – that’s where surveys come in…
So, to wrap up, we showed what the scholarly perceptions of the BMs information environment is. We found that digital resources are used extensively by academics as part of their research process and are considered vital to their research. We found preference for visual material. And a real different between a physical and a virtual visit.
We found that social media was not a priority. But we also foujnd that academics display specific information seeking behaviour and sophisticated search strategies. They come for known objects, with deep knowledge of the materials, it’s a serious and purposeful visit.
We got a better understanding of search patterns and information seeking behaviour of a specific user group. We provided a valuable guide for further development and refinement of the BM coll page – not many but really clear guidance.
And we are going to repeat this work and do this as a longitudinal research. And we are also involved in work with other poeple – National Gallery, BM and National Museum of Wales for a full 2 year longitudinal study of all types of users here. But this work completed has acted as a great pilot – and our masters and PhD studemts are invaluable here. We’ve been looking at the National Gallery and they have 2000 items(?) – far fewer than BM – but all really really famous.
And finally thank yous to Matthew Cock and David Prudames, British Museum; Claire Ross, UCLDH; Vera Motyckova, ex-student at UCLDH (now at BBC). And also to @paleofuture for an amazing 1962 comic predicting that researchers may consult materials at the Library of Congress or British Museum remotely – very nice!
Q1) How confident are you that 50% users are researchers – aren’t they more likely to fill in surveys?
A1) You’re right. This comes down to survey methodology and you just have to be honest about that in your write ups. We got a 10% response rate – about right but what about the others? But even seeing that there is real use by researchers is useful – even if only that 10%. When people talk about that stuff we don’t see references to the online version of an item but just the item so you can’t tell from the literature.
Q2) I thought that 35% of users who would be willing to engage with social media isn’t bad.
A2) Yeah but the age is mostly post grads, Facebook and Twitter users etc. So it seems low. And if you are deciding about resources – better valued for digitised resources than for investment in social media. The National Museum of Wales have millions of items and they have users no longer in Wales but are doing family and genealogy research – real balance to strike. But BM work showed as much info as possible is best.
Q4) Did you find out much about unsuccessful visits?
A4) Yes, at the end of the survey there was an “other” box. And they told us about frustrated searches, images they couldn’t find – all very useful. Important I think. Most responses were pretty positive. But on suveys poeple can share more negative comments so great to have so much positiveness
Q5) I’m from RCAHMS and we’ve been working on crowd tagging, could you say something on this…
A5) I did a tiny amount of work on Flickr community tagging work but huge interest in this. I had poeple who run these virtual museums and they millions and millions of hits. Tends to be image based stuff. But libraries and museums tend not to provide this stuff. Enthusiasts do this stuff exhaustively whilst museums don’t. More to be done here. But projects like First World War Poetry DIary did a great community sourced project digitising amateur poeple’s stuff. And the other thing is amateurs do not need the physical items – they collect digital copies of an image, they want an exhaustive collection. Archives and museums would never do that stuff. And we can do more to look at that relationship around who owns what and who gets to engage with it. I’m doing a talk on Jeremy Bentham later today – we have one woman who has done almost half the transcribing! She did it to unwind! Tapping into those key users is really interesting. And Pinterest and Tumblr etc. are picked up fast by these enthusiasts – very responsive to change. But organisations are getting better at that and being more pragmatic about sharing. The BM has a SPARQL endpoint that people can use to do their own interface on the data. That’s the future – making systems robust enough for better access.
Jen: On that note we should let Melissa go on to her next talk.
Q3) I work at National Galleries of Scotland and we struggle to work out how much data we put with the data? Collection/label stuff vs narrative perhaps.
A3) We found people want to know the size, the details. But the story behind the object may not be as important as the basic information. Provenance was important for users. But different questions for art vs. history.
::: Update: Melissa Terras has allowed the organisers of this session to share the audio recording alongside the slides so I have now embedded these here :::