May 122016
 
Participants networking over lunch at eLearning@ed

Last week I was delighted to be part of the team organising the annual eLearning@ed Conference 2016. The event is one of multiple events and activities run by and for the eLearning@ed Forum, a community of learning technologists, academics, and those working with learning technologies across the University of Edinburgh. I have been Convener of the group since last summer so this was my first conference in this role – usually I’m along as a punter. So, this liveblog is a little later than usual as I was rather busy on the day…

Before going into my notes I do also want to say a huge thank you to all who spoke at the event, all who attended, and an extra special thank you to the eLearning@ed Committee and Vlad, our support at IAD. I was really pleased with how the event went – and feedback has been good – and that is a testament to the wonderful community I have the privilege of working with all year round here at Edinburgh.

Note: Although I have had a chance to edit these notes they were taken live so just let me know if you spot any errors and I will be very happy to make any corrections. 

The day opened with a brief introduction from me. Obviously I didn’t blog this but it was a mixture of practical information, enthusiasm for our programme, and an introduction to our first speaker, Melissa Highton:

Connecting ISG projects for learning and teaching – Melissa Highton (@honeybhighton), Director: Learning, Teaching and Web (LTW), Information Services.

Today is about making connections. And I wanted to make some connections on work that we have been doing.

I was here last year and the year before, and sharing updates on what we’ve been doing. It’s been a very good year for LTW. It has been a very busy year for open, inspired by some of the student work seen last year. We have open.ed launched, the new open educational resources policies, we have had the OER conference, we have open media, we have had some very bold moves by the library. And a move to make digital images from the library are open by default. That offers opportunities for others, and for us.

Extract from the Online Learning Consortium's 2016 Infographic

Extract from the Online Learning Consortium’s 2016 Infographic (image copyright OLC 2016)

There is evidence – from the US (referencing the EdTech: a Catalyst for Success section of the Online Learning Consortium 2016 Infographic). with students reporting increased engagement with course materials, with professors, with fellow students. And there is also a strong interest in digital video. MediaHopper goes fully launched very soon, and we are taking a case to Knowledge Strategy Committee and Learning and Teaching Committee to invest further in lecture capture, which is heavily used and demanded. And we need to look at how we can use that content, how it is being used. One of the things that I was struck by at LAK, was the amount of research being done on the use of audio visual material, looking at how students learn from video, how they are used, how they are viewed. Analytics around effective video for learning is quite interesting – and we’ll be able to do much more with that when we have these better systems in place. And I’ve included an image of Grace Hopper, who we named MediaHopper after.

Melissa Highton speaking at eLearning@ed 2016

Melissa Highton speaking at eLearning@ed 2016

Talking of Learning Analytics I’m a great fan of the idea that if a thing is worth doing, it’s worth doing a 2×2 matrix. So this is the Learning Analytics Map of Activities, Research and Roll-out (LAMARR – a great mix of Hollywood screen icon, and the inventor of wifi!), and there are a whole range of activities taking place around the university in this area at the moment, and a huge amount of work in the wider sector.

We also are the only University in the UK with a Wikimedian in Residence. It is a place entirely curated by those with interest in the world, and there is a real digital literacy skill for our students, for us, in understanding how information is created and contested online, how it becomes part of the internet, and that’s something that is worth thinking about for our students. I have a picture here of Sophie Jex-Blake, she was part of the inspiration for our first Wikipedia Edit-a-thon on women in science. Our Wikimedian is with us for just one year, so do make use of him. He’s already worked on lots of events and work, he’s very busy, but if you want to talk to him about a possible event, or just about the work being done, or that you want to do.

Here for longer than one year we have Lynda.com, an online collection of training videos which the University has signed up to for 3 years, and will be available through your University login. Do go and explore it now, and you will have Edinburgh University access from September. The stuff that is in there, can be curated into playlists, via learn, usage etc.

So, Wikipedia for a year, Lynda.com for three years, MediaHopper here now, and open increasingly here.

Highlights from recent conferences held in Edinburgh, chaired by Marshall Dozier

Marshall: Conferences are such an opportunity to make a connection between each other, with the wider community, and we hope to fold those three big conferences that have been taking place back into our own practice.

OER16 Open Culture Conference – Lorna Campbell (@lornamcampbell), Open Education Resources Liaison for Open Scotland, LTW.

This was the 7th OER conference, and the first one to take place in Edinburgh. It was chaired by myself and Melissa Highton. Themes included Strategic advantage of open, creating a culture of openness and the reputational challenges of “open-washing”; converging and competing cultures of open knowledge, open source, open content, open practice, open data and open access; hacking, making and sharing; openness and public engagement?; and innovative practices in cultural heritage contexts, which I was particularly to see us get good engagement from.

There was originally a sense that OER would die out, but actually it is just getting bigger and bigger. This years OER conference was the biggest yet, and that’s because of support and investment from those who, like the University of Edinburgh, who see real value in openness. We had participants from across the world – 29 countries – despite being essentially a UK based conference. And we had around a 50/50 gender split – no all male panel here. There is no external funding around open education right now, so we had to charge but we did ensure free and open online participation for all – keynotes live-streamed to the ALT channel, we had Radio #EDUtalk @ OER16, with live streaming of keynotes, and interviews with participants and speakers from the conference – those recordings are hugely recommended; and we also had a busy and active Twitter channel. We had a strong Wikimedia presence at OER16, with editing training, demonstrations, and an ask a Wikimedian drop-in clinic, and people found real value in that.

Lorna Campbell speaking about OER16 at eLearning@ed 2016

Lorna Campbell speaking about OER16 at eLearning@ed 2016

We also had a wide range of keynotes and I’m just going to give a flavour of these. Our first was Catherine Cronin, National University of Ireland, Galway, who explored different definitions of openness, looking at issues of context and who may be excluded. We all negotiate risk when we are sharing, but negotiating that is important for hope, equality, and justice.

In the year of the 400th anniversary of Shakespeare’s death we were delighted to have Shakespeare scholar Emma Smith, who had a fantastic title: Free Willy: Shakespeaker & OER. In her talk she suggested teaching is an open practice now, that “you have to get over yourself and let people see what you are doing”.

John Scally’s keynote talked about the National Library of Scotland’s bold open policy. The NLS’ road to openness has been tricky, with tensions around preservation and access. John argued that the library has to move towards equality, and that open was a big part of that.

Edupunk Jim Groom of Reclaim Hosting, has quite a reputation in the sector and he was giving his very first keynote in the UK. JIm turned our attention from open shared resources, and towards open tech infrastructure, working at individual scale, but making use of cloud, networked resources which he sees as central to sustainable OER practice.

The final keynote was from Melissa Highton, with her talk Open with Care. She outlined the vision and policy of UoE. One idea introduced by Melissa was “technical and copyright debt”, the costs of not doing licensing, etc. correctly in the first place. IT Directors and CIOs need to be persuaded of the need for investment in OER.

It is difficult to summarise such a diverse conference, but there is growing awareness that openness is a key aspect that underpins good practice. I wanted to quote Stuart Allen’s blog. Stuart is a student on the MSc in Digital Education. HE did a wonderful summary of the conference.

Next year’s conference has the theme of Open and Politics and will be co-chaired by Josie Frader and Alec Tartovsky, chair of CC in Poland (our first international co-chair).

Learning@Scale 2016 – Amy Woodgate, Project Manager – Distance Education Initiative (DEI) & MOOCs, LTW.

I am coming at this from a different perspective here, as participant rather than organiser. This conference is about the intersection between informatics approaches and education. And I was interested in the degree to which that was informed by informatics, and that really seems to flag a need to interrogate what we do in terms of learning analytics, educational approach. So my presentation is kind of a proposal…

We have understood pedagogy for hundreds of years, we have been doing a huge amount of work on digital pedagogy, and the MSc in Digital Education is leading in this area. We have environments for learning, and we have environments at scale, including MOOCs, which were very evident at L@S. At University of Edinburgh we have lots of digitally based learning environments: ODL; MOOCS; and the emergence of UG credit-bearing online courses. But there is much more opportunity to connect these things for research and application – bringing pedagogy and environments at scale.

The final keynote at L@S was from Ken Koedinger, at Carnegie Mellon University. He suggested that every learning space should be a learning lab. We shouldn’t just apply theory, but building, doing, providing evidence base, thinking as part of our practice. He talked about collecting data, testing that data, understanding how to use data for continuous improvement. We are a research led institution, we have amazing opportunities to blend those things. But perhaps we haven’t yet fully embraced that Design, Deploy, Data, Repeat model. And my hope is that we can do something together more. We’ve done MOOCs for four years now, and there are so many opportunities to use the data, to get messy in the space… We haven’t been doing that but no-one has been. What was hard about the conference for me was that lots of it was about descriptive stats – we can see that people have clicked a video, but not connecting that back to anything else. And what was interesting to me was the articulation into physical environments here – picking up your pen many times is not meaningful. And so many Learning Analytics data sources are what we can capture, not necessarily what is meaningful.

The keynote had us answer some questions, about knowing when students are learning. You can see when people view or like a video, but there is a very low correlation between liking and learning… And for me that was the most important point of the session. That was really the huge gap, more proactive research, engagement, for meaningful measures of learning – not just what we can measure.

Mike Sharples, OU was also a keynote at L@S, and he talked about learning at scale, and how we can bring pedagoguey into those spaces, and the intersection of diversity, opportunity and availability. One of the things FutureLearn is exploring is the notion of citizen inquiry – people bring own research initiatives (as students) and almost like kickstarter engage the community in those projects. Interesting to see what happens, but an interesting question of how we utilize the masses, the scale of these spaces. We need you as the community working with us to start questioning how we can get more out of these spaces. Mike’s idea was that we have to rethink our idea of effective pedagoguey, and of ensuring that that is sustainable as being a key idea.

Working backwards then, there were many many papers submitted, not all were accepted, but you can view the videos of keynotes on Media Hopper, and there were posters for those not able to present as well. The winner of the best paper was “1A Civic Mission of MOOCs” – which gave the idea that actually there was a true diversity of people engaged in political MOOCs, and they weren’t all trolly, there was a sense of “respectful disagreement”. There were a lot of papers that we can look at, but we can’t apply any of these findings that can be applied without critical reflection, but there is much that can be done there.

It was interesting Lorna’s comments about gender balance. At L@S there were great female speakers, but only 15% of the whole. That reflected the computer science angle and bias of the event, and there felt like there was a need for the humanities to be there – and I think that’s an aspiration for the next one, to submit more papers, and get those voices as part of the event.

Although perhaps a slightly messy summary of the event, I wanted to leave you with the idea that we should be using what we do here at Edinburgh, with what we have available here, to put out a really exciting diverse range of work for presenting at next year’s third L@S!

So, what do people think about that idea of hacking up our learning spaces more? Thinking more about integrating data analysis etc, and having more of a community of practice around online pedagogies for learning@scale.

Amy Woodgate speaking about Learning@Scale 2016

Amy Woodgate speaking about Learning@Scale at elearning@ed 2016

Q&A

Q1) I think that issue of measuring what we can measure is a real issue right now. My question here is about adapting approach for international students – they come in and play huge fees, and there are employers pushing for MOOCs instead… But then we still want that income… So how does that all work together.

A1) I don’t think learning at scale is the only way to do teaching and learning, but it is an important resource, and offers new and interesting ways of learning. I don’t feel that it would compromise that issue of international students. International students are our students, we are an international community on campus, embracing that diversity is important. It’s not about getting rid of the teacher… There is so much you can do with pedagogies online that are so exciting, so immersive… And there is more we can get out of this in the future. I find it quite awkward to address your point though… MOOCs are an experimentation space I think, for bringing back into core. That works for some things, and some types of content really work at scale – adaptive learning processes for instance – lots of work up front for students then to navigate through. But what do others think about using MOOCs on campus…

Comment, Tim) I think for me we can measure things, but that idea of how those actions actually relate to the things that are not measured… No matter how good your VLE, people will do things beyond it. And we have to figure out how we connect and understand how they connect.

Q2, Ruby) Thank you very much for that. I was just a little bit worried… I know we have to move away from simplistic description of this measure, means this thing. But on one slide there was an implication that measuring learning… can be measured through testing. And I don’t think that that that is neccassarily true or helpful. Liking CAN be learning. And there is a lot of complexity around test scores.

A2)  Yes, that chart was showing that viewing a particular video, hadn’t resulted in better learning uptake at the end of the course… But absolutely we do need to look at these things carefully…

Q3) At the recent BlackBoard conference there was the discussion of credit bearing MOOCs, is there any plan to do that now?

A3) This sometihng we can do of course, could take a MOOC into a credit bearing UG course, where the MOOC is about content. What becomes quite exciting is moving out and, say, the kind of thing MSc DE did with eLearning and Digital Cultures – making connections between the credit bearing module and the MOOC, in interesting and enriching ways. The future isn’t pushing students over to the MOOC, but taking learning from one space to another, and seeing how that can blend. Some interesting conversations around credit alliances, like a virtual Erasmus, around credit like summer school credit. But then we fall back of universities wanting to do exams, and we have a strong track record of online MScs not relying on written exams, but not all are as progressive right now.

Q4, Nigel) I’m in Informatics, and am involved in getting introductory machine learning course online, and one of the challenges I’m facing is understanding how students are engaging, how much. I can ask them what they liked… But it doesn’t tell me much. That’s one issue. But connecting up what’s known about digital learning and how you evaluate learning in the VLEs is good… The other thing is that there is a lot of data I’d like to get out of the VLE and which to my knowledge we can’t access that data… And we as data scientists don’t have access.

Comment, Anne-Marie Scott) We are still learning how to do that best but we do collect data and we are keen to see what we can do. Dragan will talk more about Learning Analytics but there is also a UoE group that you could get involved with.

Q5, Paul) That was fascinating, and I wish I’d been able to make it along… I was a bit puzzled about how we can use this stuff… It seems to me that we imagine almost a single student body out there… In any programme we have enthusiastic students desperate to learn, no matter what; in the middle we have the quite interested, may need more to stay engaged; and then there are people just there for the certificate who just want it easy. If we imagine we have to hit all of the audiences in one approach it won’t work. We are keen to have those super keen students. In medicine we have patient groups with no medical background or educational background, so motivated to learn about their own conditions… But then in other courses, we see students who want the certificate… I think that enormous spectrum give us enormous challenges.

A5) An interesting pilot in GeoSciences on Adaptive Learning, to try to address the interested and the struggling students. Maths and Physics do a lot with additional resources with external sites – e.g. MOOCs – in a curated list from academics, that augment core. Then students who just want the basics, for those that want to learn more… Interesting paper on cheating in MOOCs, did analysis on multiple accounts and IP addresses, and toggling between accounts… Got a harvester and master account, looked at clusters…. Master accounts with perfect learning… Harvesting were poorer, then the ones in the middle… The middle is the key part… That’s where energy should be in the MOOC.

Q6) I was intrigued by big data asset work, and getting more involved… What are tensions with making data openly available… Is it competition with other universities…

A6) That’s part of project with Dragan and Jeff Haywood have been leading on Learning Analytics data policy… MOOCs include personally identifiable data, can strip it, but requires work. University has desire to share data, but not there yet for easy to access framework to engage with data. To be part of that, it’s part of bigger Learning Analytics process.

LAK’16 Learning Analytics & Knowledge Conference – Professor Dragan Gasevic (@dgasevic), Chair in Learning Analytics and Informatics, Moray House School of Education & School of Informatics

The Learning Analytics and Knowledge Conference, LAK’16, took place in Edinburgh last week. It was in it’s sixth edition. It started in Canada as a response to several groups of people looking at data collected in different types of digital environments, and also the possibility to merge data from physical spaces, instruments, etc. It attracted a diverse range of people from educational research, machine learning, psychology, sociology, policy makers etc. In terms of organisation we had wonderful support from the wonderful Grace Lynch and two of my PhD students, who did a huge amount. I also had some wonderful support from Sian Bayne and Jeff Haywood in getting this set up! They helped connect us to others, within the University and throughout the conference. But there are many others I’d like to thank, including Amy and her team who streamed all four parallel sessions throughout the conference.

In terms of programme the conference has a research stream and a practitioner stream. Our chairs help ensure we have a great programme – and we have three chairs for each stream. They helped us ensure we had a good diversity of papers and audiences, and vendors. We have those streams to attract papers but we deliberately mix the practice and research sessions are combined and share sessions… And we did break all records this time. This was only the second conference outside North America, and most of our participants are based there, but we had almost double the submissions this year. These issues are increasingly important, and the conference is an opportunity to critically reflect on this issue. Many of our papers were very high in quality, and we had a great set of workshops proposed – selecting those was a big challenge and only 50% made it in… So, for non computer scientists the acceptance ratio maybe isn’t a big deal… But for computer scientists it is a crucial thing. So here’s we accepted about 30% of papers… Short papers were particularly competitive – this is because the field is maturing, and people want to see more mature work.

Dragan Gasevic speaking about LAK'16 at eLearning@ed 2016.

We had participants from 35 countries, across our 470 participants – 140 from the US, 120 from the UK, and then 40 from Australia. Per capita Australia was very well represented. But one thing that is a little disappointing is that other European countries only had 3 or 4 people along, that tells us something about institutional adoption of learning analytics, and research there. There are impressive learning analytics work taking place in China right now, but little from Africa. In South America there is one hub of activity that is very good.

Workshops wise the kinds of topics addressed included learning design and feedback at scale, learning analytics for workplace and professional learning – definitely a theme with lots of data being collected but often private and business confidential work but that’s a tension (EU sees analytics as public data), learning analytics across physical and digital spaces – using broader data and avoiding the “streetlight effect”, temporal learning analytics – trying to see how learning processes unfold… Students are not static black boxes… They change decisions, study strategies and approaches based on feedback etc; also had interesting workshop on IMS Caliper; we also had a huge theme and workshop on ethical and privacy issues; and another on learning analytics for learners; a focus on video, and on smart environments; also looking for opportunities for educational researchers to engage with data – through data mining skills sessions to open conversations with with informaticians. We also had a “Failathon” – to try ideas, talk about failed ideas.

We also had a hackathon with Jisc/Apero… They issues an Edinburgh Statement for learning analytics interoperability. Do take a look, add your name, to address the critical points…

I just want to highlight a few keynotes: Professor Mireilla Hildebrandt talked about the law and learning as a a machine, around privacy, data and bringing in issues including the right to be forgotten. The other keynote I wanted to talk about was Professor Paul A Kirshner on learning analytics and policy – a great talk. And final keynote was Robert Mislevy who talked about psychometric front of learning analytics.

Finally two more highlights, we picked two papers out as the best:

  • Privacy and analytics – it’s a DELICATE issue. A checklist for trusted learning analytics – Hendrik Drachsler and Wolfgang Greller.
  • When should we stop? Towards Universal approach – details of speakers TBC

More information on the website. And we have more meetings coming up – we had meetings around the conference… And have more coming up with a meeting with QAA on Monday, session with Blackboard on Tuesday, and public panel with George Siemens & Mark Milliron the same day.

Q&A

Q1) Higher Education is teaching, learning and research… This is all Learning Analytics… So do we have Teaching Analytics?

A1) Great point… Learning analytics is about learning, we shouldn’t be distracted by toys. We have to think about our methods, our teaching knowledge and research. learning analytics with pretty charts isn’t neccassarily helpful – sometimes event detrimental – t0 learners. We have to look at instructional designs, to support our instructors, to use learning analytics to understand the cues we get in physical environments. One size does not fit all!

Marshall) I set a challenge for next year – apply learning analytics to the conference itself!

Student-centred learning session, chaired by Ruby Rennie

EUSA: Using eLearning Tools to Support and Engage Record Numbers of Reps – Tanya Lubicz-Nawrocka (@TanyaLubiczNaw), Academic Engagement Coordinator, EUSA; Rachel Pratt, Academic Representation Assistant, EUSA; Charline Foch (@Woody_sol), EUSA, and Sophie McCallum,Academic Representation Assistant, EUSA.

Tanya opened the presentation with an introduction to what EUSA: the Edinburgh University Students Association is and does, emphasizing the independence of EUSA and its role in supporting students, and supporting student representatives… 

Rachel: We support around 2000 (2238) students across campus per year, growing every year (actually 1592 individuals – some are responsible for several courses), so we have a lot of people to support.

Sophie: Online training is a big deal, so we developed an online training portal within Learn. That allows us to support students on any campus, and our online learners. Students weren’t always sure about what was involved in the role, and so this course is about helping them to understand what their role is, how to engage etc. And in order to capture what they’ve learned we’ve been using Open Badges, for which over to Tanya…

Tanya Lubicz-Nawrocka speaking about EUSA's use of Learn and Open Badges at elearning@ed 2016

Tanya Lubicz-Nawrocka speaking about EUSA’s use of Learn and Open Badges at elearning@ed 2016

Tanya: I actually heard about open badges at this very conference a couple of years ago. These are flexible, free, digital accreditation. Thay are full of information (metadata) and can be shared and used elsewhere in the online world. These badges represent skills in key areas, Student Development badges (purple), Research and communication badges (pink) and ? (yellow).

Tanya shows the EUSA Open Badges at elearning@ed 2016

Tanya shows the EUSA Open Badges at elearning@ed 2016

There have been huge benefits of the badges. There are benefits for students in understanding all aspects of the role, encouraging them to reflect on and document their work and success – and those helped us share their success, to understand school level roles, and to understand what skills they are developing. And we are always looking for new ways to accredit and recognise the work of our student reps, who are all volunteers. It was a great way to recognise work in a digital way that can be used on LinkedIn profiles.

There were several ways to gain badges – many earned an open badge for online training (over 1000 earned); badges were earned for intermediate training – in person (113 earned); and badges were also earned by blogging about their successes and development (168 earned).

And the badges had a qualitative impact around their role and change management, better understanding their skills and relationships with their colleagues.

Sophie McCallum speaking about EUSA's work on training and Open Badges at elearning@ed 2016

Sophie McCallum speaking about EUSA’s work on training and Open Badges at elearning@ed 2016

Rachel: Looking at the learning points from this. In terms of using (Blackboard) Learn for online functionality… For all our modules to work the best they can, 500 users is the most we could. We have two Learn pages – one for CSE (College of Science & Engineering), one for CHSS (College of Humanities and Social Sciences), they are working but we might have to split them further for best functionality. We also had challenges with uploading/bulk uploading UUNs (the University personal identifiers) – one wrong UUN in several hundred, loses all. Information services helped us with that early on! We also found that surveys in Learn are anonymous – helpful for ungraded reflection really.

In terms of Open Badges the tie to an email address is a challenge. If earned under a student email address, it’s hard to port over to a personal email address. Not sure how to resolve that but aware of it. And we also found loading of badges from “Backpack” to sites like LinkedIn was a bit tedious – we’ll support that more next year to make that easier. And there are still unknown issues to be resolved, part of the Mozilla Open Badges environment more broadly. There isn’t huge support online yet, but hopefully those issues will be addressed by the bigger community.

Using eLearning tools have helped us to upscale, train and support record numbers of Reps in their roles; they have helped us have a strong positive quantitative and qualitative impact in engaging reps; and importance of having essential material and training online and optional, in-person intermediate training and events. And it’s definitely a system we’ll continue to have and develop over the coming years.

Rachel Pratt talks about EUSA's training approach, working with student representatives across the University, at elearning@ed 2016

Rachel Pratt talks about EUSA’s training approach, working with student representatives across the University, at elearning@ed 2016

Q&A

Q1) Have you had any new feedback from students about this new rep system… I was wondering if you have an idea of whether student data – as discussed earlier – is on the agenda for students?

A1 – Tanya) Students are very well aware of their data being collected and used, we are part of data analytics working groups across the university. It’s about how it is stored, shared, presented – especially the issue of how you present information when they are not doing well… Interested in those conversations about how data is used, but we are also working with reps, and things like the Smart Data Hacks to use data for new things – timetabling things for instance…

Q2) ?

A2) It’s a big deal to volunteer 50 hours of their time per year. They are keen to show that work to future employers etc.

Q3) As usual students and EUSA seem to be way ahead. How do you find out more about the badges?

A3) They can be clicked for more metadata – that’s embedded in it. Feedback has been great, and the blogposts have really helped them reflect on their work and share that.

SLICCs: Student-Led Individually Created Courses – Simon Riley, Senior Lecturer, MRC Centre for Reproductive Health

I’m Simon Riley, from the School of Medicine. I’m on secondment with the IAD and that’s why I’m on this. I’m coming to it from having worked on the student led component in medicine. You would think that medicine would be hugely confined by GMC requirements, but there is space there. But in Edinburgh there is about a year of the five year programme that is student led – spread across that time but very important.

Now, before speaking further I must acknowledge my colleague Gavin McCabe, Employability Consultant who has been so helpful in this process.

SLICCs are essentially a reflective framework, to explore skill acquisition, using an e-portfolio. We give students generic Learning Outcomes (LOs), which allow the students to make choices. Although it’s not clear how much students understand or engage with learning outcomes… We only get four or five per module. But those generic LOs allow students to immediately define their own aims and anticipated learning in their “proposal”. Students can take ownership of their own learning by choosing the LOs to address.

Simon Riley talks about SLICCs at eLearning@ed 2016

Simon Riley talks about SLICCs at eLearning@ed 2016

The other place that this can raise tensions is the idea of “academic rigor”. We are comfortable at assessing knowledge, and assessments that are knowledge based. And we assume they get those other graduate attributes by osmosis… I think we have to think carefully about how we look at that. Because the SLICCs are reflection on learning, I think there is real rigor there. But there has to be academic content – but it’s how they gain that knowledge. Tanya mentioned the Edinburgh Award – a reflective process that has some similarities but it is different as it is not for credit.

Throughout their learning experience students can make big mistakes, and recover from them. But if you get students to reflect early, and reflect on any issue that is raised, then they have the opportunity to earn from mistakes, to consider resilience, and helping them to understand their own process for making and dealing with mistakes.

The other concern that I get is “oh, that’s a lot of work for our staff”… I was involved in Pilot 1 and I discovered that when giving feedback I was referring students back to the LOs they selected, their brief, the rubric, the key feedback was about solving the problem themselves… It’s relatively light touch and gives ownership.

So, here are three LOs… Around Analysis, Application, Evaluation. This set is Level 8. I think you could give those to any student, and ask them to do some learning, based on that, and reflect on it… And that’s across the University, across colleges… And building links between the colleges and schools, to these LOs.

So, where are we at? We had a pilot with a small number of students. It was for extra credit, totally optional. They could conduct their own learning, capture in a portfolio, reflect upon it. And there is really tight link between the portfolio evidence, and the reflective assignment. It was a fascinating set of different experiences… For instance one student went and counter river dolphins in the Amazon, but many were not as exotic… We didn’t want to potentially exclude anyone or limit relevance. Any activity can have an academic element to it if structured and reflected upon appropriately. Those who went through the process… Students have come back to us who did these at Level 8 in second year (highest level senate has approved)… They liked the process – the tutor, the discipline, the framework, more than the credit.

So we have just over 100 students signed up this summer. But I’m excited about doing this in existing programmes and courses… What we’ve done is created SCQF LOs at Level 7, 8, 10 and 11, with resources to reflect, marking rubric, and board of studies documents. I am a course organiser – developing is great but often there isn’t time to do it… So what I’m trying to do is create all that material and then just let others take and reuse that… Add a little context and run onto it. But I want to hold onto the common LOs, as long as you do that we can work between each other… And those LOs include the three already shown, plus LO4 on “Talent” and LO5 on “Mindset”, both of which specifically address graduate attributes. We’ve had graduate attributes for years but they aren’t usually in our LOs, just implicit. In these case LOs are the graduate attributes.

Simon Riley gets very animated talking about Learning Outcomes at eLearning@ed 2016

Simon Riley gets very animated talking about Learning Outcomes at eLearning@ed 2016

What might they look like? Embedded in the curriculum, online and on campus. Level 11 on-campus courses are very interested, seems to fit with what they are trying to do. Well suited to projects, to skill acquisition, and using a portfolio is key – evidencing learning is a really useful step in getting engagement. And there is such potential for interdisciplinary work – e.g. Living Lab, Edinburgh CityScope. Summer schools also very interested – a chance for a student to take a holistic view of their learning over that period. We spend a lot of money sending students out to things – study abroad, summer schools, bursaries… When they go we get little back on what they have done. I think we need to use something like this for that sort of experience, that captures what they have learnt and reflected on.

Q&A

Q1) That idea of students needing to be able to fail successfully really chimes for me… Failures can be very damaging… I thought that the idea of embracing failure, and that kind of start up culture too which values amazing failure… Should/could failure be one of your attributes… to be an amazing failure…

A1) I think that’s LO5 – turning it into a talent. But I think you have touched on an important aspect of our experience. Students are risk averse, they don’t want to fail… But as reflective learners we know that failure matters, that’s when we learn, and this framework can help us address this. I look to people like Paul McC… You have students learning in labs… You can set things up so they fail and have to solve problems… Then they have to work out how to get there, that helps…

Q1) In the sporting world you have the idea of being able to crash the kit, to be able to learn – learning how to crash safely is an early stage skills – in skateboarding, surfing etc.

Keynote, supported by the Centre for Research in Digital Education: In search of connected learning: Exploring the pedagogy of the open web – Dr Laura Gogia MD, PhD, (@GoogleGuacamole)Research Fellow for the Division of Learning Innovation and Student Success at Virginia Commonwealth University, USA, chaired by Jen Ross

Jen: I am really delighted to welcome Laura Gogia to eLearning@ed – I heard her speak a year or so ago and I just felt that great thing where ideas just gel. Laura has just successfully defended her PhD. She is also @GoogleGuacamole on Twitter and organises a Twitter reading club. And her previous roles have been diverse, most interestingly she worked as an obstetrician.

Laura: Thank you so much for inviting me today. I have been watching Edinburgh all year long, it’s just such an exciting place. To have such big conferences this year, there is so much exciting digital education and digital pedagogy work going on, you guys are at the forefront.

So I’m going to talk about connected learning – a simpler title than originally in your programme – because that’s my PhD title… I tried to get every keyword in my PhD title!

Laura Gogia begins her keynote with great enthusiasm at eLearning@ed 2016

Laura Gogia begins her keynote with great enthusiasm at eLearning@ed 2016

Let me show you an image of my daughter looking at a globe here, that look on her face is her being totally absorbed. I look for that look to understand when she is engaged and interested. In the academic context we know that students who are motivated, who see real relevance and benefit to their own work makes for more successful approaches. Drawing on Montesorri and other progressive approaches, Mimi Ito and colleagues have developed a framework for connected learning that shapes those approaches for an online digital world.

Henry Jenkins and colleagues describe Digital Participatory Culture that is interactive, creative, about sharing/contributing and informal mentoring. So a connected teacher might design learning to particularly use those connections out to the wider world. George Siemens and colleagues talk about digital workflow, where we filter/aggregate; critique; remix; amplify – pushing our work out into a noisy world where we need to catch attention. Therefore connected learners and teachers find ways to embed these skills into learning and teaching experiences…

Now this all sounds good, but much of the literature is on K-12, so what does connected learning mean for Higher Education. Now in 2014 my institution embarked on an openly networked connected learning project, on learning experiences that draw from web structure and culture to (potentially) support connected learning and student agency, engagement and success. This is only 2 years in, it’s not about guaranteed success but I’ll be talking about some work and opportunities.

So, a quick overview of VCU, we have an interesting dynamic institution, with the top rated arts college, we have diverse students, a satellite campus in Quatar and it’s an interesting place to be. And we also have VCU RamPages, an unlimited resource for creating webpages, that can be networked and extended within and beyond the University. There are about 16k websites in the last year and a half. Many are student websites, blogs, and eportfolios. RamPages enable a range of experiences and expression but I’ll focus on one, Connected Courses.

Connected Courses are openly networked digital spaces, there are networked participatory activities – some in person, all taught by different teaching staff. And they generate authentic learning products, most of which are visible to the public. Students maintain their own blog sites – usually on RamPages but they can use existing sites if they want. When they enroll on a new course they know they will be blogging and doing so publicly. They use a tag, that is then aggregated and combined with other students posts…

So, this is an example of a standard (WordPress) RamPages blog… Students select the blog template, the header images, etc. Then she uses the appropriate tag for her course, which takes it to the course “Bloggregate”… And this is where the magic happens – facilitating the sharing, the commenting, and from a tutors point of view, the assessment.

Laura Gogia shows the VCA/RamPages

Laura Gogia shows the VCA/RamPages “Bloggregate” at eLearning@ed 2016

The openly networked structure supports student agency and discovery. Students retain control of their learning products during and after the course. And work from LaGuadia found students were more richly engaged in such networked environments. And students can be exposed to work and experience which they would not otherwise be exposed to – from different sites, from different institutions, from different levels, and from different courses.

Connected learning also facilitate networked participation, including collaboration and crowdsourcing, including social media. These tools support student agency – being interdependent and self regulated. They may encourage digital fluency. And they support authentic learning products – making joint contributions that leads to enriched work.

A few years ago the UCI bike race was in Virginia and the University, in place of classes, offered a credited course that encouraged them to attend the bike race and collect evidence and share their reflections through the particular lens of their chosen course option. These jointly painted a rich picture, they were combined into authentic work products. Similarly VCU Field Botany collaboratively  generate a digital field guide (the only one) to the James Richer Park System. This contributes back to the community. Similarly arts students are generating the RVArts site, on events, with students attending, reflecting, but also benefiting our community who share interest in these traditionally decentralised events.

Now almost all connected courses involve blogging, which develops multimodal composition for digital fluency and multiple perspectives. Students include images and video, but some lecturers are embedding digital multimodal composition in their tasks. Inspireed by DS106, University of Mary Washington, our #CuriousCoLab Creative Makes course asks students to process abstract course concepts and enhance their digital fluency. They make a concrete representation of the abstract concept – they put it in their blog with some explanation of why they have chosen to do this in their way. The students loved this… They spent more time, they thought more on these abstract ideas and concepts… They can struggle with those ideas… This course was fully online, with members of the public engaged too – and we saw both students and these external participants did the creative make, whether or not they did the reflective blogging (optional for outside participants).

In terms of final projects students are often asked to create a website. These assignments allow the students to work on topics that really talk to their heart… So, one module can generate projects on multitasking and the brain, another might talk about the impact on the bombing of Hiroshima.

I’ve talked about connected learning but now I’d like to turn to my research on student blogging and tweeting, and my focus on the idea that if students are engaged in Connected Learning we require the recognition and creation of connections with people, and across concepts, contexts and time. I focused on Blogging and tweeting as these are commonly used in connected learning… I asked myself about whether there was something about these practices that was special here. So I looked at how we can capture connected learning through student digital annotation… Looking at hyperlinks, mentions, etc. The things that express digital connection… Are they indicative of pedagogical connections too? I also looking at images and videos, and how students just use images in their blog posts…

Because the Twitter API and WordPress allow capture of digital annotations… You can capture those connections in order to describe engagement. So, for the class I looked at there were weekly Twitter chats… And others beyond the course were open participants, very lightly auditing the course… I wanted to see how they interacted… What I saw was that open students were very well integrated with the enrolled students, and interacting… And this has instructional value too. Instructors used a similar social network analysis tool to ask students to reflect on their learning and engagement.

Laura Gogia speaking about linking and interaction patterns at VCU as part of her eLearning@ed 2016 keynote

Laura Gogia speaking about linking and interaction patterns at VCU as part of her eLearning@ed 2016 keynote

Similarly I looked at psychology students and how they shared hyperlinks… You can see also how sources are found directly, and when they access them exclusively through their Twitter timeline… That was useful for discussing student practice with them – because those are two different processes really – whether reading fully, or finding through others’ sharing. And in a course where there is controversy over legitimate sources, you could have a conversation on what sources you are using and why.

I found students using hyperlinks to point to additional resources, traditional citations, embedded definitions, to connect their own work, but also to contextualise their posts – indicating a presumption of an external audience and of shaping content to them… And we saw different styles of linking. We didn’t see too many “For more info see…” blog posts pointing to eg NYT, CNN. What we saw more of was text like “Smith (2010) states that verbal and nonverbal communication an impact” – a traditional citation… But “Smith 2010” and “nonverbal” were both linked. One goes where you expect (the paper), the other is a kind of “embedded description” – linking to more information but not cluttering their style or main narrative. You couldn’t see that in a paper based essay. You might also see “As part of this course, I have created a framework and design structure for..”… “this course” links to the course – thinking about audience perhaps (more research needed) by talking about context; framework pointed to personal structure etc.

I also saw varying roles of images in blog posts: some were aesthetic, some were illustration, some as extension. Students making self-generated images and videos incorporated their discussion of that making process in their blog posts… I particularly enjoyed when students made their own images and videos.

Laura Gogia talks about the Twitter patterns and hyperlinking practices of her research participants in her eLearning@ed 2016 keynote

Laura Gogia talks about the Twitter patterns and hyperlinking practices of her research participants in her eLearning@ed 2016 keynote

In terms of Twitter, students tweeted differently than they blog. Now we know different platforms support different types of behaviours. What I noticed here was that students tweeted hyperlinks to contribute to the group, or to highlight their own work. So, hyperlink as contribution could be as simple as a link with the hashtag. Whilst others might say “<hyperlink> just confirms what was said by the speaker last week”… which is different. Or it might be, e.g. “@student might find this on financial aid interesting <hyperlink>, now that inclusion of a person name significantly increases the chances of engagement – significantly linked to 3+ replies.

And then we’d see hyperlinks as promotion, although we didn’t see many loading tweets with hashtags to target lots of communities.

So, my conclusions on Digital Annotations, is that these are nuanced areas for research and discussion. I found that students seldom mentioned peer efforts – and that’s a problem, we need to encourage that. There is a lack of targeted contribution – that can be ok and trigger serendipity, but not always. We have to help students and ourselves to navigate to ensure we get information to the right people. Also almost no images I looked at had proper attribution, and that’s a problem. We tell them to cite sources in the text, have to do that in the images too. And finally course design and instructor behaviour matters, students perform better when the structure works for them… So we have to find that sweet spot and train and support instructors accordingly.

I want to end with a quote from a VCU Undergraduate student. This was a listening tour, not a formal part of research, and I asked them how she learned, how they want to learn… And this student talked about the need for learning to be flexible, connected, portable. Does everyone need an open connected space? No, but some do, and these spaces have great affordances… We need to play more here, to stay relevant and engaged with that wider world, to creatively play with the idea of learning!

Q&A

Q1) It was fantastic to see all that student engagement there, it seems that they really enjoy that. I was wondering about information overload and how students and staff deal with that with all those blogs and tweets!

A1) A fabulous question! I would say that students either love or hate connected courses… They feel strongly. One reason for that is the ability to cope with information overload. The first time we ran these we were all learning, the second time we put in information about how to cope with that early on… Part of the reason for this courses is to actually help students cope with that, understand how to manage that. It’s a big deal but part of the experience. Have to own up front, why its important to deal with it, and then deal with it. From a Twitter perspective I’m in the process of persuading faculty to grade Twitter… That hasn’t happened yet… Previously been uncredited, or has been a credit for participation. I have problems with both models… With the no credit voluntary version you get some students who are really into it… And they get frustrated with those that don’t contribute. The participation is more structured… But also frustrating, for the same reasons that can be in class… So we are looking at social network analysis that we can do and embed in grading etc.

Comment – Simon Riley) Just to comment on overload… That’s half of what being a professional or an academic is. I’m a medic and if you search PubMed you get that immediately… Another part of that is dealing with uncertainty… And I agree that we have to embrace this, to show students a way through it… Maybe the lack of structure is where we want to be…

A2) Ironically the people with the least comfort with uncertainty and unstructured are faculty members – those open participants. They feel that they are missing things… They feel they should know it all, that they should absorb it at. This is where we are at. But I was at a digital experience conference where there were 100s of people, loads of parallel strands… There seems to be a need to see it all, do it all… We have to make a conscious effort at ALT Lab to just help people let it go… This may be the first time in history where we have to be fine that we can’t know it all, and we know that and are comfortable…

Q3) Do you explicitly ask students not to contribute to that overload?

A3) I’m not sure we’re mature enough in practice… I think we need to explain what we are doing and why, to help them develop that meta level of learning. I’m not sure how often that’s happening just now but that’s important.

Q4) You talked a lot about talking in the open web in social media. Given that the largest social networks are engaging in commercial activities, in political activities (e.g. Mark Zuckerberg in China), is that something students need to be aware of?

A4) Absolutely, that needs to be there, alongside understanding privacy, understanding attribution and copyright. We don’t use Facebook. We use WordPress for RamPages – have had no problems with that so far. But we haven’t had problems with Twitter either… It’s a good point that should go on the list…

Q5) Could you imagine connected courses for say Informatics or Mathematics…? What do they look like?

A5) Most of the math courses we have dealt with are applied mathematics. That’s probably as far as I could get without sitting with a subject expert – so give me 15 mins with you and I could tell you.

Q6) So, what is the role of faculty here in carefully selecting things for students which we think are high quality?

A6) The role is as it has ever been, to mark those things out as high quality…

Q6) There is a lot of stuff out there… Linking randomly won’t always find high quality content.

A6) Sure, this is not about linking randomly though, it’s about enabling students to identify content, so they understand high quality content, not just the list given, and that supports them in the future. Typically academic staff do curate content, but (depending on the programme), students also go out there to find quality materials, discussing reasons for choosing, helping them model and understand quality. It’s about intentionality… We are trying to get students to make those decisions intentionally.

Digital Education & Technology Enhanced Learning Panel Session, chaired by Victoria Dishon

Victoria: I am delighted to be able to chair this panel. We have some brilliant academic minds and I am very pleased to be able to introduce some of them to you.

Prof. Sian Bayne (@sbayne), Professor of Digital Education in the School of Education, and Assistant Principal, Digital Education

I have a slight identity crisis today! I am Sian Bayne and I’m Professor of Digital Education but I am also newly Assistant Principal, Digital Education. It’s an incredibly exciting area of work to take forward so I thought I’d talk a bit about digital education at Edinburgh and where we are now… We have reputation and leadership, 2600 PG online students, 67 programmes, 2m MOOC learners, and real strategic support in the University. It’s a good time to be here.

Sian Bayne speaking about her exciting new role, at eLearning@ed 2016

Sian Bayne speaking about her exciting new role, at eLearning@ed 2016

We also have a growing culture of teaching innovation in Schools and a strong understanding of the challenges of academic development for and with DE. Velda McCune, Depute Director of IAD, currently on research leave, talks about complex, multilateral and ever shifting conglomerations of learning.

I want to talk a bit about where things are going… Technology trends seem to be taking us in some particular directions…We have a range of future gazing reports and updates, but I’m not as sure that we have a strong body of students, of academics, of support with a vision for what we want digital education to look like here. We did have 2 years ago the Ed2020 trying to look at this. The Stanford 2025 study is also really interesting, with four big ideas emerging around undergraduate education – of the open loop university – why 4 years at a set age, why not 6 years across your lifetime; paced education – 6 years of personalised learning with approaches for discipline we’re embedded in and put HE in the world; Axis flip; purpose learning – coming to uni with a mission not a major… So it would be interesting to think of those ideas in this university.

UAL/LSE did a digital online hack event, Digital is not the future, to explore the idea of hacking the institution from the inside. Looking at shifting to active work. Also a great new MIT Future of Digital Education report too. And if you have any ideas for processes or approaches to take things forward, please do email or Twitter me…

Melissa Highton, Assistant Principal, Online Learning (@honeybhighton)

I am also having quite an identity crisis. Sian and I have inherited quite a broad range of activities from Jeff Haywood, and I have inherited many of the activities that he had as head of IS, particularly thinking about online learning in the institution, number of courses, number of learners, what success would look like, targets – and where they came from – get thrown about… Some are assumptions, some KPI, some reach targets, some pure fantasy! So I’ll be looking at that, with the other Assistant Principals and the teams in ISG.

Melissa Highton talks about her forthcoming new role, at eLearning@ed 2016

Melissa Highton talks about her forthcoming new role, at eLearning@ed 2016

What would success look like? That Edinburgh should be THE place to work if you want to work on Digital Education, that it is innovative, fund, and our practice must be research informed, research linked, research connected. Every educator should be able to choose a range of tools to work with, and have support and understanding of risk around that… Edinburgh would be a place that excellent practitioners come t0 – and stay. Our online students would give us high satisfaction ratings. And our on campus learners would see themselves continuing studies online – preferably with us, but maybe with others.

To do that there are a set of more procedural things that must be in place around efficiency, structures, processes, platforms, to allow you to do the teaching and learning activity that we need you to do to maintain our position as a leader in this area. We have to move away from dependence on central funding, and towards sustainable activity in their departments and schools. I know it’s sexy to spin stuff up locally, it’s got us far, but when we work at scale we need common schools, taking ideas from one part of the institution to others. But hopefully creating a better environment for doing the innovative things you need to do.

Prof. David Reay (@keelincurve); Chair in Carbon Management & Education Assistant Principal, Global Environment & Society

Last year at eLearning@ed I talked about the Sustainability and Social Responsibility course, and today I’ll talk about that, another programme and some other exciting work we are doing all around Global Change and Technology Enhanced Learning.

So with the Online MSc in Carbon Management we have that fun criteria! We had an on campus programme, and it went online with students across the world. We tried lots of things, tried lots of tools, and made all sorts of mistakes that we learned from. And it was great fun! One of my favourite students was joining the first Google Hangout from a bunker in Syria, during the war, and when she had connectivity issues for the course we had to find a tactic to be able to post content via USB to students with those issues.

David Reay speaks about the new Online

David Reay speaks about the new Online “Sustainability & Social Responsibility” MSc at eLearning@ed 2016

So that online course in Sustainability and Social Responsibility is something we’ve put through the new CAIRO process that Fiona Hale is leading on, doing that workshop was hugely useful for trying those ideas, making the mistakes early so we could address them in our design. And this will be live in the autumn, please do all take a look and take it.

And the final thing, which I’m very excited about, is an online “Disaster Risk Reduction” course, which we’ve always wanted to do. This is for post earthquake, post flooding, post fire type situations. We have enormous expertise in this area and we want to look at delivery format – maybe CPD for rescue workers, MOOCs for community, maybe Masters for city planners etc. So this is the next year, this is what I’ll speak about next year.

Prof. Chris Sangwin (@c_sangwin), Chair in Technology Enhanced Science Education, School of Mathematics

I’m new to Edinburgh, joined in July last year, and my interest is in automatic assessment, and specifically online assessment. Assessment is the cornerstone of education, it drives what people do, that is the action they undertake. I’ve been influenced by Kluger and DeNiki 1996 who found that “one third of feedback interventions decreased performance”. This study found that specific feedback on the task was effective, feedback that could be seen as a personal attack was not. Which makes sense, but we aren’t always honest about our failures.

Chris Sangwin talks about automated approaches to assessing mathematics, at eLearning@ed 2016

Chris Sangwin talks about automated approaches to assessing mathematics, at eLearning@ed 2016

So, I’ve developed an automatic assessment system for mathematics – for some but not all things – which uses the computer algebra system (CAS) Maxima, which generates random structured questions, gives feedback, accommodates multiple approaches, and provides feedback on the parts of the answer which does not address the question. This is a pragmatic tool, there are bigger ideas around adaptive learning but those are huge to scope, to build, to plan out. The idea is that we have a cold hard truth – we need time, we need things marking all the time and reliably, and that contrasts with the much bigger vision of what we want for our students for our education.

You can try it yourself here: http://stack.maths.ed.ac.uk/demo/ and I am happy to add you as a question setter if you would like. We hope it will be in Learn soon too.

Prof. Judy Hardy (@judyhardy), Professor of Physics Education, School of Physics and Astronomy.

I want to follow up my talk last year about what we need to focus on “awareness” knowledge, “how to” knowledge, and we need “principles” knowledge. Fewer than a quarter of people don’t modify approaches in their teaching – sometimes that is fine, sometimes it is not. So I want to talk about a few things we’ve done, one that worked, one that did not.

Judy Hardy talks about modifying teaching approaches, at eLearning@ed 2016

Judy Hardy talks about implementing changes in teaching approaches, at eLearning@ed 2016

We have used Peerwise testing, and use of that correlates with exam performance, even when controlling for other factors. We understand from our evidence how to make it work. We have to move from formative (recommended) to summative (which drives behaviour). We have to drive students ownership of this work.

We have also used ACJ – Adaptive Comparative Judgement – to get students to understand what quality looks like, to understand it in comparison to others. They are not bad at doing that… It looks quite good at face value. But when we dug in we found students making judgments on surface features… neatness, length, presence of diagram… We are not at all confident about their physics knowledge, and how they evidence that decision… For us the evidence wasn’t enough, it wasn’t aligned with what we were trying to do. There was very high administrative overheads… A detail that is easily overlooked. For a pilot its fine, to work every day that’s an issue.

Implementing change, we have to align the change with the principles – which may also mean challenge underlying beliefs about their teaching. It needs to be compatible with local, often complex, classroom context, and it takes time, and time to embed.

Victoria: A lot of what we do here does involve taking risk so it’s great to hear that comparison of risks that have worked, and those that are less successful.

Dr Michael Seery, Reader, Chemistry Education. (@seerymk)

Like Chris I joined last July… My background has been in biology education. One of the first projects I worked on was on taking one third of chemistry undergraduate lab reports (about 1200 reports_ and to manage and correct those for about 35 postgraduate demonstrators. Why? Well because it can be hard to do these reports, often inconsistent in format, to assess online and I wanted to seek clarity and consistency of feedback. And the other reason to move online was to reduce administrative burden.

Michael Seery speaks about moving to online learning (image also shows the previous offline administrative tools), at eLearning@ed 2016

Michael Seery speaks about moving to online learning (image also shows the previous offline administrative tools), at eLearning@ed 2016

So Turnitin (Grademark) was what I started looking at. But it requires a Start Date, Due Date, and End date. But our students don’t have those. Instead we needed to retrofit it a bit. So, students submitted to experimental Dropbox, demonstrators filtered submissions and corrected their lab reports, and mark and feedback returned immediately to students… But we had problems… No deadline possible so can’t track turnaround time/impose penalties; “live” correction visible by student, and risk of simultaneous marking. And the Section rubrics (bands of 20%) too broad – that generated a great deal of feedback, as you can imagine. BUT demonstrators were being very diligent about feedback – but that also confused students as minor points were mixed with major points.

So going forward we are using groups, students will submit by week so that due dates ad turnaround times clearer, use TurnItIn assessment by groups with post date, and grading forms all direct mark entry. But our challenge has been retrofitting technologies to the assessment and feedback issue, but that bigger issue needs discussion.

The format for this session is that each of our panel will give a 3-5 minute introductory presentation and we will then turn to discussion, both amongst the panel and with questions and comments from the audience.

Panel discussion/Q&A

Q1) Thank you for a really interesting range of really diverse presentations. My question is for Melissa, and it’s about continuity of connection… UG, online, maybe pre-arrival, returning as a lifelong learning… Can we keep our matriculation number email forever? We use it at the start but then it all gets complex on graduation… Why can’t we keep that as that consistent point of contact.

A1, Melissa) That sounds like a good idea.

Q2) We’ve had that discussion at Informatics, as students lose a lot of materials etc. by loss of that address. We think an @ed.ac.uk alias is probably the way, especially for those who carry on beyond undergraduate. It was always designed as a mapping tool. But also let them have their own space that they can move work into and out of. Think that should be University policy.

A2, Melissa) Sounds like a good idea too!

Q3) I was really pleased to hear assessment and feedback raised in a lot of these presentations. In my role as Vice Principal Assessment and Feedback I’m keen to understand how we can continue those conversations, how do we join these conversations up? What is the space here? We have teaching networks but what could we be missing?

A3, Michael) We all have agreed LOs but if you ask 10 different lab demonstrators they will have 10 different ideas of what that looks like that. I think assessment on a grade, feedback, but also feed forward is crucial here. Those structures seems like a sensible place.

A3, Judy) I think part of the problem is that teaching staff are so busy that it is really difficult  to do the work needed. I think we should be moving more towards formative assessment, that is very much an ideal, far from where we are in practice, but it’s what I would like to see.

Q4) A lot of you talked about time, time being an issue… One of the issues that students raise all of the time is about timeliness of feedback… Do you think digital tools offer a way to do this?

A4, Judy) For me, the answer is probably no. Almost all student work is handwritten for us… What we’d like to do is sit with a student to talk to them, to understand what is going on in their heads, how their ideas are formed. But time with 300 students is against us. So digital tools don’t help me… Except maybe Chris’ online assessment for mathematics.

A4, Chris) The idea of implementing the system I showed is to free up staff time for that sort of richer feedback, by tackling the limited range of work we can mark automatically. That is a limited range though and it diminishes as the subject progresses.

A4, David) We implemented online submission as default and it really helped with timings, NSS, etc. that really helped us. For some assessment that is hard, but it has helped for some.

A4, Michael) Students do really value that direct feedback from academic staff… You can automate some chemistry marking, but we need that human interaction in there too, that’s important.

A4, Sian) I want to raise a humanities orientated way of raising the time issue… For me time isn’t just about the timeline for feedback, but also exploring different kinds of temporality that you can do online. For our MSc in Digital Education we have students blog and their tutors engage in a long form engaged rich way throughout the course, feedback and assessment is much richer than just grading.

Q5) In terms of incorporation of international students here, they are here for one year only and that’s very short. Sometimes Chinese students meet a real clash of expectations around language proficiency, a communication gap between what assessment and feedback is, and what we practice. In terms of technology is there a formative model for feedback for students less familiar with this different academic culture, rather than leaving them confused for one semester and then start to understand.

A5, David) It’s such an important point. For all of our students there is a real challenge of understanding what feedback actually is, what it is for. A lot of good feedback isn’t badged properly and doesn’t show up in NSS. I love the idea of less assessment, and of the timing being thought through. So we don’t focus on summative assessment early on, before they know how to play the game.. I agree really.

A5, Judy) One thing we don’t make much use, is of exemplars. They can be very valuable. When I think about how we get expertise as markers, is because of trying to do it. Students don’t get that opportunity, you only see your own work. Exemplars can help there…

The panel listening to questions from the floor at eLearning@ed 2016

The panel listening to questions from the floor at eLearning@ed 2016

Q6) Maybe for the panel, maybe for Fiona… One thing to build in dialogue, and the importance of formative assessment… Are you seeing that in the course design workshops, use of CAIReO (blog post on this coming soon btw), whether you see a difference in the ways people assess….

A6, Fiona) We have queues of people wanting the workshop right now, they have challenges and issues to address and for some of them its assessment, for others its delivery or pace. But assessment is always part of that. It comes naturally out of storyboarding of learner activities. BUt we are not looking at development of content, we are talking about learning activity – that’s where it is different. Plenty to think about though…

Comment, Ross) Metaphor of a blank piece of paper is good. With learning technologies you can start out with that sense of not knowing what you want to achieve… I think exemplars help here too, sharing of ideas and examples. Days like today can be really helpful for seeing what others are doing, but then we go back to desks and have blank sheets of paper.

Q7) As more policies and initiatives appear in the institution, does it matter if we believe that learning is what the student does – rather than the teacher? I think my believe is that learning occurs in the mind of the learning… So technologies such as distance and digital learning can be a bit strange… Distance and digital teaching maybe makes more sense…

A7) I think that replacing terminology of “teaching” with terminology of “learning” has been taking place. Hesper talks about the problems of the “learnification of education”, when we do that we instrumentalise education. That ignores power structures and issues in many ways. My colleagues and I wrote a Manifesto for Teaching Online and we had some flack about that terminology but we thought that that was important.

Q8) Aspirationally there would be one to one dialogue with students… I agree that that is a good aspiration… And there is that possibility of continuity… But my question was to what extent past, present, and future physical spaces… And to what extent does that enable or challenge good learning or good teaching?

A8, Judy) We use technology in classrooms. First year classes are flipped – and the spaces aren’t very conducive to that. There are issues with that physical space. For group working there are great frustrations that can limit what we can do… In any case this is somewhat inevitable. In terms of online education, I probably have to hand to colleagues…

A8, David) For our institution we have big plans and real estate pressures already. When we are designing teaching spaces, as we are at KB right now, there is a danger of locking ourselves into an estate that is not future proof. And in terms of impinging on innovation, in terms of changing demands of students, that’s a real risk for us… So I suppose my solution to that is that when we do large estate planning, that we as educators and experts in technology do that work, do that horizon scanning, like Sian talked about, and that that feeds into physical space as well as pedagogy.

A8, Sian) For me I want leakier spaces – bringing co-presences into being between on campus and online students. Whole area of digital pedagogical exploration we could be playing with.

A8, Melissa) There is is a very good classroom design service within the Learning and Teaching spaces team in IS. But there is a lag between the spaces we have today, and getting kit in place for current/future needs. It’s an ongoing discussion. Particularly for new build spaces there is really interesting possibility around being thoughtful. I think we also have to think about shifting time and space… Lecture Capture allows changes, maybe we need fewer big lecture rooms… Does the teaching define the space, or the space that designs the teaching. Please do engage with the teams that are there to help.

A8, Michael) One thing that is a danger, is that we chase the next best thing… But those needs change. We need to think about the teaching experience, what is good enough, what is future-proof enough… And where the need is for flexibility.

Victoria: Thanks to all our panel!

eMarking Roll Out at Abertay – Carol Maxwell, Technology Enhanced Learning Support team Leader, Abertay University, chaired by Michael Seery

I am Carol Maxwell from Abertay University and I am based in the Technology Enhanced Learning support team. So, a wee bit about Abertay… We are a very small city centre university, with 4025 students (on campus) and 2091 in partner institutions. We are up 9 places to 86 in Complete University Guide (2017), And our NSS score for feedback turnaround went up by 12%, which we think has a lot to do with our eMarking roll out.

We have had lots of change – a new Principal and new Vice Chancellor in summer 2012. We have many new appointments, a new director of teaching and learning enhancement, and we’ve moved towards central services rather than local admin. We get involved in the PGCert programme, and all new members of staff have to go through that process. We have monthly seminars where we get around 70 people coming along. We have lots of online resources, support for HEA accreditation and lots of things taking place, to give you a flavour of what our team does.

Carol Maxwell talks about the work of the Abertay Teaching and Learning Enhancement Team, at eLearning@ed 2016

Carol Maxwell talks about the work of the Abertay Teaching and Learning Enhancement Team, at eLearning@ed 2016

So the ATLEF project was looking at supporting assessment and feedback practice with technology, this was when our team was part of information services, and that was intended to improve the University’s understanding and awareness of the potential benefits, challenges and barriers associated with a more systematic and strategic approach to technology-enhanced assessment and feedback, we wanted to accelerate staff awareness of technological tools for assessment.

So we did a baseline report on practice – we didn’t have tools there, and instead had to interrogate Blackboard data course by course… We found only 50% of those courses using online assessment were using Grademark to do this. We saw some using audio files, some used feedback in Grade Centre, some did tracked changes in Word, and we also saw lots of use of feedback in comments on eportfolios.

We only had 2% online exams. Feedback on that was mixed, and some was to do with how the actual user experience worked – difficulties in scrolling through documents in Blackboard for instance. Some students were concerned that taking exams at home would be distracting. There was also a perception that online exams were for benefit of teaching staff, rather than students.

So we had an idea of what was needed, and we wanted to also review sector practices. We found Ferrell 2013, and also the Heads of eLearning Forum Electronic Management of Assessment Survey Report 2013 we saw that the most common practice was e-submission as well as hard copy printed by student… But we wanted to move away from paper. So, we were involved in the Jisc Electronic Marking and Assessment project and cycle… And we were part of a think tank where we discussed issues such as retention and archiving of coursework, and in particular the importance of it being a University wide approach.

So we adopted a new Abertay Assessment Strategy. So for instance we now have week 7 as a feedback week. It isn’t for teaching, it is not a reading week, it is specifically for assessment and feedback. The biggest change for our staff was the need for return of coursework and feedback in 10 working days before week 13, and within 15 weeks thereafter, That was a big change. We had been trialing things for year, so we were ready to just go for it. But we had some challenges, we have a literal grading policy, A+, A, B+ etc. which is harder in these tools.

We had senior management, registry, secretariat, teaching staff, teaching and learning staff discussing and agreeing the policy document. We had EMA champions demonstrating current process, we generated loads of supporting materials to. So one of our champions delivered video feedback – albeit with some student feedback to him that he was a little dry, he took it on the chin. One academic uses feedback on PebblePad, we have a lecturer who uses questions a great deal in mathematics courses, letting students attempt questions and then move on after completion only. We also have students based in France who were sharing reflections and video content, and feedback to it alongside their expected work. And we have Turnitin/Grademark, of which the personalised feedback is most valuable. Another champion has been using discussion forums, where students can develop their ideas, see each others work etc. We also hold lots of roadshow events, and feedback from these have raised the issue of needing two screens to actually manage marking in these spaces.

Carol Maxwell talks about the support for staff in rolling out eMarking at Abertay, at eLearning@ed 2016

Carol Maxwell talks about the support for staff in rolling out eMarking at Abertay, at eLearning@ed 2016

The areas we had difficulty with here was around integration, with workarounds required for Turnitin with Blackboard Grade Centre and literal grading; Staff resistance – with roadshows helping’ Moderation – used 3 columns not 2 for marking; Anonymity; returning feedback to students raised some complexities faced. There has been some challenging work here but overall the response has been positive. Our new templates include all the help and support information for our templates to.

So, where to now… Carry on refining procedures and support, need on going training – especially new staff, Blackboard SITS Integration. More online exams (some online and some off site); digital literacy etc. And, in conclusion you need Senior Management support and a partnership approach with academic staff, students and support services required to make a step change in practice.

Q&A

Q1) I’m looking at your array of initiatives, but seeing that we do these things in pockets. The striking thing is how you got the staff on board… I wonder if we have staff on board, but not sure we have students on board… So what did you do to get the students on board?

A1) There was a separate project on feedback with the students, raising student awareness on what feedback was. The student association were an important part of that. Feedback week is intended to make feedback to students very visible and help them understand their importance… And the students all seem to be able to find their feedback online.

Q2, Michael) You made this look quite seamless across spaces, how do you roll this out effectively?

A2) We’ve been working with staff a long time, so individual staff do lots of good things… The same with assessment and feedback… It was just that we had those people there who had great things there… So like the thinking module there is a model with self-enroll wikis… You end up with examples all around. With the roll out of EMA the Principal was keen that we just do this stuff, we have already tested it. But Abertay is a small place, we have monthly meet ups with good attendance as that’s pretty much needed for PGCAP. But it’s easier to spread an idea, because we are quite small.

Q3) For that 10-15 day turnaround how do you measure it, and how do you handle exemptions?

A3) You can have exemptions but you have to start that process early, teams all know that they have to pitch in. But some academic staff have scaled assessment back to the appropriate required level.

At this point we broke for an extended break and poster session, some images of which are included below. 

Amy Burge and Laine Ruus show their posters during the eLearning@ed 2016 Poster Session

Amy Burge and Laine Ruus show their posters during the eLearning@ed 2016 Poster Session

 

Participants explore posters including Simon Fokt's Diversity Reading List poster at eLearning@ed 2016

Participants explore posters including Simon Fokt’s Diversity Reading List poster at eLearning@ed 2016

 

Ross Ward provides an informal LTW drop in session as part of the eLearning@ed 2016 Poster Session

Ross Ward provides an informal LTW drop in session as part of the eLearning@ed 2016 Poster Session

Taking this forward – Nicola Osborne

Again, I was up and chairing so notes are more minimal from these sessions… 

The best of ILW 2016 – Silje Graffer (@SiljeGrr), ILW/IAD

ILW is in its fifth year… We had over 263 events through the event, we reached over 2 million people via social media…

How did we get to this year? It has been amazing in the last few years… We wanted to see how we could reach the students and the staff in a better way that was more empowering for them. We went back to basics, we hired a service design company in Glasgow to engage people who had been involved in ILW before… In an event we called Open ILW… We wanted to put people first. We had 2 full time staff, 3 student staff, 20 school coordinators – to handle local arrangements – and created a kind of cool club of a network!

Silje Graffer talks about the Innovative Learning Week team, at eLearning@ed 2016

Silje Graffer talks about the Innovative Learning Week team, at eLearning@ed 2016

So we went back to the start… We wanted to provide clarity on the concept… We wanted to highlight innovation already taking place, that innovation doesn’t just happen once a year. And to retain that space to experiment.

We wanted to create a structure to support ideas. We turned feedback into a handbook for organisers. We had meet ups every month for organisers, around ideas, development, event design, sharing ideas, developing process… We also told more stories through social media and the website. We curated the programme around ideas in play. We wanted to focus on people making the events, who go through a valuable process, and have scope to apply that.

Silje Graffer talks about some of the highlight events from ILW16, at eLearning@ed 201g

Silje Graffer talks about some of the highlight events from ILW16, at eLearning@ed 201g

So I just wanted to flag some work on openness, there was a Wikipedia Editathon on the history of medicine, we had collaboration – looking at meaningful connections between different parts of the university, particularly looking at learners with autism which was really valuable. Creativity… This wasn’t digital education in itself, but the Board Game Jam was about creating games, all were openly licensed, and you can access and use those games in teaching, available from OER. A great example for getting hands dirty and how that translates into the digital. And iGEM Sandpit and Bio Hackathon, are taking ideas forward to a worldwide event. Smart Data Hack continued again, with more real challenges to meet. Prof Ewan Klein gas taken work forward in the new Data, Design and Society Course… And in the Celebratory mode, we had an online game called Edinburgh is Everywhere, exploring Edinburgh beyond the physical campus! And this was from a student. You can browse all the digital education events that ran on the website, and I can put you in touch with organisers.

Next year its happening again, redeveloped and imagined again.

Q1) Is it running again

A1) Yes! But we will be using some of the redesigning approaches again.

 

CMALT – what’s coming up – Susan Greig (@SusieGreig),

Are you certified… I am based in LTW and I’m really pleased to announce new support for achieving CMALT within the University. And I can say that I am certified!

CMALT is the Certified Member of ALT, it’s recommended for documenting and reflecting on your work, a way to keep pace with technology, it is certified by peers, update certification every three years. So, why did I do CMALT? When back when I put my portfolio forward in 2008 I actually wrote down my reasons – I hoped to plan for my future careers more effectively, the career path isn’t well definied and I was keen to see where this would take me. And looking back I don’t think that career path has become more clear… So still very useful to do.

Susan Greig talking about support for CMALT, at eLearning@ed 2016

Susan Greig talking about support for CMALT, at eLearning@ed 2016

So, to do CMALT you need to submit a portfolio. That is around five areas, operational issues; teaching, learning and/or assessment processes; the wider context; communication; and a specialist area. I did this as an individual submission, but there is also an option to do this together. And that is what we will be doing in Information Services. We will provide ongoing support and general cheer-leading, events which will be open to all, and regular short productive cohort meetings. There will also be regular writing retreats with IAD. So, my challenge to you is can we make the University of Edinburgh the organisation with the most accredited CMALT members in the UK?

If you are interested get in touch. Likely cohort start is August 2016… More presentations from alt 3rd june, showcase event there in july

Making Connections all year long: eLearning@ed Monthly meet ups – Ross Ward (@RossWoss), Educational Design

Today has been a lovely chance to  get to meet and network with peers… Over the last year in LTW  (Learning, Teaching and Web Services) we’ve looked at how we can raise awareness of how we can help people in different schools and colleges achieve what they are trying to do, and how we can support that… And as we’ve gone around we’ve tried to work with them to provide what is needed for their work, we’ve been running roadshows and workshops. Rather than focus on the technologies, we wanted to come from more of a learning and teaching perspective…Around themes of Interactive learning and teaching, assessment and feedback, open educational resources, shakers, makers and co-creators, and exploring spaces… From those conversations we’ve realised there is loads of amazing stuff coming on… And we wanted to share these more widely…

Ross Ward talks about recent elearning@ed/LTW Monthly MeetUps, at eLearning@ed 2016

Ross Ward talks about recent elearning@ed/LTW Monthly MeetUps, at eLearning@ed 2016

Luckily we have a great community already… And we have been working collaboratively between elearning@ed and learning, teaching and web services, and having once a month meetings on one of the themes, sharing experiences and good practices… A way to strengthen networks, a group to share with in physical and digital shared spaces… The aim is that they are open to anyone – academics, learning technologists, support teams… Multiple short presentations, including what is available right now, but not ignoring horizon scanning. It’s a space for discussion – long coffee break, and the pub afterwards. We have a 100% record of going to the pub… And try to encourage discussion afterwards…

So far we’ve looked at Using media in teaching (January); Open Education – including our Wikimedian in residence (February); Things we have/do – well received catch up (March); Learning Design – excellent session from Fiona (April). We put as much as we can on the wiki – notes and materials – and you’ll find upcoming events there too. Which includes: Assessment and Feedback – which will be lively if the sessions here are anything to go by (27th June); CMALT (27th July); Maker Space (August) – do share your ideas and thoughts here.

In the future we are trying to listen to community needs, to use online spaces for some, to stream, to move things around, to raise awareness of the event. All ideas and needs welcomed… Interesting to use new channels… These tend to be on themes so case by case possibilities…

The final part of our day was our wrap up by Prof. Charlie Jeffrey, who came to us fresh from Glasgow where he’d been commenting on the Scottish Parliamentary election results for the BBC… 

Wrap Up – Professor Charlie Jeffrey, Senior Vice Principal.

I’m conscious of being a bit of an imposter here as I’m wrapping up a conference that I have not been able to attend most of. And also of being a bit of an obstacle between you and the end of the day… But I want to join together a few things that colleagues and I have been working on… The unambiguous priority of teaching and learning at Edinburgh, and the work that you do. So, what is the unambiguous priority about? It’s about sharpening the focus of teaching and learning in this university. My hope is that we reach a point in the future that we prize our excellent reputation for learning and teaching as highly as we do our excellent reputation in research. And I’ve been working with a platoon of assistant principals looking at how best to structure these things. One thing to come out of this is the Teaching Matters website which Amy (Burge) so wonderfully edits. And I hope that that is part of that collegiate approach. And Ross, I think if we had blogs and shorter contributions for the website coming out of those meetings, that would be great…

Charlie Jeffrey gives the wrap up at eLearning@ed 2016

Charlie Jeffrey gives the wrap up at eLearning@ed 2016

I’m also conscious of talking of what we do now… And that what we do in the future will be different. And what we have to do is make sure we are fit for the future… Traditional teaching and learning is being transformed by Teaching and Learning… And I wouldn’t want us to be left behind. That’s a competitive advantage thing… But it is is also a pedagogical issues, to do the best we can with the available tools and technologies. I’m confident that we can do that… We have such a strong track record of DEIs, MOOCs, and what Lesley Yellowlees calls he “TESEy chairs”, the Centre of research in Digital Education, an ISG gripped in organisational priorities, and a strong community that helps us to be at the forefront of digital education. Over the last few weeks we’ve had three of the worlds best conferences in digital education, and that’s a brilliant place to be! And an awful lot of that is due to the animation and leadership of Jeff Haywood, who has now retired, and so we’ve asked Sian and Melissa to help ensure that we stay in that absolutely powerful leading position, no pressure whatsoever, but I am very confident that they will be well supported. It’s pretty rare within an organisation to get 90 people to make time to come together and share experience like you have today.

And with that the day was finished! A huge thank you again to all who were part of the event. If you were there – whether presenting or to participate in the poster session or just to listen, I would ask that you complete our feedback survey if you haven’t already. If you weren’t there but are interested in next year’s event or the eLearning@ed community in general, you’ll find lots of useful links below. Video of the event will also be online soon (via MediaHopper – I’ll add the link once it is all live) so anyone reading this should be able to re-watch sessions soon. 

Related Resources

More about eLearning@ed

If you are interested in learning more about the eLearning@ed Forum the best place to start is our wiki: http://elearningforum.ed.ac.uk/.

If you are based at Edinburgh University – whether staff or student – you can also sign up to the Forum’s mailing list where we share updates, news, events, etc.

You can also join us for our monthly meet ups, co-organised with the Learning, Teaching and Web Services team at Edinburgh University. More information on these and other forthcoming events can be found on our Events page. We are also happy to add others’ events to our calendar, and I send out a regular newsletter to the community which we are happy to publicise relevant events, reports, etc. to. If you have something you’d like to share with the eLearning@ed community do just get in touch.

You can also read about some of our previous and more recent eLearning@ed events here on my blog:

 

Apr 202016
 

This is a very belated LiveBlog post from the CSCS Network Citizen Science and the Mass Media event, which I chaired back on 22nd October 2015. Since the event took place several videos recorded at the event have been published by the lovely CSCS Network folks and I’ve embedded those throughout this post.

About the Event

This session looked at how media and communications can be used to promote and engage communities in a crowd sourcing and citizen science project. This included aspects including understanding the purpose and audience for a project; gaining exposure from a project; communicating these types of projects effectively; engaging the press; expectation management; practical issues such as timing, use of interviewees and quotes, etc.

I was chairing this session, drawing on my experience working on the COBWEB project in particular, and I was delighted that we were able to bring in two guest speakers whose work I’ve been following for a while:

Dave Kilbey, University of Bristol and Founder and CEO of Natural Apptitude Ltd. Natural Apptitute works with academic and partner organisations to create mobile phone apps and websites for citizen science projects that have included NatureLocator, Leafwatch, Batmobile, and BeeMapp. Some of these projects have received substantial press interest, in particular Leafwatch (along with the wider Conker Tree Science initiative), and Dave will talk about his personal experience of the way that crowd sourcing and citizen science and the media work together, some of the benefits and risks of exposure, and some of the challenges associated with working with the press based on his own experience.  @kilbey252

Alastair (Ally) Tibbitt, Senior Online Journalist at STV, where he has been based since 2011 working both in journalism and community engagement. Aly’s background lies in community projects in Glasgow and Edinburgh, experience that informs his work writing both for STV and Greener Leith. He has particular interests in hyperlocal news, open data and environmental issues, giving him a really interesting insiders’ perspective on the way that citizen science and crowd sourcing can engage the press, some of the realities of media expectations, timings, etc. and an insight into effective ways to pitch a citizen engagement story. @allytibbett

My notes from the talks were captured on the day but, due to chairing, I wasn’t able to capture all of the discussion or questions that arose in the session. The video below captures the talks, with my notes from these below. 

YouTube Preview Image

Musings on Media and Communications for Citizen Science Projects – Dave Kilbey, Natural Appitude

I’m not an expert but I have been working in this area for some time so these are some musings informed by my work to date.

I’ve worked on a variety of projects, which started with a project called NatureLocator – all basically mobile apps, but also website. We try to make it as simple as possible for people to take part in these projects, and we try to do that working with experts so that the data we collect is useful and purposeful. So our projects include work on invasive species, work with the biological monitoring centre. So effectively we work with researchers, organisations, and engaging the public in what we do. And we do that with design of bespoke smartphone apps and websites. In theory Innovative but actually much of this is established – although BatMobile is an exception – as was never really good enough to launch. And public engagement is central to what we do, and from that naturally comes much of our engagement with media.

We spend a lot of time and money on design and usability, because if they aren’t easy to use and appealling then participants won’t use them or use them again. The apps are for contribution, the website is for looking at the data – that’s more of an unprovoked engagement…

So the content on media on communications is this bit, which I’m calling “Smurfs… and the wrong kind of conkers”.

So I thought about why we want media coverage in the first place? It’s obvious but it matters… And these are selfish through to altruistic…

We want this to get the project (and us) noticed – we want to share what we do, and to get the project out there (important for a business too). You want to engage an army of volunteers – you can’t have citizen science without citizen scientists, you need people engaged. You want to attract more funding – crucial in a university context. Success metrics – which include impact – we are measured on how many people took part, engaged etc. and as researchers we are also measured on media presence to an extent. But there is also the aspect of personal satisfaction, and that matters.

On a more altruistic basis is increase knowledge of a concept or problem – we’ve really had that feedback on our invasive plant species work. Citizen science is increasingly about finding solutions to problems – there are all sorts of things like examination of proteins being gamified, so participants contribute regardless of knowledge. We also want to inspire interest, perhaps even the next generation of researchers – we are all passionate about what we do, and want to share that…

But the crux of the matter is that media isn’t always as important in the ways you’d expect.

If your project isn’t ready, the media coverage will be a real pain. There is a project called Ash Town done more of less as a media stunt… The organisation using the data wasn’t ready, the data wasn’t ready… and they had a backlog of verification and that disillusioned participants… The feedback loop wasn’t there but they had to take advantage of that moment. So I tend to be quite conservative about when I share projects, I want them ready.

Quite a few of our projects have had mass media interest and that can be brilliant but they cause a big spike and are largely unfocused… Normally you want a focused set of interested participants. It can be helpful but long term it’s less clear how it is helpful for finding those participants. By contrast micro media and focused marketsing and events, such as conferences, lead to better engagement – and the data from targeted audiences tends to be much better. For example there was a big issue of giant hog weed in the media this summer – we had more records than ever before… but 80% of that data was incorrect. Normally the data in Plant Tracker is 90% accurate. That was due to lots of people finding out about giant hog weed and recording lots of false positive. NOt neccassarily a problem, but an issue for data centric projects.

So we find drip feeding/organic networking works best for us. But as they say “Any publicity is good publicity?”… Maybe…. Mostly we’ve had good coverage,

To use a fishing analogy I see the mass media as ground bating – causing a general feeding frenzy, but then you have to think about how you are baiting your hook to make use of this… So it’s all about how you follow up…

So, with our first app, Leaf Watch, we had loads of media coverage. This project was small scale before with maybe 500 records a year, without the photos or georeference. So we set up a smartphone app with that sort of data for verification interested… And we had 5000 records… But also a lot of noise… 3 bottom pictures, and worse… even a smurf!

So, how to attract publicity… Again, I’m no expert… Often it’s about finding an interesting story to tell that has relevance at this point in time – is there a hook to draw people in, trigger their imagination. For the Uni of Bristol it was often our Public Relations Office that often got us the gig. Me, on my own using my Twitter feed, is going to get the Times interested… So utilise your existing resources in your organisation, they have some great powerful contacts etc. to call on. And I have a colleague who does a good job of researching likely journalists and contacting them directly…

Really much of this feels random, but it’s about a lot of events coming together, and stuff in the outside world… Looking for those opportunities to tell your story to an audience that’s ready to listen… (And do get in touch).

Engaging the Media – Ally Tibbett, STV

I work at STV, and have a background in community projects and volunteering activities. I currently work at STV, also setting up a fledgling news site.

So I wanted to set the context of engaging with media… ANd I wanted to set the scene. Many newspapers are losing 10% circulation, broadcast TV are doing better, but still online transition. But most media company websites are booming – our STV pages collectively reach a few million people a day. So still a lot of reason to get word out there. And it’s worth planning that as you do your citizen science project. You need to think about where you will find the people you do want to engage with. More and more people get their news via social media. Many read news via mobile device. It’s getting more visual with vides, images, infographics. Big interactive graphics are great, but hard to scale to a phone so many media companies keep it simple..

So I’ve tried to set this up as a timeline… How you might engage the media… Before your project. When recruiting participants – who do you want to reach, is it a specific geography? Age greoup? demographic? that should influence both the scial media platfors and media companies you use. What is the benefit for participants? What is the long term goal. Is ther ean interesting back story – and what change will it bring about. And plan out a communication calendar – can you hook into, e.g. International Authors day. Editors are always looking for a new angle on events, or a local angle on a national news story. And even if that doesn’t fit your timing it can be helpful. The other thing to think about is what digital assets can you share/produce. A press release is nice, but a press release with bangs and whistle, with infographics or images etc. That is brilliant – helps journalists know why they should engage now. It’s about the infotainment, not just the data. And it could be as simple as a slideshow, or animated gifs, or data we could map. Thinking about citizen science projects I’ve already worked on, I thought of a project on happiness on different neighbourhoods – we persuaded them to share some data. If you do want help producing maps etc, then there are skilled journalists who can help. We’ll need a Shapefile. And we need that data to be open to support more open interactive stuff…

So, assuming you had a nice launch and a little publicity boost… How do you engage dring th eproject? Well citizen engagement can be more than just research – can they promote project fro you on social media. You need a #hashtga to generate social media buss and help you collate conversation. Can you give progress reports to journalists who covered the launch and those you hope will cover final results. And building that buzz from the outset, can mean there is a story, and help show th eimpact of your prokect. Also, thnk about things that cannot be shared – could be copyright or child protection etc. issues. And as you aggregate content around the hashtag and curate the best, remove anything with an issue. Tools like STorify let you do this.

From my point of view one of the best ways to engage the press is when there is a result, a discovery… The media thrives on a wee bit of controversy etc. So Neive Short from CRESH at Edinburgh looks at mapping alchohol etc. and social issues – she is a campaigning academic, taking her studies to policy makers, and that, for instance, is always of interest. So air quality or air pollution crowd sourcing project would certainly have some of those qualities, those cases to engage policy makers. Too often we get press releases about “we did a study… we might be able to do something in the future…” but we need a concrete story really…

A note on press releases… They are fundamentally quite useful. Do send them out. Keep them short. Include multiple short quotes. have a clear top line, be clear about what you’ve done. Comes with a variety of visuals in different formats – landscape, portrait, infographics, animated films etc. And supplying images in multiple formets – making our job to package it easier – makes a big difference. Is the story important enough for us to send someone out to take new images? Maybe not. BUt actually don’t send 6MBs of materials is not good – so send a press release linking to resources.

So, journalists. Do send releases etc to a generic news email addresses. Use tools like Twitter and LinkedIn to find journalists with an interest in your subject, message them direct. Provide advance warning, reminders, photo and filming opportunities. Don’t do it at the weekend – no TV will come. Do it at a lunchtime on a weekday… PRactical stuff. If no one shows up, don’t worry about it, do send them pictures etc. And if there is one place that you really really want to be featured in, offer it as an exclusive and see it works. Obviously I’d like that to be me… BUt that’s something useful to hold back ni that way…

And, lastly, humour works. If you can find something daft, and can present it in a funny way… Our story “What if Back to the Future was set in Glasgow” is the second most ready story on our website having gone up yesterday. Most read story in the last year on STV was a very tall man who using the bathroom had a hand dryer calamity – that did great and almost made the front page of Reddit. We can be too serious… Be fun. Share the 15 things that happened in this project that were most funny, say… Humour works.

And with that we turned to some really interesting questions and discussion – huge thanks to all who came along and took part in this.

Whilst he was in Edinburgh for this event Dave Kilbey was also able to give an interview for the CSCS Network website, which you can watch there, or in the embed below:

YouTube Preview Image

Huge thanks to Dave and Ally for making the time to come along and speak to the CSCS network who I know really appreciated their presentations and sharing of experience. Huge thanks too to the lovely CSCS network team for providing a space for this event and support for our speakers and their travel. 

Mar 232016
 
Screen Capture of the Data Design and Society website

Today I attended the University of Edinburgh Data, Design & Society (DDS) course’s final presentations session, having been invited by Ewan Klein, who is the course organiser.

Data, Design & Society is an innovative programmes across three departments of Edinburgh University: the School of Informatics; the School of Social and Political Studies; and Design Informatics. Students on this programme (which is a 20 credit bearing Level 8 course) have been focusing on specific real world projects which, this time, have been focusing on food and food sustainability. All of the course materials are available publicly online, along with more information on all of the projects.

The format for this session was group presentations of the projects and for each of these I’ve captured the group name and comments, but not all of the students names. If you are interested in following up with any of these do feel free to contact the teams via Ewan (ewan [AT] inf.ed.ac.uk).

Please note: I took these notes live during the presentations so please do be aware that there may be some corrections to come, and that there is much more information about all of the challenges and responses on the DDS site

Good Eats

Good Eats wants to encourage students to consume healthy wholesome food. On the whole students are not getting the nutrition they need. The Healthy University Project found that only 25% of students at UoE get five fruits or veg per day. They ran their own survey on undergraduates and postgraduates. We looked at factors influencing decisions and found that price was by far the most important factor (over 80%) but convenience was also important (45%). We did find interest in healthy eating though, and around 50% of students were preparing food at home for themselves at least 5 days a week.

Good Eats also ran a focus group. There is general concern about their food and would like healthy and sustainable eating. But they consider eating healthily is more expensive, takes more time, and energy. So we wanted to ensure that we designed a solution that was healthy, quick and cheap. We looked at ways to convey information – brochures, website etc. But we thought that a tailored personal solution was going to be key, including some interaction, so we focused on an app. The app would enable convenience, it would be accessible, versatile to engaging on different levels, interactive, and it also allowed us the potential to include other types of media.

So, the app would act as “a cookbook in your pocket” with interactive shopping list, and a wide variety of information suiting students from different backgrounds and cultures.

So, we started to design the app. We had a main screen, and you could look at settings – metric and currency conversion; favourites list. For each recipe there is an ingredients list, methods list with integrated timers, and there is also a shopping list that you could customise – or add directly to from the recipes.

We then ran a participatory design session with users. We had really good feedback – they particularly liked the idea of being able to add ingredients to their shopping list, and the convenience. We asked students if they would really use this app and they indicated that they would look up recipes, and prepare lists the night before cooking so that they could pick up ingredients around lectures.

During the project we looked at lots of ideas, we built on our feedback from our survey and focus group and also from our mock presentation. We think this has great potential and really enjoyed working on this project.

And finally, a quick demo of the app store listing, the main menu, the settings screen, the recipe pages – that helps you navigates. There is a favourites list. And we have an information on recipes – cost per serving, timing, etc.

Q1) Can I download it?!

A1) Not yet but, we wish!

Q2) Did you look at other food and recipe apps and resources to build that and were there particular things you chose to take or not take from those?

A2) We did look at other apps and sites but didn’t directly take anything from that. We did use Spoon University, which is sort of a similar idea as a website, but that is focused on cooking and eating at college and not so much focused on nutrition. So we kind of used that and other sites as foundation for what we wanted to do.

Q3) How much of a behaviour change did this involve? Are students cooking?

A3) That first survey indicated 75% of students were cooking 5 times a week or more. So we wanted to improve the cooking, not change how often they cook. Talking to our focus group we asked what they cooked… They said easy things like macaroni cheese, cookies, burgers – things they could make for friends so we specifically looked for healthy recipes.

Q4) You said that students are not getting enough fruit and veg – did you integrate ways to encourage this in your design of the app?

A4) We looked at using the database to recommend healthy ingredients and alternatives. We also talked to food managers about improving on e.g. Sainsbury’s recipes.

Q5) Content – recipes customised like that. Also financial sustainability.

Q6) Might be good to talk about how to find some of the healthy food – so you don’t waste time on trying to find kale etc…

A6) We did discuss what could be in there… Like social media and local settings, stores to buy healthy foods etc.

Save the cups!

Right now, as consumers, we knew that coffee cups are not being recycled. More than 3bn coffee cups are thrown away in Britain each year, and fewer than 1/400 are recycled. So we wanted to see what we could do to address that, and also to look at what University of Edinburgh could do.

At University of Edinburgh over 2 million coffee cups are sold, only 2000 keep cups are sold. Coffee cups are not recyclable, keep cups are not well publicised.

From our focus group we found people are concerned at the situation but they are also not clear on what to do – they have to dry out a cup before putting in the recycling. We considered adding a 5p charge for cups, or to decrease the cost of keep cups. Give discounts on keep cups or give first years a coffee cup when they begin their studies. But we were told that making policy changes can be slow so we focused on behaviour change. So we decided upon a poster, which would highlight the 20p discount per drink sold if you use a keep cup.

So, for our design ideas we got together a group of four people to critique our design ideas. Our first posted highlighted that if you took all disposable cups wasted at University you could make 200 keep cups – that was a bit too bland. We also tried to focus on what happens to coffee cups after being wasted – that coffee cups thrown away a year could fill a whole classroom – but that was too abstract. We had a further design focused on global impact of waste – featuring a polar bear – and people cared but felt it was far disconnected from coffee cups.

So our next poster design was “Do you like coffee?” and highlighted the 20p saving. People felt motivated by saving money – it was the most effective of the posters – but they felt 20p was too little. But we knew we couldn’t change pricing. So we decided to focus on the economic angle but highlight the savings more clearly. So we developed a poster that continued that message, saying “if you drink coffee every day you save over £50 a year”. And our previous posters were on a brown paper background, that was associated with environmental issues, so we went for a cleaner look and feel more in line with economic angle.

So, if we compare our final design with a current UoE design… That highlights waste and cost (£7) of a keep cup. From our research we think our poster would be more effective. Our participants thought £7 sounded expensive so could be a deterrent rather than a motivator, whereas we highlight savings per year.

So our conclusion is that by putting up more and particularly better posters the University could do more to contribution to waste, and maybe make a dent in that 2bn coffee cups wasted per year.

Q1) Did you think about using the poster in a virtual space such as Facebook – where you could click to buy… Maybe removing barriers to buying.

A1) We didn’t think about that. The one that is up in the library is actually in the queue area in a cafe… You see it as waiting in line, highlighting what you could save.

Q2) The numbers are kind of staggering, so if you get it right it could really make it a different. Did you think about that price – it does seem a lot – but also on carrying keep cup around and that being a potential behaviour change that is needed.

A2) In our research we did talk to people who had keep cups… a lot did it for financial reasons and a lot of buyers are staff members who using them on desks. Students can be more reluctant to do that, concerned with spills in laptops. And we did ask about policy change – e.g. for disposable cups being recycled cups – but that is really slow. But we did suggest reducing the keep cup price, or handing out to first year.

Q3) Do you think students would actually carry these around?

A3) We think so and they indicated that they might.

Q4) What about the branding of the cups themselves – there are lots of coffee shops in Edinburgh, each with their own branded keep cups. Did you look at all at the branding of the cups, or of the issue of people actually using their cups across different shops – since students (and staff) don’t just frequent one place.

A4) Looked at reduction of usage of cups, we focused on within the university and policy in place… Didn’t think about interacting with the city as well as the university.

Pimp My Pollock

Pollock halls is the main catered residence halls for UoE students, serving around 2000 students a day in a buffet style self-service restaurant (JMCC). They have a number of initiatives to try and eliminate food waste. They have a zero waste to landfill policy, they compost and use that on campus. They do good stuff but they don’t engage students in that. So our goal is to foster student awareness and engagement.

There is an issue to solve here, There is a cost of around £2000 and 8000 kg (the weight of a Tigon!) waste per month because students put too much food on their plate. They are cooking almost twice the food that is eaten. They do try to highlight waste on screens – but that isn’t totally credible and the maths isn’t quite correct when comparing waste to number of food items.

So we looked at 9 different ides – including things like smaller plates or no trays – but the feedback was that change like that is difficult and slow to do. So we looked at communicating to and involving behaviour change in students. Our focus groups fed back though that being served food might help with the waste, that the environment looking better might make the difference too. Students also said that they didn’t know what a good job the university already does with waste, and again talked about the environment. And that they wasted food because it didn’t taste good. So we need to change environment to change behaviour. So, we decided on… drumroll…

Pimp My Pollock. A video/presentation, social media campaign and redesign of JMCC to change attitude and behaviours. We wanted a video that could be played during freshers week, to include RAs (staff/senior students that support students in the accommodation), to help raise visibility of staff and the good work already being done. The social media campaign would build upon existing interest. There is already a very popular @sexdrugsjmcc Instagram account with images of the food that is used playfully and is managed by the community – definitely not the university. So we thought of perhaps using Facebook to highlight reductions in food waste, fun images, maybe Spotify playlists for the JMCC too, to engage students more.

There is also a perceived behavioural control issue if you have two conflicting views of the same thing so that our impressions match up with positive work taking place – hence redesigning the space. We also want to make the space itself so it is more inviting, makes better use of space, and help highlight waste through infographics/posters etc.

Q1) Is anyone working on behaviour change in the management of this space? I am particularly surprised about the size of plates thing – that’s a proven thing.

A1) I work at the sustainability department and what we’ve tended to find from accommodation services, managed separately and differently. Trying to manage infrastructural changes are not met well. So with coffee cups… When we found recyclable coffee cups they said not cost effective. Haven’t personally tried with plate size but happy to feed that forward to that team…

Q2) Had you thought about ambassadors approaching people when eating about how much on their plate – I know staff do that sometimes when trays are put away…

A2) We thought about that… Hence the idea of the Facebook page… Hopefully that would help without that issue of it being staff. In terms of the policies already in place students don’t know about that so an induction, and engagement with food waste issue coming from students rather than staff would be more effective.

Q3) On small plate and trays I know that the service team see the plate size as reasonable… And they see that plate size as reasonable… and returning going up again when having so many students going through, student satisfaction.

A3) RAs who have lived there longer they didn’t think that plat size etc – popular in focus groups – was realistic. They felt that being served was more likely to be successful as then you can take smaller portions without needing to negotiate that.

Trayless Dining in Pollock Halls

Our idea is trayless dining in Pollock Halls. They have 2000 students eating at the JMCC dining room but they are currently catering for nearer 4000 because of waste. We wanted to help address that, and reduce the waste going to compost. There are some pre-existing initiatives. The JMCC Love Food Hate Waste initiative – more for retailers and producers – so we wanted to focus on students.

We canvassed student opinion. Many didn’t know how much they were wasting, even returning a second time with trays. So our idea is simple but there is supporting evidence that removing trays would have an impact. We had a focus group of 4 students at Pollock Halls – they weren’t aware and didn’t care, there was apathy to waste. Students were more positive to outsiders changing their behaviour, rather than coming from them. We wanted changes to environment. Students in our group saw plate size changes as too aggressive. Removing trays seemed acceptable.

So, we did participatory design process with 20 volunteers and got them to photograph their results. We asked them to go trayless and we did see a reduction in food waste… But there were logistical challenges. We think a few days of doing this would get them to adapt. We followed up with an online survey – 40% were happy for that change; 25% didn’t care; 35% were unsure. That seemed prety good compared to initial apathy.

Generally students were willing for some changes, and would have little influence on dining experience as they get the same product, and this could have a long term impact. The American University saw a major improvement on waste and washing trays etc. San Diego State University saw a 4.9% cost reduction from going trayless – including food waste and cleaning. And this can have a health impact too.

But we did see some contrasting opinions. We asked about whether removing trays would be inconvenient – we did have someone saying that multiple trips back and forth would be inconvenient. A staff member suggested that JMCC is too small of an area to implement trayless dining compared to US food halls. Main issue was behavioural changes towards waste from front of house. And their conveyor belt is build for trays not plates.

We didn’t see immediate fixes here so we thought about implementation – could be trickle down and trickle up. For Trickle Down we found a 6 point plan for going trayless: keep them available in case required; provide trays for disabled students; convert staff and employees – they must be onboard; gather feedback – there are concerns to hear and engage with; create a smooth transition – we think that implementing programme at the beginning of the academic years because that will be their first experience, as freshers, with JMCC; audit – and make available so students and staff can actually see the impact and the positive impact.

Bottom up implementation is preferred by the staff… It is supposedly already happening in Love Food, Hate Waste campaign… But we didn’t see much evidence. So, in conclusion… Trays are the solution to the JMCC waste problem because its easy and cheap to implement.

Comment 1) I think these suggestions are great and I’ll feed them back to colleagues, will keep trying to persuade people to pilot schemes…

Q2) You talked about starting at the beginning of the year – that’s more bang!

A2) Actually when change is implemented mid year, students are initially upset. Think you can avoid that when no other experience…

Q1) So maybe a pilot in Freshers week

Q3) You guys took on feedback from previous session for today, really great.

Q4) The comment about JMCC being too small to go trayless was intriguing… Was there any evidence of the actual layout in use at the US universities cited, or of the size of some of these and why that makes a difference?

A4) Evidence of smaller cafeterias that have successfully implemented this. One of us has personal experience

Ewan) Part of argument was throughput… As for 2000 students they need 3 sittings.

A4) Staff were happy for students to lead the change themselves, they were fine with that so if we can persuade students to do that that could work!

For these last three presentations there was no time for questions after all of the earlier discussion and engagement – but there were definitely people keen to ask questions and discuss all of the projects presented. 

Healthy Meal Deals

We wanted to increase consumption of healthy packaged foods on campus. We looked at several solutions: healthier alternative meal deal; adapting store layouts; healthy loyalty cards. There are meal deals on campus but we wanted to add to these with healthy meal deals. There are various current surveys that find students gaining weight during their university time, and the food and behaviours during this time guide them later in life. And surveys have shown that students relied too heavily on convenience foods because – grabbing food between lectures etc.

And in the UK there is a well known health crisis around BMI, it’s across all classes – it’s not as high in highly educated groups but still highly effected.

So, before we introduce our suggested deal we want to show you current meal deals. Currently you can select a sandwich from a range, crisps from a range, and sugary and/or caffeinated drinks. The meal has more sugar and fat than you should, but most calories come from the crisps and sandwiches. The sandwiches are high in fat but many are actually proportionately low in calories. Similarly sugar and calories – way worse. 30g is maximum sugar per day, many options have more than the appropriate 10g/meal. Looking at nutrition labels you can see those sandwiches are high in fat and sodium.

So, we wanted to look at switching out options here using what’s already easy to supply. So we considered alternatives where sandwiches are wholemeal bread – not ideal but more balanced; water or tea and maybe fresh juice; and fruit or yoghurt instead of crisps.

Promotion wise we wanted to persuade students to make those healthier choices, and to have that campaign actually across campus, not just in shops/cafes. We tweaked designs a lot with focus groups. The price isn’t concrete – couldn’t chat with manager until today – but likely £3 area and students indicated willingness to pay up to £4. And we also found good responses to comparing the healthy meal deal to the standard meal deal on nutrition rather than price basis. Then in-store we wanted to promote e.g. mixed nuts (rather than chocolate) they’d give protein and good fats – with a wee monkey but also a real citation – which students said they wanted as evidence. We also had some playful posters highlighting the benefits of fruit, of tea, etc. As well as promotion of wholemeal bread, to make it healthier….

So, our next step is to have discussion with retail managers – we meet him today! So that will be in our report. But we wanted to find out if it was realistic enough. But we expect obstacles to be commercial interests – if school has contract with providers of goods in the deal that good be a barrier; bureaucratic procedures; price.

In conclusion we wanted to include a meal deal that was inclusive of healthy options and that would increase demand for convenient but healthy food options. And it competes with but doesn’t replace current meal deals.

FoodHub

We will talk about the fast hack (all teams took part in) at the beginning and how we progressed from there. We were given the task of increasing the rate at which students select healthy food options on campus and engage with sustainable food initiatives. During our work we had a three part survey – two parts focused on our ideas, a third part focused on awareness. That awareness section was where we had most interesting data – there was apathy, the campaigns felt quite insular in terms of who was aware and engaged etc. We felt there was poor promotion on campus and students couldn’t name sustainable food initiatives on campus. We also found students had a lot of priorities and sustainable food wasn’t high on that list.

We suggest FoodHub, a united front for all of the many existing initiatives around sustainable cheap healthy food on campus. We thought about delivery through Facebook, app, text, etc. And we also wanted to consider active vs passive information – looking information up is time consuming, but prompts can be invasive. We went into a participatory design process with this in mind. So, we ran a trial text service as part of this, sharing messages from existing initiatives on campus. Then we did one to one feedback with participants.

We had great focus group qualitative data but we wanted something more quantitative data too, and to understand how texts might work and how. Our survey showed that a third of students don’t plan food but buy on the day; but we also wanted to accommodate those planning in advance. People found finding cheap food on campus relatively doable, cheap sustainable food more challenging.

So, we ran this for a week and we had results. 37% attended 1 or more event and all said they’d attend again; 100% would recommend to friends; 91% thought it provided a cost effective alternative; 63% indicated they might seek out food themselves. But what didn’t work? Well we sent 14 notifications and one 1 to 2 responses – this was either people who already had plans, or who were out at KB where events were less accessible. But we also saw 100% no change in attitude towards sustainable food options. People have busy schedules, we had a number of individuals already engaged in sustainable foods already in our group – so events not novel.

When we went into participatory design process we were thinking about an app, but actually texts seemed more effective – more embedded in day etc. And for an app they would have to download, use, keep on phone. In terms of active vs passive platforms. The majority of people wanted Facebook for more passive service, and text for more active materials. The combination was definitely the most possible. People also wanted some sort of review of events that could help guide peers (Yelp style), probably part of the Facebook component.

So, where to go from here? We are running it for a month to see if attitudes change; we want to add more features; and we want to promote and share the service to a wider range of students on campus.

The RA Connection

We are going to talk about what we did, the way we approached this. We made a toolkit for RAs (Resident Assistant) to help them support students to make healthy decisions on campus. We talked to students, RAs, Foodsharing, SRS office, and we engaged with surveys, focus group, informal interviews, and participatory design workshop with RAs.

When we throw away food we could be eating it has a financial and environmental impact. The University has a policy for zero food waste – but students are not included in that. Students care about this but when asked what they consider when purchasing foods students say convenience and costs are main concerns when considering buying sustainable food. So, our idea was to work with Edinburgh Food Sharing. So, what is Food Sharing?

Food Sharing takes unused or unsold food from individuals and businesses, that is passed to Food Sharing, is then redistributed to Food Sharing. This is mostly still edible (e.g. day old bread) – half of all food waste is edible, and 40% of UK students report skipping meals because of costs – so these can be complimentary issues. In terms of changing behaviour there are personal, social and material environment aspects. We thought that changing individuals isn’t the best route, but that changing social would be more effective, hence using the RAs.

The RAs help students when they first arrive at university. 170 RAs are based at 35 sites across the University and they are obligated to run at least one sustainability event per year. And free food is a great way to engage students. So we decided to educate RAs with sustainability by helping them running their events, we used our participatory design session to engage with RAs. We set SMART goals for this: Engage 50 students from multiple halls in (1) food collecting and distribution based on but not overlapping with FoodSharing (2) events to cook, prepare and share that food.

To help RAs we wanted to do some work for then – handouts, posters, and event forms for their Learn space run by ResLife. The handout explains food sharing. The event forms cover Cooking from Scraps – a workshop attendance followed by running their own; and Food Sharing Month – to raise awareness. The posters highlights workshops. The second is about food sharing to raise awareness.

So, we wanted to raise awareness, support RAs and make it easy for them to do more. And it was received really well by the RAs we spoke to.

And with that all the presentations were concluded and we’d hit our 11am finish time.

Huge thanks to Ewan and the whole DDS student and staff community for having me along – there were some fantastic ideas presented and I really enjoyed seeing the different approaches taken – some much more design orientated, some much more technical. The projects will now go on to write up their work into reports and the projects will be shared on the course website.

Mar 142016
 

Back on 2nd December 2015 I attended a Digital Scholarship event arranged by Anouk Lang, lecturer in Digital Humanities at the University of Edinburgh.

The event ran in two parts: the first section enabled those interested in digital humanities to hear about events, training opportunities, and experiences of others, mainly those based within the College of Humanities and Social Sciences; the second half of the event involved short presentations and group discussions on practical needs and resources available. My colleague Lisa Otty and I had been asked to present at the second half of the day, sharing the range of services, skills and expertise EDINA offer for digital scholarship (do contact us if you’d like to know more), and were delighted to be able to attend the full half day event.

My notes were captured live, so all the usual caveats about typos, corrections, additions, etc. apply despite the delay in me setting this live. 

The event is opening, after a wee intro from Anouk Lang, with a review of various events and sessions around Digital Humanities, starting with those who had addtended the DHOxSS: Digital Humanities Summer School at Oxford in summer 2016.

Introduction to Digital Humanities – Rhona Alcock

Attended with bursary from College. Annual event for a range of interests and levels. Was an introductory strand on DH. Then another ~9 that were much more specialist. The introductory session was for those new to the field, or keen to broaden understanding. Gave an overview of other strands (some quite technical). Learnt a huge amount for that week. Came back and write up notes – 25 pages of them and I’ve referred back to them a huge amount. Main topics I benefited from was on planning DH research projects, on light weight usability testing (and QA). Great session on crowdsourcing and comining in your research area. Also some great sessions on knowledge exchange and engagement. More than anything what I had was the sense of connecting with others interested in DH and what they could get from it and do with it. And loads of new ideas for what I do, new audiences to take it to. Ractical advice to take it forward, tools etc. So, highly recommended.

David Oulton, College Web Team

I went to the summer school too. Found a big gap in the range of tools and resources (WordPress, Drupal, ESRI, ArcGIS, etc). There are so many tools that are just out there and can be used for all sorts of things, that can be downloaded, etc.

I’ve created a list on pbworks (dhresourcesforprojectbuilding.pbworks.com). And there are lots of resources listed there. I’m keen to encourage academics to just go and try these things, that can be set up quickly and easily.

Digital Medieval – Jeremy Piercy

Worked with new digital Bodleian interactive tools. Interactive scans of images that allow you to see hidden writing with a varieties of light – often can’t do on an artefact. Also learned a lot about ArcGIS, and how useful that can be. Also curation issues – many of our documents will eventually cease to be. This allows ongoing curation of those items. We have various high quality images and scans that will “always be accessible”. Portability of data is a key issue – not tying your work to a particular interface that may later fail/change/etc. Need to be able to move information around. I wasn’t the only person at that…

Gavin Willshaw, Digital Curator at the Library

I also went to Dig approaches to medieval and renaissance studies. Was quite specialist in many ways. Lots of hard work but learned a lot. I was also keen to understand how DH researchers work and what the library can do to support that with collections and tools. Was also some sessions on DIY digitisation – mini projects around doing that, managing that data. Tools such as Retro Mobile – ways to see what lies beneath images. Also some quite good introductory overviews of areas like TEI and IIIF for interoperability etc. Also several workshops to try stuff out directly. Really enjoyed seeing new stuff – e.g. hyperspectral imaging. Also a session on how museums can use wifi signals to track visitors movements and tailor what they do to that experience. And fed into discussions we’ve been having about Beacons and Blue Tooth in the library. From my point of view it was a really interesting mixture of tools and skills and experience. And I’m now looking for how the library can get more involved. Again, would highly recommend.

Humanities Data: Curation, Analysis, Access and Reuse – Rocio von Jungenfeld

I work in the data library, but am also finishing my PhD at ECA. I was looking at data tools and analysis tools. To compare what we do to what others do. And the recommendations from other places. Had very interesting speakers. Combination of HATHHI trust, and also OxII. They gave really practical advice on software out there, schemas, metadata, methodology, great insights to data tools and analysis. New tools to me, e.g. Gephi, and was very useful. Good experience overall – was a big Edinburgh contingent there, stuff done together. Interesting people and good lectures. Would recommend. My notes are available.

Harriet Cornell, Edinburgh Law School

I’m a post doc in HCA, and project officer for the political settlements programme. It was a complex workshop. Went for this one not introduction as I felt up to date. But this was at the sharp end in terms of the technical stuff. Galloped through technical stuff, would have liked more time on software. But reflecting afterwards it was great. Trying Open Refine – that I didn’t know about it – but also how we label and tag data and research. Really useful. Four things I took away:

  1. The capacity for DH projects – having that director of the HATHII Trust was great that that could happen
  2. Tools, particularly Open Refine – so useful. I know that Anouk and Anna have run workshops on this but it’s brilliant.
  3. Labelling and tagging. I do lots of blogging on WordPress and thinking about SEO, and just thinking about that in a different way was great.
  4. Design, Curation, Research and Longevity – thinking about the time and cost of planning and making things properly sustainable, after e.g. 10 years.

If I did it again I’d have done the introductory workshop. But this was great if you were happy to get down with Owl, ontologies, Python etc. I was tweeting from the session with the #dhoxss tag.

Linked Data for the Humanities – Anouk Lang, School of Literature, Languages and Culture

When you are a researcher your data is your baby. My lovingly curated research database with rich information about which historical figures were writing to whom, from which places, at which times. The problem is that if you want to share that data, your stuff is hard for others to use. The solution is ontologies and linked open data.

So what is an ontology? It is a structured way of understanding the context of an object – e.g. for the British Museum it might be where it was from, who acquired it and where and when, when it is from, where is it located, what is it made from. So we have linked data. Which we express as a “triple” – a subject, predicate, and object. So for a James Joyce letter then you have a lot of known individuals already – James Joyce is out there. And then locations wise there are lists of locations – you want an authority list (someone elses ontology of places). And then the predicate (e.g. when James Joyce was born) is also already available…

So, data is stored in a “triplestore” and you can query it using SPARQL, which lets you ask about name, place, location etc. There is a structure for SPARQL queries. That lets you query stuff within others databases.

So, if you are interested in using stuff in others’ databases or how to share your data with others, then you want to learn about Linked Open Data and SPARQL.

DHSO – Jim Mistiff

PhD student in English Lit. Went to the summer school and did the Linked Open Data and RDF course. Before PhD I was a Drupal developer, so had an interest from that so interesting to see it from another angle. I’ll be looking at specific application of LOD. Specifically for a text heavy usage for my own PhD. Not a perfect solution but an interesting starting point for a conversation.

Getting some definitions out of the way. LOD is a scary term for literary humanists. Any texts I’m using is a data object really. Breaking a long text up is a more useful way to think of it. Open Data helps you share data with others (and vice versa). The Linked part lets you interlink stuff. So if, say, a DB at Edinburgh (modernist data), and one at UVIC (who have modernist correspondence) you can link those together to form one complete set.

One of the results of doing LOD is that your dataset or database or a version of the internet that is easier for machines to read. Right now the internet is a set of dumb links – LOD allows text data objects to be more machine readable. I think it will be easier to explain that through an example.

So, I’m writing my PhD on Hugh MacDiamid. I’m interested in later stuff, particularly a poem called In Memoriam James Joyce. I’m interested in the construction. He wrote the poem through borrowing/plaguerising from other sources which are not credited. E.g. from an ad in the Writers and Artists Yearbook 1949. I’ve gone through that poem and found about 60% of the sources. That is a very interlinked text that maps nicely onto the idea of LOD. So, what I’d like to do is to datafy MacDiarmid. What I propose in terms of what Anouk covered is to take the text, word by word… and take the two lines from the poem, break them into triples… Can do it character by character… can automate… Can then apply Stanfords NLP to it… and then identify automatically when you’ve got a name of a real person in there, the name of a text… A very vague linked data model here… Bits in boxes (in his diagram) that are in other sources. So a mention of Pape – means Capt A.G. Pape and we could grab info from DBPedia. And then a text by that author could be pulled in/connected to. Etc. This makes assumption about what else is out there. But what it gives us is a rich version of the poem. We tend to read many texts in these ways – looking up definitions, references etc. Not suggesting change in core tasks, but using technology to enhance what we do. We could turn a LOD version of the poem into a hyperlinked version that pops up those obscure references etc. Much of what we already do, but in an automated way.

My suggestion expects and relies on other people doing parts of the work. What we can all do is get it closer to LOD than it is. There are five steps (see 5stardata.info by Tim Berners-Lee). Step one is get it online – e.g. a PDF on the web. The next step is to structure that data – a table rather than an image of a table for instance. Next step is open format – so that table in CSV rather than Exel. Next step is using RDF to point to things. Final step is LOD – the stuff of linking from proper names and quotations to other names and data stores. And then we have LOD.

DHSI, University of Victoria, Canada – Anouk Lang

This is premier DH summer school but costly to get to. Cheaper with student membership of computation in humanities(?).

I did three workshops – there for 3 weeks. I’m going to talk about Programing for Human|ist|s. My area uses R, Python etc. I did an intense week long Python course. So you start with a spec. For us we decided to construct a script that will visit a website and pull out certain bits of information relating to discussion posts (username, data of posting, content) and write those to a spreadsheet so they can be used for analysis. So, a web scraper.

Then you write Pseudocode. And that includes pulling in other people’s stuff – loads of Googling Stack Exchange for code and libraries.

So, Beautiful Soup does loads of web scraping. Then wanted it to write to a file. So then, you can build the code and run it. Then creates a file. Now when I did that I did pull out the appropriate text. It looks like a mess, but that’s a starting point. I spent the rest of the summer consolidating things, and doing some other things. So that included building something fun using a Markov Chain Generator – to feed text in, and produce lovely parodies. Can then use Python to automate Twitter posts. So we did a fun PatrickTwite Twitterbot (to go with a book launch).

Programming History Live – Anna Groundwater, History, Classics and Archeaology

I’m a historian. I’m here to talk about Programming History Live, which I attended in London in October. My first encounter was with the website. And that is a fantastic website (programminghistorian.org) – free, open, very enabling. It takes you through tutorials in lots of software you can use. Great range from Zotero, Antconc to do corpus analysis tool, to Python. Stuff on data cleaning, network analysis. Using Omeca to do online exhibits – will be used for a masters renaissance course. Comes from best DH ethos. I recommend W3C for LOD and Open Data. Also clear tutorial on SPARQL on Programming Historian. Also has web scraping tutorials.

My interest is network analysis. Martin During uses Palladio – free online network analysis tool which visualizes those networks. The site takes you through step by step. Gives you working examples with data from the website so that you do it yourself. Actually doing stuff is a great way to learn. Now recommended to several dissertation students who will use Palladio and then on Gephi. One thing to add is that these amazing exciting tools are compelling and exciting but there is theoretical underpinning that you need to understand if you use them. And Programming Historian also covers areas of that.

So, fantastic resources here (and Anouk also gave a lovely plug for EDINA’s geospatial tools and expertise, including my colleagues QGIS training).

Session on BL was led by James Baker (now at Sussex, was at BL). He’s also an SSI fellow – gives you £3k for DH work and profile there. I learned a lot on Antconc, on TEI, shell and widget for web scraping.

Antconc was with Anouk, and the tutorial written by Anouk and Heather Froehlich at Sterling. Antconc helps you analyse a corpus. It’s free to download to look at your corpus. So, for example a set of movie reviews (test data from Programming Historian) that has “shot” marked up across the reviews. I like this because it combines the patterns of big sets of data, with the ability to see the exact context. Combines distant and close reading concepts. And here “shot” is shown in its many definitions and uses. Concordance is seeing that word with surrounding words (and you can choose surrounding number of works). I’m interested in this because of cognitive geography of James I and VI using text analysis with Antconc. Some work already done on Agatha Christie using Atconc (in a paper on language and dementia by Ian Lancashire [PDF]).

Other useful resources: James Baker did a great reflective post on his blog, Cradled in Caricature. Also there are a range of people I recommend following on Twitter around digital scholarship, digital humanities, etc. including: @heatherfro, @williamjturkel, @adam_crymble, @ruthahnert, @melissaterras.

And that brought Section 1 to a close. Section 2 of this event – which I was presenting at – was collaboratively noted. See: https://docs.google.com/document/d/1U1CTcTi7hC-1gVq5HD762k2MCgSXaXDfPp5KsDNfOlk/

Mar 132016
 

This afternoon I’m at the EdinburghApps Final Pitch event, being held at the University of Edinburgh Informatics Forum. As usual for my liveblogs, all comments and edits are very much welcomed. 

EdinburghApps, is a programme of events organised by Edinburgh City Council (with various partners) to generate ideas and technology projects addressing key social challenges. This year’s Edinburgh Apps event has been themed around health and social care (which have recently been brought together in Scotland under the Public Bodies Joint Working Bill for Health and Social Care Integration).

The event has run across several weeks, starting with an Inception weekend (on 6th & 7th Feb, which I blogged some of here), then a midway catch up/progress day (held on 27th Feb – you may have seen me tweet from this), and culminating in today’s final pitch event, at which we’ll hear from previous winners, as well as this year’s teams. The challenges they have been addressing around health and social care challenges fall under five headings (click to see a poster outlining the challenge):

Sally Kerr, Edinburgh City Council

Welcome to our final pitch event!

EdinburghApps is designed by the Council to explore how new approaches and new ideas can inform what we do. So, to start with, we are going to hear from some of our previous winners.

ARC-Edinburgh – Anne Marie Mann & Ella Robbins

Anne-Marie: We started this app to address Addiction recovery back at Edinburgh Apps in October 2014 – which we won!

So our app – a smartphone app just called ARC  (http://www.arcapp.co.uk) – is  an App to support those in Addition recovery, helping them to track progress, boost motivation, and connect to the Recovery Network in Edinburgh.

Key features of our app are a guide to local meetings, AA, NA, etc. We also have a motivation and reflection section which includes motivational quotes, mindfulness resources, and we also have a “Need Help?” section which connects the individual to our Emergency section. In this section we connect the user to their key contacts, they select these at set up and can send a pre-populated text asking for support.

But there is more here. We had an idea, now we have an app, a company, a community… And Robin is going to talk more about that.

Ella: I don’t think when we first had our idea we knew what would happen next. We worked with Jana at the City Council to create a proposal for a developer – we aren’t developers we just had an idea. We hired a developer – through Anne Marie – and he’s been the third part of this project the whole way through, and that’s Dave Morrison, University of St Andrews.

When we had the team we researched the market. We had access to a close friend with addiction issues who was able to give us an insight into needs and requirements. But we looked at what else was out there. We connected to Dave Williams at the council who connected us to Serenity Cafe, which helps addicts in recovery.

We then set up our company, which we run outside our full time work and care responsibilities. We then went into an intenside user requirements and design process – drawing out every screen of our app before anything was built. We created a project plan, we worked out a marketing plan, and we set about launching our app.

The Council’s role was funding – which was great – but also project management. We had regular meetings to check in and check progress. The council were also essential to that relationship to Serenity Cafe, and that local and specific expertise of Dave Williams. Those contacts, access to market research, and knowledge and experience helped us hugely, particularly to overcome challenges as we went along. The Council provided guidance. On a practical level the Council also undertook printing and distribution of marketing materials and crucial advocacy.

In terms of our reflections on this process… It has been hard work and took longer than we thought. I work in marketing in my day job so this was a huge change and learning opportunity for use. We’ve had to manage a whole range of stakeholders who we wouldn’t normally have worked with, managing expectations, undertaking user requirements, etc. was a huge opportunity. It was a real chance to help people of Edinburgh and has been enormously rewarding.

So, the app is out now and we’ll be giving it a big proper launch very soon!

Q&A

Q1) Can you see yourself doing another app now that you’ve done this?

A1 – Anne Marie) Ella just had a promotion at work, I’m just finishing my PhD, so not right now but I can see us doing more in the future.

A1 – Ella) Absolutely, sometime in the future, but not right now.

Run the City – Jenny Tough

This came out of Edinburgh Apps 2015, our team was Kate, Jenny and Hilde (aka Small, Medium and Tall). We all lived in different cities and had travelled to other places a lot so had lots of ideas about what we might do – probably 8 ideas, a bunch we pitched, but the one we settled on was Run the City…

So the idea was that running can be a brilliant way to explore a new city and get to know it, and as a traveller it would be great to have some guidance on the best routes etc. So, we proposed a mobile app that would be engaging, and have a minimum of 5 routes through the city, and would interoperate with other running apps – so you can capture all your running stats as you normally would. It was going to need to work on iOS and Android, and be easy to add these routes to.

So, myself and Jamie Sutherland (@Wedgybo) eventually took things forward – both of us are seasoned international runners.

We did some scoping on what runners would want and they really wanted a mixture of green routes and city routes, to not just be the key tourist areas. And that there needed to be different distances and difficulties, as well as th ebest local spots to run. I started out dropping key pins on the map based on Council data. But we also tried lots of routes out – running those routes, testing them out, making sure that worked.

The kind of data we were using was data on monuments in Parks and greenspaces. There were also trees with stories, parks in the city (with opening hours etc) and we came up with five routes…

The first of these routes is the City Centre Highlights and History, which starts on Calton Hill but also takes in Grassmarket etc. The second route is Edinburgh Green Route – for those wanting to enjoy great places to run but not neccassarily interested in the history. The third is around Hermiston Gait, which is actually beautiful. The fourth is the Water of Leith – and we had audio we could draw on here which was brilliant. And finally we had the Seven Hills of Edinburgh – a really difficult route but essential as an unofficial race does this route every year.

Jamie used Ionic framework which is based on AngularJS and ues Cordova for hybrid app. And we used FireBase to create the routes – and that looks really simple for me editing routes in the app.

We rang weekly test runs – in place of meetings! Edinburgh Apps gets you fit!

We sent the app to beta testers as it was, without instructions for accurate results. And there was mixed feedback on the runs and on the technical side of the app too.

In terms of what we found were difficult, and what we learned. We found audio placement difficult to define for different paces (i.e. walkers vs very fast runners) – and that only worked by testing it at those paces. The catchment area of audio points was also extremely hard to fine tune (e.g. which side of the road). But there was also the issue of the seasonability of Edinburgh – daylight time being an aspect, but also things like differences in route for festivals etc since footfall changes a lot. We also found that app simulator really didn’t give us a good idea of what worked and what didn’t – th eonly way to do that was test it with running.

The future for run the city. The MVP was recently launched and is available in the App Store right now. We have route development in siz new international cities currently underway. But doing more here is really a challenge when fitting this around other day jobs and responsibilities. So we are also testing monetisation strategies – events, in-app purchases, advertising to make that development work possible.

So, do try the app, give us your feedback.

Q&A

Q1) What is the audio?

A1) It’s the directions – turn left, turn right, etc. But also the things you are seeing and experiencing.

Q2) And how easily could that be changed? Is the audio geocoded? Have you considered iBeacons if they become more popular/available?

A2) The audio is tied to pins on the map added in FireBase. We have been considering iBeacons certainly.

Q3) Could you crowdsource the routes?

A3) Sure, but it can take a lot of work to develop the routes. But the running community online is big and active so I definitely think that that’s the way forward.

Sally: And now we have the really exciting part of the day, the pitches from our teams! So, lets start with Game of Walks…

A Game of Walks (#agameofwalks) – Gary

The team for this project was Elena (@atribeofneli), Katie (@hiccuo42), Lorna (@LornaJa23511553), Mischa, Gary (@garycmartin), Mohammed.

The project we were walking on with Sustrans was to encourage children to walk more. The idea is that with a school groups we gamify the walk to school. And to also include some level of STEM, as well as art as they get to design some parts of the system. The second weekend was rather fun as we prototyped the system.

The idea is that children are in different team groups, collecting a particular animal shape. Then they get to choose the animal shape for the next week’s challenge. The idea is that you place these devices across the walk to school you encourage walking to school, use of safe walking routes, and some gameplay.

So we are using Arduino with sensors… And walking part triggers the light. The units wait a set period, then select randomly but equally a shape to show (of three). And then triggering will show another shape. Each animal shows around 10 minutes – and you need to collect it. If it’s someone else’s shape then you don’t collect it. So other walkers, cats, dogs etc. may trigger the system but it should be random and unbiased. And when they capture that shape maybe they share it on their blog, league tables within the school etc. And the units use little gobo selectors so you can theme and change those as you want (e.g. easter, christmas, halloween), etc.

So the units are all 3D pieces (18-20 hours per unit and all the pieces). They aren’t quite ready for outdoors yet, but the battery life isn’t bad – 35-45 hours right now but could easily be set up to do a week. And you could also set up the units to only capture/be active during school run hours.

So, where we are now is that we want to do some school events – fairs or festivals or similar – to test them in a contained environment. But I’d be keen for feedback from teachers, teaching assistants, etc. who would be keen to use these with kids in a real environment.

Q&A

Q1) You said they aren’t waterproof at the moment?

A1) Not at the moment… You could take these and insulate the electronics on the inside so that they don’t corrode. If you wanted them more long term you could do more. The idea is to make these cheap and accessible – it’s about £12 in 3D printing material, and about £10 electronics, so relatively cheap and therefore not a big deal if they go missing. But actually you could fit most of the electronics in a poster boards – on a single image on the paper with a wireframe in that poster – which would be lovely. So, the form factor (units) isn’t essential.

Lots you could do here, like installing units that capture footfall data when game isn’t in place so that you have a baseline of data to give you some idea of how busy it is on a given route, and if the Game of Walks is making a difference.

Sally: We did test these units with colleagues at the council… And discovered just how competitive our adult colleagues were!

Meet & Eat: A recipe for Friendship – Beata and Annabella

Beata: The idea is basically dinner for strangers! Our mission statement was to help prevent loneliness amongst Edinburgh’s student population. The challenge owner was the NHS who highlighted the issue of loneliness, and that that is often about transitions in life of all sorts, including moving away from home/becoming a student. And this is a big problem. 68% of adults say that they feel alone, either often, sometimes or always. And 18-34 age group is most affected. Lack of personal contact can be as harmful to health as smoking 15 cigarettes a day. So we really wanted to find a way to help.

Annabella: We thought that a great way to address this would be through food – as we all need to eat. So, our example Meet and Eat user is Jin. Jin is a 20 year old engineering student at Heriot Watt. Studies are fine, but he misses his family and friends from home. He sometimes finds it hard to make friends outside of class – initially language was an issue but it isn’t now… But he doesn’t have that network of friends and support. But Jin walks through university and sees a poster on the wall for Meet and Eat. He signs up and decides to join a dinner at his student union. He feels safe going to an event there and decides, being Japanese, he’s going to take sushi as his dish for the dinner. He meets new friends with things in common, and they can take it from there.

So, that’s the idea basically. Students are often early adopters of tech but we wanted to have a location for events that was safe and neutral – and accommodating of students who don’t have room for, say 5 people.

So, we tried to run two test events. The first was to be at Glasgow School of Art but that was in reading week. We ran another in Fountainbridge. We only had one student along but he gave us great feedback. He said first years are much much more open to this. Freshers week is when people are open to meeting people, starting events there would make people more likely to come. And we need better advertising.

Moving forward we would like to popularise the concept using existing social media, university intranet and forum platforms. We’d like to create a welcome pack for partnering with universities and include that in freshers week. And maybe that could lead to student Meet & Eat societies. If we get that buy in we think we could go forward with the app idea, but we need more market research and marketing support.

What we need is marketing assistance, links to universities – we have links with GSA and Napier. But we also need business advice, and we’d like more people for our team. We have work, university… a cat… But not sure how best to fit this in – although we’ve been inspired by the presentations that we’ve already seen.

Q&A

Q1) At least some students in first year of Edinburgh have catered food, not as likely to be able to participate.

A1 – Annabella) That’s a good point, which we hadn’t considered.

Q2) The office for social responsibility and sustainability in Edinburgh sponsored Global Sustainability Jam which led to an app called Fridge Friend – aimed at reducing waste by sharing with others.

A2 – Annabella) When we did some market research we also looked at supermarkets who recycle or discount food. We thought offers etc. might be encouraging and motivating.

Q2) There is also a thing called Food Share in Edinburgh who you might want to look at.

A2 – Beata) We looked at that but we think we need people engaged before we can do some of those partnership. In our research we came across Freedom who also use food waste in their cooking.

Q3) How do people get in touch if interested?

A3 – Annabella) We have meetandeatscotland@gmail.com

A3 – Beata) And a Facebook group as well.

Chattercare – Archie and his Dad 

This was initially designed to address people with cognitive issues… We are all social hubs, connecting with friends and families and neighbours… But when people have cognitive disadvantages they lose connections, those bonds are broken… People lose touch..

So, our idea is to enable communication between different people. So, the person with cognitive disadvantages can connect, but those people can also connect and exchange information between each other. We were really thinking about informal communication. From my perspective, when my great aunt had a stroke, you find yourself looking after someone with no idea of where to start… How do you wash a person in a wheelchair? What’s the new medication and possible side effects – how do I share that with others involved in care? For my great aunt she kept saying “miss miss” and had no idea what that meant – but actually she was wishing people a “Merry Christmas”.

So, how do you share that information? There are interest groups across similar carers; there are people caring for an individual – often many people involved; and messaging for one to one engagement; and we wanted some adaptive technology enabling the individual with cognitive difficulties to take part to. And so, that’s our idea.

And now… A live demo…

We are using a platform called Rocket Chat (Note: this looks like/may be a close relative to Slack) which is available for PCs, Macs, Web browsers, Tablets, Mobiles (iOS and Android). But we require lots of modifications… We will just show some examples here…

Lets call our home help “Jane Austen”. So Jane subscribed to a general #wheelchairusers channel, but she also is part of a homehelps private chat group for more specific questions.

“Mary Shelley” is our supervisor for home helps… And she subscribes to #wheelchairusers as well as #strokerecovery. But she is also part of direct message conversations with “Barbara Cartland” – the daughter of a patient who is interested in pensions. And also a private group for “Jack Faust” – an individual who needs care and help, this would be private to those caring for him. So Barbara Cartland asks for an update and his grandson “Billy Boy” sends an update and image from his visit.

So, what is ChatterCare since there is an application already there? Well it would be about customisation, and the idea would be that all communications are in one place; there is an opportunity for some oversight – so for instance the Stroke Recovery group could be monitored by the Council, to share authoritative information, expel myths, share resources known to be good. And eventually we’d really want some adaptive tech. It would be great to have the individual with cognitive difficulties directly involved, but they will all have very different needs and requirements, which is why that would be a later thing requiring further development.

Note: no questions here, so onto our final team… 

Open Doors – Laura & Team Open Doors

Loneliness is a huge issue in the UK and it needs to be dealt with soon. Over 1.7 million people over 65 can go a week without having contact with someone they know, of these 1.1 million can go a month without that sort of contact. So, our idea is an app called OpenDoors which will be simple and intuitive and is designed for older people.

Elderly people are quite keen to use new technology, but modern technology can have too many confusing functions and applications that they will never need. So, for this app we plan to use very large icons, make it visual and intuitive, add only the necessary functions and features. And we want it to be very consistent so the users always know what they are doing.

We talked to people who tried to do this before and we think the biggest challenge would be getting people to join this sort of social network. There are now 11 million people in the UK over 65 (AgeUK 2016) but only 28% using social media. So, we want to start with Elderly people in Edinburgh, working with family members as elderley people are more likely to use technology if a family member uses it and introduces it. We also plan to promote our service and network at offline events, including those run by the council. And we plan to have a listing of local events to encourage meeting and engagement. We will also look at TV ads, as TV is used by older people to manage loneliness.

We think this idea also has the potential to save the NHS money, since loneliness can have such detrimental mental and physical health effects.

Our initial idea was that we would create a simple button-like device to access Open Doors but, for safety reasons, we decided a standard tablet or mobile app would be more productive. The users of our app will be both the elderly individuals and anyone who is familiar with the mainstream mobile devices.

We haven’t tested the app yet but we have interviewed elderly people, researchers, and UX experts to get their input. We also have an event coming up at the end of the month. And we have designed the prototype app, to include clear easy to use functions, chat, etc. But to make our idea a reality we would need to develop our OpenDoors app to also work offline, so that it is more flexible.

Rahma: The app is very simple, big clear icons, and you can look at family members, view our friends very easily, make a call, or view chat. And, for the keyboard we have bigger icons/keyboard so it’s easier to type. Personal profiles let you add information. But this is a prototype…. We want to make it a real app that could be sold or available for free. So, for now we will develop the app and

Q&A

Q1) I would imagine that for your audience typing could be a challenge so autocomplete could be useful. Have you thought about customising that autocomplete/autocorrect for your users? My phone has autocorrect and autocomplete options… But those are biased to the model of what they think the user will say – so Californian tech comes up high in the options list. For your target population could you create a more appropriate model?

A1 – Laura) That would be possible. We were thinking of having voice commands for those with visual impairments. We haven’t considered what you were saying exactly, but it’s a really good piece of advice.

Q1) There is a team at Cambridge who helped Steven Hawking with this.

Q2) Most of us use a whole variety of tools right now… There is quite a wide list of tools in use in our family circle. If we all had to use one tool, we probably wouldn’t do that, but if that could stitch together existing tools that might work…

A2 – Laura) That’s what we want to do, to connect up some key tools but make it easier to engage with and use, making it more simple to use.

Q3) Great presentation. I have a comment about your user base… How will you develop your user base here? You need to think about how you get those early adopters first, to build up that interest to get to first 100 or 1000 users. Relying on Facebook or Twitter to find those family members won’t work.

A3 – Laura) Our marketing strategy is, for early adopters, to engage with the city, with the Council, and find users there. For app development and testing, and hopefully then expand out from there. Perhaps starting with computing clubs etc.

Sally: We have sadly reached the end of Edinburgh Apps and all the pitches will be on YouTube, and with the Council and Challenge Setters. My next step is to connect you to the right service owners, to help with next steps etc.

I want to thank all of the teams who took part. I know how much work it takes to get to this stage. I want to thank you personally for that work. And I also want to thank everyone who came along to support, to listen, etc. And, what we have for all the teams are some goodie bags. And I’d like all of the teams to come up here for huge round of applause!

Thank you again to you all! And do keep an eye online for all the videos!

And with that (and much rustling of goodie bags) we are done… ! 

Mar 032016
 
Jisc Digifest conference pass.

Today I am in Birmingham for day two of Jisc Digifest 2016 (catch up on Day One here). I’m particularly hear wearing my Jisc’s 50 most influential higher education (HE) professionals using social media hat, helping to share the event with the wider sector who aren’t able to be at the ICC.

There is also an online programme so, if you aren’t here in person, you can not only follow the tweets on #digifest16 and the various blogs, you can also view special online content here.

As usual, this is a liveblog so all corrections, additions, comments, etc. are very welcome. 

At the moment my expected schedule for day one (this will be updated throughout the day) is:

9.00 – 10.00 Plenaries – the power of digital for teaching and learning

The chair for this session is Sarah Davies, head of change implementation support – education/student, Jisc.

Heather MacDonald, principal, Loughborough College

I have missed the beginning of Heather’s talk, so catching up as she addresses the issue of Area Reviews in FE… Heather is talking about the uncertainty of mergers, and of needing to be confident going forward, ready to embrace a technology led future.

Technology, however, is also a real and substantial job threat. But this intelligence is only artificial – until recently it took huge amounts of computation to recognise an image of a cat. We need to get out there and teach to create the next generation of creative and innovative future employees. We need to address the needs of this changing world through chnging pedagogies, through empowering students – perhaps to organise and teach themselves. But what would Ofsted say about that? Well, it matters, a good Ofsted report is very important for FE Colleges, but I would rather have creative and innovative teaching methods. That means we have to, as Tim Marshall said last night, bring the regulators up to speed more rapidly. We should be looking for solutions through the digital lens of technology

Professor John Traxler, professor of mobile learning, Institute of Education, University of Wolverhampton 

Prior to today some of what I will say has been pre-trailed on the blog. I was quoted as saying that “mobile learning” has stalled… But I essentially want to raise the issue of “mobile learning” and just the regular matter of learning with the tools that we have. I was making that distinction around a couple of issues… One is that the money had run out, and that money and that will had fuelled the rhetoric of what we did with innovation in the first decade of this century; the second is the developments and changes in mobile technology itself. About 15 years ago mobile was delicate, fragile, expensive, scarce, something for institutions, and to promulgate their solutions. But the money ran out. And we also focused too much on what we were building, less on who we were building it for… But meanwhile mobile has made the transition to cheap, robust, easy, universal, personal. It’s hardly notable anymore. And whatever constitutes mobile learning now is not driven from the top, but by our students. And the technology moves fast but social practices and behaviours moves even faster, and that’s the harder thing to keep up with. People share, disrupt, discuss… That happens outside the institution…Or inside the institution but on an individual basis.

This technology is part of this fluet, transient, flexible, partial world. It enables people to help each other to learn. And web access is significantly moving to mobile devices rather than desktop machines. But what does that do for the roles of educational designers, teachers, etc. What people call “phone space” is very different to cyber space. Cyber space is a permitted space, back to the world. Whereas phone space is multimodal, you are having conversations, doing other things, crossing roads, travelling… And this is a very different learning space from a student sat at a computer.

Now, looking back I’d consider “mobile learning” rather backward looking, something of the last decade. I think that we, as professional educators, need to look outwards and forwards… And think about how we deal with this issue of abundance – how do we develop the criticality in our students to manage that. And we should question why they still come to us for face to face experiences, and to think about what that mean. Hence, I’m not that bothered if mobile learning actually is dead.

Ian Dolphin, executive director of the Apereo Foundation

We are a registered not for profit in the US, we have been described as an Apache Foundation for Education – that’s not quite right but gives an idea of what we do. We provide software including SAKAI, Xerte, and OpenCast (capturing and managing media at significant scale). But enough about us…

Next generation digital learning environment… Lots to say there but I will be focusing on a conversation that has opened up in the United States, and the relationship of that conversation to developing the discussion around Learning Analytics.

That conversation was started by Educause, which looked at the VLE – the benefits but also the drawbacks of being inflexible, of being very course or teacher-centred. And that work highlighted what a new VLE might want to look like – flexibility for different types of courses, that it should support collaboration across and between institutions, that it should support analytics for advising, and that this new environment should be a much more personal environment than what has gone before.

The analogy here perhaps is of Groundhog day. These are issues we have heard before over the last 10 years. But why do I think the environment is different now? Well, we are are more mature in our technology. We have gotten smarter and better at lightly working tools in and out of different environments. We are pragmatic about bringing functionality in pragmatically. And, lastly, we are starting to learn and develop a practical use of big data and learning analytics as a potential tool for personalisation.

I just want to pause to talk about academic analytics – about institutional trends, problems, etc. versus learner analytics – which are specific and personal, about interventions, retention etc. And we are already seeing some significant evidence about the effectiveness of learning analytics (see recent Bricks and Clicks report), with examples from the UK and US here. If one looks at the ends of the continuum here we are starting from prediction for retention intervention, but moving towards predictions for personalised learning.

There are several approaches to learning analytics at the moment. One is to buy in a system. We are taking a very different approach, developing a platform that uses various flexible components. That helps ensure data can move between systems, and that’s an issue Jisc has been raising – a national and international issue. And I think yesterday’s opening session was absolutely right about the importance of focusing on people, on humans. And if you look at the work Jisc has done, on ethical issues and informed consent, that is having an impact nationally and internationally.

We work with the society of analytics research. And there is a Solar analytics maturity framework. We have partnered with Solar and Jisc on our work and, to finish, I’d like to make a shameless plug for our Solar colleagues for LAK’16 which takes place in Edinburgh this summer.

Chrissi Nerantzi, principal lecturer in academic CPD, Manchester Metropolitan University

I asked all of you, and those online, to help me to design the cover for “Wandering While Wondering” and I want to thank you all for that. We are doing a speed chat on Twitter and I’d invite you to participate there.

My background is in languages, and in being a translator. But mostly in the world of books… I felt privileged when authors of books I was translating were alive, and available to talk to – but usually asking for help wasn’t possible. Reference books are still important for me… But the world of web 2.0 has changed how we can ask for help, the resources we have to hand. And I stopped working in translation when the world of digital became much more exciting. I’m now a learning developer working with colleagues to do exciting things with technology.

I’ve been asked about my work on playful learning… technology shouldn’t be driving but we should be able to play with new things – like Google Cardboard – and discover how to use these things in teaching! But not everyone feels playful… And sometimes people’s wings are cut by the culture, by experience… Sometimes we have to remember that we are physical beings, with room to play and discover (Chrissi is throwing balloons around the room and walking the auditorium at this point).

We are very lucky in my university to have real support for creativity… We have a creative module, where teachers and learners work and learn together, locally and globally on projects. We can be tempted to put our learners and academics in boxes, but we have to get out of our silos, put our egos aside, and work for the common good. We all want to do it but often the fear is there, colleagues play it safe because they don’t know what is going to happen (by genuine coincidence one of the circulating balloons pops at this point!). Being silly, being playful, gives us that freedom to get out of our boxes, to engage, to empower, to educate.

Sarah: How do you give staff the space and encouragement and permission to play, in such a time-pressured environment.

Chrissi: We need to make that space by getting rid of stuff, to encourage experimentation. We stuff the curriculum with stuff… But that space to play is essential, we need space to play, to experiment, to think of new ideas…

Heather: I fully agree with that. You need to understand what happens in your organisation, and highlighting and encouraging good practice. Build models of DNA, rather than studying them. Go outside, rather than doing everything in the lab…

Chrissi: That is really important, that need to use the space creatively. We have outside spaces we can use… Even a change of scenery can refresh our minds. We have to have that openness.

Ian: I tihnk this is also true in digital space. One of the problems for staff in digital spaces is that you need to make business cases to the IT department, get approval, and then you can do something… I think moving to something like the App Store for educational technologies, where you can experiment more.

John: I’m nervous that for students being on campus endorses their identity as students – being a distance learner or part time learner is a very different identity – and play might not do that for them. Students can have more serious conceptions of learning. But I’m also nervous of play as prescriptive practice – our students come from a huge range of cultural backgrounds and in some of those contexts play is seen as frivolous, we have to be aware of that and not go in too fast and miss the opportunity to engage.

Q&A

Q1 from David on Twitter) You haven’t talked much about the purpose of education, can you say a bit more about this.

A1 – John) If we are merely serving the economy in our work with students, we have to be aware of that, what that means etc. And that’s a concern about VLE – they won’t use those spaces as soon as they graduate. And that’s also part of criticality – our students have to have the skills to adapt and develop, to manage their

A1 – Sarah) They say that we can expect to have

Q2 from John Kirrimuir on Twitter) When the evidence for a technology is scant, how do we manage the risk of playing with it.

A2 – Heather) Use your nouse, but it is about risk taking.

A2 – John) We had a lot of small or limited experiments without evidence bases… But you do create that in those experiments…

A2 – Sarah) And it’s weighing up what you want to do in your teaching and learning.

Q3) I work in academic development around criticality and digital literacy skills, and this is very much about teaching students how to learn. That goes ok. But how about dealing with CAVEs: Colleagues Averse to Virtually Everything… those who don’t want to learn.

A3 – Chrissi) I think we need to immerse colleagues into these possibilities, being part of that community, not just modelling practice. If it’s something we look at rather than trying and engaging, then it doesn’t work. It’s hard to do but  if we work together we can change everything. There are habits, including bad habits, but it takes time and can happen.

A3 – Heather) It’s also about providing a safe environment to experiment and take risks. A number of institutions have created “learning laboratories” to trial ideas, to film or share those… A safe space which can work well. There is also quite a bit of student pressure, in FE at least, as they are getting something different down the hall.

A3 – Ian) I think that criticality and willingness to fail is absolutely essential for creativity, for many forms of learning… That has to permeate the institution.

A3 – John) I’m nervous that we are too introspective, deciding what learning is or is not… that we should not define this.

Comment, Sarah) In that context where learners have so many tools and opportunities in their own hands, what is the role of the university?

A4 – John) Well we can give them a degree, a crude generalisation but still true.

And, with that slightly controversial comment, we are done for this session. 

New directions in open research

Chairing this session is Neil Jacobs, head of scholarly communications support, Jisc.

Neil: We have a very distinguished panel for this session on new directions for open research: Tom Crick, Ross Mounce, and Cameron Neylon. We’ll have all three talks and then opportunity for discussion and questions.

Cameron Neylon, professor of research communications, Curtin University, Australia

I’ve been involved in open research practices for a long time now but I think things have really started to change. Open data was fringe, maybe 5 years ago, now mainstream. Open research is increasingly of interest. And we are starting to think more about open source software, to deeper engagement. We started with people outside the standard practice, and it has now moved into the mainstream of what the institutions we work within do. Institutions are academic institutions, but also funders, also institutions in terms of the way that we communication what we do.

We have seen a move towards institutionalising these practices, primarily towards Open Access. And I’m sure many of you have faced your institutions trying to institutionalise those processes of open access… And as we do that it’s important to think about what we want these institutions to be in the future, what that should look like. We’ve been retrofitting this stuff into institutions, what they do, mechanisms for recognition and reward. What does it mean to institutionalise these things? What happens when we haven’t thought about the unexpected consequences of this. And I’m as guilty of this as anyone… Maybe those mandates haven’t played out how we’d expect in how researchers do their work.

I have moved from being at a publisher and back to being a researcher. And I’m now a researcher in a cultural studies department rather than my original research area in the sciences. And seeing the world through that cultural lens changes things. I can see the ways in which we see things as “cultural challenges” in particular disciplinary communities… In doing this… Well we say it’s cultural… Maybe it’s digital… But  I think we’ve also used that as an excuse not to probe too deeply what’s going on. To ask about the culture of the research administrators in our community… These staff are familiar with traditional platforms, and publishers, and mechanisms… As we institutionalise open access practices, what does that mean about the cultures are, and how they can change. How do we move beyond “this is a cultural clash” to asking the question of how different cultures are in play, and how they might (and might not) change.

Ross Mounce, postdoctoral research associate, University of Cambridge

I’m going to talk about some of the tools and facilities already available for open practice. And I’ve been part of setting up the Research Ideas and Outcomes (RIO) Journal (riojournal.com), so I’m going to show you a promotional video for this. (The video is highlighting that the platform publishes research proposals, but also supports openness throughout the process of research. Whilst this plays I’m also going to note that I’m delighted to see that Ross is sporting a Software Sustainability Institute T-shirt today. Find out more about SSI’s brilliant work on openness and open software here, and through the JORS journal).

So, 90% of research proposals are never funded. And even those that are funded you see only a small part of that – maybe an abstract. As a researcher that makes it hard to see what is going on, to avoid repetition, and to collaborate. So RIO has been set up to share research proposals, as well as data, traditional papers etc. We aren’t the only ones – there is also F1000 Research and ThinkLab. And to find out more I’d recommend Mietchen 2014 (The Transformative Nature of Transparency in Research Funding).

Tom Crick, professor of computer science and public policy, Cardiff Metropolitan University

That was a really good framing from Cameron and Ross on the larger institutional stuff. We know that there are bigger funding and sustainability issues. But I wanted to come at this from a more personal perspective… As an academic and researcher I’ve always cared about openness and cross-fertilisation, at a personal level but also on a bigger governance and policy level. Openness promotes excellent research and science. How do we make sure that we know… There are clear economic levers, and broader economic and societal levers… So how do we ensure this approach is both top down and bottom up, and that that all works.

My job combines this odd combination of computer science and public policy, working with Welsh Government, with NESTA on data-driven policy making, particularly in Wales… Looking at how governments can do much more bespoke and evidence based policy making. Wales are keen to be a more agile economy, but how to do that. We have looked at available data sets – real time effective data for public policy making – and that is very hard. There’s a lot of “Open” out there, but having skills, tools, infrastructure and ability to use this effectively is harder. There is a UK Government push to openness, there is a big Welsh Government push to openness… And this work was only possible because there is so much open data out there… That’s a great way to look, to find structures and infrastructures that have impacts on us as academics but how do we really promote interdisciplinary practice, to have that funded without having to shape specific angles for funding from specific research councils.

We need to think about funding, reward, recognition, and governance structures that really can promote open research. Ross is actually wearing a t-shirt for the SSI, who I was going to mention, and they are supporting the research base, the openness of software. How can we embed a culture of openness throughout academia? I look forward to your questions and comments?

Q&A

Q1 – Amber Thomas, Warwick) I have questions about drivers for openness. In sciences you talked about reproducability, in social sciences it’s about data literacy perhaps, in Humanities it’s much more about collaboration… How do you see this playing out?

A1 – Cameron) I think that thing of cultures, and of social as well as disciplinary and community drivers. I think we need to think about the stories that researchers are telling themselves. As someone new to humanities research I find discourse analysis really powerful, getting those things to align will get us towards progress.

A1 – Tom) A month ago I was chairing a panel at a Jisc/AHRC Digging Into Data workshop, that’s a clear interdisciplinary initiative. We had a data scientist, a poet, and a German Central Banker, all talking about projects funding under the DiD call… But there were some issues of common language or understanding but there was a commonality of what they wanted to find out. When you find cross cutting things – and data can be a great focus for that – you can find alignments of culture, a common aim to solve common real world problems. Maybe one of the things we can pitch openness as is for the greater good, but that doesn’t always work… But open can mean new possibilities and opportunities and solution.

A1 – Cameron) It can be tempted to talk about the objects, but actually we need to talk about who we want to talk to, why, what the research is… less about data etc.

A1 – Ross) Disciplinary boundaries really don’t matter that much, they are quite artificial. Some of the most creative and brilliant academics I know are totally interdisciplinary. I’m an SSI fellows, I can talk to other fellows who are historians, or are in totally different areas, but we find common interests and concerns.

A1 -Tom) Universities try to pump prime this stuff with early career researchers, but there has to be more fostering of that across the board, at other levels.

Q2 – Adrian Stevenson, Jisc) You gave examples of research, but I wonder if you have good examples of research being done in different ways because of the open agenda.

A2 – Ross) Yeah, there’s Timothy Gowers Polymath project, who put out his questions, did the work with the community, and entirely bipassed traditional examples.

A2 – Cameron) A classic example was the e-coli outbreak where raw data was released to GitHub, and all started working collectively on that. Similar has happened for Ebola and Zika. But those have been “turning on the open thing” as needed. But the real examples are often from open source software which have been built that way to avoid a lack of funding for anyone else to do it. If you take that radically open approach you can find you are outside the traditional reward and recognition structures. I know people in this room have been frustrated by people publishing on things which have already been shared/blogged before but not noted in the same way for being in that format. So that is a challenge.

A2 – Tom) As Cameron said, we can get too focused on data/DOI etc. when actually you need that big challenge, that big goal, that research that you are working towards. It’s not just to tick a box on a form. Whether that scales up and down, looking at standard research models that’s more challenging, but if you can start to tackle this super-nationally and nationally that gets things moving… Need that shift before we see things as out there, as radical.

A2 – Cameron) I am part of a project called the Open Collaborative Science in Development project, where we do research, we go to communities to find the needs. But when we say “open” we have to think about “to whom” and “to do what”. And that matters as openness can hit local political agendas at odds with the research agenda – for instance over water quality. But there are increasing numbers of examples, in different ways.

Q3 – Daniela, Jisc) Cameron, you were talking about institutionalising practices, and I was wondering if you could say more about your vision for what this is, and the better practices that could be institutionalised.

A3 – Cameron) Let me start with something that hasn’t worked as well as it should have, the RCUK open data requirements. The mandate there was ahead of where the institutional support and resources were in place. We can’t always build platforms to respond to change rapidly enough. Framing things as compliance issues created a compliance culture, and that doesn’t engage researchers to actually engage and change things. So, what could we do or should we do? We have to build institutions better. Our institutions were build for mid century needs and they need to change in terms of governance and structures. I think we have to look very closely at the way that institutions administratively engage with researchers. We need more research to actually understand what is happening there. I see a culture of researchers, administrators, and funders that are seemingly at odds, even though they don’t want to be… We have to dig into that before we can improve structures. But that’s a hard problem. Within the frameworks we have, what can we do to ensure researchers and administrators both meet each others needs, and can understand those needs. Administrators have to engage in compliance, standards etc. Researchers have a different set of concerns. How can we align, and do that at scale, to find how we can collaboratively work to solve these problems… Rather than fight based on misunderstandings and misconceptions. More collaborative practice would be really powerful, and that really changes the way that senior university administrators see their role – away from CEO type roles towards Community Leaders.

A3 – Tom) I can give an institutional profile. I am half time academic, and half time working on our university’s commercialisation and enterprise work. I have access to some of the levers. A lot of this working is about promoting the benefits of collaborative research, research tools… Understanding that collaborative models lead to funding, that there are easy workflows and infrastructures – I write all of my papers on GitHub with other people… Once you make it easy to do this, you can promote that and cascade it down as well. We care about compliance, but also try to be proactive in changing research culture across the institution and disciplines.

A3 – Ross) Collaboration is easier when you share research easier – the proposal, RDM plan, the data… Right now research proposals are really quite closed, and researchers want to keep it that way for competitive reasons. But actually sharing early helps with compliance.

A3 – Cameron) Compliance is a useful lever… But researchers tick the boxes and then forget about them… In the case of DMPs, how do we make those collaborative between the researcher, funder, and institution, so that they are part of the record of what is expected, what happens, part of the manifest of data at the end of the project. So, this stuff depends on local context but also DMPs can be dead ends now, we need to make them a real living breathing part of the conversation.

A3 – Ross) DMPs aren’t open, and that’s the issue… I am a data parasite, I use other people’s data… I’m a bioinformaticist. And if I see that data is being generated, and will be released… I will look for that data, I’m contributing to that compliance data and I’ll report that if the data isn’t available. We really have to open up DMPs.

Q4 – Rob Johnson, Research Consulting) Tom, I was really interested in your enterprise director role… And in my own work I speak to different communities but opening up data and research opens up lots of enterprise  opportunities.

A4 – Tom) Actually I’m deputy enterprise director. Now enterprise can look like consultancy – which many academics don’t feel interested in that – or in commercialisation which can seem far from their research and teaching. But it’s about not being bogged down in terminology, but understanding the challenge you want to solve, the people you want to talk to… But not badging that in ways that may put people off. It’s hard though. I got involved through research data management… That’s tricky but means you can kick interesting projects together.

Q5 – Paul Stokes, Jisc) We’ve heard a lot about enterprise, economics… But ultimately someone has to pay for all this. Do you have examples of direct economic benefits of open work that helps make the business case.

A5 – Cameron) I can give small scale examples. I left PLoS a year ago and have become a consultant half time, that’s very

Public Genome Project – massive economic activity from that. But we keep missing the big things… GeneBank, PubMed etc. as they are so big and important, forgetting that they had to be fought for. And it can be hard to look at smaller stuff, and to account for that impact. But I also want to push back a bit about the economic impact for each part of the research, perhaps we should make the case that some infrastructure is not about a clear financial return – greater participation in democracy has an economic impact but not a clear financial one. So we should think beyond the financial ROI. But, that said, there are big examples, there are big platforms, there are great REF Case studies… But we have to be wary of focusing on financial ROI.

A5 – Tom) Thinking about open data in government, and in Wales we have projects around innovation in public services. If thinking of open data as financial ROI for private sector, that’s tricky and requires finesse. But there is public infrastructure that others can build upon… It’s a non zero cost but it doesn’t have to revenue generating, or producing financial ROI.

Q6 – Dashia, Cambridge) I am a PhD student and there seems to be a trend for publishing papers into social media – what does that mean for openness… Rather than repositories. People want research visible, not as clear if open.

A6 – Ross) ResearchGate and Academia.edu are not open and not the same as a repository… Many researchers don’t understand that difference between free and open. And they are indicative of a failure from institutions to present repositories in easy to use, beautiful looking ways…

A6 – Cameron) It’s symptomatic of failures of institutions and join up in institutions. There is a kind of Napster for research at the moment which is similarly emerging from disruption from an unmet demand. There are opportunities around that demand, and that could be met by the community at low cost… But we often miss that opportunity and that’s where those commercial platforms set up. The outreach director at Mendeley says “Academia is unwilling to invest in anything until it’s so important they can’t afford it”! And that usability etc. is part of that. What do we need to do to invest properly, particularly to move away from direct immediate financial ROI as the requirement.

A6 – Tom) Social media is another way to facilitate and enable openness in a general sense. For my discipline we have preprints in ArXiv. It seems crazy to write a paper collaboratively and openly and not put it there. But I use Twitter, I blog as an academic – good practice myself – but five years ago that was generally seen as a waste of time. Now we’ve seen a step change of the perception of the researcher. Social media is being promoted in institutions through courses etc. But those tools connect you to networks, to those beyond your discipline, and it makes your work visible.

Introducing the UK research data discovery service

Christopher Brown, senior co-design manager, Jisc

The context for data discovery here is that there is CKAN, but also an Australian national data discovery service. So, in this project we looked at what else was out there, and we selected CKAN as the solution. The aim was for a UK data discovery service to meet Jisc customer requirements. And this is a project of Jisc, the Digital Curation Centre and the UK Data Archive and their UK data centres.

We have 9 HEIs (including University of Edinburgh) and six data centres engaged in this project. And we have a governance structure and a researcher group advising us to ensure our solution meets researcher needs. Why do this? Well to make data more visible and transparent, to promote the HEI or Data Centre research, to encourage re-use and sharing of data, to validate research, but also to support various mandates on the sharing of data. And this sharing creates the potentiall for greater cross-disciplinary and cross-institution collaboration.

So, we gathered user stories through workshops, for instance around the researcher, the project manager, the machine (as user of M2M services), the data repository, the system manager, the funder. So, this accommodates a range of use cases, for instance discovering data or a lack of data as a researcher, to inform the shape of future research.

In order to make this happen we are aggregating metadata around a core schema that maps individual data repositories to the UK National Repository Discovery Service. We engaged with participants in workshops and online meetings, gathered those stories, selected CKAN software (used for CKAN and ANDS) and created a statement of requirements. And we now have a publicly abailable alpha site: http://ckan.data.alpha.jisc.ac.uk/.

There are a number of issues, including the quality of data, the completeness of the service – as you want a service like this to be as complete as possible. There are issues around open access, licensing, copyright. There are copyright issues around some metadata too. Access to external data may require a log in. And we need to ensure functionality meets requirements.

So, we have an alpha site, and we are using a rapid development cycle towards a Beta. To ensure we don’t leave system testing until the end, and to ensure our system meets user requirements. It’s all open and acceptable. There is an open metadata schema document (http://bit.ly/1QZVMCo) and you can also see the scope of the datasets in a shared document.

The timeline here is that we will be working on the business case and Beta site in Spring, before possibly going into production.

And now, for the demo. And, as I say, this is clearly an alpha site.

Chris: I’m Chris, developer for this project. We decided to use CKAN… The landing page presents a search of datasets, you can also explore by organisation – which shows all of those involved, including universities, research centres, data centres. You can search the data sets by keywords, and filter those by various things – institution, license, tag or format. When we look at a particular example we can see a description, the associated resources, the tags for that data, and then you have the additional information here. In a future version we’ll add four more metadata fields as well.

If we take an example document, you can either preview or download the data. The preview uses a plugin to let you take a quick look at the data, as well as a description of the data. And then, if you want to download the data you can download the file(s).

Christopher B: We don’t store the data, we just aggregate the metadata, the download link will take you to the data in the host data repository – whether at an HEI or data centre.

Chris: Some of the data we have has geospatial coordinates and you can use a map to select a region to explore/filter the search results to a particular area.

On the site, in addition to the pages shown, we also have About and FAQ sections.

Christopher B: That FAQ section is being populated as we go, with questions about the data, the metadata, etc.

Q&A

Q1) How are licenses handled?

A1 – Chris) We are looking to have a standardised list to filter by.

A1 – Christopher B) We are working with existing data and that creates challenges – as there are different versions of the same licenses, and different licenses all in place. So we are trying to encourage standardisation.

Q2) Is the widget that previews files limited to specific files?

A2 – Chris) Yes, limited to PDF, excel and CSV files right now. There are other plugins that could be brought in for other formats potentially. I’m working on a visual search, using thumbnails – the VADS data set will support that. When we preview that there will be images.

A2 – Christopher B) Again, that’s about available metadata – VADS has that, but not all services do. Have to balance what we can do for some data with what can be done across all data sets.

Q3) Could you explain what needs to happen technically for an organisation to take part in this?

A3 – Christopher B) Mostly OAI-PMH.

A3 – Chris) Mainly harvested by CSW – Catalogue Service for the Web. Also supports OAI-PMH – a common standard for harvesting metadata, that supports a range of common metadata schemas including Dublin Core. And we set up periodic parses of data. Large data sets are checked every week, smaller data sets are updated every day. The metadata shown is what we can work with from organisations involved, and working with organisations to standardise.

A3 – Christopher B) There will be requirements for organisations of what is needed to supply data – like the European Data portal – but that guidance isn’t there yet. Coming soon though. There will also be the ability to search for any metadata field from the advanced search feature – coming soon.

Q4) Is it intended to be entirely open access to search?

A4 – Christopher B) Interesting question. Lots of additional functionality that could be solved by institutional login. But yes, right now, it’s open to use and search. But for the data download you will need to login.

Chris: I should say that there are different levels of production of metadata, depending on the institution. Is there anything else that people would like to see. That complex search functionality for instance?

This is available online already so do use it, test it, and give us feedback. And understand that it is in alpha. But it’s all open – CKAN is in GitHub, our schema is open, and as it develops we’ll be publicising the service more.

Q5) Do you have any way to check that the data archives being harvested are actually live an online?

A5 – Chris) That’s a great question. I do check the output from the harvest and any issues are automatically emailed to the admins at the appropriate organisation.

A5 – Christopher B) We are also producing reports, for instance of duplicate titles. And again the reports are being emailed back to the appropriate organisation responsible for that metadata. Also, one last thing… People have asked if you can save search results and you can do that through the URLs for those search results.

A5 – Chris) The reports are in development at the moment. I can show you a preview of these… These include duplicate titles – no good having the same data titles several times – so this flags all data of the same name, and can work with institution to resolve that. And other reports include view reports for the service.

Q6) Is there anything that indicates that data is open access before I download it?

A6 – Christopher B) Yes, you can filter by that, like you would on Flickr. But that will be in a later version as we need that data from the various contributing organisations.

Do keep an eye on the project as it develops and thank you all for coming!

13.30 – 14.30 Plenaries: the power of data

What can data mining the web tell us about our research?

This session is chaired by Dr Paul Feldman, chief executive, Jisc.

Euan Adie, CEO, Altmetric 

I’m going to be talking about the power of data, specifically data that can support the research process. I work for a company called Altmetrics (which around half the room indicates they’ve head of).

Data can help us capture several aspects of research:

  • Quality – is this good research? Is it well researched? Is it reproducable?
  • Engagement – is this research reaching it’s audience, the people it should reach.
  • Impact – what do the audience do with that research once they encounter it.

These three things are different… They are related but not all one thing. You can have a great piece of work that gets no attention, that’s valid, that happens. And you can have a terrible piece of work that gets lots of attention – happens particularly in mainstream media coverage of science.

Today, I won’t talk about quality but will instead focus on engagement and impact. What I will say is that quality is often best assessed by someone who knows.

Impact is hard to define… It’s a bit like obscenity… You know it when you see it! And impact has different meanings for different people. For researchers it can be about reaching peers, for funders it is often about reaching out more widely thoug. As a very broad definition, impact is about showing that your work makes some kind of difference. And that matters in the REF because people want to know what has happened as a result in funding and investing in research. We have a responsibility in the academic and research community to show how our work has impact.

So, how do we know if our work has impact? Well it has impact if it is cited in Science or Nature of PNAS. But citations are just one measure of impact, that doesn’t account for wider practical applications of our work. So if I publish in Nature, I’ll get picked up by all kinds of citation tools. But what if my research is published in a document like the World Health Organisation Guidelines for the treatment of Malaria? It’s research based work, developed by a panel of experts. Why should you get credit for a journal article, but not a policy document?

Maybe you publish in humanities or social scientists… What if your work becomes a book like “Thinking, fast and slow” or “Freakonomics”? If my work is cited, is a chapter etc. why does that not count? What about patents? Shouldn’t I get credit for that impact? So, all of this data can be collected, as can Social Media and discussion of work there… There are traces that act as an indicator that can be presented to individuals for them to make a decision on what matters.

Now, I left social media until last because people often think of Altmetrics as being about tweeting and blogging and so on. It’s a very high volume source, but not always the most important for impact (more so for engagement perhaps). But there is value there too – tweets about a research paper by Greenpeace and Barak Obama shows real impact for that publication.

The data is online, there’s a lot of it to draw upon. But it’s not always immediately useful. The right tools, plus human context, moves us towards something useful and important. And so, we come back to altmetrics and the altmetrics manifesto… This document was written in 2010 by several people, including Jason Preen, Cameron Neylon, and was about these tracks and traces, it talks about recommendations and discovery and understanding that too. And various people are looking at this… Altmetric (that’s me), Impact Story, PLOS, and Plum Analytics.

The Altmetric journal is now on about 6000 journals and will let you explore the usage of the academic work… That brings together newspapers, magazines, tweets, blogs, etc. For example, we can look at the Gravitational Waves article that came out recently.

Now, I should talk about numbers. The score isn’t so important, it is the pooling of the data which you can then look at and explore in more detail. Only you know what impact you are really looking for.

Another solid example was on UAVs and drones, and in Altmetrics they found that their work was being cited in a government document. That’s impact and engagement and useful to know about.

Now, I’ve said this is all good but you have to be cautious about quality. Knowing the Altmetrics is just a starting point… The data needs critical engagement. And these measures are complimentary to the traditional measures – if peer review is the bread and butter, Altmetrics is the jam. And Alt suggests alternative… but it’s complementary. Metrics suggests quantitative as the focus but that’s not right either.

And when we talk about Altmetrics we need to be cautious of not making the same mistakes others have made before… With Bibliometrics we can all things of examples where they have been used poorly, but also good usage too. The Leiden Manifesto for Research Metrics was a Comment piece in Nature that I recommend reading and engaging with, it talks about what you should (and should not) focus on.

You’ve heard me talking, but the best way to understand this stuff is to try it for yourself – look up your work or that of your researchers. Look at http://altmetric.it/. And if you are a librarian you can access a bigger tool, free for libraries, http://www.altmetrics.com/.

The fourth paradigm: Data-intensive Scientific Discovery and Open Science – Tony Hey, chief data scientist, Science and Technology Facilities Council

Nice to be back at a Jisc event, 10 years ago I used to chair a Jisc Committee on research. Jisc have done some great work in supporting research – this conference is more on teaching and learning but they still play an important role in research.

Research is increasingly moving towards data intensive research, and in fact about 10 years back the UK had the eScience initiative which put us ahead for being part of this. Much of science is now Data-Intensive. The Higgs Boson researchers, your Bio Informatics folks are intense data users, with petabytes of data. But we have a long tail of science and many use off the shelf tools with discreet data sets. But in those data intensive space we see not only scale but variety, combining of data from different sources and specialists. We also have data from sensors. And we have volume but also velocity of data accumulating.

Probably one of the easiest ways to start is to talk about the Sloane Digital Sky Survey, which kicked off in 1992 and “finished” in 2008. That project took surveys of more than 1/4 of the night sky, producing 200 GB of data per night. It was two surveys in one – images and spectra. There were nearly 2m astronomical objects captured. But astronomers haven’t the time to look at all of that…. And that data had “absolutely no commercial value” so the data was published openly on the web and was published before the research was really done.

This project led Jim Grey from that project to consider an idea like eScience (but not). And I’d talk about this as the Fourth Paradigm. We started in science with Experimental Science, describing natural phenomena; then we had Theoretical Science – Newton’s Law, Maxwell’s Law etc; Computational Sciene emerged in the last few decades accommodating simulation of complex phenomena. Now we have Data-Intensive Science. That started at the beginning of the 2000s and is about scientists dealing with data from instruments, networks, sensors…

So, for instance… Genomics and personalised medicine. You can use genetic markers (e.g. SNPs) to understand causes of disease. My old team at Microsoft Research looked at SNPs working on Wellcome Trust data for second complex diseases… And analysis with state of the art machine learning enabled some valuable insights into causes and patterns of disease.

Another example… I live in Seattle, which is on the edge of an earthquake zone. There is an electro-optical cable on the sea bed which captures data from a network of sensors etc. The issue is no longer too little data, but so many sources that it is hard to know what to analyse and explore in more detail. And researchers like John Delaney do work on oceans, volcanos, and the ecosystem that exist around that. That data is transforming ocean sciences.

Latestly, and this is where I work, we have CEDA: the Centre for Environmental Data Analysis, which pools data from different sources. And there we have the JASMIN infrastructure, which puts petabyte data storage next to petabyte flop processing.

If you want to know more, this book, The Fourth Paradigm: Data-Intensive Scientific Discovery, is (as far as I know) the only book by Microsoft published under a Creative Commons license.

So, that’s data intensive science. But we are moving to a more open world too…

My colleague Jim Grey works with the National Institute of Health in the US. They mandated that you should deposit the data from your research in the US National Library of Medicine. When that was a recommendation they had about 25% of data, when it became law it went up to nearer 60-70%. Now they withold the next grant on condition of data deposit and that is driving deposit more and more each month. So data is increasingly being openly shared.

Now there are problems… Amgen identified 53 landmark publications in bioinformatics and tried to reproduce them. Only 7 could be replicated with similar findings, the rest could not. That doesn’t mean it’s wrong, just that it can’t be reproduced. So that’s kind of an existential crisis.

Now, in February 2013 there was a US White House Memo on increased public access to the results of federally-funded research. This was a directive requiring that federally funded research had to be publicly available, and that the data also has to be publicly available.

Meanwhile the EPSRC has high expectations for Data Preservation. There the requirement is that research data is securely preserved for at least 10 years after it has been shared. Also the Digital Curation Centre, established 2004, was set up to give guidance and advice, and they are key to some of Jisc’s work in research data management, and will remain important.

There is also an issue with the maintenance of data links. We need sustainable links which endure. Research from early 2000s, according to a specific piece of research, was in 44% of cases already unreachable due to broken links. And we see tools like ORCID helping you find specific work, identified consistently and clearly.

Last year I presented at IDCC and Jim Frew, who was presenting there, talked about progress in data curation in the last 10 years. He said the biggest change is funding agency mandate, there are better curated databases and metadata now – but not sure that the quality fraction is increasing. Frews laws of metadata: First law: scientists don’t write metadta; Second Law: scientsts can be forced to write bad metadata! I would say that physicists are a bit ahead of the game here… But still much to do.

So, I hear a lot about libraries. I personally think they should be central to the university. But as Dean of engineering my students hardly went there (other than to study and have coffee), so what is the function of the library? I’m going to talk about their role a bit… I think they need to be central to provision to teaching and research as much as possible. In research I believe the library should be ensuring they keep a copy of the research output (another Jisc example, OpenDOAR).

So, what are we lacking? Is it true that the UK is world leading? Having spent 10 years in the US… Well when I went to Caltech for my PhD, they barely knew about the US, let alone Oxford where I had previously been. So we have to be cautious of statements of world leading.

So, in the US the NSF Task Force on “Campus Bridging” (2011) have been establishing a vision to ensure there is speed in the network as if the data and the processing were next door to each other. This is Science DMZ architecture is a network design pattern, improving the baseline end-to-end performance through ongoing global adoption. That connects universities across the States, but I haven’t seen that done here. There has been an attempt to implement Science DMZ approach, but there is a need for universities to support that too.

Also in the US there is a Pacific Research Platform which is developing infrastructure to achieve data transfer speeds comparable to being located in the same place.

So, moving into the final part of my talk… What is a data scientist? Well there are at least three skills that make up at that role (not that all data scientists have to have all of these) but we need scientist career paths, for data scientists not just PhDs and PostDocs… The final vision, like the National Library of Medicine and PubMed Central (the version adopted in the UK), is that of Jim Gray who has the vision for All Scientific Data Online, that allows reuse of data, and use across disciplines and increase scientific velocity. I see Jisc doing good work there, but I am concerned about that end to end aspect.

Paul Feldman

Over the last two days we have been looking at the power of digital. I have been in awe of the innovation and passion I’ve seen over the last two days. Digital is disruptive, but education hasn’t seen that disruption in quite the same way. Only 15% of businesses turn to UK universities for support and training in digital skills. There are huge opportunities – are you ready? It’s imperative to understand the opportunities for digital transformation across the organisation in all aspects of the organisation in teaching, learning, research but also operational aspects. With the right leadership we can be great, perhaps the best in the world. Now either we can grab that opportunity or it will be lost. So, how do we win it:

  1. Digital Learning
  2. Data
  3. Digital Content

World class research can’t flourish without the Janet Network – we believe it’s already world class but also investment to get to that full vision (as is the case in the US). We are investing in higher bandwidth, in reimagining the network. But there are opportunities to make the best of data, skills, and to ensure use and reuse across disciplines and boundaries.

Digital Learning is transforming everything we do… Have you heard of Colin Hegarty, a maths teacher at Preston Manor school in London. He is in the running for a million dollar prize for his YouTube videos teaching maths to secondary school kids. The sector needs to develop this kind of process so it is not limited to pockets of best practice.

Our learning spaces have to be fit for purpose… Learning spaces must blur boundaries between learning, working, and leisure. Connectivity and wifi is keen – that’s especially a challenge in the workplace. So, can we make Eduroam ubiquitous outside academia, so it’s there in coffee shops, in apprentice employers.

We are at risk in not investing enough in learning and skills. We are paying special attention to FE, for FELTAG, but also have to focus on HE and Research. But back to Apprenticeships. Digital Learning is key to delivering the ambitious apprenticeships the government wants to establish by 2020. Apprentices can feel quite lonely on placement, so we’ve been working to develop a social network for apprentices to help them feel supported and connected.

Turning to data we need to engage with learning analytics – the UK can’t afford to overlook this. We are setting up – a world first – a UK National Learning Analytics service and framework for HE and FE. Wouldn’t it be great if all organisations in HE and FE took part in this, so that we can understand pathways and their role especially for widening participation.

Finally securing access to the best Digital Content is key. You will have heard from my colleagues this week on negotiations around journals, and working with SCONUL on the UK National Digital Library. We are on a real cusp of something here… The future of Digital is in your hands. Digital changes lives. Technology enables students like Alex (featured in a video yesterday) find it life changing for what it enables him to do and participate in. We have huge opportunity and we have to make use of it.

And with that we finish with another view of the new Jisc video, More Power to You. 

YouTube Preview Image

Responsible metrics for research

Chairing this session is Catherine Grout, head of change – research, Jisc.

Catherine: My role is to deliver innovative and smooth services for research data. Before our speakers I’m just going to give you a brief overview of the importance of metrics. Using metrics in an appropriate and responsible way, underpins the research process, grant applications, resource allocation, research assessment, research impact etc.

Advances in technology enable us to use other types of data – sometimes under the heading of altmetrics. But there are potential pitfalls. The HEFCE Metric tide report: http://bit.ly/hefce_metrictide/ was really important in setting out a future where traditional peer review could sit alongside new metrics. It made recommendations about how metrics which might be deployed more effectively. But there are many challenges and pitfalls along the way…

Stephen Curry, professor of structural biology, Imperial College

I was part of the steering group for that HEFCE Metric tide report, which I’d encourage you all to read if you haven’t already. That report certainly reinforces that metrics are here to stay. Quantitative measures are part of all we do, including research.

We did, in working on the report, questioned even using the word metrics. Metric suggests you know what you are measuring. I personally prefer indicator. But “metric” was the brief given to us by the minister.

If we do need to use metrics they must not stand alone. Metrics are informative but must sit alongside peer review. And institutions need to be transparent about use. That means being transparent and properly engaging staff in metrics. They must clearly state principles for assessment. They must not delegate measures of excellence to league tables or journals. And metrics must be put in place with staff. Data should be transparant – one of the biggest isues of league tables etc. And it must build on DORA and the Leiden Manifesto, about the principles for the responsible use of research metrics.

So, this was a great document that I was proud to be part of this. But words are just words. And we have a deep cultural issue with metrics. Despite the REF not intending to force researchers towards high impact journals, our researchers certainly feel that pressure. There are difficulties to be resolved. And the best way to do this is to clobber people with the evidence to make your case. The impact factor number is something simple to mask something complex. Data on Nature Materials citation performance shows how variable impact really is. 50% of citations came from a very small number of journals – it’s a long tail… And that happens across journals. ALL journals publish papers with high citation counts, ALL journals publish papers with low citation counts. Citation distribution data should be shared for all publications – I’m very pleased to see the Royal Society doing this, others are following, all should.

We also need to understand that the correlation between JIF (Journal Impact Factor) and article citations for individual scientists are poor (Seglen 1997, BMJ, 314, 498-502). That paper is 20 years old but it isn’t known and we have to ensure that is known. We don’t want to reinforce the unintended and bad effects of metrics. This isn’t the fault of those who publish metrics’ fault but we have to reimagine what we do in academia.

We should be working towards high quality research that is highly reproducible, not just novel or shaped for high JIF journals. I’d like to see open publishing and efficient publishing at all stages of the process. And I’d like to see mentoring and citizenship recognised – e.g. contributions to peer review. I take Ron Vale’s cue that

“as stewards of our profession, academic scientists have a collective responsibility to consider how to disseminate knowledge through publications and how to advance (careers)…”

Cameron Neylon, professor at the Centre for Culture and Technology, Curtin University, Western Australia

I guess we should check our metrics for this session in the form of social media sharing… Or maybe even email sent (a negative measure).

Now, I’m going to disagree somewhat with my learned colleague here. We know that metrics are very flawed. The data we have are largely meaningless.. They don’t tell us what we say we need to know about, or indeed what we do know about. As scientists we are supposed to be testing what is known. So two words for you… Excellence and Quality… Try and define them in your head without using either of them… I think you’ll find that difficult.

Everyone wants to be Excellent… But at the root of this is not just numbers that are inaccurate or unhelpful, but it’s also about a system of research and research funders who don’t know what we’re talking about. In Ewan’s talk earlier he referenced Justice Potter’s comment of not knowing what pornography is, but he knows it when he sees it. And Excellence and Quality are assessed in a similar way. Researchers either know Excellence or Quality when they see it, or can point to those who can identify that. That’s ok in a small community. But that’s a problem and loses meaning when we try to communicate that more widely to funders, to policy makers etc. Applying those ideas to all research is just nonsense… But we’ve internalised this very deeply. Stephen said this is a cultural issue…. A very deep cultural issue…

So, again, think… Can you rank two universities? Two researchers? Two papers? Two grants? You probably said “no” at first… But actually for grants it must be true as we do that. Imagine a world where that is not true. Where proposals are purely about allocating funding to all, not competing in these ways. The idea of ranking is so deeply felt but actually when you pick that apart it just isn’t true. We believe it because we think we have limited resources… I’m not even sure that’s true… The assumption that we only fund the best best is political but deeply embedded.

Lets go back to Justice Potter… He said something that was actually a bit different from the well known quote. He says “It is not possible to explicitly define something as obscene, as there are too many things that might be obscene” but that “for a certain set of agreed principles advanced, I can assess that this doesn’t meet any of them”. So he actually said, he can see when he doesn’t see it.

Now in research we are great at knowing what should not be funded, but we are terrible at knowing what should be funded. We have different priorities. We have to think in terms of qualities, not quality… But that is challenging… That’s about changing research funding, the shape of careers… And that’s pretty impossible. But what can we do?

Well we can move – using my new humanities view on the world – from people to peoples, from quality to qualities… Pluralising changes how we think about the sentences we use – try it! It changes the way we think, The stories we tell ourselves, our narratives, the way we think… that changes… So changing the words is the start to change the culture… Supported by evidence, supported by analytics.

And we have to ask ourselves again and again what we are measuring as performance. The idea of being good researchers and good scholars.

Semantometrics – Dr Petr Knoth, research fellow, Knowledge Media Institute (KMi), The Open University

I would like to talk about an idea we had at the Open University about two years ago, which came out of frustration at current research metrics. We wanted to gamify this a bit. We wanted to find a way to do this in a way that wouldn’t change their work. The metrics should adapt for different behaviours in which we work. One way to do this is to use the full text but I’ll get to that in a moment…

At the moment we have many metrics, but the main advantages of these is that they are very explicit and easy to calculate… But there is insufficient evidence that they measure quality at all. They all developed in an axiomatic way… No one queries the way they are calculated and we need to… We need to move to more data driven model.

So, I just want to show you the rejection rates for peer reviewed journals (n=570) with impact factors. This shows no correlation between rejection and impact factor. And one of the challenges of peer review is that in ranking papers the reviewer cannot read absolutely everything – as Cameron said it’s easy to reject the poor quality work but much harder to understand good quality work.

Metrics are also difficult because there is a time delay. The REF takes publications only from the last 5 years. Citation based metrics are almost impossible to use well for, say, the work published in the last year. Altmetrics are useful but they are measuring popularity, for understanding interest, but is that appropriate for decisions on career progression?

The problem with all these metrics is that they measure interactions (in different ways), without qualifying how important the interactions are. They are very limited. So the idea of Semantometrics is to judge the full work – we are asking them to be good communicators rather than good researchers. So, we can take full text and use NLP to detect good research practices followed, we can detect paper types, we can analyse citation contexts – tracking the progression of facts. We can detect the sentiment of citations. We can normalise by the size of the community who will access the research.

So we have developed an approach for Semantometrics… Each publication builds on prior knowledge and by analysing text we can measure the link to the discussions in the past and those going forward. The higher the distance from previous work, the more impactful the publication. This can be measured, and is natural – it doesn’t distort science. You can find out more at our website semantometrics.org.

To finish I want to talk about what I think we need to do with research metrics. We need to take a data driven approach, where we see how we can test impact metrics on data sets. In data retrival there is the concept of the “golden set” or “ground truth” and we need that to take this approach. But there are also human judgements to be made here. And then there are many factors to take into account – financial impact, etc. as well.

On a related note we did work on the WSDM Cup which was work on new research metrics based on the Microsoft Academic Graph (>120m papers). There is no full text there but the judging of that prize was online through Bing search engine, and through human judgement. This type of model for advancing research metrics seems like a good approach.

So, to summarise, full-text is necessary for research evaluation. Semantometrics are a new class of methods. We are studying on method to generate new metrics, but there isn’t only one solution so we need to explore further.

Q&A

Q1 – Neil Jacobs, Jisc) Really interesting to hear the different perspectives today… So how do we have this conversation, do we have shared vocabulary even?

A1 – Cameron) It’s the same class of problem as any interdisciplinary approach. It’s actually worse than you suggested: we have some common vocabulary but we mean different things by it! We need deep questions: why do we use public funds for research? What do we expect to get out of that? And how do we assess our performance against that to enable us to do better in the future? We need to have the fights and create new ways to do this… In my head I hear my inner-researcher say “this is interesting, we need to do more research” and that doesn’t seem right either.

A1 – Stephen) Cameron’s point on quality over quantity is well made. It’s so hard to compare work across disciplines… You can’t choose but actually we have to… Right now our society chooses to spend a lot of money on biomedical research, reflecting society’s values. There is a wider public conversation to be had about what we do fund. That conversation has to include historians, it has to involve the public… Part of that process has to include numerical evaluations, and I think that’s ok as long as it’s only part of the criteria.

A1 – Petr) In terms of shared vocabulary, I particularly think thats about impact, rigour, etc. We don’t know what we value at the moment. We have to understand that researchers – especially early career researchers – don’t always choose their work, they are often assigned to it or taking the available role. And they are rewarded based on that. Maybe we should reward quality for their actual work instead. We have to break impact down into what we actually value, and how we will use them for specific purposes.

A1 – Cameron) Stephen, those of us in this space though that the Metric Tide report was excellent, but presumably you had that conversation on vocabularies…

A1 – Stephen) We did and there was disagreement. In natural sciences there was some sympathy for citation counts, but in the humanities you can be looking at an artistic performance where citation count simply isn’t relevant. In Australia they used metrics for sciences, not for other things… But we saw that two-tier approach that can value one side of research more than another as being quite dangerous. The impact element of the REF has been interesting first time around, and good for the sector… But we could see no consistent pattern that could be pulled out as metrics to use consistently in the future. But it’s good that there are so many potential types of impact that can come out from research.

Q2 – Catherine, Jisc) In that Metrics Tide report the findings are quite wide ranging. How do you feel about the next steps? What are the most important recommendations with highest priority to take forward now?

A2 – Stephen) We have 20 recommendations in total. Some of them are cultural. We did aim to draw a line in the sand about where we are and where we want to be for best practice. Some are straightforward and technical – data formats, encouraging all to sign up to ORCHID etc. But we didn’t see an easy alternative to REF peer review panels. Some of the other recommendations around transparency and conversations within universities, those can start right away. At Imperial these conversations have already begun – triggered by the unfortunate suicide of Professor Graham. We have talked about what should be incentivised, where we wish to see contribution. So we are having those conversations and want to take those forward, and I hope other universities do that too.

A2 – Cameron) I think the report did all it could in terms of recognising and pushing those cultural shifts. But an aspect that is needed is the data structures and infrastructure, how things have been calculated, and ensuring that data is available for access for the full range of scholarly criticism and engagement – to apply the same standards to our metrics as we should do for any other scholarly data set.

A2 – Petr) I think that this shows that impact is not just economical, but has real impact on people’s lives. The other thing is transparency. I have experience of comparing just citations. We did work comparing Google and Microsoft citation data and they were totally different, so the transparency is the critical bit. If people are judged on data that is incomplete that is hugely problematic. And if you do research on data sets, you are often not cited (the data set is instead) and that also needs to be taken into account. The ability to go back and recalculate values is important. So I support the infrastructure issue too.

Q3 – Catherine, Jisc) I wanted to ask about “snowball metrics”, which weren’t recommended in the report. I was wondering what your view of snowball metrics are, and it’s place in this sort of area. If there is scope for that.

A3 – Stephen) I don’t know a lot about snowball metrics. We had a submission from John Green, who heads up that approach for a collaboration of universities and Elsevier. We didn’t want to back any single approach but this one has a degree of openness, so healthy to that degree. But it is a project pushing the line that just adding up the numbers tell you all you need to, to shape the strategy of the university etc. And this is where I really encourage universities to have those sorts of discussion of what this means.

A3 – Cameron) There is some underlying issues here. It comes back to public infrastructures for information too. The good thing is that it is a set of recipes – analytical recipes for normalising citations to the income of the university, etc. I have to be honest that the measures available there aren’t that useful for strategic decisions. I’m sure the snowball people would suggest that I create those recipe. What concerns me is that that is a collaboration between universities and a single data provider, so it relies on a specific data structure (Scopus). It is different than Thomson Reuters. And this is no use of public data yet. We need that data to be publicly owned and available and we need to be able to scrutinise that.

A3 – Petr) I’m not familiar with that project as much, but I would say that when I compare data from Scopus and Mendeley – different data sets from the same supplier – they do not match up or agree. And that is one supplier. So what happens when we compare data from different sources?

And with that, and a closing thank you from Catherine, Digifest is a wrap for 2016. Thanks to all at Jisc for organising and to all the lovely speakers, fellow delegates, etc. that I had the pleasure of seeing/meeting/catching up with this week!

 March 3, 2016  Posted by at 8:17 am Events Attended, LiveBlogs Tagged with: ,  No Responses »
Mar 022016
 
The stage at Jisc Digifest 2016

Today and tomorrow I am in Birmingham for Jisc Digifest 2016 which I’ll be liveblogging here. I’m particularly hear wearing my Jisc’s 50 most influential higher education (HE) professionals using social media hat, helping to share the event with the wider sector who aren’t able to be at the ICC.

There is also an online programme so, if you aren’t here in person, you can not only follow the tweets on #digifest16 and the various blogs, you can also view special online content here.

As usual, this is a liveblog so all corrections, additions, comments, etc. are very welcome. 

Plenaries: the power of digital for change

Dr Paul Feldman, chief executive, Jisc

Good morning and warm welcome from me and the whole Jisc team to Digifest 16. A warm welcome to those, like me, here for the first time. Digifest is all about the power of digital in education. That video of More Power to You is all about a subtext coming from Jisc over the next few months about people and technologies.

Now I’ve been in post only a few months and in that time I’ve been finding out about how you are using digital. And Digifest 16 is all about seeing the power of digital in practice. You, as well as others not able to be here today, and our online audience will do just that. Some of those articles we shared ahead of today have already had 800 views, and we want to carry that conversation on after today.

Before I hand over to our Jisc Chair I also want to thank our sponsors and partners for Digifest: Pervasive, Danny Boyle, ITR, Optix, Saville AV.

And with that, I will hand over to David Maguire, Chair of Jisc, to talk a bit more about Jisc and what we do.

Professor David Maguire, chair, Jisc

Welcome from me, and from all members of the board to Jisc Digifest. I will be talking about the power of digital, and that you have to have vision as well as pragmatism.

So, Jisc, the body for further and higher education which represents all things digital. We represent shared digital infrastructure, services, advice and expertise. We work with around 1000 organisations around the country. We have a national network infrastructure with about 18 million users in any given year. That is well known. It is perhaps less well known that over 50% of all UK library spend on e-resources comes through Jisc. And we save the sector around £203M annually – about twice what Jisc actually spends to do that.

Jisc is of the sector, for the sector. We do three main things for you. We run shared digital infrastructure and services – including the Janey network but also things like learning analytics, research data management. We provide Sector Wide deals with IT vendors and commercial publishers – examples here include Microsoft 365, Amazon Web Services, Prevent web filtering. And we provide expertise and advice.

One of the challenges we face is the huge growth in interest – a six-fold growth in traffic in the use of Janet since 2010. That growth means we also need to invest in the infrastructure, to ensure that we update our infrastructure to keep it suitable to meet those needs.

I also want to talk about University digital challenges.

Right now there is lots going on – a Digital Wild West. We have BYOD (Bring Your Own Device) – easy to desire but hard to deliver well, Wikipedia Scholars – everybody can find out everything now, limited IP respect – and as custodians we have responsibilities to the IPO, to copyright holders, for things that happen on our network.

We see students moving faster than university policies/systems/practices/staff. The answer isn’t to slow students down, but to be ready to do that.

We have to keep up with demand – building industrial strength solutions. We’ve all heard of academics building something in a weekend, but it takes a huge amount of work to take an idea and turn it into a robust and flexible solution. That’s a challenge across MOOCs, VLEs, student systems, Learning Analytics.

Breadth vs Depth are also challenges for us to address. How do we cater to specialists and generalists at the same time?

Now, the education sector is rather a small technology sector for vendors to create appropriate robust solutions for. We need common information systems to reduce the cost of building systems that meet the sectors requirements.

So, Jisc is working on some ideas and solutions to address those challenges. Right now there is a Janet mid-term upgrade to ensure we can continue to meet that rising demand on the network. We are looking at Learning Analytics. We are also working on more technology and content agreements – to reduce the cost of delivering the same services to you. We are looking at Open Access – currently costing more to access content in Open Access journals than in old proprietary systems. We have FE area reviews. Big push on research data management in readiness for the next REF. And Paul and I are keen to ensure Jisc remains at the heart of Technology-enhanced learning.

Paul Feldman: On technology-enhanced learning there is no point doing that unless you think about the context and the spaces that that learning takes place in. Which takes us to our next speaker…

Professor Andrew Harrison, professor of practice at University of Wales Trinity St David and director, Spaces That Work Ltd

I’m delighted to be here to talk about the interaction of space and pedagoguey. I’m a psychologist by training but I’ve been working in education for the last 20 years. The internet has changes notions of place, time and space. What excites me particularly is I see more blurring between learning, and working, and leisure, blending in new ways.

Now, some people are predicting that these changes make physical campuses unsustainable – there is a famous quote from Peter Drucker on this. Now I don’t believe that but I think that traditional categories of space are becoming less meaningful as space becomes less specialised. I could give you a 30 minute talk on the importance of corridors! The meaning and function of spaces are being challenged. We are under more pressure to use spaces more effectively. And we are really bad at utilising spaces. The typical space usage in HE is 25-30% so we need to try and use space more intensively, and to make that space flexible for less specialised use. So we need some specialised learning spaces, but more generic learning spaces and also more informal learning spaces.

So, how can space support learning and teaching? Ideas about learning and teaching are changing, so what sort of space do we need to create to support interaction and active participation? How do we make spaces integrated, multidisciplinary? How do we support distributed learning that can take place anywhere, any time. And how does that physical space relate to our digital spaces? We need to create spaces that support the pedagaguey – thinking spaces, designing spaces, creative spaces, etc.

But, where I get really excited is your world. Where digital is not just equipped by technology but informed by it. Virtual and physical are not opposites – they are part of the same thing. Even when you are in a virtual space you are still situated. And we have to acknowledge that and respect the continuing usefulness of face to face experiences. My own work particularly looks at spaces that support blended learning. Universities regularly have to reshape campuses to support these types of activities – typically spaces are bigger, with better lighting, acoustics, technology. In HE shared learning spaces tend to be boring – they seem to be thought of as a neutral rather than telling stories, rather than being designed and that’s an integral part of the space.

Now this image (three images of classrooms) is here to scared University Estates teams. A traditional didactic learning set up means rows and rows of students – very efficient. More participative spaces maybe enable pair working. But a more active pedagoguey means a room that seats fewer people in clusters. As we move to this type of teaching and learning we may need fewer spaces, but larger spaces. And this shows the importance of estates and teaching staff working together to design learning spaces.

Some of my favourite examples of great spaces are the Melbourne University Learning Lab – a flexible, adaptable space, and the IED Blended learning classroom in Karachi. At the moment I’m working on the University of Wales Trinity Saint David, Swansea. We have an amazing opportunity there to entirely redevelop a quarter of the city. Part of that relates to understanding the UWTSD library offer, and a VLE that enables on campus and off campus experiences that are comparable. And Wales is leading the world in moving to a single VLE so that all universities and libraries will be connected.

When we talk abot the creatoin of learning-centred communities we can see different levels and types of learners as very different, but I’m excited to think about hybrid spaces blending schools and museums together. There are huge opportunities to explore what that mean.

To summarise I think the future learning experience is much more layered. It is about flexible activity zones to support learning, living and working. And it is about users choosing appropriate settings and technology for the tasks they want to achieve (a study by a US chair supplier years back found students sitting on the tables more than the chairs!). Needs for spaces change throughout the day. Those needs also change depending on the learners context, background, mode.

So, successful digital learning spaces are about:Space; Place; Process; Experience – including those moments before and after class, how do we create a student journey that celebrates learning and its place within wider society.

Paul: This takes us to our next speaker Donna Lanclos – it’s her third Digifest and she is here to provoke us, as I’m sure she will!

Professor Donna Lanclos, associate professor for anthropological research,UNC Charlotte

I was really interested in what Andrew said. We today, just like our students, are here benefiting from being face to face as well as the digital being available. That digital possibility doesn’t take away from that shared experience.

We do need to think about digital as a space. Society-wide we have to think about what does it mean to do the things that we do face to face, when things can be done online. What does it mean for teaching and learning when we can take those spaces around in our pocket. Teachers can worry about attention… But that misses the point, it’s about where people are, what they are doing. That concern about attention is the outcome of a flawed system of handing out tools and telling people to use it. Instead we have to understand what these different experiences means.

What does it mean for those without access to these digital places? When really important stuff happens in those spaces, but not everyone can have access to. It’s not the “digital divide”, it’s segregation. We have to deal with that. We have to move the conversation away from tools and practice. We have to talk about place and experience. Lets talk about people… They can be present, they can be online… they can be engaging multimodally. What does the existance of these digital spaces mean for engagement. What is happening in this room is more than you just sitting there. How does what you do on your life make you more here? That’s something we can leverage, and use. We have to not be afraid of that… Asking students to switch off devices is the modern equivelent of “look me in the eye young man” – and that never worked either!

The theme over the next few days is about leveraging the digital for change. And I’d ask you to question why change, for whom are we changing. When I run a workshop we often end up talking about tools – people are comfortable about talking about that. But at some point the conversation moves from the tools, to the people being engaged through those talks. And we don’t signal that, it just happens. It naturally falls out of that chat, and we quickly turn to ideas of presence and community. So, think about who you engage with in digital spaces… And that will vary hugely depending on your experience (as is the case for engagement in any space).

We have to think about, if we want to change the nature of engagement, then we need to think less about what you have to do, and more about with whom you have to speak. If you don’t want to change, then that is valid and you have to make that case. Likewise, if you want to change things, make that case. Ensure you have moments of reflection to think about that. More than that we need to think carefully about the roles of leaders to make space for that kind of reflection and change, and for there to be safety around the risks of change. Change happens because you are willing to take risks and see what comes next. Predicting the future locks us into something, blocks off other possibilities.

We were asked to to say what we thought you could do to get the most from Digifest. So, I recommend the mapping sessions, as I love those. But make sure you engage with human beings. I’d encourage you not to get out there to talk to people who hand you a tool and tell you how to do it, instead engage with those people asking you what you need to do, who you need to connect to. I would like these sorts of events to be about intention and purpose. You can go to any trade show, you, the online participants, this is the core of Digifest. I would frame this event in terms of the human experience that you can engage with, and not digital as a tool.

So, eventually technology will come into the conversation… But not starting with that gives you a much more interesting conversation to have.

Q&A

Q1) About the flexible spaces, and the need to include technology… Actually that’s difficult in terms of challenges around power. Moveable flexible furniture makes power hard to manage.

A1 – Andrew) Power is an issue. Battery life helps, tablets help. We also have trolleys for device charging as well, and floor boxes can help. But not all sessions need technology… You can have technology zones around the edge, and flexible space in the middle.

A1 – Donna) Ideally you’d be supported to develop a pedagoguey that works whether or not the power comes up… I spoke with someone who hit a power issue and noted a colleague “taught anyway, without a powerpoint” as if that was extraordinary. We have to manage the untethered spaces… And reassure colleagues who are nervous of failure. No matter what happens you still teach and your students still learn.

Q2) I’m just having difficult visualising students you are talking about. You seem quite generic about different students you are talking about… Thinking about FE and HE I don’t hear cultural inclusion in any of your talks. We have such a variety of competencies and confidence… Can you give me a dimension of different types of students… You all seem to be talking about young students.

A2 – Andrew) I agree completely. The average age of students here is over 21, in New Zealand (where I’m from) it’s 25. The reality is that we have a much more complex set of students, expectations, skills that the institution needs to embrace.

A2 – Donna) My institution has a very diverse student body. Institutions have responsibility to have intentions around what they want their students to achieve. Of course they come in with a wide variety of preparations and experiences, but that shouldn’t mean we don’t have expectations for them. Funding can of course limit the degree to which we can target our work. We have a responsibility to teach the students that show up – not just providing technology support but also teaching and learning support. We should be less driven by student expectations coming in, and be more driven by our intention and ambitions.

A2 – David) I’m not aware of any technology that asks the user what age they are. Technology is neutral in this. And there are real opportunities for all kinds of students here.

Q2) We are seeing grants for disabled people coming in in September, and it would be really interesting to see how we can do more to assist them.

A2 – Paul) Absolutely and Jisc have a real role in promoting inclusion, including accessibility technologies. You’ll have seen in the video an example of inspiring use of technology to widen participation. One of the things that worries me about social inclusion. The first place I visited in this role was Sheffield University. I was incredibly impressed with the quality of technology kit that students had. One of the things we included in our submission to the Department of Culture, Media and Sport, is the need to ensure that we are socially inclusive and that it is not just well off middle class kids who have access to great kit.

Q3 – Dave Starkey, Portsmouth) You talked about collaborative learning spaces and that they take up more space. We have some flexible spaces at Portsmouth and they don’t seem to take up more space.

A3 – Andrew) Yes, on a student number basis they do take up more space – just to have desks, to have space between groups for group working. That’s less densely packed than rows of chairs with flip desks. The way Universities handle this is to timetable more intensively but having fewer bigger spaces. We are planning on 2.7m per student in teaching studios like this, rather than the typical 1.8m per student in traditional classrooms.

Q3) For us we do see some aoustic blend across groups, but that can be beneficial as they learn from each other…

A3 – Donna) We are seeing a huge demand for these types of rooms – asking to teach at 8.30 in the morning to get into the rooms. Making these active spaces available has huge impact. I mean, in what universe did we think densely packed spaces were a good ideas.

Q4 – Brian Kelly, UK Web Focus) Based on those presentations I’m not sure what this event is selling, the talk of a single infrastructure… 20 years ago we had a focus on national provision…

A4 – Paul) Yes, we have national provision to an extent, but we are here to help you do this stuff.

A4 – David) Yes, we provide infrastructure nationally. And there are some economies of scale. But we are very much about promoting best practice and opportunities. We don’t have an agenda here, other than what you as a community tell us.

A4 – Paul) Generally Jisc is moving away from big services. Janet is really important but generally we are focusing on best practice, on tools and expertise that you couldn’t afford as individual institutions, but which you can take and adapt and embed in your organisation. We want to know the spaces you want, the investments we can make to support you to teach your students, whether in HE or FE. Helping us understand what we can do to help you, for you to employ locally in your organisation, is what we want to understand.

Q5 – David White) To what extent do you think digital can make people better teachers or researchers?

A5 – Donna) It can’t.

A5 – Andrew) I think I agree. It can enable and enhance things.

A5 – Donna) Digital doesn’t do anything. It’s people that do things.

A5 – David) I basically agree but for some people digital can capture the imagination and motivate teachers and learners. It can in that sense make people better teachers. If we think the solution to all known problems is digital, that’s not the point. You still need good pedagogies, good learning objectives, etc.

A5 – Donna) I think technology can’t be seen as the solution, and we shouldn’t think of teaching and learning as a problem. It’s a process.

A5 – Paul) I think I would disagree to an extent. The student experience was so much about capturing information when I was a student. Now there is such availability of information that there is that space for discussion, for participation. You need great inspiring lecturers and teachers. But technology enables even less good lecturers and teachers to do a better for their students.

Q6) You’ve talked about the learning experience. But digital is transforming the research experience. There is such increasing availability of data. Digital is transforming the way we do research and that wasn’t reflected in those talks.

A6 – David) Absolutely. I touched on data access and research data management – where Jisc is hugely active. We are looking at informing the next REF and how we can play a role in that. Some of the things going on in Janet are focused on support for big data, for CERN, shared data centres for High Performance Computing, for the Crick centre, etc.

A6 – Andrew) From a space point of view research spaces are changing just as radically. The interdisciplinary drive is a big part of that too.

A6 – Donna) There are absolutely parallels between teaching and research staff. Again that issue of dealing with people through digital places to do the work they need to do with their research, but those motivations are still the same, even as technologies change.

Q7) The best practice you are advocating goes against the government’s practice to fit students in like sardines, to save money per head.

A7 – Paul) I’m not sure I’d agree that that is the agenda…

A7 – Andrew) All the universities I work with are trying to do more for less. But there is also a rebalancing of use of space… And reimagining or reinventing existing spaces to deal with larger numbers, to improve occupancy. But financially that is challenging too. The fee structures coming in does seem to have really changed the importance of the estate to attract good students and staff. Space is getting more attention at all levels.

A7 – Donna) I hate that particular government agenda. In fact I’m a bit “from the future” in that respect as we’ve had that in the US for longer. I would like to see more support and advocacy from Jisc for the sector for better teaching and research spaces and practices. There is a role for advocacey… So that collectively we don’t agree to do more with less, but to leverage shared agendas to push back on that. Or at least to call governments on their claims that they care about education.

A7 – David) It was ever thus. We have always asked for more. I would say that technology can be beneficial helper here, to reduce costs of delivery, to be more effective in what we do. Operating in the virtual world is more cost effective than a physical space. We can bring in wider audiences, and we can reach more people digitally.

A7 – Paul) My view, having come from the commercial world, is that the government is trying to apply the values of the commercial world on the education sector. But I would ask you to put pressure on your own organisational decision makers as they have a lot more power to make opportunities and to show leadership within that agenda.

Paul: And on that controversial question we are done here. So, go out and use our 30 minute break to engage with people!

Improving digital technology skills in FE: the CPD serviceAdvice and practical assistance  – Sarah Dunne, senior co-design manager, Jisc; Clare Killen, consultant; Peter Chatterton, consultant; Georgia Hemings, co-design support officer, Jisc

After an introduction from Sarah, Claire is kicking off the session with feedback from students at college who are keen to make better and more effective use of technology. Teachers are looking to engage learners, to do that wherever they learn – whether classroom or home. But teachers are always short on time. For some traditional teaching modes are still their focus.

The sector is also facing challenges: FELTAG suggests the sector moves 10% of guided learning hours online – but who will create the content. There has to be motivation and support for staff in moving to  a blended model. We also need to make space for elearning development, providing flexible training. In house access to training and support varies. Lots of content is available but there are challenges about making that work.

Peter: We are keen to hear your views, starting with the question: What are your biggest challenges in developing digital capabilities and opportunities?

Comments from the audience include: an abundance of strategies but not necessarily the time and resources to make that happen. And the challenge when things dont work all for 100% of the time – ensuring confidence and trust aren’t negatively impacted by that. 

Peter: What about content?

Comment: Theres information out there… but you cant just take that and put it up on the VLE. So you have to make it c;ear what can be used, how to make that easy, and what you have to do to use this sort of content. 

Sarah: Im going to talk about what we are planning to do, this is ore aspirational at this stage as this session is part of our planning process.

So, FELTAG is informing our work – it isn’t the sole driver but it is useful, particularly the findings on digital capabilities. Indeed Jisc has been doing work already in this area, underpinned by our framework for capabilities, which breaks this area into six key aspects.

So, to address some of these needs we will have a discover tool which enables you to assess your own digital capabilities, to understand which resources will be of most relevance, where there is scope to develop your skills. And this will helppeople access advice and support.

Second, we will have a Learn area, directing you to resources, with community ratings and reviews. This will be frames around specific digital capabilities and themes.

And we will have Build activities – an online activity builder app – a tool to assist with embedding digital approaches to learning and lesson planning. This will be later in the year, but will let you upload content, choose materials by level, etc.

And we are supporting Meet opportunities so that you can review and rate apps and learning resources, to develop your knowledge base and contribute resources, providing opportunities for collaboration and sharing of experience.

And finally, we are very conscious of the need to Find and Reuse a route directly through to learning objects and instruction on how to repurpose and reuse objects on various platforms – and we are currently working with organisations to identify those resources.

And with that Sarah hands over for questions, and Im switching sessions as the sound levels in Hall 3 are making it hard to hear this session – especially audience comments. 

Showcasing research data tools

I have snuck into the Showcasing Digital Research Tools demo session as there are a number of interesting speakers lined up here. At the moment John Casey is talking about the Clipper project. As I’ve recently blogged a workshop on this project I recommend reading that post for the background. 

John Casey is now doing a live demo – you can explore the demo site at: http://reachwill.co.uk/clipper2.1/clipper-editor/.

The Clipper tool uses APIs from major video platforms such as YouTube. I can search for a video, select it, and make it part of my project. I can choose to select a clip from that video – based on time markers. And I can title and annotate that clip. And because you access the player from the site these videos come from, you can use only videos you have appropriate access rights to. So, for instance, I’ve tried this with Jisc MediaHub and it works as playing a video in Clipper will direct you to login then view the content.

Giving researchers credit for their data – Neil Jeffries, Bodleian Libraries, Oxford University

This is a project aiming to encourage researchers to deposit their data in repositories, but also to get more value out of their data and other shared data. We have several partners in this work, including organisations focused on sharing methodologies rather than the research data itself, and those working with publishers.

The idea is that this tool is a “carrot” for data deposit. There is a “submit data” button in the repository – this means that repositories get more data deposits and better metadata. And the researcher gets an additional publication/citation possibility – and preservation of that data. Publishers working with this syetsm get more data paper submissions, etc. And we know that connecting that deposit to publishing can be a powerful motivator for researchers.

So, to make this happen we have various connectors built (or planned) to tools where data will be coming from. Within the repositories a deposit generates a page, QR code, links to data etc. And we have a “Data Paper Companion” space. When a researcher submits data we connect that to their Orchid ID, their data is viewable and explorable by journal, project, etc. For any data set supporting licenses, declaration of interests, metadata, etc. is shown on the page, along with a summary of data. As a user you can elect to download a sample or the full data. When you find a data paper (e.g. The Elton Archive) you can find the data associated with that, you can also find the information on publications etc.

As the publisher of that data you can also edit the record, add new associated data sets, etc. And, once everything is organised you can choose to submit your data paper to a journal such as F1000 Research. If you choose to do that your data and details are pulled through to their submission system, where you can make edits, add content, etc. but all of your data assets have been brought through for you making this quick and easy.

So, the idea is to encourage greater deposit of data, and the

We have various data sharing and publicatoin platforms…  Mendeley, FigShare, DSpace repositories, etc. on board.

Q&A

Q1) Is that Bodleian project live yet?

A1 – Neil) No, we aren’t scheduled to be done with phase 3 for another 6 months but we should have an update then. The idea that this is a route through to publishers though. We have made our source code available already, though we still have some work to do on connectors – Sword connectors will be build by the appropriate module owners though. And I know that Jisc is looking at a centrally provided service to enable this.

The Jisc project manager in the Pod also notes that there will be a showcase for this work, and you can follow #dataspring for further updates on all the projects.

Having had a chance to chat with the lovely folk at Guidebook (info, etc. on their website if you are curious) I’ve headed to a slightly different session, on open citation. 

Introducing the open citation experiment – Drahomira Herrmannova, doctoral researcher, Knowledge Media Institute (KMI), The Open UniversityVerena Weigert, senior co-design manager, Jisc

Verena: I’m here to introduce Drahomira who has been designing the open citation experiment, to test a new approach that evaluates the full text – the meaning of the citation. The idea is to overcome draw backs of conventional citation metrics, and takes advantage of the availability of full text.

This project was the first large scale analysis of this new type of metrics, based on over 1 million articles. Drahomira will say a bit more about the approach taken, and show a demonstrator website.

Drahomira: Thank you for the introduction. This experiment uses full text methods to understand research metrics – using Semantometrics.

So, what are Semantometrics? They are a new class of metrics for evaluating research. This is different from research metrics and altmetrics, both of which measure engagement. Whilst those counts have been widely used and adopted, despite criticism, but technology and the availability of full text make different metrics possible, that look at the full text rather than just the usage/engagement from outside sources.

So Semantometic contribution measures are based on the idea of measuring the progress of scholarly discussion. The hypothesis states that the added value of publication p can be estimated based on the semantic distance from the publications cited by p to the publications citing p. So this measure traces development and bridging of ideas and concepts (http://semantometrics.org/).

This work with Jisc was a comparative study with analysis carried out to investigate the properties of the contribution measure. The experiment were carries out on a dataset obtained by merging data from the Connecting Repositories (CORE), the Microsoft Academic Graph (MAG) and Mendeley. After merging the datasets there are 1.6 million publications (though 12 million starting data set).

So, I will now show you the demonstrator website – already online and public. We’ll also share our research around the project on the same site. What I’m going to show you is the visualisation made possible through semantometrics. So, we can, for instance, look at network diagrams showing nodes and networks across publications. And in this model the mode important paper is the one which bridges two different communities or areas of thought. We measure the distance of publications cited by a specific publication, and we look for the contribution value to a particular domain, and more broadly. We look at specifics of groups or clusters of publications, and the distribution between them.

So, papers in both sets may be dispersed… and that isn’t necessarily impactful. But a paper with a very narrow range of citations that opens ideas up to a much wider range of papers and communities may be very impactful.

I prepared some examples on some publications, with visualisations that put the paper at the core, then show linkages out to papers… And distance is semantic distance between ideas. Those visualisations show the links between papers, but also indicate the field itself – the diversity of the areas in which the publication sits.

I selected examples which generate interesting graphs… But there is more detail in the report, which is available on the website. Two of these graphs address contribution and citation count. These show a big difference… Very few papers have high citation counts but many papers have high contribution. We were looking at correlations between the datasets… We were interested in looking at readership – using Mendeley data. Citation count and readership have a high correlation – not surprising. On the average values we see that above a certain value of citations, publications receive always above average contribution scores. That confirms what we might imagine to be true – that impactful papers are cited more. But it also reflects that lower citation scores may represent smaller more specialist research communities.

Q&A

Q1) Have you factored in negative citations – citing papers that are being critiqued or argued against?

A1) No, but that is citation sentiment and that is a research area that we know about and are interested in.

Q2) Do you factor in the age of a citation?

A2) No, not at the moment, but again something to consider.

Q3) An observation. I’m glad you’ve done this observation on an open data set, as Thomson Reuters impact scores for REF are hopeless, as they are closed and proprietary. Your work finally opens that up, and that’s great. There is some discussion on the REF, and the cost of running that. And discussion of whether there is a light touch REF – with more metrics and less human time. What impact could you see this work having in a lighter touch REF?

A3) One of our motivations here was to see how metrics could be use. A big advantage here for REF. Whilst there are issues – like negative citations etc. It can be hard to compare publications. But we need to better understand what exactly research metrics capture, whether metrics are stable – whether recently after publication is representative or not. We can develop new metrics that takes account of time. Lots of promise… But we really have to understand what the metrics tells you. On the openness I agree with you. What really helped us was that… Originally we missed the citation network so I have to say Microsoft really helped. But Mendeley is very managed by people, Microsoft is very noisy data.

Q3) At the moment we have to take publishers word for it..

A3) Sure, but we have to be aware of the downsides of public data sets.

Q4) I’m assuming this was a global corpus – how did you account for language as that can be so difficult to do with semantic processing and analysis?

A4) That would be really interesting. My colleague is an expert in this area and we hope to do more work on that.

Q5) What do you see as important next around the stability of the metrics?

A5) We are looking at the stability of the metrics at the moment. But we believe they should be more stable for citations, but contribution we think that that will change more over time. One of the other challenges here is how one handles uncited publications… The advantage of semantics is that the data is there from the moment of publication, so in terms of understanding contribution that’s immediately available. I think this can be used to distinguish key papers, and to understand distance between publications. We can place a value on each citation already.

Verena: We have arranged a workshop in March with domain experts, and a report will be coming out at the end of March. And we’ll tweet some of those links.

Jisc’s investment in digital content for humanities: understanding the impact on research outcomes – Paola Marchionni, head of digital resources for teaching, learning and research, Jisc; Peter Findlay, digital portfolio manager, Jisc; Professor Eric T Meyer, senior research fellow, Oxford Internet Institute; Dr Kathryn Eccles, research fellow, Oxford Internet Institute.

Paola: Welcome to this session. We will hear first from Eric Meyer at OxII at some work on Early English Books Online and House of Commons Parliamentary papers which really shows the impact of those resources. Then we’ll hear from my colleague Peter Findlay on the portfolio of services, and how things are changing going forward, and hwo we are looking for new ways of teaming up, and we’d love your feedback on that.

Eric Meyer: The project I’ll be talking about was funded by Jisc and ProQuest but I just wanted to start with a bit of background on our work. So in 2008 – but updated since – we created a toolset called TIDSR, supported by Jisc, to understand the impact that various digitised resources were having. Jisc had digitised a whole series of collections and wanted to understand the impact they were having. I won’t go into details but this includes quantitative and qualitative methods for assessing impact. This isn’t a downloadable thing, it’s instructions to do these things no matter how much experience you have (or don’t have) in doing this. You can see case studies there and explore those citing and using the toolsets.

So, back to the current study. The thing about measuring impact is that the size of the audience isn’t the only thing you want to look at, a small audience that is the right audience might be big impact. It is not easy to compare the impact of different resources for some of these reasons. In this work we looked at Early English Books Online and House of Commons Parliamentary papers, and we’ve looked at these collections before in different ways. These are quite different collections in lots of ways and we wanted to see what kind of impact they were both having on teaching, learning and research.

So, the first highlight may be obvious but is worth stating: the context of the use of digital resources is changing but these changes are incremental and have a long development cycle prior to the realisation of impact. But one of the interesting things about digital resources for humanities is that they seem quite familiar. So a scanned page is similar to physical page in many ways. It’s not like in the social sciences where an analysis, a network diagram, etc. might be something quite different. It’s also worth noting that EEBO enables studies that wouldn’t be available in the same ways before. So, if you were an undergraduate humanities student you might not be able to access special collections, to private materials at all – possibly not until well into a PhD – but now these digitised copies do give you that access.

We also saw that, whilst the serendipity of the previous space didn’t happen, new types of serendipity occur digitally. And having these resources available allow people to wander beyond their direct focus and explore related areas for new possibilities and directions to look in.

So, we also found that usage of both EEBO and HCPP has been increasing over the past decade. But HCPP has seen a less steep incline – and that is because it seems to have found it’s audience more quickly, whereas EEBO has only found it’s audience gradually. EEBO released full text relatively recently – that will be a factor – but has also been used more creatively, which we will come to later.

While researchers at top universities are most likely to use EEBO and HCPP, less research-intensive HE institutions also benefit from both collections. We knew that usage was high (particularly for EEBO) in research intensive organisations and we wanted to explore if that was just the “rich getting richer” or if those benefits were more widely spread. In fact the resources are used across HE and, to an extent, FE. One of the interesting aspects was that the ordering of (anonymised) institutions varied between the two services, this isn’t just about departments but also because of the type of usage.

One of our case studies is Open University and they are very high HCPP users but are not high EEBO users. And my colleague Katherine spoke to them and we found that HCPP materials were invested into other courses – around information literacy for instance – which made a significant difference to their use. We also saw usage from less expected subject areas of these collections, for instance literary heritage, conservation, preservation etc. courses also using materials in EEBO.

Researchers rely heavily on specific digital collections that they return to regularly, which is resulting in incremental changes in scholarly behaviour. Now Google was important but specific databases and collections was ranked much higher than any other way of finding resources. Users of HCPP and especially EEBO gave us lots of feedback on what the resource could and couldn’t do, and what they liked about them. Lots to learn from how you embed these tools in universities.

Resource use in the humanities is extremely diverse, and this makes providing access to needed resources and tools particularly challenging – we asked researchers to list some of these and there were so many resources there. The thing about EEBO is that it’s something that stakeholders in particular areas that have come to rely upon it. By contrast HCPP is an important secondary resource for many people who use it in a really different way.

The citation evidence that is available shows a growing literature that mentions using EEBO or HCPP, and these publications in turn are reasonably well-cited. Now we looked across lots of publications and citation data here, but these databases take a while to be updated. We see spikes of citations – outliers – but generally there has been an upwards direction of publications etc. But humanities publications have a long gestation period – it can be 8 years for history for example – but the number and growth look pretty good across both resources.

The number and range of disciplines that refer to EEB and HCPP is much more diverse than expected. We have visualisations here that help illustrate the spread. The ideas move beyond core humanities disciplines – for instance HCPP publications in medical and medical history areas for instance.

Researchers are more concerned with the content and functionality of the digital collections than in who provides access. That’s a challenge. The library is invisible for many students and researchers… They say they don’t use the library and then when you highlight subscription collections they aren’t aware these come from the library – they think it’s Google. So, that’s a problem as that isn’t transparent until users lose access, change organisation etc.

The UK is unusual for providing national-level access across institutions through Jisc’s national purchasing. Now we know that the UK punches above its weight in terms of academic impact. This obviously isn’t down just to this set up, but that national purchasing agreement and access to resources does contribute to the UK’s global prominence. And they have potential democratising effects – you may see some institutions, some FE institutions too using these resources less, but still using them. And there is opportunity to encourage greater use of resourcing in teaching.

Shifts to humanities data science and data-driven research are of growing interest to scholars, although there is still plenty of room for growth in this focus on digital humanities, particularly in teaching. For EEBO that usage increase really reflected that opening up of xml texts, the hack events and social media presences around that change which really encouraged use – projects such as Trading Consequences.

Digital collections have become fundamental to modern scholarship – for the summary and full report see: http://microsites.oii.ox.ac.uk/tidsr/case-study/2016-idc.

Please do take a look at the full report, give us your comments and questions. Do read the report and feedback.

Q&A

Q1) We did a digitisation project of Newton’s notebooks and they were being used but the citations are citing the paper copies as if they’ve seen them physically – which they haven’t – rather than digitally, so how do you deal with that.

A1) That is really a big issue. There are scholars on both sides here… Some claim they wouldn’t cite the library they used for a book… And MLA’s advice to cite “online” not URLs isn’t helping any. We did a previous report of splashes and ripples suggested human readable, easy URI’s as mattering. But this idea of there being something dirty about digital is still there… There is less bias maybe but the confusion remains. Some resources give helpful suggested citations with URIs, but not all by any means.

Q2) How do you compare the kind of data mining impacts and the other direct impacts of resources? I was involved with the Trading Consequences project and I know those data mining projects use a lot of data and look quite different, but how does that compare with more direct impact.

A2) Direct and qualitative projects aren’t really comparable. So it’s about individual resources demonstrating what they can do. We did some work on a very niche resource a few years ago, with very low usage, but for teachers this resource on how dancers built a portfolio was invaluable. So it’s being able to assemble a bunch of different kinds of imapcts a resource can have, and demonstrate to funders etc.

Comment) That matters when looking at subscriptions and the value of those.

A2) We have built this toolkit and over the years people almost without exception come back and say how fun it is to use the toolkit, to find out more about their users, to think about how they use these things, to reflect their usage and interest. So this is an opportunity to reflect. The other quote I remember from years ago from a humanties scholar was that “this is the first time I’ve studied anyone who could talk back to me” as she was used to working on dead people, so she found this really exciting.

Comment) The other aspect of EEBO was, when we got the service, it saves time and money. This researcher was absolutely thrilled by it.

A2) The speed and volume of these things was the initial easy sell of these things, then we’ve tried to layer additional things beyond that.

Q3) We are looking at impact of our resources, are you still working on this?

A3) We have done lots of work before, hope to do more. One of the reasons I hired Kathryn back in 2007 was that she was a proper academic historian but she was new to this online world and her impact has been absolutely vital to this sort of work.

Q4) How about aggregated search points… Sometimes when staff and students search for resources they often get multiple materials, they find other ways in… How do you tae account of this.

A4) This is the stuff we tend to get from interviews. In a previous study we found that people were often arriving relatively deep in their website – coming through discovery tools – so we did work with them to help ensure users understood what to do next, to make their site more sticky by changing the page set up so you were signposted to the next issue, the context, the stuff to carry on exploring. We often think of people arriving at a website front door, but often they find a much less expected way in.

Q5) I work for a publisher like ProQuest and today someone asked me about the Return On Investment of our services… Is that something you have looked at?

A5) We’ve tended to shy away from that as you’d have to make so many assumptions. Maybe if we had an economist on board… We have looked at some to see how income related to impact but that’s the nearest to that idea.

Paola: The nearest thing that we have seen is to try to represent how much it would cost to visit physical resources, travel etc.. But of course if that was the requirement they might not access them at all.

A5) We also have services where two resources from across the world are compared side by side – that’s not something you can do any other way.

Q6) I wanted to ask a question about creative digital reads, by artistic rather than academic communities – particularly thinking of CC licensed and public domain resources. I work with the BL finding out how people use public domain collections in innovative ways. People sometimes thing that having the freedom to do things makes them concerned they might be doing something wrong… But we are really interested in creative use.

A6) You could compare images to see usage.

Q6) Peter Balnam(?) has been doing something like that.

A6) We do ask people in our surveys whether they have reused or repurposed resources… But there is lots of scope for that though – hence that EEBO hack event etc.

Q6) At British Libary Labs we expected lots of academic engagement and we have unexpectedly had a big response from artists and creative professionals.

A6) And i think that’s a thing you can think about… Hack events, Wikipedia editathons, etc. can also show real impact on the greater world.

Peter Findlay: Showing the impact of digitisation Jisc has funded over the years has always been a big challenge… When we had proposals in for this sort of work we did’t know what would happen… So this is all really exciting. We are now in a position where we can see this kind of impact but with the current changing public sector funding, the ability to do this has become a real challenge. The overarching question for us is about the future business models for digital resources.

The focus of institutions is also shifting. Even when value is demonstrated it can be hard to get that message across to decision makers in those institutions. And I’d like to explore with you how important it is to have access to these kinds of collections. These resources are part of people’s work every day… To make this happen again we have to work more closely together, in terms of what we do and in terms of how we fund it.

We’ve also been thinking about what kind of models we might contemplate. We’ve been thinking of a sort of Kick Starter for digital content – with Jisc as negotiator for collections. So, less about digitisation, more about collectively purchasing and finding mechanisms to select and identify content together so that they can be purchased. Not just a purchasing consortium, we are also interested in tools for analysis of content. So Jisc Historic Text is a platform for instance where we’d like to put more content.

A slight adjustment for that would be Jisc seeking core funding to kick that off. We could go to charities, foundations, etc. Essentially we are talking about us together purchasing content or, if you have it, distributing content. We have also been thinking of Jisc as publishers – for institutions together as a collective to enable reduction of costs, a bit like the open platform for the humanities ideas. AGain, this would focus on platform, development, and ongoing support through, say, some form of subscription (for want of a better word). We’d also need to think about cost recovery for any platfrom that is set up, so that it can be sustained

Our third model is Jisc becoming more a supporting institution for the development of tools around the analysis of content, lab activities, mechanisms for small initiatives that can be followed up afterwards.

We’ve been having some great discussions, I’m just nothing the feedback to the rest of the room. 

Group 1: If digital collections were not available, nothing comparable would be available – they enable courses and work not otherwise available. For the BL where impact is hard to demonstrate in general, can be easier for some specific resources though. Impact of individual services is possible, and works – as per Eric and Katherine’s work. Humanities researchers often aren’t aware that resources cost money, they don’t think about costs often. Archives do get switched off and disappear. Legacy resources sometimes get ported to institutions who when they can no longer resources – opportunity there. There are resources available, and they can be marketed to students, but they aren’t always what is wanted. Cambridge commented that the impact stimulates funding. Preservation can be a motivation for sustainability – so others preserving content takes burden off institution. Crowd funding good but may mean small voices and needs may get crowded out. Concern from institutions about content on others’ platforms. Idea that institutions could support platforms… They digitise then share centrally would be one model – making things more effective for institutions, easier to promote, and brings platforms and content together, enabling publishers to put content on platforms too.

Group 2: We thought about current models… For my institution – we just had one or two of us from libraries. In a library, for us, buying a one-off is better than an ongoing subscription in hard economic times. That way you can keep it, rather than adding to yearly subscription burden. Pitching at the end of the financial year might be best, as that is when budget may be available. Over 90% of budgets year on year is committed to journals, databases, ebooks, we have very limited funds for new stuff. And we are keen for more materials to be online, for teaching and learning as well as research. We were quite keen on Kickstarter model… Mixed opinions on whether you need finance directors on board to make that work – although library directors have some autonomy. So, if you had a Kickstarter type website were libraries could show interest in new resources, but also offer a way to suggest ideas, capture gaps in provision etc. Also thought about ad hoc models… Pay per view being one option. Also talked about car leasing – lease then option to buy… Trying to borrow ideas from different sectors.

Group 3: Not a huge amount of library experience on our table either. Talked a bit about how we use wishlists (collections development request list) for new things to buy. So many new things appear and we always need to prioritise wishes. Jisc Colletions is crucial to a lot of what we do – the NESTE2 agreement for purchasing for example. We are also part of other consortiums for purchasing as well. We thought one way to think about material for digitisation might be to share wish lists, in an anonymised way to help deal with competitive drivers that can make collaboration more tricky. Also larger scale digitisation projects as a possibility here. Going back to wish lists those could also come out of a collective gap analysis, rather than looking at products already on the market. And demand is key to securing funding for any kind of digisation project, and we need to think of sustainable business models, and the ability for institutions to articulate what is important to us.

Peter: That was very interesting. Thank you very much for those insights, and we will continue to have those conversations with you. Thanks to all of our speakers and to ProQuest for co-funding this work.

 

The case for learning analytics – Phil Richards, chief innovation officer, Jisc; Michael Webb, director of technology and analytics, Jisc; Niall Sclater, learning analytics consultant

Phil: I’m chief innovation officer for Jisc and we are here to talk about the case for Learning Analytics… To start with I want to talk about what we mean by learning analytics. Google and Facebook track our actions, our interactions, etc,, using that to attract advertisers etc. as they are hugely revealing. Learning analytics is a bit like that, it’s about patterns and understanding data from our interactions with VLEs, library accesses, etc.

Michael: We really are talking about using the kind of big data that Phil was describing. We are looking, in our project, at areas such as improving retention but we also want to move towards adaptive learning.

Predictive learning analytics are statistical analysis of historical and current data derived from the learning process to create models that allow for predictions that can be used to improve learning outcomes. Models are developed by mining large datasets and seeking patterns of behaviour. That’s quite different from the “have they logged into the VLE in a week” type approach.

So, just a moment on retention. We have a huge number of students dropping out at various stages of their education and that recruitment and loss of students is expensive and problematic. 70% of students reporting a parent with HE qualifications achieved an upper degree against 64% if students reporting no parent with HE qualifications for instance. But we can address some of those issues.

I wanted to talk about some US examples that have inspired us. Marist College in the US, funded by the Bill & Melinda Gates Foundation, undertook work supported by Bill and Melinda Gates Foundation. They investigated how use of academic early alert systems impact on final course grades and content mastery. The outcome was interesting… Analysis showed positive impact on final course grades. But they also found that any intervention that alerted students they were at risk was effective.

The New York Institute of Technology also found a successful way to predict students at risk. And there has been the building of a Predictive Analysis model built for US universities which seems to have a high level of identification of at risk students. At Purdue University the signals project, based on performance, background, etc. was effective but that work has been critiqued. University of Maryland found that students being able to compare VLE activity with that of other students were 1.92 times more likely to be awarded grade C or higher compared to students who did not use it.

So, there is clear evidence of the usefulness of predictive models and their potential benefits for student outcome. And Jisc have a learning analytics project which has three core strands: Learning analytics architecture and service; Toolkit; Community – we have a mailing list and blog you can follow. The toolkit includes two main activities: the code of practice and the discovery phase. The Code of Practice was one of our most successful early pieces of work for this project. Failed learning analytics projects tended not to be down to technical issues but for ethical issues. And that code of practice has had great reception, including from the NUS.

We have done some research, which was in the Times Higher, that shows that students are happy to share data for learning analytics. We have a good reason for doing this, a clear code of practice and that means students will buy in.

So, what are we doing? Well we are building a national architecture, defining standards and models and implementing a core service. Why do that? Standards mean that models, visualisations and so on can be shared; lower cost per institutions through shared infrastructure; and this lowers the barrier to innovation, as there is consistency.

Our architecture is built around dashboards; but also alert and intervention system – we’ve defined an API that ensures interventions are captured and feed into the data store; we have student consent app – for how their data is used, preferences etc; and a student app. Then, at the centre we have a learning records warehouse – cloud based system built on open standards – and the learning analytics processor that sits on top of that. The kinds of data we are collecting includes self-declared student data; student information system; VLE; Library; and other things that may be useful (“???”).

To do this work we are partnering with the commercial sector, and our partners are Unicon (on open source stuff), Marist, Blackboard, Tribal, TherapyBox, HT2 (cloud solution provider). And that partnership has led to some really useful things already, including Blackboard making data more available.

So, the service includes dashboards – visual tools to allow lecturers, module leaders, senior staff and support staff to view. This includes student engagement, cohort comparisons, etc. Based on other commercial tools from Tribal and Marist. The student app is bespoke development by Therapy Box, and this interface is very much based around fitness apps. The first version will include overall engagement, comparisons to others including other high achieving students, self-declared data – including student-defined goals, consent management. We are inspired by gaming too – you get trophies for cool stuff!

The Service alert and intervention system, based on open source tools from Unicon/Marist (Student Success Plan) allows management of interactions around alerts.

The data collection falls into two types… Relatively static student record data, and the big ever changing activity data. We’ve taken slightly different approaches to those two data sets. So we have information on the student (ETL) based on HESA and FLR(?) in FE space and consistent with HEDIIP, and you can see our work on there on GitHub. For the activity data we are collecting via TinCan (xAPI) which lets you get quite detailed data. We’ve commissioned a Blackboard module, have supported a Moodle plugin etc.

Now the idea of an xAPI “recipe” is a shared way of describing activities. So the data from accessing a course is the same whether Moodle or Blackboard is used. So, same holds true for “borrows a book” etc.

We have had 72 expressions of interest from the sector. We have 26 organisations, across a diversity of organisation types are engaged in the activity. We have over 1 million records collected in real-time. We needed historic data for this project so we’ve also working on historical data collation from Moodle and Blackboard to enable those predictive models that require data to work on.

Across different stakeholders there are different priorities. For Russell group universities it may be about widening participation and support for students achieving 2.1 or better. For a teaching lead organisation it may be about focusing on interventions in teaching and learning, to improve retention.

Phil: Every year universities have to make around 7000 different measures reporting to HEDIIP. And this project can help aggregate that data, and to give back analytics to the individual institutions based on the architecture we have come up with. And this is the first project to create something like this which provides access to all the information needed for a HEDIIP return. One of the concerns about HEDIIP future reporting is that it may become more frequent… Currently that’s annual. If automated these returns could be quarterly or more regularly. Now learning analytics is a great reason to upload data more regularly for HESA and other agencies, and to benefit from learning analytics as part of that.

The way we’ve set this project up is very similar to the way UCAS has used Amazon Web Services. Until a few years back their website spiked dramatically on A-Level results day and the cloud scaling makes that possible without issues on their server.

Part of making this work is about keeping data private and carefully managed. They want to benchmark and compare. The way we have addressed this is by having users declare that they are happy to share their data if aggregated and anonymised into pools of, say, 10. But we need to have data in place to do that. We need to build up number of contributors.

Now you can look at this for interventions for individual students, or explore by cohort or background etc. Maybe there is potential for new metrics like this to feed into the new proposed TEF.

Some interesting potential in the medium term. Just to talk more about unified data definitions… Our basic standard for that more general data is the HESA model. And we’ve done some work with HESA on the national HE business intelligence service – a fully live production service that has been available from Autumn 2015.

The government is scrutinising subscription organisations like Jisc, like HESA, ever more so, and there are some real opportunities here. We took part in a HEFCE learning gain call in May 2015, which was around standardised tests, etc. and we have work to do there at the moment.

A quick move to genomics…

In Iceland everyone knows their ancestry and the Iceland government has gathered all the genomic data into deCODE and Iceland’s genetic data bank. This system uses reference data, undertakes analytics number crunching and outcomes include understanding the pathways and outcomes.

So, just to borrow that model… Maybe our learning analytics warehouse can be our DNA bank for higher e-learning. The background data would include demographics, early learning and employment outcomes. The analytics and number crunching, towards deeper understanding of elearning, metrics for engagement learning gain, personalised next generation e-learning.

In a recent report with pro Vice Chancellors said that HE was getting more global, more digital, more competitive. But none claimed the UK was taking a lead here. In universities we still use tools we have been using for decades, but the rest of social sciences have moved leaps and bounds ahead… Why not do this with our data?

So, Micheal talked earlier about personalised learning. So, right now we do capture data on how we learn, how your brain works, etc. And maybe sharing that earlier enables a truly personalised next generation elearning that helps you determine the pathways you need to take – for instance a student with low social capital wanting to study architecture might see what the different paths might be… That could really improve social mobility and close some gaps.

In the short term we’ve seen that interventions for not dropping out… seem to really help at risk students who are particularly likely to be widening participation students, which could really help bridge some of those gaps. Maybe this is the kind of work that can put the UK out there as leaders in this field.

I hope that’s given you a good case for why we are doing this work now. Where it might lead in 2 years, and where it might lead in 5 years.

Q&A

Q1) Why has Jisc decided to do learning analytics from ground up, rather than work with an existing provider. And I was disappointed not to see UK examples in that mix – we have good examples, some better than US examples shown there.

A1 – Micheal) We aren’t building from ground up, we are combining existing tools and components. We are putting architecture together to make things work better.

A1 – Phil) We have put together an open architecture, but we have worked with providers… Those were selected through a public sector procurement process (as we are duty bound to do, at least until the referendum) and these are the companies that came out. And some companies have approached us wanting to take part, and we will open up the project to more commercial companies later this year. We want to help you avoid vendor lock in but to work with as many providers as possible. Why are we doing that? It’s what over 1000 people we spoke to in the scoping process ranked most highly.

A1 – Michael) Why US examples – just wanted to use some different examples and I’ve shown the UK ones before – you’ll find them on the website.

Q2) I work at a learning analytics start ups, really great to hear what Jisc are doing, and great to hear about that focus on widening participation. I’m really interested in what the big barriers are: is it cultural, ethical, technical?

A2 – Micheal) It’s a mix of all those things. Some can be solved relatively easily – getting data in and out. Student records systems still tricky but will get easier. Senior staff buy in really matters, a key part of our discovery phase was getting buy in and understanding their system. The pattern is that there is no pattern…

Q3) A follow up question… You spoke about Russell Group universities and the possibility of a positive effect on widening participation, can you say more about that?

A3) We ran a scoping process and one of the use cases presented by this type of organisation was specifically about widening participation and also narrowing that gap between those achieving 2.2 versus 2.1.

Q4) You mentioned models elsewhere being mappable to here… library data and VLE data. What about other types of explicit engagement like citations etc.

A4 – Micheal) Yes, want to do that. But actually assessment data is an important early start there.

A4 – Phil) Some commercial companies aren’t interested in shared or common metrics but we saw evidence in the States that it can work, and enable benchmarking. We think that has real value and that that doesn’t preclude commercial vendors from also providing more granular and bespoke solutions.

And, with that, day one at Jisc is done. I’ll be tweeting to #digifest16 for the remainder of the evening for the dinner etc. I will be back on the blog again tomorrow.

 

Feb 262016
 

Today I am at the British Library (BL) Labs Roadshow 2016 event in Edinburgh. I’m liveblogging so, as usual, all comments, corrections and additions are very much welcomed.

Introduction – Dr Beatrice Alex, Research Fellow at the School of Informatics, University of Edinburgh

I am delighted to welcome the team from the British Library Labs today, this is one of their roadshows. And today we have a liveblogger (thats me) and we are encouraging you to tweet to the hashtag #bldigital.

Doing digital research at the British Library – Nora McGregor, Digital Curator at the British Library

Nora is starting with a brief video on the British Library – to a wonderful soundtrack, made from the collections by DJ Yoda. If you read 5 items a day it would take you 80,000 years to get through the collections. One of the oldest things we have in the collection are oracle bones – 3000 years old. Some of the newest items are the UK Web Archive – contemporaneous websites.

Today we are here to talk about the digital Research Team. We support the curation and use of the BL’s Digital collections. And Ben and Mahendra, talking today, are part of our Carnegie Funded digital research labs.

We help researchers by working with those operating at the intersection of academic research, cultural heritage and technology to support new ways of exploring adn accessing the BL collections. This is through getting content into digital forms, supporting skills development, including the skills of BL staff.

In terms of getting digital content online we curate collections to be digitised and catalogued. Within digitisation projects we now have a digital curation role dedicated to that project, who can support scholars to get the most out of these projects. For instance we have a Hebrew Manuscripts digitisation project – with over 3000 manuscripts spanning 1000 years digitised. That collection includes rare scrolls and our curator for this project, Adi, has also done things like creating 3D models of artefacts like those scrolls. So these curators really ensure scholars get the most from digitised materials.

You can find this and all of our digitisation projects on our website: http://bl.uk/subjects/digital-scholarship where you can find out about all of our curators and get in touch with them.

We are also supporting different departments to get paper based catalogues into digital form. So we had a project called Collect e-Card. You won’t find this on our website but our cards, which include some in, for instance, Chinese scripts or urdu, are being crowd sourced so that we can make materials more accessible. Do take a look: http://libcrowds.com/project/urducardcatalogue_d1.

One of the things we initially set up for our staff as a two year programme was a Digital Research Support and Guidance programme. That kicked off in 2012 and we’ve created 19 bespoke one-day courses for staff covering the basics of Digital Scholarship which is delivered on a rolling basis. So far we have delivered 88 courses to nearly 400 staff members. Those courses mean that staff understand the implications of requests for images at specific qualities, to understand text mining requests and questions, etc.

These courses are intended to build capacity. The materials from these courses are also available online for scholars. And we are also here to help if you want to email a question we will be happy to point you in the right direction.

So, in terms of the value of these courses… A curator came to a course on cleaning up data and she went on to get a grant of over £70k for Big Data History of Music – a project with Royal Holloway to undertake analysis as a proof of concept around patters in the history of music – trends in printing for instance.

We also have events, competitions and awards. One of these is “Off the Map”, a very cool endeavour, now in its fourth year. I’m going to show you a video on The Wondering Lands of Alice, our most recent winner. We digitise materials for this competition, teams compete to build video games and actually this one is actually in our current Alice in Wonderland exhibition. This uses digitised content from our collection and you can see the calibre of these is very high.

There is a new competition open now. The new one is for any kind of digital media based on our digital collections. So do take a look of this.

So, if you want to get in touch with us you can find us at http://bl.uk/digital or tweet #bldigital.

British Library Labs – Mahendra Mahey, Project Manager of British Library Labs.

You can find my slides online (link to follow).

I manage a project called British Library Labs, based in the Digital Research team, who we work closely with. What we are trying to do is to get researchers, artists, entrepreneurs, educators, and anyone really to experiment with our digital collections. We are especially interested in people finding new things from our collections, especially things that would be very difficult to do with our physical collections.

What I thought I’d do, and the space the project occupies, is to show you some work from a researcher called Adam Crymble, Kings College London (a video called Big Data + Old History). Adam entered a competition to explain his research in visual/comic book format (we are now watching the video which talks about using digital texts for distant reading and computational approaches to selecting relevant material; to quantify the importance of key factors).

Other kinds of examples of the kinds of methods we hope researchers will use with our data span text mining, georeferencing, as well are creative reuses.

Just to give you a sense of our scale… The British Library says we are the world’s largest library by number of items. 180 million (or so) items, with only about 1-2% digitised. Now new acquisitions do increasingly come in digital form, including the UK Web Archive, but it is still a small proportion of the whole.

What we are hoping to do with our digital scholarship site is to launch data.bl.uk (soon) where you can directly access data. But as I did last year I have also brought a network drive so you can access some of our data today. We have some challenges around sharing data, we sometimes literally have to shift hard drives… But soon there will be a platform for downloading some of this.

So, imagine 20 years from now… I saw a presentation on technology and how we use “digital”… Well we wont use “digital” in front of scholarship or humanities, it will just be part of the mainstream methodologies.

But back to the present… The reason I am here is to engage people like you, to encourage you to use our stuff, our content. One way to do this is through our BL Labs Competition, the deadline for which is 11th April 2016. And, to get you thinking, the best idea pitched to me during the coffee break gets a goodie bag – you have 30 seconds in that break!

Once ideas are (formally) submitted to the BL there will be 2 finalists announced in late May 2016. They then get a residency with some financial (up to £3600) and technical and curational support from June to October 2016. And a winner is then announced later in the year.

We also have the BL Labs Awards. This is for work already done with our content in interesting and innovative ways. You can submit projects – previous and new – by 5th September 2016. We have four categories: Artistic; Commercial; Research; and Learning/Teaching. Those categories reflect the increasingly diverse range of those engaging with our content. Winners are announced at a symposium on 7th November 2016 when prizes are given out!

So today is all about projects and ideas. Today is really the start of the conversation. What we have learned so far is that the kinds of ideas that people have will change quite radically once you try and access, examine and use the data. You can really tell the difference between someone who has tried to use the data and someone who has not when you look at their ideas/competition entries. So, do look at our data, do talk to us about your ideas. Aside from those competitions and awards we also collaborate in projects so we want to listen to you, to work with you on ideas, to help you with your work (capacity permitting – we are a small team).

Why are we doing this? We want to understand who wants to use our material, and more importantly why. We will try and give some examples to inspire you, to give you an idea of what we are doing. You will see some information on your seat (sorry blog followers, I only have the paper copy to hand) with more examples. We really want to learn how to support digital experiments better, what we can do, how we can enable your work. I would say the number one lesson we have learned – not new but important – is that it’s ok to make mistakes and to learn from these (cue a Jimmy Wales Fail Faster video).

So, I’m going to talk about the competition. One of our two finalists last year was Adam Crymble – the same one whose PhD project was highlighted earlier – and he’s now a lecturer in Digital History. He wanted to crowdsource tagging of historical images through Crowdsource Arcade – harnessing the appeal of 80s video games to improve the metadata and usefulness fo historical images. So we needed to find an arcade machine, and then set up games on it – like Tag Attack – created by collaborators across the world. Tag Attack used a fox character trotting out images which you had to tag to one of four categories before he left the screen.

I also want to talk about our Awards last year. Our Artistic award winner last year was Mario Klingeman – Quasimondo. He found images of 44 men who Look 44 with Flickr images – a bit of code he wrote for his birthday! He found Tragic Looking Women etc. All of these done computationally.

In Commercial our entrant used images to cross stitch ties that she sold on Etsy

The winner last year, from the Research category was Spatial Humanities in Lancaster looking for disease patterns and mapping those.

And we had a Special Jury prize was for James Heald who did tremendous work with Flickr images from the BL, making them more available on Wikimedia, particularly map data.

Finally, loads of other projects I could show… One of my favourites is a former Pixar animator who developed some software to animate some of our images (The British Library Art Project).

So, some lessons we have learned is that there is huge appetite to use BL digital content and data (see Flickr Commons stats later). And we are a route to finding that content – someone called us a “human API for the BL content”!

We want to make sure you get the most from our collections, we want to help your projects… So get in touch.

And now I just want to introduce Katrina Navickas who will talk about her project.

Political Meetings Mapper – Katrina Navickas

I am part of the Digital History Research Centre at the University of Hertfordshire. My focus right now is on Chartism, the big movement in the 19th Century campaigning for the vote. I am especially interested in the meetings they held, where and when they met and gathered.

The Chartists held big public meetings, but also weekly local meetings advertised in the press and local press. The BL holds huge amounts of those newspapers. So my challenge was to find out more about those meetings – how many there were advertised in the Northern Star newspaper from 1838 to 1850. The data is well structured for this… Now that may seem like a simple computational challenge but I come from a traditional research background, used to doing things by hand. I wanted to do this more automatically, at a much larger scale than previously possible. My mission was to find out how many meetings there were, where they were held, and how we could find those meetings automatically in the newspapers. We also wanted to make connections between papers, georeferenced historical maps, and also any that appear in playbills as some meetings were in theatres (though most were in pubs).

But this wasn’t that simple to do… Just finding the right files is tricky. The XML is some years old so is quite poor really. The OCR was quite inaccurate, hard to search. And we needed to find maps from the right period.

So, the first stage was to redo the OCR of the original image files. Initially we thought we’d need to do what Bob Nicholson did with Historic Jokes, which was getting volunteers to re-do them. But actually newer OCR software (Abbyy Finereader 12) did a much better job and we just needed a volunteer student to check the text – mainly about punctuation not spelling. Then we needed to geo-code places using a gazeteer. And then we needed to use a Python code with regular expressions to extract dates and using some basic NLP to calculate the dates of words like “tomorrow” – easier as the paper always came out on a Saturday.

So, in terms of building a historical gazeteer. We extracted place names run through: http://sandbox.idre.ucla.edu/tools/geocoder. Ran through with parameters of Lat and Long to check locations. But we still needed to do some geocoding by hand. The areas we were looking at has changed a lot through slum clearances. We needed to therefore geolocate some of the historical places, using detailed 1840s georeferenced maps of Manchester, and geocoding those.

In the end, in the scale of this project, we looked at only 1841-1844. From that we extracted 5519 meetings (and counting) – and identifying text and dates. And that coverage spanned 462 towns and villages (and counting). In that data we found 200+ lecture tours – Chartist lecturers were paid to go on tours.

So, you can find all of our work so far here: http://politicalmeetingsmapper.co.uk. The website is still a bit rough and ready, and we’d love feedback. It’s built on the Umeeka (?) platform – designed for showing collections – which also means we have some limitations but it does what we wanted to.

Our historical maps are with thanks to the NLS whose brilliant historical mapping tiles – albeit from a slightly later map – were easier to use than the BL georeferenced map when it came to plot our data.

Interestingly, although this was a Manchester paper, we were able to see meeting locations in London – which let us compare to Charles Booth’s poverty maps. Also to do some heatmapping of that data. Basically we are experimenting with this data… Some of this stuff is totally new to me, including trialling a Machine Learning approach to understand the texts of a meeting advertisement – using an IPython Notebook to make a classifer to try to identify meeting texts.

So, what next? Well we want to refine our NLP parsing for more dates and other data. And I also want to connect “forthcoming meetings” to reports from the same meeting in the next issue of the paper. Also we need to do more machine learning to identify columns and types of texts in the unreconstructed XML of the newspapers in the BL Digital Collections.

Now that’s one side of our work, but we also did some creative engagement around this too. We got dressed up in Victorian costume, building on our London data analysis and did a walking tour of meetings ending in recreating a Chartist meeting in a London Pub.

Q&A

Q1) I’m looking at Data mining for my own research. I was wondering how much coding you knew before this project – and after?

A1) My training had only been in GIS, and I’d done a little introduction to coding but I basically spent the summer learning how to do this Python coding. Having a clear project gave me the focus and opportunity to do that. I still don’t consider myself a Digital Historian I guess but I’d getting there. So, no matter whether you have any coding skills already don’t be scared, do enter the competition – you get help, support, and pointed in the right direction to learn the skills you need to.

Farces and Failures: an overview projects that have used British Library’s Digital Content and data – Ben O’Steen, Technical Lead of British Library Labs.

My title isn’t because our work is farce and failure… It’s intentionally to reference the idea that it can be really important early in the process to ensure we have a shared understanding of terminology as that can cause all manner of confusion. The names and labels we choose shape the questions that people will ask and the assumptions we make. For instance “Labs” might make you imagine test tubes… or puppies… In fact we are based in the BL building in St Pancras, in offices, with curators.

Our main purpose is to make the collections available to you, to help you find the paths to walk through, where to go, what you can find, where to look. We work with researchers on their specific problems, and although that work is specific we are also trying to assess how widely this problem is felt. Much of our work is to feed back to the library what researchers really want and need to do their work.

There is also this notion that people tell us things that they think we need to hear in order to help them. As if you need secret passwords to access the content, people can see us as gatekeepers. But that isn’t how BL Labs work. We are trying to develop things that avoid the expected model of scholarship – of coming in, getting one thing, and leaving. That’s not what we see. We see scholars looking at 10,000 things to work with. People ask us “Give me all of collection X” but is that useful? Collections are often collected that way, named that way for adminstrative reasons – the naming associated with a particular digitisation funder, or from a collection. So the Dead Sea Scrolls are scanned in a music collection because the settings were the same for digitising them… That means the “collection” isn’t always that helpful.

So farce… If we think Fork handles/4 Candles…

We have some common farce-inducing words:

  • Collection (see above)
  • Access – but that has different meanings, sometimes “access” is “on-site” and without download, etc. Access has many meanings.
  • Content – we have so much, that isn’t a useful term. We have personal archives, computers, archives, UK Web domain trawl, pictures of manuscripts, OCR, derived data. Content can be anything. We have to be specific.
  • Metadata – one persons metadata is anothers data. Not helpful except in a very defined context.
  • Crowdsourced – means different things to different people. You must understand how the data was collected – what was the community, how did they do it, what was the QA process. That applies to any collaborative research data collection, not just crowdsourcing.

An example of complex provenence…

Microsoft Books digitisation project. It started in 2007 but stopped in 2009 when the MS Book search project was cancelled. This digitised 49K works (~65k volumes). It has been online since 2012 via a “standard” page turning interface ut we have very low usage statistics. That collection is quite random, items were picked shelf by shelf with books missing. People do data analysis of those works and draw conclusions that don’t make sense if you don’t understand that provenance.

So we had a competition entry in 2013 that wanted to analyse that collection… But actually led to a project called the Sample Generator by Pieter Francois. This compared physical to digital collections to highlight the issues of how unrepresentative that sample is for drawing any conclusions.

Allen B Riddell looked at the HathiTrust corpus called “Where are the novels?” in 2012 which similarly looked at the bias in digitised resources.

We have really big gaps in our knowledge. In fact librarians may recognise the square brackets of the soul… The data in records that isn’t actually confirmed, inferred information within metadata. If you look at the Microsoft Books project it’s about half inferred information. A lot of the Sample Generator peaks of what has been digitised is because of inferred year of publication based on content – guesswork rather than reliable dates.

But we can use this data. So Bob Nicholson’s competition entry on Victorian Jokes led to the Mechanical Comedian Twitter account. We didn’t have a good way into these texts, we had to improvise around these ideas. And we did find some good jokes… If you search for “My Mother in-law” and “Victorian Humour” you’ll see a great video for this.

That project looked for patterns of words. That’s the same technique applied to Political Meetings Mapper.

So “Access” again… These newspapers were accessible but we didn’t have access to them… Keyword search fails miserable and bulk access is an issue. But that issue is useful to know about. Research and genealogical needs are different and these papers were digitised partly for those more lucrative genealogical needs to browse and search.

There are over 600 digital archive, we can only spend so long characterising each of them. Microsoft Books digitisation project was public domain so that let us experiment richly quickly. We identified images of people, we found image details. we started to post images to Twitter and Tumblr (via Mechanical Curator)… There was demand and we weren’t set up to deliver those so we used Flickr Commons – 1 TB for free – with the limited awareness of what page an image was from, what region. We had minimal metadata but others started tagging and adding to our knowledge. Nora did a great job of collating these images that had been started to be tagged (by people and machines). And usage of images has been huge. 13-20 million hits on average every month, over 330 M hits to date.

Is this Iterative Crowdsourcing (Mia Ridge)? We crowdsource broad facts and subcollections of related items will emerge. There is no one size fits all, has to be project based. We start with no knowledge but build from there. But these have to be purposefully contextless. Presenting them on Flickr removed the illustrations context. The sheer amount of data is huge. David Foster Wallace has a great comment that “if your fidelity to perfectionism is too high, you never do anything”. We have a fear of imperfection in all universities, and we need to have the space to experiment. We can re-represent content in new forms, it might work, it might not. Metaphors don’t translate between media – like turning pages on a screen, or scrolling a book forever.

With our map collection we ran a tagathon and found nearly 30,000 maps. 10,000 were tagged by hand, 20,000 were found by machine. We have that nice combination of human and machine. We are now trying to georeference our maps and you can help with that.

But it’s not just research… We encourage people to do new things – make colouring books for kids, make collages – like David Normal’s Burning Man installation (also shown at St Pancras). That stuff is part of playing around.

Now, I’ve talked about “Crowd sourcing” several times. There can be lots of bad assumptions of that term. It’s assumed to be about a crowd of people all doing a small thing, about special software, that if you build it they will come, its easy, its cheap, it’s totally untrustworthy… These aren’t right. It’s about being part of a community, not just using it. When you looka at Zooniverse data you see a common pattern – that 1-2% of your community will do the majority of the work. You have to nurture the expert group within your community. This means you can crowdsource starting with that expert group – something we are also doing in a variety of those groups. You have to take care of all your participants but that core crowd really matter.

So, for crowdsourcing you don’t need special software. If you build something they don’t neccassarily come, they often don’t. And something we like to flag up is the idea of playing games, trying the unusual… Can we avoid keyboard and mouse? That arcade game does that, it asks that idea of whether we can make use of casual interaction to get useful data. That experiment is based on a raspberry pi and loads of great ideas from others using our collections. They are about the game dynamic… How we deal with data – how to understand how the game dynamics impact on the information you can extract.

So, in summary…

Don’t be scared of using words like “collection” and “access” with us… But understand that there will be a dialogue… that helps avoid disappointment, helps avoid misunderstanding or wasted time. We want to be clear and make sure we are all on the same page early on. I’m there to be your technical guide and lead on a project. There is space to experiment, to not be scared to fail and learn from that failure when it happens. We are there to have fun, to experiment.

Questions & Discussion

Q1) I’m a historian at the National Library of Scotland. You talked about that Microsoft Books project and the randomness of that collection. Then you talked about the Flickr metadata – isn’t that the same issue… Is that suitable for data mining? What do you do with that metadata?

A1) A good point. Part of what we have talked about is that those images just tell you about part of one page in a book. The mapping data is one of the ways we can get started on that. So if we geotag an image or a map with Aberdeen then you can perhaps find that book via that additional metadata, even if Aberdeen would not be part of the catalogue record, the title etc. There are big data approaches we can take but there is work on OCR etc. that we can do.

Q2) A question for Ben about Tweeting – the Mechanical Curator and the Mechanical Comedian. For the Curator… They come out some regularly… How are they generated?

A2) That is mechanical… There are about 1200 lines of code that roams the collection looking for similar stuff… The text is generated from books metadata… It is looking at data on the harddrive – access to everything so quite random. If no match it finds another random image.

Q2) And the mechnical comedian?

A2) That is run by Bob. The jokes are mechanically harvested, but he adds the images. He does that himself – with a bit of curation in terms of the badness of jokes – and adds images with help of a keen volunteer.

Q3) I work at the National Library of Scotland. You said to have fun and experiment. What is your response to the news of job cuts at Trove, at the National Library of Australia.

A3 – Ben) Trove is a leader in this space and I know a lot of people are increadibly upset about that.

A3 – Nora) The thing with digital collections is that they are global. Our own curators love Trove and I know there is a Facebook group to support Trove so, who knows, perhaps that global response might lead to a reversal?

Mahendra: I just wanted to say again that learning about the stories and provenance of a collection is so important. Talking about the back stories of collections. Sometimes the reasons content are not made available have nothing to do with legality… Those personal connections are so importan.

Q4) I’m interested in your use of the IPython Notebook. You are using that to access content on BL servers and website? So you didn’t have to download lots of data? Is that right?

A4) I mainly use it as a communication tool between myself and Ben… I type ideas into the notebook, Ben helps me turn that into code… It seemed the best tool to do that.

Q4) That’s very interesting… The Human API in action! As a researcher is that how it should be?

A4) I think be. As a researcher I’m not really a coder. For learning these spaces are great, they act as a sandbox.

Q4) And your code was written for your project, should that be shared with others?

A4) All the code is on a GitHub page. It isn’t perfect. That extract, code, geocode idea would be applicable to many other projects.

Mahendra: There is a balance that we work with. There are projects that are fantastic partnerships of domain experts working with technical experts wanting problems to solve. But we also see domain experts wanting to develop technical skills for their projects. We’ve seen both. Not sure of the answer… We did an event at Oxford who do a critical coding course where they team humanities and computer scientists… It gives computer scientists experience of really insanely difficult problems, the academics get experience of framing questions in precise ways…

Ben: And by understanding coding and

Comment (me): I just wanted to encourage anyone creating research software to consider submitting papers on that to the Journal of Open Research Software, a metajournal for sharing and finding software specifically created for research.

Q5) It seemed like the Political Meetings Mapper and the Palimpsest project had similar goals, so I wondered why they selected different workflows.

A5 – Bea Alex) The project came about because I spoke to Miranda Anderson who had the idea at the Digital Scholarship Day of Ideas. At that time we were geocoding historical trading documents and we chatted about automating that idea of georeferencing texts. That is how that project came about… There was a large manual aspect as well as the automated aspects. But the idea was to reduce that manual effort.

A5 – Katrina) Our project was so much smaller team. This is very much a pilot project to meet a particular research issue. The outcomes may seem similar but we worked on a smaller scale, seeing what one researcher could do. As a traditional academic historian I don’t usually work in groups, let alone big teams. I know other projects work at larger scale though – like Ian Gregory’s Lakes project.

A5 – Mahendra) Time was a really important aspect in decisions we took in Katrina’s project, and of focusing the scope of that work.

A5 – Katrina) Absolutely. It was about what could be done in a limited time.

A5 – Bea) One of the aspects from our work is that we sourced data from many collections, and the structure could be different for each mention. Whereas there is probably a more consistent structure because of the single newspaper used in Katrina’s project, which lends itself better to a regular expressions approach.

And next we moved to coffee and networking. We return at 3.30 for more excellent presentations (details below). 

BL Labs Awards: Research runner up project: “Palimpsest: Telling Edinburgh’s Stories with Maps” – Professor James Loxley, Palimpsest, University of Edinburgh

I am going to talk about project which I led in collaboration with colleagues in English Literature, with INformatics here, with visualisation experts at St Andrews, and with EDINA.

The idea came from Miranda Anderson, in 2012, who wanted to explore how people imagine Edinburgh in a literary sense, how the place is imagined and described. And one of the reasons for being interested in doing this is the fact that Edinburgh was the world’s first UNESCO City of Literature. The City of Literature Trust in Edinburgh is also keen to promote that rich literary heritage.

We received funding from the AHRC from January 2014 to March 2015. And the name came from the concept of the Palimpsest, the text that is rewritten and erased and layered upon – and of the city as a Palimpsest, changing and layering over time. The original website was to have the same name but as that wasn’t quite as accessible, we called that LitLong in the end.

We had some key aims for this project. There are particular ways literature is packaged for tourists etc. We weren’t interested in where authors were born or died. Or the authors that live here. What we were interested in was how the city is imagined in the work of authors, from Robert Louis Stevenson to Muriel Spark or Irvine Welsh.

And we wanted to do that in a different way. Our initial pilot in 2012 was all done manually. We had to extract locations from texts. We had a very small data set and it offfered us things we already knew – relying on well known Edinburgh books, working with the familiar. The kind of map produced there told us what we already knew. And we wanted to do something new. And this is where we realised that the digital methods we weree thinking about really gave us an opportunity to think of the literary cityscape in a different mode.

So, we planned to textmine large collections of digital text to identify narrative works set in Edinburgh. We weren’t constrained to novels, we included short stories, memoirs… Imaginative narrative writing. We excluded poetry as that was too difficult a processing challenge for the scale of the project. And we were very lucky to have the support and access to British library works, as well as material from the HathiTrust, and the National Library of Scotland. We mainly worked with out of copyright works. But we did specifically get permission from some publishers for in-copyright works. Not all publishers were forthcoming, and happy for work to be text mined. We were text mining work – not making them freely available – but for some publishers full text for text mining wasn’t possible.

So we had large collections of works, mainly but not exclusively out of copyright. And we set about textmining those collections to find those set in Edinburgh. And then we georeferenced the Edinburgh placenmmaes in those works to make mapping possible. And then finally we created visualisations offering different viewpoints into the data.

The best way to talk about this is to refer to text from our website:

Our aim in creating LitLong was to find out what the topography of a literary city such as Edinburgh would look like if we allowed digital reading to work on a very large body of texts. Edinburgh has a justly well-known literary history, cumulatively curated down the years by its many writers and readers. This history is visible in books, maps, walking tours and the city’s many literary sites and sights. But might there be other voices to hear in the chorus? Other, less familiar stories? By letting the computer do the reading, we’ve tried to set that familiar narrative of Edinburgh’s literary history in the less familiar context of hundreds of other works. We also want our maps and our app to illustrate old connections, and forge new ones, among the hundreds of literary works we’ve been able to capture.

That’s the kind of aims we had, what we were after.

So our method started with identifying texts with a clear Edinburgh connection or, as we called it “Edinburghyness“. Then, within those works to actually try and understand just how relevant they were. And that proved tricky. Some of the best stuff about this project came from close collaboration between literary scholars and informatics researchers. The back and forth was enormously helpful.

We came across some seemingly obvious issues. The first thing we saw was that there was a huge amount of theological works… Which was odd… And turned out to be because the Edinburgh placename “Trinity” was in there. Then “Haymarket” is a place in London as well as Edinburgh. So we needed to rank placenames and part of that was the ambiguity of names, and understanding that some places are more likely to specifically be Edinburgh than others.

From there, with selected works, we wanted to draw out snippits – of varying lengths but usually a sensible syntactic shape – with those mentions of specific placenames.

At the end of that process we had a dataset of 550 published works, across a range of narrative genres. They have over 1600 Edinburgh place names of lots of different types, since literary engagement with a city might be a street, a building, open spaces, areas, monuments etc. In mapping terms you can be more exact, in literature you have these areas and diverse types of “place”, so our gazeteer needed to be flexible to that. And what that all gave us in total was 47,000 extracts from literary works, all focused on a place name mention.

That was the work itself but we also wanted to engage people in our work. So we brought Sir Walter Scott back to life. He came along to the Edinburgh International Book Festival in 2014. He kind of got away from us and took on a life of his own… He ended up being part of the celebrations of the 200th aniversary of Waverley. And popped up again last year on the Borders Railway when that launched! That was fun!

We did another event at EIBF in 2015 with James Robertson who was exploring LitLong and data there. And you can download that as a podcast.

So, we were very very focused on making this project work, but we were also thinking about the users.

The resource itself you can visit at LitLong.org. I will talk a little about the two forms of visualisation. The first is a location visualiser largely built and developer by Uta Hinrichs at St Andrews. That allows you to explore the map, to look at keywords associated by locations – which indicate a degree of qualitative engagement. We also have a searchable database where you can see the extracts. And we have an app version which allows you to wander in among the extracts, rather than see from above – our visualisation colleagues call this the “Frogs Eye View”. You can wander between extracts, browse the range of them. It works quite well on the bus!

We were obviously delighted to be able to do this! Some of the obstacles seemed tough but we found workable solutions… But we hope it is not the end of the story. We are keen to explore new ways to make the resource explorable. Right now there isn’t a way where interaction leaves a trace – other people’s routes through the city, other peoples understanding of the topography. There is scope for more analysis of the texts themselves. For instance we considered doing a mood map of the city, scope to see that. But we weren’t able to do that in this project but there is scope to do that. And as part of building on the project we have a bit of funding from the AHRC so lots of interesting lines of enquiry there. And if you want to explore the resource do take a look, get in touch etc.

Q&A

Q1) Do you think someone could run sentiment analysis over your text?

A1) That is entirely plausible. The data is there and tagged so that you could do that.

A1 – Bea) We did have an MSc project just starting to explore that in fact.

A1) One of our buttons on the homepage is “LitLong Lab” where we share experiments in various ways.

Q2) Some science fiction authors have imagined near future Edinburgh, how could that be mapped?

A2) We did have some science fiction in the texts, including the winner of our writing competition. We have texts from a range of ages of work but a contemporary map, so there is scope to keying data to historic maps, and those exist thanks to the NLS. As to the future…  The not-yet-Edinburgh… Something I’d like to do… It is not uncommon that fictional places exist in real places – like 221 Baker Street or 44 Scotland Street – and I thought it would be fun to see the linguistic qualities associated with a fictional place, and compare to real places with the same sort of profile. So, perhaps for futuristic places that would work – using linguistic profile to do that.

Q3) I was going to ask about chronology – but you just answered that. So instead I will ask about crowd sourcing.

A3) Yes! As an editor I am most concerned about potential effort. For this scale and speed we had to let go of issues of mistakes, we know they are there… Places that move, some false positives, and some books that used Edinburgh placenames but are other places (e.g. some Glasgow texts). At the moment we don’t have a full report function or similar. We weren’t able to create it to enable corrections in that sort of way. What we decided to do is make a feature of a bug – celebrating those as worm holes! But I would like to fine tune and correct, with user interactions as part of that.

Q4) Is the data set available.

A4) Yes, through an API created by EDINA. Open for out of copyright work.

Palimpsest seeks to find new ways to present and explore Edinburgh’s literary cityscape, through interfaces showcasing extracts from a wide range of celebrated and lesser known narrative texts set in the city. In this talk, James will set out some of the project’s challenges, and some of the possibilities for the use of cultural data that it has helped to unearth.

Geoparsing Jisc Historical Texts – Dr Claire Grover, Senior Research Fellow, School of Informatics, University of Edinburgh

I’ll be talking about a current project, a very rapid project to geoparse all of the Jisc Historical Texts. So I’ll talk about the Geoparser and then more about that project.

The Edinburgh Geoparser, which has been developed over a number of years in collaboration with EDINA. It has been deployed in various projects and places, mainly also in collaboration with EDINA. And it has various main steps:

  • Use named entity recognition to identify place names in texts
  • Find matching records in a gazeteer
  • In cases of ambiguity (e.g. Paris, Springfield), resolve using contextual information from the document
  • Assign coordinates of preferred reading to the placename

So, you can use the Geoparser either via EDINA’s Unlock Text, or you can download it, or you can try a demonstrator online (links to follow).

To give you an example I have a news piece on the buriel of Richard III. You can see the Geoparser looks for entity recognition of all types – people as well as places – as that helps with disambiguation later on. Then using that text the parser ranks the likelihood of possible locations.

A quick word on gazeteers. The knowledge of possible interpretations comes from a gazeteer, which pairs place names to lat/long. So, if you know your data you can choose a gazeteer relevant to that (e.g. just the UK). The Edinburgh Geoparser is configured to provide a choice of gazeteers and can be configured to use other gazeteers.

If a place is not in a gazeteer it cannot be grounded. If the correct interprestation of a place name is not in the gazeteer, it cannot be grounded correctly. Modern gazeteers are not ideal for historical documents so historical gazeteers need to be used/developed. So for instance the DEEP (Directory of English Place Names) or PELAGIOS (ancient world) gazeteers have been useful in our current work.

The current Jisc Historical Text(http://historicaltexts.jisc.ac.uk/) project has been working with EEBO and ECCO texts as well as the BL Nineteenth Century collections. These are large and highly varied data sets. So, for instance, yesterday I did a random sample of writers and texts… which is so large we’ve only seen a tiny portion of it. We can process it but we can’t look at it all.

So, what is involved in us georeferencing this text? Well we have to get all the data through the Edinburgh Geoparser pipeline. And that requires adapting the geoparser pipeline to recognise place names to work as accurately as possible on historical text. And we need to adjust the georeferencing strategy to be more detailed.

Adapting our place name recognition relies a lot on lexicons. The standard Edinburgh Geoparser has three lexicons derived from the Alexandria Gazetteer (global, very detailed); Ordnance Survey (Great Britain, quite detailed), DEEP. We’ve also added more lexicons from more gazeteers… including larger place names in Geonames (population over 10,000), populated places from Natural Earth, only larger places from DEEP, and the score recognised place names based on how many and which lexicons they occur in. Low scored placenames are removed – we reckon people’s tolerance for missing a place is higher than their tolerance for false positives.

Working with old texts also means huge variation of spellings… There are a lot of false placenames/false negatives because of this (e.g. Maldauia, Demnarke, Saxonie, Spayne). They also result in false positives (Grasse, Hamme, Lyon, Penne, Sunne, Haue, Ayr). So we have tried to remove the false positives, to remove bad placenames.

When it comes to actually georeferencing these places we need coordinates for place names from gazetteers. We used three place names in succession: Pleiades++, GeoNames and then DEEP. In addition to using those gazeteers we can weight the results based on locations in the world – based on a bounding box. So we can prefer locations in the UK and Europe, then those in the East. Not extending to the West as much… And excluding Australia and New Zealand (unknown at that time).

So looking at EEBO and ECCO we can see some frequent place names from each gazeteers – which shows how different they are. In terms of how many terms we have found there are over 3 million locations in EEBO, over 250k in ECCO (a much smaller collection). The early EEBO collections have a lot of locations in Israel, Italy, France. The early books are more concerned with the ancient world and Biblical texts so these statistics suggest that we are doing the right thing here.

These are really old texts, we have huge volumes fo them, and there is a huge variety of the data and that all makes this a hard task. We still don’t know how the work will be received but we think Jisc will put this work in a sandbox area and we should get some feedback on it.

Find out more:

  • http://historicaltexts.jisc.ac.uk/
  • https://www.ltg.ed.ac.uk/software/geoparser
  • http://edina.ac.uk/unlock/
  • http://placenames.org.uk/
  • https://googleancientplaces.wordpress.com/

Q&A

Q1) What about historical Gaelic place names?

A1) I’m not sure these texts have these. But we did apply a language tag on a paragraph level. These are supposed to be English texts but there is lots of Latin, Welsh, Spanish, French and German. We only georeferenced texts thought to be English. If Gaelic names then, if in Ordnance Survey, they may have been picked up…

Claire will talk about work the Edinburgh Language Technology Group have been doing for Jisc on geoparsing historical texts such as the British Library’s Nineteenth Century Books and Early English Books Online Text Creation Partnership which is creating standardized, accurate XML/SGML encoded electronic text editions of early print books.

Pitches – Mahendra and co

Can the people who pitched me

Lorna: I’m interested in open education and I’d love to get some of the BL content out there. I’ve been worked on the new HECoS coding schema for different subjects. And I thought that it would be great to classify the BL content with HECoS.

Karen: I’ve been looking at Copyright music collections at St Andrews. There are gaps in legal deposit music from late 18th and 19th century as we know publishers deposited less in Scottish versus BL. So we could compare and see what reached outer reaches of the UK.

Nina: My idea was a digital Pilgrims Progress where you can have a virtual tour of a journey with all sorts of resources.. To see why some places are most popular in texts etc.

David: I think my idea has been done.. It was going to be iPython – Katrina is already doing this! But to make it more unique… It’s quite hard work for Ben to support scholars in that way so I think researchers should be encouraged to approach Ben etc. but also get non-programmers to craft complex queries, make the good ones reusable by others… and have those reused be marked up as of particular quality. And to make it more fun… Could have a sort of treasure hunt jam with people using that facility to have a treasure hunt on a theme… share interesting information… Have researchers see tweets or shared things… A group treasure hunt to encourage people by helping them share queries…

Mahendra: So we are supposed to decide the winners now… But I think we’ll get all our pitchers to share the bag – all great ideas… The idea was to start conversations. You should all have an email from me so, if you have found this inspiring or interesting, we’ll continue that conversation.

And with that we are done! Thanks to all for a really excellent session!

Feb 102016
 

Today I am at a Supervising Dissertations at a Distance workshop, co-hosted by eLearning@ed and the Institute for Academic Development. The session is based on a research project and is being facilitated by Dr Jen Ross, Dr Philippa Sheail and Clara O’Shea.

As this is a liveblog the usual caveats apply – and corrections and comments are welcome.

Jen Ross (JR): This event came about from some research that myself, Phil and Clara have worked on looking at online distance learners going through the dissertation process at a distance. So we will talk a bit about this, but also we have an exciting new development that we’ll be showing off: a board game based on our research!

So, myself, Phil and Clara worked on this project, funded by the Principal’s Teaching Award Scheme, with our colleagues Sian Bayne, Erin Jackson and Gill Aitken.

This work was done with 4 online distance programmes – clinical education, clinical management of pain, digital education and law. We had 18 semi-structured interviews conducted with graduates almost all via Skype. We undertook thematic analysis of transcripts. We also had 3 focus workshops/conversations with supervisors which enabled us to trigger reflection on the interview data.

So, to start with I want to talk about the “campus imaginary”, after Taylor’s idea of the “imaginary”, and Goggin’s definition of shared beliefs and understandings (rather than imaginary imaginary). Drawing on these we came up with the idea of the “Campus imaginaries” – the shared understanding of the campus and the organisation for those not physically here. We have nick-named this “when it was good it was very very good, but when it was bad it was the internet”. Why? People had lovely things to say, but when they didn’t they often attributed this to being an online distance learner, even when describing quite common dissertation experiences.

For instance June talks about struggling with time to do her dissertation around full time work – she attributes this to being an online distance student. Eva felt she had a good experience but that the supervision wasn’t great, it was adequate but she felt that it could have been better. And she also attributed this to being a distance student.

Terry says: “If you are full time you can just pop in and see your supervisor, or you speak to his secretary and book an appointment to see him. I don’t think there is a limit for a full time student.” [this gets audible laughs in the room given the realities of supervision on and off campus]

Now, that is funny but it is also poinagnt. That imagined idea of the physical space isn’t helpful for Terry and his expectations around supervision, of the support and time available, and those perceived differences between (idealised) physical and distance experience.

Arnott, meanwhile had a poor experience with their supervisor and felt that maybe being able to talk face to face might have helped that.

Nieve didn’t complete the dissertation, exiting with diploma. She felt (in retrospect) that doing some of the degree online, and some on-campus would have helped her as she felt lonely during her dissertation, and wanted to have the opportunity to share experience with other dissertation students. But again we can recognise that as a concern of many on campus students too.

So the themes that came up here, specifically in relation to online distance dissertations are also very familiar: unexpected obstacles; issues with motivation; supervisory relationships; time and space to focus; isolation; doubt. I think we have to do better at being supervisors helping students to understand what they can expect, that they can talk to us about all of these things, that we can support them (and that we don’t have secretaries!)

Phil Sheail (PS): I’m going to talk about the sense of “hospitality at a distance” – of hosting each other as distance students and supervisors, in learning spaces that overlap with homes.

Ruitenberg (2011), drawing on Derrida, in a great paper called “The empty chair: education in an ethic of hospitality” in Philosophy of education. She talks about hospitality as a demand for openness to the arrival of something and someone we cannot forsee: a demand that is impossible to fulfil, but that confronts all of our decisions and actions…”

I think this concept is relevant as whilst I was doing interviews there were so many different students, from different backgrounds and cultures… and it forces us to question some of our ideas of hospitality and of being a good host. Ruitenberg also talks about the figure of the teacher in “at-home” education. And the ethics of the university, the spaces of education are not the teachers

Amplification – you have to amplify yourself to put across your normal sense of enthusiasm, and that works well online.

One of the other things I did on a project with support services – disability office, careers, etc. and that connects to this idea of hospitality, and very particularly the idea of arrival, of welcome. So, we’ve been thinking about

Q: For intermittent learners, students might be engaged in a programme that they started 6 years ago, and starting a dissertation in that context.

A: Well when you start dissertation you may have a supervisor that hasn’t taught you… And there can be a dependency in that relationship between student and supervisor which can be challenging…

Q: Some of our supervisors are not Edinburgh staff members but those from NGOs etc.

A (JR): That was the case with one of the programmes we looked at. There it’s almost a welcome for supervisors too, and what does that mean in terms of making a space for dissertation, and establishing that complex relationship.

A (PS): Even if you are away from the institution, your supervisor is in a hospital etc. it’s important that the University does welcome you, particularly if things go wrong in that relationship, so they know where else to turn.

Martin, a supervisor, talked about the importance of a good and deliberate welcome for students.

In the example you just gave, of students who take a long time… Some students have complex care requirements. June again comments that she had gone through marriage breakdown, family crisis, health issues, but that for her, the degree was actually useful as a consistent presence in her life.

Now we’ve talked about welcomes and being supportive… But not all students actually want that. Terry comments that he wasn’t keen for hand holding and wouldn’t be whether he was full time, part time or online. And we have to remember that not all students want the same thing here.

JR: So we are going to turn now to how we can think of other ways to imagine the campus, alternatives that make students welcome. And also around fostering connections and counteracting negative disconnections. So, over to Clara…

Clara O’Shea (COS): The Dissertation Festival is an idea that Marshall and I came up with and made happen. We started this in 2011 – so reading Jen and Phil’s work backwards into what we do. This idea came out of the experience of loneliness and disconnection which can take place as a student going through the dissertation. We wanted something to support students through the dissertation process.

So, we try to run this festival 6-8 weeks before dissertations are due (usually August) so the festival is generally in May/June. The festival runs in Second Life – so we meet in a virtual space with sunshine, beach, virtual champagne and sushi. And this is just to be welcoming, warm, to make students feel comfortable.

So, the idea is that students come into the space, they present their work – 2 or 3 in an hour or hour and a half period, usually somewhat themed to foster connections, allow sharing of resources, etc. We checked student availability but also tutor availability – and opened the sessions up to others on the programme, and those beyond the programme. Participants do their presentation on voice chat for about 15 minutes. Questions come in in text chat – the presenter may reply during the talk or afterwards, which we also help facilitate.

So, last year we had some sessions on game based learning, multimodality, etc. We also had some tutor and alumni sessions on academic writing, on surviving and thriving through the dissertation, and also literature hunting. All of these sessions are synchronous but they are also recorded. Those recordings and the sessions are also complimented by a wiki (on PBWorks) where comments, further information, etc. can be shared. Each student has a page on the wiki with video, transcript, etc. But they also played with other ways to articulate their idea… We have them write haikus – they hate writing them but then find them really useful. They also play with images as well.

We also have a new innovation since last year called “The Visualisations Gallery”. This is to encourage students towards multimodality… We had tutors, current students, alumni all sharing visual ways to imagine their research.

And, even if a visitor can’t access that wiki, you can leave comments in Second Life.

The dissertation festival gives students a few things. It gives students a touchstone when things are quiet, a way to stay connected with the community. Students not yet at dissertation stage have the opportunity to see what that looks like, how that works. We’ve had students making connections, reading over a draft for each other. It gives students a chance to touch base with other supervisors… Which means accessing other expertise, to fill the gaps, to suggest other content.

So, when Jen talked about campus imaginaries, I think maybe this gives an imaginary that is more realistic and helpful. Places like Second Life give a useful, shared delusion of the campus. We all experience that very differently depending on their own timezone, location, the version of software they are running… It’s an illusion we all buy into. But arguably that is the experience of being on campus anyway.

On a practical basis we move those virtual logs, we adapt the voice presentation to the speakers needs, etc. But every time people come into Second Life they bring in their home space – the sounds, the distractions – and share that. It makes that special overlapping space. The space changes every time anyone comes in and out, and the dialogic space that participants create. And I think that’s where hospitality fits in.

Q&A

Q1: Can you say more about the interviewees – how many students, how many supervisors. I would like to know more about similarities or differences between supervisors and students.

A1 (JR): The interviewees were all students. The supervisors gave input through workshops, where they reflected and responded to student comments. Those haven’t been written up as quotes yet but inform our understanding here. One thing that struck me was that supervisors often also feel a sense of dislocation from supervisees… For instance maintenance of an authoritative supervisory role when you and the student are Skyping each other from home, you see the students kids running about, etc. And that giving those relationships a different character and nature perhaps.

Q2: For us the distance is often not as important about the fact that they are intermittant adn part time.

A2: That longer process does mean more can happen… Which can mean more likelihood to need to take an interruption of studies, and struggle to fit things in.

Q2: As a coordinator one of my challenges is managing supervisor expectations – that students don’t work full time for 10 months.

A2 (PS): Certainly some students took a while to get going… Changes in work or work priorities can impact on projects, especially work-based projects. One of our students had moved through 3 continents whilst doing their work.

A2 (COS): The festival can be useful for providing an additional deadline. Students often struggle to prioritise their own research over their work commitments etc. Students can also have unrealistic idea of their own – and their supervisors – availability during the dissertation process. When my students start we talk  through those things that

A2 (PS): We did have students feeling they were out of sync with other students. In one programme regular Skype chats were available but being ahead or behind made that chat less useful… They got into this idea that only students at the same pace/stage can share. There was also that issue Clara mentioned about being unclear on how much time they could expect from supervisors, or how much they were allowed. More clarity there might help.

A2 (JR): One of the most interesting things for me was seeing the difference in practice between programmes. Some started at the same time, some were rolling… But no matter how rigid the system some students always went out of sync. It was interesting to see how many ways there are to organise a programme and a dissertation process, you can only organise so far.

Q3: Are there resources we can give supervisors meeting students for the first time that they haven’t taught before?

A3: We have a dissertation planner that is for students to adapt, to help them manage the process, to understand availability of students at a given time, etc. These are on the website too. So things like work commitments, times when supervisors are away…

Q3: That sounds more like its for students. What about supervisors.

A3: There are resources for PhD supervision but if you talk to Velda (IAD) she will be able to comment.

A3 (PS): I think for student services it is important to have routes for students to access them online. Careers, counselling, disability and chaplaincy all have some some of page for what they can do for online programmes now, and are looking at ways to offer services online. I had a student I spoke to in this research who had a horrible personal time, and she was surprised that counselling was never suggested

Comment (LC): There are resources you can embed in Learn for your courses that point to those support services.

Q4: Is 6-8 weeks really enough time for capturing the problems?

A4: I think it’s about right. We’ve tried later – and that’s too late. We’ve tried earlier but students get nervous about what they can present. It seems to be around 8 weeks is about right. And, if they aren’t ready at that point then students are in trouble and need to have conversations with supervisors. At that stage they can’t change methodologies though… But our research methods course ends with an assignment which is a proposal for research which triggers those sorts of theoretical and methodological conversations early, and raise any major concerns on timing etc.

JR: And now…. We will have a short break but then when we come back we will be playing Dissertation Situation: the board game based around our work! This is a primarily discussion based game.

So, the thing that is useful to know is that the scenarios in the game have come from data generated in this project. So these are real world problems (slightly fictionalised). They have happened, they are likely to happen again.

Cue board games… 

Q&A

Q1: We want an online version!

A1: We did talk about that – either to share and then print off, or to play online.

Comment: We should get students to play too!

Comment: I think that this would be really useful for supervisors, to know they are not alone, but also for students to understand what can arise.

Q2: I was also going to say that students should place. For us we didn’t get through the whole game in the time… And that was fine… But for me it wasn’t important to finish necessarily – that’s a game design thing perhaps, and a timing thing.

A2: I did some rough calculations… But it was guesswork.

PS: Any areas that weren’t useful for particular disciplines?

Comment: Yes, the data question doesn’t really apply to psychology in the same way, or for law. But literature related question on losing data would apply.

PS: One of things we were aware of was that some online distance learners wouldn’t have bandwidth – eg programmes with students in sub-Saharan Africa – to play this game online – or only a simple version. But actually a download version might work.

Comment: Would also be good to share this, or a list of scenarios to supervisors off-campus, not affiliated with the University.

Comment: I think it might be easier to get to students than supervisors…

Comment: And concerned it could be seen as patronising… But you could call it a simulation.

Comment: For new supervisors etc. you could set up a wiki with the questions, and have discussion there…

Comment: I think it would be interesting to know what potential there is for moderating, fact checking, or connecting this to other resources, things that new or outside supervisors just may not know are there. Some pragmatic solutions also potentially put a supervisor at some risk, or raise controversial issues, so knowing where to put those, flag those up… What to do next etc. would be great.

And with that we are done with a really interesting session. Huge thanks to Jen, Phil and Clara for this workshop. Do feel free to follow up with them about that game – it was a really useful tool for discussion. 

Related Links