Today and tomorrow I am at the Institutional Web Managers Workshop taking place a the University of Reading. I’ll be liveblogging so usual caveats apply re: spelling, names etc. I welcome any comments and corrections and will do a tidy up post event.
IWMW is being introduced by Sean ? of University of Reading. Employment in ICT is 300% above average around Reading so you’ve very much picked the right place for your conference.
Context of HE, never experienced such changing times as currently in, now have market place or sort of market place with student choice at heart of the system. What does that mean for you and for university? We have to get better at digital marketing – the web and social media is central to that, reputation is vital, and the way that courses will be delivered will change. That in many ways is your challenge. We live in very challenging times and that is your opportunity. How will
Introduction by Brian Kelly
Brian introduced himself and opened by saying the event had run for 15 years. This year there are several sponsored places and much amplification of the day for those not able to be here in person, and those that want to come back and review videos etc. later on.
Last year we discussed concerns and indeed “the axeman cometh” – one of our colleagues last year blogged about being made redundant shortly after last year’s IWMW.
Technical developments – a quick show of hands shows most attendees have one smartphone, many have two, some of have three, and one has four! (OK, lets be fair that person with four is Ben Butchart, my colleague at EDINA, who is running a parallel session on Augmented Reality and smartphones). This is really an opportunity to experiment. Students will have expectations of smart phone access to courses.
Centralisation – is being discussed this is already happening in Wales.
Demonstrate our value – we do a good job but we need to be able to demonstrate that. And to be ready to respond to competition and privatisation. We need to think politically.
Is this is the end of community? Well perhaps not (see image of Frintr mosaic of Brian’s community) we are used to sharing and being part of communities around events, on mailing lists, in our social networks etc. We do not live in a vaccuum (see image from Tony Hirst of Brian’s network an where there are clusters) – there may be visualisations of communities that we can see in this way.
Innovation – over the next two days we’ll be talking about innovation, having sessions about it. We have the mobile web – there is so much potential in these devices. About half of the audience have switched to mobile devices for consuming information. Today we are trying out Shhmooze to share presence and enable further networking.
Recently UKOLN/CETIS ran a survey looking at approaches to delivering institutional websites to mobiles. Many said they planned to deliver it in the same way as large screen devices. But there was a diversity of approaches from apps to SMS. Is this diversity a strength or should we be more consistent.
Openness – KIS (Key Information Sets) – these are the
Do we need institutional websites anymore? Most Coca Cola visitors go to their Facebook page rather than their website.
Mike is up to summarize the DevCSI event (#iwmw11hack) that preceded IWMW 2011. One way to engage: integrate estates management data. KIS is another way and we can integrate with other systems. Dave Chaps and Ben O’Steen created a heatmap of student housing with costs. Even just converting data – e.g. catering spreadsheets – and turning it into something more useful, like a website, it adds real value.
Brian: I think this sort of work shows the value of local developers in a way that meets local institutional needs.
Tomorrow in the final session I’ll be asking more of you to get up and share what you have learned. Please contact Brian or Marieke if you are interested.
We know what you do, so what is it worth? – Ranjit Sidhu, Statistics into Decisions (SiD)
This is a follow up to Ranjit’s talk at last year’s IWMW event asking “so what do you do?”. This is as much about mental
tempted by the dark side:
“what if the data shows us in a bad light?” – the data is out there, someone will do this for you (see the recent headlines about the cost of university websites vs. their effectiveness). A vaccuum can be much more dangerous. We need to get ahead of the curve, have information out there so we’re not sitting targets.
“universities have a wider role and simply cannot be measured that easily” – but your role is being scrutinised. We want to value what web teams do so that you have a good way of saying cost | benefit
“I have this data but no one is listening” – this is the one we should be focusing on. It’s not about the data, it’s about delivery of that information as well.
“with so much information and data at our finger tips why is online analysis seen as being so ineffectual?” – to start off with we need to be aware that “web analytics is crap”. And it’s not clear to others what the difference it between the various statistics are. Why should someone away from the web team have to look at the same reports and the same figures. We need to get beyond what we can provide and think about how we should be sharing that with others.
The other thing we’re talking about is automation – people often say “well we send these reports weekly to them…”. That’s spam. If that’s all your doing just stop, these message will not be read and you are creating a negative feeling about your work. We have all this information coming in, we think we need to automate, but we don’t process it properly. Automation can be good but if it’s not processed it’s garbage.
The Quest for Perfection – there is so much data available we want it to be perfect. Just because you don’t have every bit of data doesn’t mean that you cannot find valueable in that data. We need to forecast and create models that show the value of what we do.
We have online information but it is only meaningful if combined with offline information. Offline data by it’s nature is imperfect too. But we can create new models of how to communicate that data. It’s not perfect but it’s good and it will get better.
So, to summarize: the language is arcane, information is automated, it is dislocated from the business, and it’s badly designed – it’s the human factor, design matters. So, here is a dashboard we have created for Strathclyde – make the data fit what you want. This is an international recruitment board, we do a social media board, we have prospective students board. You have to cut that data into useful pieces.
So looking closer – we do top line figures. Total visits, total non EU visits – this connects to pricing policies of universities. We do demographic breakdown of gender, age and interests. And, the crux of the matter, we have pound signs. Try and think about if someone did something offline what would the cost be, how can we show the money we have saves, or use projections of converting accesses to student applications. And ca project income from different countries and an estimation of saved costs in sending out prospectuses. Can also do projections about how improving presences could lead to greater income – helps justify costs of web development.
Going micro and macro. For some universities we are providing information for particular departments and faculties – of real value to those teams. Just making an attractive dashboard to present stats can actually make you money from internal charging.
To know how you compare to others you need to compare your activity to sector statistics. It will give you a sense of where to improve, where your strengths are, where you are particularly different. There are obvious groupings but lots of very different groupings – such as interest in particular international students etc.
Sector stats dashboard now on display – Total visits to a particular website, total uk visits, total internal traffic – most universities see between 41% and 44% of traffic – they are customers but a different type of customer. Prospective visits is generated by looking at traffic to UK universities, and international traffic as a percentage of that. Then you can see the top 10 countries traffic and a board of stats on the average revenue breakdown for the top 4 non-eu countries, calculating potential worth per visit. The one thing universities should not be worried about is traffic. You have huge sites, its the conversion that’s the problem. You can take that cost and compare it to the offline cost of recruitment.
From knowing the value of new visitors you can look to create campaigns and find the costs and benefits of that campaign.
The key thing about these statistics is that this is a good way to get a proper voice, to show the real value of what online does. But you think about what a website must deliver and what the commercial equivalent would cost to run – you are undervalued, this stuff is a good way to get a louder voice about what you do. You can start a conversation that can really show the value of what you do.
We did a sort of mindmap for homepage – a lot of people wanted to know how traffic is going through the homepage [this is data you can find through Google Analytics through entrance and exit routes). You can find where people get lost en route. This visual shows stats that indicate that Baidu is bigger than Bing – what does that mean for your website and how you are developing it?
Also International behaviour differences – they stay longer, they look at a lot of content and they are more likely to be first time visitors – what does that mean for what you are doing with the site – do you have the right calls to action in the right places?
So, do you take up the challenge? You need to get the data so you are ready when asked. You need to start somewhere.
Q1) Nicola Osborne, EDINA [me] IS it fair to compare the costs of paper materials with those of online? It has been a long time since the main route for many information sources has been paper. I just think that you would never send out prospectuses to many of these people, even if in the UK so it’s perhaps not the right comparison.
A1) Yes, it’s fair. You have to justify your costs as a department and you need to map that to costs of offline.
Q1) I agree with the principal but just don’t think you have the comparison quite right on this one.
Q2) conversion factor of paper may be significantly different – paper is more effevtice.
A2) Sure, we can develop those models to become more accurate. Right now that data doesn’t exist.
Q3) Kevin Meers online: can measurement fall through the gaps if you have multiple departmens responsible for different areas of online
A3) Yes, it can and it’s important that you avoid that
Q4) Was interested in what you said about losing students through extra clicks. We have a limited amount of space on the homepage so we can’t link to everything
A4) I don’t have definitive answer but it’s interesting that the top clicks are repeated a stage down the website.
Q5) Matthew Hoskins, University of Leeds – we have focused on UG recruitment, now focusing on PG recruitment and selective recruitment. How do you use top line figures to find quality students, that great research postgrad say? These are really small niche audiences – how can you use these techniques to reflect that.
A5) Until they do something we don’t know if they are a good or a bad student. The best way to tailor to small niches you need to tailor to
Q6) Brian: If you want a dashboard for your institution what do you do?
A6) We do make money from the institutional dashboard. The sector stats are £50 each – the cost of providing. We have 10 on those stats at the moment but the more institutions we get on there, the better the stats will be.
Marketing and Other Dirty Words – Amber Thomas, Programme Manager, JISC
I want to raise some potential. Others in your institution are working with open data and content and open educational resources. There are a lot of people persuing open ways of working. It’s good but how do we connect what people are doing to what the university is doing and the central university web presences.
The comfort zone is around open access to research, open innovation, open educational research, public good, academic autonomy, public good – the language of values. BUT we also have our dirty words around impact, brand, CBA, synergies, metrics, marketing, KPIs, Business cases – the language of the market. We need to connect these up. There are pragmatists that are already trying to do this.
Cue Prisoner style T-shirt: many have objections to these sorts of languages. People don’t like the idea of comparison to benchmarking. What we can measure is not necessarily what is important. Impact can be far downstream, far in the future, there is a risk of reducing us to the short and near term. Metrics don’t respect the long tail. Formalising social media can institutionalise and de-personalise it. There is a lot of truth there and reasons to be cautious but there are very real costs. This is about staff time. People working in services, academics and we don’t want to waste the time of students.
Buzzword Bingo time! Impact, Marketing, Metrics – others on the agenda will be looking at this in much more detail. Of course there are direct and indirect elements here. Costs are often indirect. Time from IT and the library is often taken for granted in universities. It is hard to measure costs, moving activity from one part of the institution to another
Shift focus to listening and stats side of social media, not just putting stuff out there.
Marketing – translating reputation into value sounds good but it’s hard to prove that.
Impact – a huge hot potato. In digital content and social media it’s a massive issue. In research impact is a massive issue. In learning and teaching impact is a massive issue. The Splashes and Ripples report and the Listening for Impact report both look at what has/can be down to show the impact of work. Decisions need to be taken on assessing research impact. Institutional Managers are certainly thinking more about how we map the value of the service to the REF. There is also something about the presence of a particular academic or department – big issue and big opportunity. People got together to look at how we can move to more sophisticated metrics – Beyond Impact.
There are now much clearer models for how web tools and social media impact on the student experience. It’s becoming a lot more about fees, it’s about student satisfaction, and it’s about retaining students as drop outs are expensive. There is role in that to providing model content for students. There is also something here about the KIS data and being aware of how your university will be represented on the wider web. See University College Falmouth openSpace – about how a resource on screenwriting went viral. This is a small institution that made a big impact in their subject area. Not only universities – Amplified Leicester looked at social capital and cohesion and how that can be mapped against the REF.
Marketing: a lot of what we use social media for is marketing anyway – it’s marketing, market intelligence and exchange (learning, scholarship, CRM). It’s worth mentioning in terms of impact – sometimes it’s about measuring our activity, sometimes it’s about measuring the effects of what we’ve done. And social media can also effect Search Engine Optimisation (SEO). Who is the social in social media? There are people within the institution (who we have varying levels of control over), we have researchers, teachers, services, students, and those outside the institution. At some recent events I’ve noticed academics are happy to talk about branding (and personal branding) but not yet about marketing.
There is a fear that metrics are so important that it seems they will lead to a decision but it is so much more complicated than that. We have to demystify the role of metrics in that process. They are crude, they are only one aspect, but they are important for making decisions.
The view ahead: options for responding to the new agenda – it’s NOT an option to deny it is happening, to leave it to others or to pay lip service. It is about producing numbers, to produce stories, to deepen our listening approaches, to improve our metircs, to broaden our impact model – to include those outside of academia in particular, and to extend the impact timeframe.
So, I think those languages of values and marketing are bridgale:
- Profiling academic expertise – often they have an online presence, or several – hard to know the one up to date profile though. Some are very media friendly, few universities have a blog roll, and how many of those academics link to their other materials – their slideshare, their publications etc. There is a real opportunity there around academic profiles.
- Supporting REF metrics will be important – focus on your internal function as advisors.
- Enhanced research publications – including data, visualisations etc. This is possible and you can help them make a big splash. Making good PDFs and beyond to good quality engaging research
- Cross linking open content to open course data – it is linked in the VLE but it loses it’s link to the teaching content. Why not link from the content to the courses? Make richer links and why not use this as taster content to help filter applications.
- Social Media listening tools – become the centre of expertise in this
- Web analytics and visualisation – graphics are important and engaging. Visualise all sorts of data, even in the HR system perhaps. Develop that skill.
Think about how you can make the most of how the web works – linking, APIs, feeds, web analytics, data visualisation, stories. There are things you learn about what is tweeted, shared, automatically pulled into dashboards etc that you can make work better. Story telling often comes up around open data – to explain how that data is valuable – skills in telling stories are important.
Think about fees, student recruitment, student satisfaction, impact, the REF, internationalisation.
Q1) Chris Gutteridge, Southampton: we’ve found that open data gets you open web pages, the most valuable bit of it has been webpages from the data that are linked together. That’s been the surprise benefit of this
A1) Lorcan Dempsey has been looking at the role of the library and how it is changing and talking about the inside out library – curating internal resources for the public. There is a huge opportunity for this sort of activity.
Q2) Brian: you said that people in all departments come in and want to get statistics and data and get information like the KIS. You seem to be saying there are opportunities for web teams here, how should this be taken forward.
A2) Thinking of an outside example local councils have been having massive restructures and outsourcing. One of the things that they’ve been doing is social media cafes and surgeries and honing their skills in these spaces. So much depends on your team and where they are based. A general theme there is not about what we provide but about monitoring the use of things outside your control to change.
Quick JISC plug – calls that are out and coming:
- Activity Data Programme
- DevSci Programme – visualisation skills very much in that space
- Call (England and Wales) – opening up course data
- Call imminent from Open Educational Resources – institutional resources and linking to web presences
- Call coming out on technology enhanced organisational support
- Call coming about discovery infrastructure, preservation, research information management, and libraries and mobile technology
With that we broke for lunch (which was yummy!) and are now back for the afternoon sessions:
Using activity data to support your users – Tom Franklin, University of Manchester an Franklin Consulting
What is activity data? Well it’s everything in a log file. The long answer is that it’s every log in, every search, a rich source of data around usage.
We are well behind the game here. Supermarket loyalty cards are all about tracking activity data and get huge value out of that data. Visa have been using activity data for years, they used it to detect fraud and now all the credit card companies do that. Amazon tracks know what you’ve brought and they know what others have brought. Google collects and uses our activity data.
But… this data is only useful if you know who the people you are collection data on are.
So in Student recruitment – what are potential students doing on our website? Where do they go? What are their routes? can you tie schools’ IP addresses to UCAS applications? [should you?] You can use that information to improve the university website, to support candidates while they are on the site? to improve the usability of the site? Certainly to improve student recruitment.
Student retention is incredibly important, very expensive. How can we use data we are already recording to prove success, to improve retention. Purdue university have been doing some work on this. They have a traffic light system for individual students and analysed those students against behaviour generally associated with doing well. They look for worrying patterns, bad patterns, disastrous patterns. By automatically alerting students about how they are doing and what they could do better they are finding that students are doing better.
Demonstrating value – library impact data project – relationship between library use and results; using a variety of data – turnstile activity, library management system, EZProxy service, student record system. The relationship was demonstrated (not causality). See: http://library.hud.ac.uk/blogs/projecs/lidp/. Shows some significant differences. There are all sorts of other questions you can ask. You can see if there is a pattern of borrowing the books that dictates success – timing of borrowing perhaps. You have similar data in your library too!
At Cambridge they are using SAKAI and identifying behaviours of staff and students that are associated with success (or not). A similar piece of work is going on at Leeds Met. But if you can demonstrate that there is an effect then you can encourage staff and students to use the VLE perhaps. see: http://vledata.blogspot.com/.
Research Impact – is about going beyond Google Analytics. How can we increase use of the institutional repository. Help people find the information that they need. There is a project at the University of Aberystwyth in Wales to try and increase the use of materials in the virtual welsh repositories network. They are using an Amazon like approach to encourage people to access more than one paper. People typically go from Google Search or Google Scholar and nab a paper but then do little else. So they are using a combination of a recommender system – people who accessed x also accessed and also similarity based recommendations (project is AEIOU – Aceross Welsh Repository Network). This data can also help us make easier user journeys through a site.
Resource utilisation – can we make that better, more efficient? What resources are being used? By whom? What are the patterns? Can you support users to make better use of resources? Are the subscriptions optimal?
So, it all sounds great and easy… what could go wrong?
Data protection – when you collect personal data you need permission first. Some of it you are required by law to collect but are people consenting? You also have to say why you are collecting data, they must be able to give consent to the uses and sharing of data. With staff and students that’s not so hard to do and you have a duty to support their studies which may be a good enough reason. Sharing data has risks. To connect it up usefully anonomisation may be needed but that can impact on usefulness. There is also a big debate on whether IP address is personal data or not. You can anonomise IP addresses but not perfect. Interesting issues there that you at least need to be aware of.
Licensing – if you are sharing data you need to think about what the license may be, who is to be allowed access to this data. Some of the projects that have anonomised.
One of the things that happened at the beginning of these projects was that searching for this data many found they were not collection and/or not storing that data. How can you improve what you are doing by understanding your users better? There will be some effort in analysing data but determine the costs and the benefits, persuade your boss to fund it. Good luck!
Q1) Ranjit Sidhu, SiD: Looking at Twitter while you were talking there is a lot of focus on privacy here. I wondered if you want to comment on it. I think people get their knickers in a twist for no reason – especially when compared to Amazon etc.
A1) It’s not just me it’s also the Data Commissioner – we should be doing more with this data. People are getting too uptight about privacy. If you are supporting what people want to do then it shouldn’t be a problem.
Q1) Something like an IP address is publicly addressed.
A1) The issue is that we can use the IP as a proxy then we could see lots of people as the same thing – this is one of the challenges with the openurl router as providers see one url/proxy (in a more literal meaning) address. You could see weird unconnected other behaviours.
Q2) Sam Trafford, Edgehill University: did you say something on data on which schools produce good students
A2) No one is doing this yet but it seems like something we could do.
Q2) I think you have huge potential for misuse there – for profiling schools that do not bring in money.
Comment from Amber) From the government open data agenda they are sharing loads of data and lots of ways of modelling it so that it’s sensible rather than being about league tables.
Q3) Brian Kelly, UKOLN: Think I agree with the idea of a risk assessment about making data available. In the context of the cookie legislation the government minister said they were looking for a business friendly
Q4) Amber Thomas, JISC: Is there potential to look at people logging in via Facebook, Twitter, Google logins etc?
One of the audience members from Central School of Speech and Drama is going to make their student portal login able by third party services.
Q5) Ben Butchart, EDINA: I understand that with the OpenID logins for sites like Twitter, the site has to declare what information is giving up and what data will be used as I understand it.
Search Engines versus Instititional Impecuniousness – David Hawking, Funnelback
David has been working in the National University of Australia for 40 years in a variety of roles but recently I’ve had to learn some of Amber’s dirty marketing words. I joined Funnelback which was established in 1999 and we hope to have about 8 members of staff soon.
Who are we? David gives us an overview of the rich history of the UK academic community. But academia is under huge pressures to perform. Fee paying foreign students have become a big focus of the university budgets. These students make a major contribution to economies. They aso increasingly look to force cash contributions from students.
HE Institutions are businesses and there are two fundamentals to running a successful business (a) need to maintain and increase revenue and (b) needs to increase staff productivity (without lowering standards). The need to attract better students, more funding, undertake commercialisation and reduce unnecessary time/costs.
Some examples: London Metropolitan University
If I search for a subject I will get a set of search results related to the courses as a sub section of search results. This is trying to deliver those extra conversion.
Another example: Universite Paul Sabatier
A comment already made today: universities are just like an ecommerce site. A user comes in, looks around and leaves the site unsatisfied. There’s nothing worse than a site that gets lots of data from you but gives you nothing more. You want to improve your conversion rate.
There is an opportunity to adapt results or presentation to a visitor’s region based on IP address. Issue: matching careers with courses – people will be searching for courses by job titles.
Those were course finding examples but now I’m going to talk about expertise finding. This project is Australia’s Knowledge Gateway combining 8 universities in the country. All Australian institutions need to opt into the HERDC – this is much like the REF. They are collecting and reporting all that data but it can facilitate discovery of strength, attraction of staff, attraction of graduate students, industrial investment, endowments. How do businesses, potential research students etc. find out who they should contact for more etc.
So here is the Autralia’s Knwoledge Network. This ranks researchers based on activity around their work.
This tool will be rolled out so that each institutions can use this in their own scoped context.
Image and marketing – websites are vital to marketing a university. Search is vital to web experience. Key differences between universities and corporates. Websites may contain critical information but may not confirm to corporate image. Arguably papers shouldn’t be presented only subject to the needs of the corporate image though.
Business Intelligence – what are people searching for on your sites? Are your campaigns working?
Productivity wise – student enquiry handling is always near the top of university search logs: access to course materials, course planning, accomodation, exams, timetable., library etc.
Many universities now make video recordings of lectures – do people want to access lectures by day and time or also by topic or concepts?
Staff efficiency – locating policies and procedures, staff directory, online services, parking permits, leave forms, gathering material for grant proposal, finding collaborators. An issue: new staff and position handocer, search of restricted access material, academic staff are often away from the office.
Tools for the SearchMaster
Tools like autocomplete, alternative suggestions for spelling, synonyms, and query blending. Another common problem is that there may be nothing on our websites – we can help by providing “best bets” of what we think is being sought.
Tune your ranking – established methods for doing this and can make substantial impact. Test sets here that can be used – e.g. accomodation queries, etc. and really increase effectiveness. There are hundreds of websites, millions of pages to track.
A researcher requests “The Emmottiser” which tries to explain for the query what results appear and why. People actually get surprised about lack of traffic even when their page on the library does not include that text at all. This tool helps suggest words to include in your page.
“The accessoriser” the Australian government has required WCAG2 compliane testing and reporting over time can be build into your search. Not all of these things can be reported on automatically.
Search profile – should search look the same for all visitors to the university webpage or should they have differentiated places.
Successful search deployment requires the right ingrediants, the right process, a dash of presentation and tasty presentation.
Q1) Ben Butchart, EDINA: I liked the example of a PhD looking for a researcher or supervisor. I guess a lot of institutions had to sign up to that, was wondering about the politics of that
A1) The Group of 8 are keen to have the credit for setting up that search in the first place. All institutions have to have that data available, all are obliged to make that data available. The Group are also keen to bring in other institutions.
Q2) Dan Watson: how much of this is out of the box, how much of that is customised
A2) Most of that is essentially out of the box… in the coming version (in August).
Q3) With the autocomplete it seems like that was both query selection and answer suggestions
A3) Yes, you need targeted suggestions weighted to the content on the university site. We suggest replacement text if a letter is wrong. We add in results for other areas (e.g. staff info) as possibilities – we’ve had some ideas about how that one box completion could help. But there is a downside of that – may miss out on log information as you may go right to a page rather than a results section. Maybe the speed of access will improve conversion though.
Q4) Please talk dirty about sales and the like. I carried out a survey of search engine usage in Russell Group Universities. Google is the most common option to go for, a mix of others in use. If people like what they see how can they persuade managers etc. to look into this.
A4) Funnelback’s first customer was the National University of Australia but after a while we left the university market alone for a while as Google was there and free. But universities are more interested again in how to tailor search better on their websites. If you look at what you want to achieve from search – business objectives, who the audiences are, high rates of conversion, that’s where we can come in.
Next up we have coffee and then Parallel Sessions – I am giving one of these – in the main theatre of all places – so expect an end to blogging until Day Two!