Mar 242011

Tomorrow I am attending the ScraperWiki Hacks & Hackers Day Glasgow (#hhhglas) and thought I would gather some resources here on the blog for my own reference and for any fellow social media types who like to play around with mashing up various APIs, feeds, data etc.

ScraperWiki are the organisers for tomorrow’s events but I’m pretty new to ScraperWiki and most of my data munging experience has been with UNIX scripting and Perl (rather than Python or Ruby). I really do like the visual pseudocode experience of tools like Yahoo! Pipes but there are loads of online tools for transforming, manipulating and combining data and even Google Docs can be deceptively useful for working with data.

I gave a presentation at Haggis and Mash this year highlighting various EDINA APIs so for more information on our APIs and machine harvest-able services (not all listed below) take a look at my slides:

  • AddressingHistory API (Query 1765, 1865 and 1905 Edinburgh Post Office Directory data with geolocation info)
  • Jorum API (Access educational resources in Jorum based on collection, submitter, etc.)
  • Open Access Repository Junction APIs (information on HE and FE organisations and their repositories, mainly but not exclusively UK)
  • OpenStream API (Ordnance Survey Open Data products for UK HE – requires free registration)
  • Unlock (middleware service) lets you geocode text and placenames – you can do similar with the Data Science Toolkit

Obviously I have omitted a lot of very useful social APIs and tools like Google maps etc. I’ll try to add to this list with other (less obvious) useful APIs and tools but if you have some ideas for using any of the above or can recommend some great data or tools I should be trying out do leave a comment below.

 March 24, 2011  Posted by at 6:50 pm Social Media at EDINA Tagged with: , , , , ,  No Responses »
Mar 222011

EDINA are currently running a project called “Linked Data Focus” which is exploring how our projects and services can engage with Linked Data – whether as new published data sets or schema or as a way to enhance and improve our own content and services through connections to other data.

Part of my role is to look at new technologies and ideas that we could be working with. Linked Data offers some really fantastic opportunities for creating new services, tools and websites from data that is distributed across the web and across institutions. There is fluctuating interest in these technologies but the core concept that data should have context, should be interoperable (something I particularly appreciate as a former data munger* for SUNCAT) and should connect to other relevant data/metadata/resources seems very robust.

Last month I was delighted to be able to attend a “Linked Data Learn-In” held by the LD Focus tean as I was keen to hear how our own work was taking shape and to hear from the guest speaker Wilbert Kraan of JISC Cetis.  I have just posted up my notes from the session on the Linked Data Focus Blog – do take a look. (Although the notes may look quite technical/acronym filled non techies can be reassured that I have loaded the post with through to project and tool pages, documentation and Wikipedia definitions.)

*Yes, “munging” is a real word and it refers to programmatically tweaking, manipulating and standardising data or metadata. SUNCAT collects data from almost 80 libraries so there is a fair bit of behind the scenes magic required to make all that metadata work together in one catalogue.

 March 22, 2011  Posted by at 4:02 pm Social Media at EDINA Tagged with: , , ,  No Responses »