Adventures with joined-up culture data

Jon Pratty is Relationship Manager (Digital and Creative Economies) of Arts Council England. Earlier this month, he drew on his vast experience to discuss linked open data and culture with the Open-data Brighton and Hove Group. This is an edited version of his talk.

I feel the same kind of excitement about Open-data Brighton and Hove as I did when I first got webby back in 1993-94.

In those days, we sat isolated, unconnected, on islands of digital stuff, gradually brimming half-meg hard drives, occasionally making it into public places via strange things like Geocities or The Well.

Training as a journalist in 1995, I realised joining up people and content was a key dynamic of online journalism.

Sitting in The Telegraph newsroom in 1998, I saw ticking away on a screen and knew it was one of the best things I’d ever seen.

It was aggregated content from a few newsrooms then sending out feeds. It is still one of my key sites; it is like standing on top of a hill and clocking content as far as the eye can see.

Taking my online skills to 24-hour Museum [now] in January 2001, I wanted the site to plug into the newsnow dynamic.

By 2003, our tech wizards at SSL had built us the first dedicated RSS feed of museum and gallery content in the UK, and probably beyond.

Lots of lessons were learned about the importance of titling and keywording remotely-published content, and also when to publish, depending on when your RSS feed refreshes.

An early lesson was to trust your instincts. Most people in public services and higher education said we should use RSS 1.0 or Atom for feeds.

A sideways glance at the mushrooming RSS culture in publishing and journalism showed RSS 2.0 was the way to go. So that’s the way we went. Go with the majority when it comes to joining things up. Don’t put your money into Esperanto when everyone is speaking English.

Other people in the culture sector wanted to join things up, too.

In 2002, the 24-hour Museum’s funders, the Museums, Libraries and Archives Council [MLA], led a project to join up some databases to allow cross-searching.

At the 24-hour Musem, I could see it was a good idea, but the tech used needed to be rich and useful, yet universally accessible. Money was given by MLA and the project was called the 24-hour Museum Metasearch Project.

Learning from that? Well, it’s still ongoing.

Back then, the tech chosen was OAI harvesting, with Z39.50 as a standard. Baffling for me, because I’m not into the tech. The big question in my mind was: why would you do it?

The project leaders decided to join dissimilar data sources from different subject museums. I wanted to know how you bring to life content from different sources, not how you patch together the data.

It seems so simple today. Why join databases together with great big pipelines of mechanical connections – when, really, the first discussion should have been audience-focused.

Someone needed to ask: “What do we want to say? And to whom do we want to say it?”

Also problematic was that data interactions were not live. We’re talking about harvesting here. The databases we joined up were speaking to each other only once a week. This was not the real-time web we love today.

Perhaps the biggest problem for me was that museums or galleries had to pay to engineer a connection to the metasearch project.

Just imagine paying £5,000 to join Twitter!

At some levels, these sort of projects are still being suggested even now. It’s got to be free or at least very easy for cultural organisations or individual artists to export data. The next generation of open-source CMS need to have options for multiple data outputs, RSS, API, or whatever.

The metasearch project pilot led eventually to CultureGrid, our current integrated culture data collection, via something called the Integrated Architecture Project and the People’s Network Discovery Service. These were waypoints towards a stronger strategy that is now morphing into interesting things all the time.

While CultureGrid still has its roots, unfortunately, in OAI technology, there are some cool things being done with a company called Knowledge Integration, which has built something called a terminology engine on the side of CultureGrid that may one day operate as a kind of taxonomy generator.

So what’s the latest picture in terms of cultural data online?

Basically, it’s coming together. Efforts within the UK and US museum geek community concentrate mainly on discussing [endlessly] ways to engineer linked data perfectly.

This might produce wonderful connections between content one day. And it could be a pathway to the lovely patch of sunlit digital downland called the semantic web.

But here’s a reality check: there’s lots of data out there to work with and try to join up in meaningful ways for audiences.

As I saw with the metasearch project and CultureGrid, it’s easy to join similar databases using one technology.

The hard thing is to join dissimilar content in meaningful ways: older content, legacy stuff, archives, .pdfs, ancient databases, different types of files, weird digital standards.

This is the real gross pathology of the digital landscape.

Two years ago [as a consultant] I proposed to JISC a relatively simple data-mining and indexing effort to bring back to life £70 million-worth of Lottery-funded heritage and museum websites, two-thirds of which now lie sleeping but with useful data and content. It wasn’t greenlighted; it wasn’t a great proposal.

But I still think data-mining, powerful search techniques, vocabularies, taxonomy work, and lightweight indexing will help us join data together more effectively than building clunky connections between sources of data.

So what’s out there, right now, for free, in Brighton and Hove?

Plenty: copyright-free culture news, listings, venue info, features, reviews, blogs and more. Lots of RSS 2.0 content and an API output of culture data can be got from sources like Culture24.

Sounds fantastic! But is it really? Some big issues are now appearing over the horizon. What do people want from data like this? Who wants it anyway? Developers? Publishers? Hyperlocal news sites? Individual web-users? Looming large in any open-data conversation needs to be the issue of trust.

As punters with smartphones at the ready, we expect info about places, times, trains, buses, gig tickets and so on to be accurate. It’s got to be trustworthy.

My learning from Culture24 was that info about culture vitally needs to be correct. Families travel to make museum visits. The info they use has to be accurate. Culture24 have 10 years of data, content and really great relationships with thousands of museums, galleries and heritage sites all over the UK.

If arts organisations want to join the information-publishing space and partner with media organisations, for example, they’ll be expected to offer guarantees of quality and availability in a service-level agreement. Arts companies like Culture24 understand how to motivate museums and galleries to enter and check their own listings, venue and exhibition data.

That last point is key. The experts about info about an event are the people running it. They know when it’s happening. If these people are running lots of events they can control the quality of the data they are exporting. Users of the info can trust it to be right. That’s where brand values begin, with data services. Is it right? Yes, we checked it.

Does this cost money? It does, but in culture venues this could come under the overall marketing costs. It’s not a reason to give the job of getting listings right to other people.

Just think: if you are the one really famous museum about Elizabethan culture in Britain – like the Mary Rose Museum – you’re in the best place to be the one true source of info about that subject.

You have what I call data equity. You’re the one place where the real thing is kept. You own the island. If you get the basics of your data right – titles, content, quality, standards – you’re the go-to guys for that data. At that point, since you control IP, quality, consistency, you can make partnerships with other media, or offer it free, or do what you want.

It becomes another part of your cultural offer. It’s a valuable commodity. It’s a data brand – something we need to consider how to market, and how to signify in the future as having value.

This is all quite new really. Many funders and most arts organisations don’t have a data-sharing policy or strategy. It’s evolving as we go along.

If arts organisations want to be out there, we need to be able to offer reliable, trustworthy, data that matches what others are putting into the open-data mix.

Posted in Talks | Leave a comment

311 services and “citizen-relationship management”

Instinctively, I believe Brighton and Hove should have its own “311” service, a means by which citizens with non-emergency needs can access the information they require about public and community services, in an easy and speedy fashion.

Furthermore, I feel that such an innovation is intimately bound up with the creation of an open-data city. Data generated by citizens seeking a service can identify gaps in provision, build a knowledge base for immediate use, and inform strategic solutions in the longer term.

The purpose of this post is to give an overview of “311” services, particularly in the United States, and to support those who argue for such a service to be introduced in Brighton and Hove.

The first 311 service appears to have been introduced in Baltimore, Maryland, in 1996; the biggest one, in New York City, was initiated in 2003.

The 311 number was reserved for non-emergency municipal services throughout Canada in 2004; the first Canadian service was in Calgary, Alberta in 2005.

My interest in 311 and open data was reinforced by a remarkable article in Wired magazine in November 2010: What 100 million calls to 311 reveal about New York. It includes this paragraph:

“Launched in March 2003, 311 now fields on average more than 50,000 calls a day, offering information about more than 3,600 topics: school closings, recycling rules, homeless shelters, park events, pothole repairs. The service has translators on call to handle some 180 different languages.”

Unsurprisingly, customer satisfaction is outstanding.

Last year, several of the chief information officers leading the open-data cities movement in the United States threw their weight behind an “Open 311” initiative.

Vivek Kundra, the first US chief information officer (a pioneering appointment by Barack Obama in March 2009), said:

“Too often, people grumble that their complaints about government – be it city, county, state, or federal – get swallowed by the bureaucracy. Open 311 is an answer to that problem, placing the role of service evaluator and service dispatcher in the power of citizens’ hands.”

The Open 311 approach enables new web-based applications that use real-time data to allow citizens to track the status of repairs or improvements, while also allowing them to make new requests for services. The possibilities of such an approach are articulated in early services such as SeeClickFix.

There are already significant lessons to be learned from the North American experience. Specifically, it is clear that first-class, round-the-clock service – by telephone, web and mobile – is fundamental, supported by robust citizen-relationship-management software.

In the United States, the Public Technology Institute has designated nine local governments as “citizen-engaged communities”. The resources the institute provides have included Six Key Strategies: Multi-channel Contact Centers.

Closer to home, Vicky Sargent‘s work for Socitm (Society of IT Managers), as reported in The Guardian, seems to point in the right direction. So does the appointment of Joe Harley, as the UK’s chief information officer.

But it’s cities such as Brighton and Hove that can lead the way. And a citywide 311 service could be a great first step. What do you think? Email

Posted in 311 services | Tagged | 1 Comment

Inaugural ODBH meeting on February 8 2011

There was a great turnout for the inaugural meeting of the Open-data Brighton and Hove group at The Quadrant in Queen’s Road, Brighton BN1 3FA.

This post does not seek to be anything other than a bullet-point list of action points or issues discussed by the 40+ people who attended. By summarising them, I hope it will help inform discussion at our next meeting on Tuesday, March 8.

The main points and issues, which I’ll add to in the light of any feedback, are:

  • Catalogue and categorise Brighton and Hove data that is already, to some degree, “open” (with a particular focus on data held by instituitons/organisations that receive public funding);
  • Identify various strands/themes of interest to ODBH members, including: transport, health, education, voluntary/community, police/crime, cultural (museums, galleries etc), environment;
  • Mobilise all-party, non-partisan support for an open-data Brighton and Hove from canddidates in the city council elections on Thursday, May 5;
  • Possible creation of a Google spreadsheet where datasets – and their uses – can be identified or recorded. Will a Google Group add value (in addition to this blog and the Open-data Brighton and Hove Meetup page?
  • Use of Twitter hashtag #ODBH. See also @OpendataCities;
  • Preparation for CityCamp Brighton on March 4-6 and the £10,000 prize;
  • Delicious links relating to open-data cities;
  • Research into – and analysis – of most prominent open-data cities, including San Francisco, New YorkWashington, Toronto; also monitor/collaborate with other open-data city campaigns such as the one in Manchester;
  • Does Brighton and Hove need its own CIO and/or its own Chief Digital Officer, similar to Rachel Sterne in New York? Does it need its own datastore?
  • Which institutions/organisations have a leadership role in the move to an open-data Brighton and Hove? Pivotal role of Brighton and Hove City Council; Paul Colbran (@PaulColbran), the city council’s Head of ICT, contributed to the meeting, for which thanks;
  • Is there a need for a “311-style” non-emergency service? See: Open311 and “What 100 million calls to 311 reveal about New York“;
  • Possible future speakers at ODBH: Chris Taggart (@CountCulture), of OpenlyLocal + OpenCharities + OpenCorporates (provisionally booked for Tuesday, March 8). Also Roger French, managing director of Brighton Buses? A CIO from a North American open-data city? Who else?

The inaugural meeting attracted significant interest. Here is a selection of articles, reports and blogs:

My next post will try to highlight some links to open-data cities around the world – and to articles about developments in such cities.

Posted in Meetings | Leave a comment

Open-data Brighton and Hove

In advance of the inaugural meeting of the Open-data Brighton and Hove group, I thought it would be helpful to start a blog on which I could record and debate developments in #Opendata. Links to articles on the web will be stored on the Cogapp Delicious links for Open-data Cities.

Posted in Uncategorized | 1 Comment