How open data has changed journalism

To mark Open-data Day (December 3), solicited my views about how open data has changed journalism. This is what I wrote:

Data – whether open or not – has always fuelled journalism. Data that is increasingly “open” (in the fullest sense of the term) will transform journalism.

Ironically, a lot of the best revelatory journalism of the past has depended on journalists unearthing data (ie “stuff”) that others want to keep locked. Ideally, by lawful means. Therefore, openness may remove some of the mystique that journalists delight in, as people who know things that only those “in the know” know.

An open-data tsunami will mean that more journalism will be about interpreting – and putting into context – data that is open to all, at least in its rawest, unrefined form.

To an even greater degree, journalism will be about adding value to data by transforming it into information. The best journalism will be to add value to information, to provide insight, even wisdom.

Openness of data will change the behaviour of individuals and organisations. But not immediately and not in every case. Would MPs have played fast and loose with their expenses if they knew data about each claim would be published openly and in real time? Sad to say, it is quite possible some would.

Much good journalism has involved shedding light on data that was routinely (although not widely) available and which was only rarely studied or analysed.

Importantly, some of the best journalism has involved making connections and spotting patterns. I’m thinking of earlier parliamentary abuses, such as the “cash-for-questions” scandal of the mid-1990s, before Hansard was on the web, and when it was rarely read in print by journalists.

Those were the days when typewriters and telephones – rather than computers and the internet – were the primary journalistic tools. When bars and restaurants – rather than offices and desktops – were the venues for journalistic enterprise.

With more data openly available – along with more tools easily available for mining, sifting and interpreting it (as in the case of the Wikileaks material) – there are many more needles to be found in the burgeoning haystacks of unstructured data.

But even when every day is #Opendata Day, the best stories may remain hidden in full public view – until one of the new generation of journalists stumbles expertly across them.

Posted in Uncategorized | Leave a comment

Aristotle and an open-data Brighton and Hove

Aristotle got it right.

“Man is a political animal,” the ancient Greek philosopher wrote nearly 2,500 years ago. He believed human beings were suited to living in a “polis” [πόλις] or city-state – large enough to be self-sustaining, but small enough for lives to be lived on a human scale.

Ancient Athens, for example, had a population of roughly 250,000, similar in size to Brighton and Hove. A [male] citizen could walk across it in a single day, frequently bumping into people he knew or recognised, many of whom took decisions that directly affected him.

Fast forward to the 21st century and cities dominate the planet. By 2050, 70 per cent of the world’s population will live in cities.

Increasingly, the most successful cities will be the ones that are the most connected, where digital technologies are seamlessly integrated into everyday life, as omnipresent and as invisible as the air we breathe.

Such technologies will enhance and enrich not only the complex network of relationships between citizens, but also the connections between cities. The world will, I believe, be characterised more by a network of networked cities than by the current patchwork of nation states.

This transformation has already started, particularly in North America, with cities such as New York, Washington, San Francisco and Vancouver leading the way in a global “open-data“ revolution.

So what is an “open-data city”? In Brighton and Hove, all political parties have agreed the value of public data is greatest when it is freely and openly shared, without unnecessary licensing restrictions.

They said: “We envisage a city in which every individual and business can use and re-purpose public data to help create a more vibrant and sustainable future, with more efficient public services, more effective voluntary organisations, and more enterprising private businesses.”

Our city is leading the country in moves towards opening up data so that the outstanding expertise of local citizens can build the applications and services that will help shape all our futures.

In effect, data is the “straw” that makes the bricks to build the walls of mansions that are currently unimaginable.  Just as it was impossible to imagine Google or Facebook when the web was in its infancy in the early 1990s.

Therefore, the Open-data Brighton and Hove Group – which has more than 100 members, including developers, designers, usability professionals, journalists, and artists – is organising the UK’s first Open-data Cities Conference.

To be held in Brighton and Hove next April, it will bring together protagonists from the 20 or so cities that also have populations of 250,000 or more and – like our own city – are unitary authorities or metropolitan districts with significant responsibilities for public services.

The conference, with more than 200 attendees from across the UK, will look at the potential impact of open data on public services, media and culture, and the voluntary and third sectors.

Posted in Meetings | Leave a comment

Open data: last chance for city newspapers

In a previous life, I was the first Fleet Street journalist to leave national newspapers for the internet in the mid-1990s.

Consequently, I was delighted to be asked to write what turned out to be the cover article in InPublishing magazine about  how the rise of open-data cities may offer local newspapers the last chance to re-invent themselves.

As I prepare for the next Open-data Brighton and Hove Group meeting on Tuesday, September 13, I wanted to explain how I see how the media landscape might – or could, or should – change in our city in the coming years.

You can full read the full article here. Or download my podcast here.

Open data: last chance for city newspapersLooking at the plight that many newspapers find themselves in, I wrote: “Put bluntly, I believe the many of the businesses that run local newspapers are guilty of serial breaches of trust.

“They have violated the trust of journalists – by paying them poorly, equipping  them badly, and failing to provide a viable platform on which they can exercise  their journalism; they have violated the trust of their communities – by cutting  costs and withholding investment, thus failing to satisfy the democratic needs of active and informed citizens; finally, for what it’s worth, a number of executives appear to have broken faith even with shareholders, by seeking to suck as much value as possible out of a moribund industry – in salaries, pensions and share options – while privately recognising that the game is up. That the party is over.

“The world has changed. And media companies have failed to re-invent themselves to take advantage of an unprecedented level of disruption.

“So what is to be done? And where to begin?”

Focusing on our own city, I argued:

“Brighton and Hove – like other open-data cities – believes ‘the value of public data is greatest when it is freely and openly shared, without unnecessary licensing restrictions’. All political parties in the city have signed up to an open-data manifesto that envisages ‘a city in which every individual and business can use and re-purpose public data to help create a more vibrant and sustainable future, with more efficient public services, more effective voluntary organisations, and more enterprising private businesses’.”

Against the backdrop of falling circulations and revenues, he suggests city newspapers should re-invent themselves. Ceasing to be products, they could become providers of data-driven services – not only on the web, but also on every device imaginable, including internet-enabled television sets in every living room

I concluded: “They could catalogue and collate the data, publish the data, store the data in ‘data mines’, visualise the data, enhance the data, and link the data from which services yet to be dreamed of will be built, for the benefit of local businesses as well as consumers.

“This, however, will require traditional media companies to re-invent themselves, to re-establish trust with their employees, partners and communities, and to work with public and private organisations of all sorts.

“Once again, they must invest in journalism, journalists, and journalistic enterprise; they must give their journalists the tools and the technologies to do the job, to meet the needs of their communities. They must not outsource innovation – but they must accept they will need outside help to be successful.

“If they do this, if they work with those striving to shape the future of our open-data cities, they could once again put themselves at the heart of the communities they seek to serve.”

Posted in Media | 1 Comment

Update: Examples of Brighton and Hove data

Some helpful work by a number of #ODBH members – particularly by Paul Brewer (@pdbrewer) – has inspired me to compile in one post some of the links that have been tweeted by various people since last week’s meeting.

Most recently, with an appeal for people to research the websites of all Brighton and Hove public organisations, Paul has made an impressive start on an “Open Data Wikilist“.

Glyn Huelin (@glynhuelin): 2003 election results in a datasheet. He sourced these from

This fits nicely with Paul’s work on the 2007 election results in Brighton and Hove. Which were sourced from here, on the city council’s website.

Paul also re-processed school-level GCSE results trends 2007-2010.

Mark Sheppard (@m_sheppard) linked to this map of the 21 wards in Brighton and Hove

That’s all for now. But lots more to come, I’m sure.

Posted in Elections, Meetings | 2 Comments

A good meeting – and some election action

Despite some confusion by the barman (!), our meeting upstairs at the Western Front last night provided food for thought – and nutrition for action.

Another remarkable turnout from officers of Brighton and Hove City Council, led by Paul Colbran, resulted in optimism that details about the 179 candidates for a total 54 council seats of in a total of 21 wards will be published shortly by the local authority as open data (either .csv or .xls).

Today, I’d be grateful for any feedback what occurred to me overnight, namely that the basic data should include at least:

Candidates (possible datafields – based on information required on “Form 1a Nomination Paper”):

▪                Electoral division/ward

▪                County/district/London borough/county borough

▪                Date of election

▪                Mr/Mrs/Miss/Ms/Dr/Other

▪                Surname

▪                Forenames

▪                Commonly used surname

▪                Commonly used forenames

▪                Description

▪                Home address 1

▪                Home postcode

In addition, I am also envisaging data should be published openly about polling stations, including:

▪                Electoral division/ward

▪                Polling district

▪                Polling station

▪                Polling station address 1

▪                Polling station postcode

For reference, Cogapp – the agency I work for – built this application for the General Election for The Sunday Times last year (launched in the run-up to election day and updated in real time on the night):

I’m sure someone will create something more granular for Brighton and Hove – if we can get the data out. If and when the data is released, I’m certain we can together trumpet its availability to the local developer community and showcase any good work done.

I haven’t been able to find the recent innovation/visualisation I dimly recalled and referred to at last night’s meeting.

But the following example by Stroud would have been wonderful…..if only the data had been in .csv or .xls and not .pdf!!

Eden council does it quite neatly:

Please get in touch if you who see any good examples of open local elections data for 2011 – and/or if you think other data relating to candidates, polling stations or wards might be helpful.

ODBH members are probably already aware of these:

Separately, the main party websites locally are:

▪                Conservative Party:

▪                Green Party:

▪                Labour Party:

▪                Liberal Democratic Party:

Since last night’s meeting, Paul Brewer (@pdbrewer) has re-processed the 2007 election results in Brighton and Hove into a flat spreadsheet. The website version is here.

Finally, our meeting benefited from a surprise visit  – and valuable input – from London-based Glen Wintle (@glynwintle), of the Open Rights Group. To whom, I pass on all our collective thanks.

Posted in Elections | 5 Comments

Categories for an open-data catalogue for Brighton and Hove?

In advance of the next meeting of the Open-data Brighton and Hove Group on Wednesday, April 13, I thought it might be useful to analyse the categories/classifications of data adopted by a number of local authority “datastores”.

I have looked in detail at the approaches adopted by: Kent County Council; Lichfield District CouncilLondon Datastore; Manchester City Council; Trafford Council; and Warwickshire County Council. (Interestingly, – unlike in the United States – relies on tags and free-text search rather than structured categories.]

My analysis shows a total of 60 categories, including near-duplications (marked with an asterisk *). You can see the full analysis in a two-worksheet spreadsheet.

From these – and any other (?) – categories , I hope the Open-data Brighton and Hove Group will be able to agree a broad framework, appropriiate to the specific context of our city, that could form the basis for a data catalogue (albeit one that constitutes, at this stage, more of a wish list than a reality).

Once a framework is agreed, we could perhaps begin to identify the sub-categories into which datasets might be catalogued, eg Art and Culture/Libraries or Education/Schools etc.

Art and Culture
Benefits and Council Tax
Births, Marriages, Deaths
* Boundary/Area-based
* Boundaries
* Business and Economy
* Business Support and Licensing
Championing London
Children and Young People
* Community
* Communities and Neighbourhoods
Council Tax
* Crime
* Crime and Community Safety
* Crime and Public Safety
Customer Access
* Education
* Education and Learning
* Education and Schools
Elections and Democracy
* Employment
* Employment and Skills
* Environment
* Environmental Services
Events, Leisure and Tourism
* Financial Information
* Finance
Food Safety
General Information
Heritage and Culture
Jobs, Careers and Training
* Leisure
* Leisure and Culture
Local Democracy
News and Events
Parking, Travel and Roads
* Planning
* Planning and Development
* Recycling, Rubbish and Waste
* Recycling
Supplier Spend
Sustainable Environment
The Council and Democracy
* Transport
* Transport and Highways
Young People
Posted in Categories | 6 Comments

An historic moment: all-party support for Open-data Brighton and Hove!

It was a truly historic moment – and a fantastic result! And I’m not talking only about Wayne Rooney’s goal last night in Manchester United’s victory over Chelsea.

In fact, I missed most of the game, because I attended the Brighton and Hove Chamber of Commerce’s hustings at City College.

The event – a full house – marked the moment when leading figures of all four main political parties formally signalled their endorsement of the following statement of support for an Open-data Brighton and Hove:

“We support efforts to transform Brighton and Hove into an open-data city – a community in which all publicly-funded organisations strive to engage with citizens to build a more creative, prosperous and accountable city.

“We are committed to doing everything we can to ensure Brighton and Hove City Council takes the lead in soliciting the ideas, input and creative energy of all our citizens.

“We believe the value of public data is greatest when it is freely and openly shared, without unnecessary licensing restrictions.

“We envisage a city in which every individual and business can use and re-purpose public data to help create a more vibrant and sustainable future, with more efficient public services, more effective voluntary organisations, and more enterprising private businesses.

“We are convinced the creation of an open-data Brighton and Hove will not only increase public participation and economic activity, but also promote transparency and accountability.

“As a result, we believe an open-data Brighton and Hove will be a better-informed, more openly democratic, inclusive city.”

Present last night and/or consulted were:

  • Conservative Party: Mary Mears and Brian Oxley;
  • Labour Party: Gill Mitchell [and Mel Davis];
  • Liberal Democrat Party: Paul Elgood [and Jonathan Eke];
  • Green Party: Bill Randall and Jason Kitcat.

As members of the Open-data Brighton and Hove Group know, you will know the statement of support draws on the recent declaration by Enschede in the Netherlands, as well as a version of the motion originally adopted by Vancouver and the words of President Barack Obama when launching the open government initiative in the United

States in 2009. After the next meeting of the Open-data Brighton and Hove Group on Wednesday, April 13 – at a venue still to be decided – we will seek other signatories (eg city MPs, individual council candidates, leaders in other public organisations).

I have already alerted The Argus to this remarkable development. Which attracted attention from across the country when I tweeted, from the hustings. 

Finally, I will be talking about the Open-data Brighton and Hove at the following events: OpenTech 2011 on Saturday, May 21 (along with fellow open-data activists from Manchester and News Rewired conference on Friday, May 27.

Please retweet this blog post and encourage members of your network to seek support from every candidate in the forthcoming city council elections.


Posted in Announcement | 3 Comments