Presentation/Discussion: Eric Gundersen, Development Seed (www.DevelopmentSeed.org)
Wed 28 July, 9am
Rapporteur: Tanya Notley

‘Tools and strategies for data collection & visualization’

Eric started by sharing some of his personal background and explained how his studies and interests first took him to Peru in 2003. Fujimore had just been ousted and the political climate was changing, while internet access was also increasing. He took open source tools being used by Howard Dean (‘Drupal powered Dean Space’) in US and introduced these to NGOs in Peru to see how they could be utilised in micro-finance. It was this experience that led him to co-found Development Seed (http://developmentseed.org/). [Eric has a master’s degree in International Development and has dual bachelor’s degrees in Economics and International Relations.]

Today though he said he’d talk about large datasets to allow for a discussion on challenges researchers in room facing with this. He also wanted to discuss the value of open data.

At Development Seed they use open source tools, particularly Drupal (http://drupal.org/), a Content Management System. They work with government bodies, international institutions and large NGOs and they build websites (closed and open); most of these sites include a map which uses open data to visualise information.

What is Open Data?
July 4, 1966. Freedom of Information Request law first signed by Lyndon Johnson. In many ways this lay the foundation for a more open government in the US context and also for open data. [for more detailed explanation see: http://bit.ly/9X6EB5]

More recently DC dataset took things further when they provided all kinds of datasets on a website to the public. They launched a Competition ‘Apps for Democracy’ and announced ‘we are giving you some sets of data’ and you have 30 days to build a tool that uses this data.

This was an experiment to see what would happen and it went against the dominant practices at the time where this kind of data was not made really accessible. Perhaps some of it was available in different places, or you could access it with a FOI request, but it wasn’t made available in a usable format in the one place.

So what happened?

Some people built a map that was a carpool finder, another built a a parking meter finder. Eric worked on the mapping of bike routes, shops and bike theft along the roads. (http://outsideindc.com/bikes). They also built a more light-hearted ‘stumble safety’ where they mapped all the bars around their office. You can filter events based on time you are drinking and you can then find out thefts and other crime which appear around the bars. Stumblesafely.com

But actually there was an unexpected result of doing this mapping. Some time later someone told them ‘you know the cops really love your site’. They had started using this map to see where and when they would patrol, which makes sense. But the way people might use your data is not always unproblematic. In another example the police changed the way they reported things because they knew how this data was being used.

So by opening up this data, the city benefited by putting this info out there and allowing people to do something with it.

Eric believed that when governments open up data this supports innovation within the private sector and supports the government because people use their data in ways that can be helpful, useful.

Nov 19th 2009
Karzai is sworn in as President in Afghanistan. The govt bodies working on election monitoring opened up data about votes (but he says sadly in PDF format). So number of datasets were available: master polling centre list (where they are and for what districts); Premium results; The ECC complaint data. So there were three open data sets. Development Seed were employed by National Democratic Institute (http://www.ndi.org/) to map this data so they could understand voting patterns better. The particular aim was to locate and visualise voting irregularities from the elections. Initially they made this as private site, but showing partners. Later they made public. http://afghanistanelectiondata.org/data. (technical walk though here: http://developmentseed.org/blog/2009/dec/17/opening-afghanistans-election-data-open-source-data-browser)

[Note from Eric: Adobe PDF not a good way to open up data! Need crazy scraping tools to use data embedded in a pdf!]

Development Seed started by ‘Geocoding’ the voting open data they had. Then they mapped this data so people could filter the information by location, ethnicity and voting patterns. People can export datasets for their own use as well from the site.

The site was built with tools they have developed over time (Drupal + Managing News) but generally for a project like this you can say it will cost around $100,000USD for customisation. While the tools they develop are open source, organisations pay for their add on services, expertise and customisation.

What happened?
Created snow maps of country, using radar and satellite imagery from NASA and US military: Some specific blog posts go into details on how to make maps like this: 

Terrain Mapping in Afghanistan: Summer in Kabul
http://developmentseed.org/blog/2009/nov/02/summer-kabul

Snow Cover Hillshade Maps: Winter in Afghanistan
http://developmentseed.org/blog/2009/nov/05/snow-cover-hillshade-maps-winter-afghanistan

While this was useful, it’s also useful to consider what happens when you allow people to add data on the places they live. This is the value of Open Street Maps. “the OSM community is huge and growing” [watch this video showing OSM edits made in 2008:http://vimeo.com/2598878]

Eric mentions that the founder of OSM just got series B $12.5 billion funding for CloudMade, which adds value on top of OSM. So this shows it’s also possible to build services around the OSM data. www.cloudmade.com/press/2010/07/27/cloudmade-raises-12-3-million-in-series-b-funding/

Last spring over 141,000 miles of new mapping data was contributed to OSM from an organisation called Africover (FAO).

More examples:
Layers in OSM of Haiti [http://haiti.openstreetmap.nl/]
US Geological Study (GS) showing earthquake intensity.

In Haiti, good maps did not exist when the earthquake struck earlier this year. This is one sign of the lack of stable infrastructure which in the end led to so many deaths when the earthquake hit.

At Open Street Maps parties people sit round and trace satellite imagery. [to watch how to do this: http://vimeo.com/3365503]

So after Haiti and the base layer maps were created, people started adding collapsed buildings which was info being collected by various people on the ground. Here is more on Open Street Mpas in Haiti:

What’s Next for the Humanitarian OpenStreetMap Team:
http://developmentseed.org/blog/2010/jul/15/whats-next-humanitarian-openstreetmap-team

Humanitarian OpenStreetMap Team Report from Haiti: Week 2:
http://developmentseed.org/blog/2010/apr/07/humanitarian-openstreetmap-team-report-haiti-week-2

Humanitarian OpenStreetMap Team Report from Haiti:
http://developmentseed.org/blog/2010/mar/30/humanitarian-openstreetmap-team-report-haiti

Speeding up OpenStreetMap-based Map Development with OSM Bright Template:
http://developmentseed.org/blog/2010/mar/23/speeding-openstreetmap-based-map-development-osm-bright-template

‘Flexible Data Workflow’
Development Seed came to work with an NGO (Interaction) to add value to the Haiti OSMs to show things like a ‘who is working where map.’

Haiti: Mapping Who is Working on What, Where:
http://developmentseed.org/blog/2010/jul/21/haiti-mapping-who-working-what-where

Post-Earthquake Map Tiles for Haiti:
http://developmentseed.org/blog/2010/jan/29/post-earthquake-map-tiles-haiti

Why do people, including for profit groups and government open up data?
There are a number of Incentives to do so:
1.cost saving…things might happen with your data that are useful and you don’t have to pay for this. So you are fostering innovation without really doing much.
2.contributing to knowledge…by making the data open you are effectively stopping ‘recreate the wheel’ problem where people have to start from scratch to collect the same data. By opening it up you are also allowing people to collectivise knowledge including local knowledge. They add meaning to the data when they look at it and use it in new ways.

Questions:
1. What about balance between privacy, surveillance and transparency and also between social good and commercial values? For example, I don’t want my house online, or to appear to the public sitting on my balcony on Google Maps. All this openness can be used for corporate interests as well and where do I have a say in that?

Yes private sites use this open data and sometimes also make their own closed, private, even problematic maps. Yes it’s political, maps are inherently political. What exists on a map and what does not is political. Why is voting data not available for Sudan? There is a vested interest in not releasing data as well. We need to scrutinise this, yes.

2.There is an example of a dataset of diseases and at some point someone created a map around that. Then they started to work with community groups who had experience with working with environmental diseases and they started defining the units of analysis and started doing mapping. They took this info and it became part of a political campaign that lobbied against polluters. So it’s not just the mapping and tools but the process of enabling effective use that will change adverse conditions affecting peoples lives.

Yes. How can you take data and tell complex stories. Open Layers tools allow that to happen, allow people to say ‘hey it’s interesting to look at the data with this perspective because it tells a different story’. Anyone can take open maps tiles to tell other stories. Open standards, open layers, open data allow people to do this. I think we’ll see a lot of changes in next 18 months of exactly this happening. It’s really developing.

Statement: For those of us doing mapping work at community level, one thing that would be useful is business cases for moving from PDF to open data formats. You get a lot of friction for asking people to change and move away from this format. I think people need to be convinced.

Question: What is Cloud Made?

Company run by founder of Open Street Maps www.cloudmade.com
In the past you saw private data mining companies buy datasets from govt and package it for different audiences. Now going to see different approach emerging because there are business models that can be built around open data as well.

A different kind of example is the Open Seeds ‘managing news’ software which is a news aggregator. It pulls together news sources based on the search terms you specify and it maps them. People using this are groups like human rights watch and Al-Jazeera who use it to manage news when certain events happen and they need to track and follow how and where this is being reported to be up-to-date.

www.managingnews.com. 

Customizing Maps and Geo Data in Managing News :
http://developmentseed.org/blog/2010/feb/19/customizing-maps-and-geo-data-managing-news

Managing News Beta 9 Released:
http://developmentseed.org/blog/2010/apr/12/managing-news-beta-9-released

Still govts needs to be convinced of the value of opening up data. 10 years ago they were paid very well for selling data they collected from us. And you could say fair enough because it paid for the cost associated with collecting this data. But now think those kinds of data mining companies will not become irrelevant but they will change tactic, work in different ways. After all the need for services around the data will grow not cease to exist: insurance companies will still want to buy data about x, y and z and these companies can still provide that with open data.