Open data

See also Data, Computing#Stats and big data, Computing#Data structures

General

data, noun

• facts and statistics collected together for reference or analysis: there is very little data available
  • the quantities, characters, or symbols on which operations are performed by a computer, which may be stored and transmitted in the form of electrical signals and recorded on magnetic, optical, or mechanical recording media.
  • Philosophy things known or assumed as facts, making the basis of reasoning or calculation.

• Forbes: A Very Short History Of Data Science

• http://en.wikipedia.org/wiki/Data_set
• http://en.wikipedia.org/wiki/Hyperdata
• http://en.wikipedia.org/wiki/Semi-structured_data
• http://en.wikipedia.org/wiki/Metadata_standards

• Data, Information, Knowledge, and Wisdom - some abstractions

• http://sunlightfoundation.com/blog/2013/10/02/government-apis-arent-a-backup-plan/

Articles

• http://worldofdata.org/
  • http://worldofdata.org/2013/01/11/going-to-data-visualization-school/

• http://www.ocsi.co.uk/news/2013/04/25/we-dont-need-data-we-know-our-areas/

Learning

• http://schoolofdata.org/
  • http://schoolofdata.org/data-expeditions/guide-for-guides/

Open Data

See also WebDev#API, Open, Semantic web

• http://en.wikipedia.org/wiki/Open_data
• http://en.wikipedia.org/wiki/Open_Data_in_the_United_Kingdom

• http://okfn.org/opendata/
• http://okfnlabs.org/

• http://www.reddit.com/r/opendata

• http://databridgeuk.wordpress.com/findings/

• http://linkedgov.org/
  • http://wiki.linkedgov.org/index.php/Main_Page
  • http://wiki.linkedgov.org/index.php/The_economic_impact_of_open_data

• https://en.wikipedia.org/wiki/Triplestore

• YouTube: Prof Sir Nigel Shadbolt - The Fifth Paradigm: From Open Data to Social Machines

Data sources

• http://www.w3.org/wiki/TaskForces/CommunityProjects/LinkingOpenData/DataSets

• http://www.reddit.com/r/datasets/

• http://semwebquality.org/

Hubs / platforms

• CKAN is a fully-featured, mature, open source data portal and data management solution. CKAN provides a streamlined way to make your data discoverable and presentable. Each dataset is given its own page with a rich collection of metadata, making it a valuable and easily searchable resource.
  • http://ckan.org/features/visualise/

UK

• NCVO UK Civil Society Almanac Datastore

• http://www.data-archive.ac.uk/find/hasset-thesaurus/skos-hasset This new resource is an outcome of the Jisc-funded SKOS-HASSET project, led by staff at the UK Data Archive at the University of Essex, which owns and manages HASSET. Like dictionaries, thesauri describe the changing world around them; this is why the UK Data Archive continues work to ensure HASSET is up to date. Simple Knowledge Organisation System(SKOS) makes the thesaurus machine-readable. It is the version of Resource Description Framework (RDF) specific to classification resources. It encodes these products in a standardised way to make their structures comparable and to facilitate interaction.

Government

• Data.gov.uk is a key part of the Government's work on Transparency which is being lead by the Transparency Board. Data.gov.uk implementation is being led by the Transparency and Open Data team in the Cabinet Office, working across government departments to ensure that data is released in a timely and accessible way. This work is being supported by Sir Tim-Berners Lee & Professor Nigel Shadbolt. There are a number of technical partners involved in the project to date. These include the Comprehensive Knowledge Archive Network (CKAN): CKAN runs the catalogue at data.gov.uk/data as well as a growing number of open data registries around the world. It is a project created by the Open Knowledge Foundation to make it easy to find, share and reuse open content and data. The CKAN software provides a web interface, programmer's API, feeds notifying of changes, and a browsable history of all changes. The API is documented here: http://data.gov.uk/data/api.
  • http://data.gov.uk/linked-data

• data.gov.uk: Who is doing what? - This page lists the domains which publish and maintain linked data and short term projects developing the government use of linked data. Most sectors have one or more SPARQL endpoints, which enable you to perform searches across the data; you can access these interactively on this site.

• Office for National Statistics: datasets and reference tables
• statistics.gov.uk - not open

• London Datastore has been created by the Greater London Authority (GLA) as an innovation towards freeing London’s data. We want citizens to be able access the data that the GLA and other public sector organisations hold, and to use that data however they see fit – free of charge. The GLA is committed to influencing and cajoling other public sector organisations into releasing their data here too.

• http://www.datagm.org.uk/ - manchester

• Open Data Communities - Open Access to Local Data. This site is the UK Department for Communities and Local Government's official Linked Open Data site. It provides a selection of statistics on a variety of themes including Local Government finance, housing and homelessness, wellbeing, deprivation, and the department's business plan as well as supporting geographical data. All of the data is available as fully browsable and queryable Linked Data, and the majority is free to re-use under the Open Government Licence.

• http://api.opencorporates.com/documentation/Home

Education

• http://ukdataservice.ac.uk/

• http://linkedup-project.eu/

• http://ldfocus.blogs.edina.ac.uk/

• http://datashare.is.ed.ac.uk/

BBC

• http://www.bbc.co.uk/developer/technology/apis.html

• http://www.bbc.co.uk/blogs/internet/posts/News-Linked-Data-Ontology
• http://www.infoq.com/presentations/bbc-data-platform-api

• http://www.johngoodwin.me.uk/mashups/6radio.html

• http://api.bbcnews.appengine.co.uk/

• http://www.programmableweb.com/api/bbc/mashups

Other

• http://www.nestoria.co.uk/help/api

Scotland

• http://linkedscotland.org/case-studies
• http://datahub.io/dataset/linked-scotland

• https://sites.google.com/site/scottishlinkeddataswig/

National

• A Digital Ambition for Scotland - October 22 2010
• Scotland's Digital Future: A Strategy for Scotland - Strategy setting out how the Scottish Government will ensure Scotland takes full advantage of digital technology.

"Action 2.4 We will develop proposals with partners for releasing more government information and data for use by the public. Initial proposals to be developed and implementation to begin by end of July 2011. We invite suggestions for areas where the greater availability of public data could lead to new services or innovative applications " - March 3 2011

• http://www.scotland.gov.uk/Topics/Government/sustainabilityperformance/Data
• http://cofog01.data.scotland.gov.uk/ - linked data
  • http://blog.swirrl.com/articles/golspie-energie-use-data-in-scottish-government-buildings

• http://www.govcampscotland.com/conversation/digital-future

• Scottish Index of Multiple Deprivation - Data Sources and Suitability

• http://bigdata.holyrood.com/agenda
  • http://www.youtube.com/watch?v=5el8G6Xo1Lc

• http://www.sns.gov.uk/

Health

• http://www.isdscotland.org/Products-and-Services/eDRIS/DSLS-Consultation/

• ALISS stands for Access to Local Information to Support Self Management. It’s a wide-ranging project taking a number of approaches to making it easier to find local self management support.

Local

• http://openlylocal.com/
  • http://openlylocal.com/councils/open?country=Scotland

Ireland

• http://data.fingal.ie/
• http://opendata.ie/
• http://www.cso.ie/en/databases/

Europe

• http://open-data.europa.eu/
• http://ec.europa.eu/information_society/policy/psi/open_data_portal/index_en.htm
• http://publicdata.eu/
• http://lod2.okfn.org/eu-data-catalogues/
• http://latc-project.eu/datasets
• http://www.oecd.org/statistics/
• http://www.eea.europa.eu/data-and-maps

• http://opendatachallenge.org/

• http://www.europeana.eu/portal/api-introduction.html

• http://www.earthobservatory.eu/Ontologies_and_Linked_Geospatial_Data

• https://kenai.com/projects/envision/pages/Home

USA

• http://www.data.gov/opendatasites
• http://id.loc.gov/

• http://www2.census.gov/census_2000/datasets/

• http://www.govtrack.us/developers/api - json

Gloal

• http://datahub.io/

• http://www.google.com/publicdata/directory

• http://en.openei.org/wiki/Main_Page

• http://www.mpi-inf.mpg.de/yago-naga/yago/ - YAGO2s is a huge semantic knowledge base, derived from Wikipedia WordNet and GeoNames. Currently, YAGO2s has knowledge of more than 10 million entities (like persons, organizations, cities, etc.) and contains more than 120 million facts about these entities.

UN

• http://data.un.org/
  • https://data.undp.org/

WordNet

• http://wordnet.princeton.edu/
  • http://wordnet.princeton.edu/wordnet/related-projects/
  • https://en.wikipedia.org/wiki/WordNet - english language semantic relations
  • http://globalwordnet.org/

• http://www.stevenloria.com/tutorial-wordnet-textblob/

• http://www.w3.org/TR/wordnet-rdf/
• http://semanticweb.cs.vu.nl/lod/wn30/

Crowdsourced

DBpedia

• http://en.wikipedia.org/wiki/DBpedia
  • http://dbpv.wordpress.com/

• http://aksw.org/Projects/DBPSB.html

• http://www.iswc2013.semanticweb.org/sites/default/files/iswc_poster_24.pdf

• http://www.faviki.com/pages/welcome/
  • http://www.w3.org/2001/sw/sweo/public/UseCases/Faviki/
• http://clarkparsia.com/pelorus

WikiData

• https://www.wikidata.org
  • https://www.wikidata.org/wiki/Wikidata:Glossary
  • https://www.wikidata.org/wiki/Wikidata:List_of_properties/Summary_table
  • http://meta.wikimedia.org/wiki/Wikidata
  • http://semanticweb.org/RDF/Wikidata/
  • https://github.com/mkroetzsch/wda
  • https://meta.wikimedia.org/wiki/Wikidata/Development/RDF

• https://github.com/goldsmith/Wikipedia

Geo

• http://linkedgeodata.org/About

Other

• http://svonava.com/post/62186512058/datasets-released-by-google

• http://ventus.project.asu.edu/
  • http://www.nature.com/news/scientists-ask-public-to-hunt-for-power-plants-1.12969

• http://echoprint.me/data_download - music id

Commercial

• http://www.whoownsscotland.org.uk/ - has to cover land registry cost?

• http://www.internetworld.de/var/ezwebin_site/storage/images/media/images/social-media-prisma/2350312-1-ger-DE/SOCIAL-MEDIA-PRISMA.jpg

Development

• https://github.com/linkeddata

JavaScript

• http://www.w3.org/community/json-ld/
  • http://www.i-programmer.info/news/87-web-development/5749-json-linked-data.html

• https://github.com/digitalbazaar/jsonld.js - node

• http://dailyjs.com/2010/11/26/linked-data-and-javascript/

• http://blog.swirrl.com/articles/imd-mapper-a-new-linked-data-and-javascript-app

• http://isawnyu.github.io/awld-js/

APIs

See WebDev#API

• http://www.programmableweb.com/apis
• https://www.mashape.com/
• http://www.apihub.com/
• http://publicapis.com/ [1]

Mining

• http://en.wikipedia.org/wiki/Data_mining

• https://news.ycombinator.com/item?id=6868479
• https://news.ycombinator.com/item?id=6869870

• http://www.dcc.ufmg.br/livros/miningalgorithms/DokuWiki/doku.php?id=contents
• http://www.slideshare.net/anilmlis/semantic-web-mining
• http://www.mops1.com/oracle/event/pasig/downloads/PASIG_2010-Simon.pdf
• http://www.public.asu.edu/~hdavulcu/CSE591_Semantic_Web_Mining.html

• http://en.wikipedia.org/wiki/Data_wrangling
• http://en.wikipedia.org/wiki/Data_analysis
• http://en.wikipedia.org/wiki/Data_management
• http://en.wikipedia.org/wiki/Data_governance

Scraping

• https://scraperwiki.com/

• http://kimonify.kimonolabs.com/ [2]

• https://github.com/scrapinghub/portia
  • http://blog.scrapinghub.com/2014/04/01/announcing-portia/ [3]

Tools

• http://vis.stanford.edu/wrangler/

• http://openrefine.org/ - google refine

• http://idcubed.org/open-platform/platform/
  • https://wiki.idhypercubed.org/wiki/ProjectMustardSeed - A Framework for developing and deploying secure cloud applications to collect, compute on, and share personal data

• Recline Data Explorer and Library - A simple but powerful library for building data applications in pure Javascript and HTML.