Open data

From Things and Stuff Wiki
Jump to navigation Jump to search


Too shuffle into semantic, etc.

General

data, noun

  • facts and statistics collected together for reference or analysis: there is very little data available
    • the quantities, characters, or symbols on which operations are performed by a computer, which may be stored and transmitted in the form of electrical signals and recorded on magnetic, optical, or mechanical recording media.
    • Philosophy things known or assumed as facts, making the basis of reasoning or calculation.

See also Coding#Serialization_and_markup, Coding#Data_types_and_structures, Coding#Stats_and_big_data

Learning

Open Data

See also WebDev#API, Open

"in 2009 and 9 the American Government launched data.gov - 'a comprehensive catalogue of data provided by federalo agencies and represents transparency and accountability in groups and officials... how the government spends tax dollars' - (YAU 2011)

Data.gov.uk - Opening up Government More than 9000 data sets

"As of April 2010 the following UK Government departments and agencies have provided data sets to data.gov.uk:BusinessLink, the Cabinet Office, the Department for Business, Innovation and Skills, the Department for Children, Schools and Families, the Department for Communities and Local Government, theDepartment for Culture, Media and Sport, the Department for Environment, Food and Rural Affairs, the Department for International Development, the Department for Transport, the Department for Work and Pensions, the Department of Energy and Climate Change, theDepartment of Health, the Foreign and Commonwealth Office, the Home Office, Her Majesty's Treasury, Lichfield District Council, the Ministry of Defence, the Ministry of Justice, the Northern Ireland Office, theOrdnance Survey, and the Society of Information Technology Management.

"All data included in data.gov.uk is covered either by Crown Copyright, the Crown Database Right or have been licensed to the Crown. In turn, all data available on data.gov.uk is available under a worldwide, royalty-free, perpetual, non-exclusive license which permits use of the data under the following conditions: the copyright and the source of the data should be acknowledged by including an attribution statement specified by data.gov.uk, which is 'name of data provider' data © Crown copyright and database right. the inclusion of the same acknowledgement is required in sub-licensing of the data, and further sub-licenses should require the same. The data should not be used in a way that suggests that the data provider endorses the use of the data. And the data or its source should not be misrepresented"

Semantic Web

See also Open web#Semantic

  • Semantic Web - a “Web of data,” the sort of data you find in databases. The ultimate goal of the Web of data is to enable computers to do more useful work and to develop systems that can support trusted interactions over the network. The term “Semantic Web” refers to W3C’s vision of the Web of linked data. Semantic Web technologies enable people to create data stores on the Web, build vocabularies, and write rules for handling data. Linked data are empowered by technologies such as RDF, SPARQL, OWL, and SKOS.
  1. Use URIs to denote things.
  2. Use HTTP URIs so that these things can be referred to and looked up ("dereferenced") by people and user agents.
  3. Provide useful information about the thing when its URI is dereferenced, leveraging standards such as RDF, SPARQL.
  4. Include links to other related things (using their URIs) when publishing data on the Web.

or

  1. All kinds of conceptual things, they have names now that start with HTTP.
  2. I get important information back. I will get back some data in a standard format which is kind of useful data that somebody might like to know about that thing, about that event.
  3. I get back that information it's not just got somebody's height and weight and when they were born, it's got relationships. And when it has relationships, whenever it expresses a relationship then the other thing that it's related to is given one of those names that starts with HTTP.

On the Semantic Web, vocabularies define the concepts and relationships (also referred to as “terms”) used to describe and represent an area of concern. Vocabularies are used to classify the terms that can be used in a particular application, characterize possible relationships, and define possible constraints on using those terms. In practice, vocabularies can be very complex (with several thousands of terms) or very simple (describing one or two concepts only).

There is no clear division between what is referred to as “vocabularies” and “ontologies”. The trend is to use the word “ontology” for more complex, and possibly quite formal collection of terms, whereas “vocabulary” is used when such strict formalism is not necessarily used or only in a very loose sense. Vocabularies are the basic building blocks for inference techniques on the Semantic Web.

  • Agile Knowledge Engineering and Semantic Web (AKSW) is hosted by the Chair of Business Information Systems (BIS) of the Institute of Computer Science (IfI) / University of Leipzig as well as the Institute for Applied Informatics (InfAI). Goals: Development of methods, tools and applications for adaptive Knowledge Engineering in the context of the Semantic Web. Research of underlying Semantic Web technologies and development of fundamental Semantic Web tools and applications. Maturation of strategies for fruitfully combining the Social Web paradigms with semantic knowledge representation techniques.
  • Sindice - Data Web Services. Millions of websites mark up their content using RDF, Microformats, Microdata, Schema.org, RDFa, Opengraph and more. Sindice helps you find, understand and integrate with their content.

Linked Open Data

  • W3C: Linking Open Data - The Open Data Movement aims at making data freely available to everyone. There are already various interesting open data sets available on the Web. Examples include Wikipedia, Wikibooks, Geonames, MusicBrainz, WordNet, the DBLP bibliography and many more which are published under Creative Commons or Talis licenses. The goal of the W3C SWEO Linking Open Data community project is to extend the Web with a data commons by publishing various open data sets as RDF on the Web and by setting RDF links between data items from different data sources. RDF links enable you to navigate from a data item within one data source to related data items within other sources using a Semantic Web browser. RDF links can also be followed by the crawlers of Semantic Web search engines, which may provide sophisticated search and query capabilities over crawled data. As query results are structured data and not just links to HTML pages, they can be used within other applications.
  • Linked Data is about using the Web to connect related data that wasn't previously linked, or using the Web to lower the barriers to linking data currently linked using other methods. More specifically, Wikipedia defines Linked Data as "a term used to describe a recommended best practice for exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF." This site exists to provide a home for, or pointers to, resources from across the Linked Data community.
  • LodLive project provides a demonstration of the use of Linked Data standards (RDF, SPARQL) to browse RDF resources. The application aims to spread linked data principles using a simple and friendly interface with reusable techniques.

RFD

RDF is a general method to decompose any type of knowledge into small pieces, with some rules about the semantics, or meaning, of those pieces. The point is to have a method so simple that it can express any fact, and yet so structured that computer applications can do useful things with it.

The basic unit of RDF is a statement called a triple. One can think of a triple as a type of sentence that states a single "fact" about a resource. RDF allows you to define statements about things (or resources), in the form of subject-predicate-object expressions (known as RDF-triples due to the 3 constituent parts).

The different forms for representing the RDF data are:

  • RDF/XML
  • Notation-3 (N3)
  • Turtle - a simplified, RDF-only subset of N3.
  • N-Triple
  • RDFa
  • TRiX
  • TRiG
  • JSON-LD

RDF/XML

Here's some RDF XML: 

<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"

xmlns:ns="http://www.example.org/#"> 

 <ns:Person rdf:about="http://www.example.org/#john">
   <ns:hasMother rdf:resource="http://www.example.org/#susan" />
   <ns:hasFather>
     <rdf:Description rdf:about="http://www.example.org/#richard">
       <ns:hasBrother rdf:resource="http://www.example.org/#luke" />
     </rdf:Description>
   </ns:hasFather>
 </ns:Person>
</rdf:RDF>
  • HTTP Vocabulary in RDF 1.0
  • TriG - RDF Dataset Language. A concrete syntax for RDF as defined in the RDF Concepts and Abstract Syntax ([rdf11-concepts]). TriG is an extension of Turtle ([turtle]), extended to support representing a complete RDF Dataset.

N3

Here's some N3 RDF: 

@prefix : <http://www.example.org/> .
:john    a           :Person .
:john    :hasMother  :susan .
:john    :hasFather  :richard .
:richard :hasBrother :luke .

Turtle

N-Triple

RDFa

2004. RDFa 1.1 reached recommendation status in June 2012.

  • RDFa is an extension to HTML5 that helps you markup things like People, Places, Events, Recipes and Reviews. Search Engines and Web Services use this markup to generate better search listings and give you better visibility on the Web, so that people can find your website more easily.

JSON-LD

JSON-LD was created by people that have been directly involved in the Linked Data, lowercase semantic web, uppercase Semantic Web, Microformats, Microdata, and RDFa work. It has proven to be useful to them. There are a number of very large technology companies that have adopted JSON-LD, further underscoring its utility.

Other

News

Libs

Vocabularies

SIOC

  • SIOC initiative (Semantically-Interlinked Online Communities) aims to enable the integration of online community information. SIOC provides a Semantic Web ontology for representing rich data from the Social Web in RDF. It has recently achieved significant adoption through its usage in a variety of commercial and open-source software applications, and is commonly used in conjunction with the FOAF vocabulary for expressing personal profile and social networking information. By becoming a standard way for expressing user-generated content from such sites, SIOC enables new kinds of usage scenarios for online community site data, and allows innovative semantic applications to be built on top of the existing Social Web. The SIOC ontology was recently published as a W3C Member Submission, which was submitted by 16 organisations.

ResumeRDF

SCOT

  • SCOT is an acronym for Social Semantic Cloud of Tags. The name was chosen to emphasise the goal of providing a consistent framework for expressing social tagging at a semantic level in machine-understandable way. The SCOT ontology provides a model for expressing the main concepts and properties required to describe information for tagging activities (e.g., users, tags, resources, etc.) on the Semantic Web. This document contains a detailed description of the SCOT Ontology.

Data Cube

  • RDF Data Cube Vocabulary - There are many situations where it would be useful to be able to publish multi-dimensional data, such as statistics, on the web in such a way that it can be linked to related data sets and concepts. The Data Cube vocabulary provides a means to do this using the W3C RDF (Resource Description Framework) standard. The model underpinning the Data Cube vocabulary is compatible with the cube model that underlies SDMX (Statistical Data and Metadata eXchange), an ISO standard for exchanging and sharing statistical data and metadata among organizations. The Data Cube vocabulary is a core foundation which supports extension vocabularies to enable publication of other aspects of statistical data flows or other multi-dimensional data sets.

Other

Tools

  • W3C: RDFImportersAndAdapters
  • W3C: ConverterToRdf converts application data from an application-specific format into RDF for use with RDF tools and integration with other data. Converters may be part of a one-time migration effort, or part of a running system which provides a semantic web view of a given application.
  • Redland is a set of free software C libraries that provide support for the Resource Description Framework (RDF).
    • Raptor is a free software / Open Source C library that provides a set of parsers and serializers that generate Resource Description Framework (RDF) triples by parsing syntaxes or serialize the triples into a syntax. The supported parsing syntaxes are RDF/XML, N-Quads, N-Triples, TRiG, Turtle, RDFa 1.0 and 1.1, RSS tag soup including all versions of RSS, Atom 1.0 and 0.3, GRDDL and microformats for HTML, XHTML and XML. The serializing syntaxes are RDF/XML (regular, and abbreviated), Atom 1.0, GraphViz, JSON, N-Quads, N-Triples, RSS 1.0 and XMP.
  • rdfstore-js is a pure Javascript implementation of a RDF graph store with support for the SPARQL query and data manipulation language. node.js

other

Heterogeneous Information Sources on the Web

OWL

SPARQL

LOV

GRDDL

  • GRDDL is a mechanism for Gleaning Resource Descriptions from Dialects of Languages. It is a technique for obtaining RDF data from XML documents and in particular XHTML pages. Authors may explicitly associate documents with transformation algorithms, typically represented in XSLT, using a link element in the head of the document. Alternatively, the information needed to obtain the transformation may be held in an associated metadata profile document or namespace document.

VOAF

Web Observatory

Python

JavaScript

REST

See WebDev#API

Validation

Search

Other

See also MediaWiki#Semantic

  • URIBurner - A data virtualization service that transforms data hosted in a variety of data spaces and formats into standards compliant Linked Data Objects for uniform access, integration and management. The underlying technology is Virtuoso's in-built Linked Data Middleware (aka Sponger) that uses URLs as data source names for its powerful data ingestion and transformation services that result in highly navigable Linked Data Object graphs. Post transformation, each Data Object is endowed with a dereferenceable identifier (Name) that resolves to its actual representation via its URL (Address). The Sponger then re-presents Data Object descriptions via HTML documents (the default behavior) or in a variety of raw data graph forms that include: CSV, N-Triples, Turtle, N3, RDF/XML, JSON, CXML, OData (Atom and JSON) etc.

Data sources

Hubs / platforms

  • CKAN is a fully-featured, mature, open source data portal and data management solution. CKAN provides a streamlined way to make your data discoverable and presentable. Each dataset is given its own page with a rich collection of metadata, making it a valuable and easily searchable resource.

UK

  • http://www.data-archive.ac.uk/find/hasset-thesaurus/skos-hasset This new resource is an outcome of the Jisc-funded SKOS-HASSET project, led by staff at the UK Data Archive at the University of Essex, which owns and manages HASSET. Like dictionaries, thesauri describe the changing world around them; this is why the UK Data Archive continues work to ensure HASSET is up to date. Simple Knowledge Organisation System(SKOS) makes the thesaurus machine-readable. It is the version of Resource Description Framework (RDF) specific to classification resources. It encodes these products in a standardised way to make their structures comparable and to facilitate interaction.

Government

  • Data.gov.uk is a key part of the Government's work on Transparency which is being lead by the Transparency Board. Data.gov.uk implementation is being led by the Transparency and Open Data team in the Cabinet Office, working across government departments to ensure that data is released in a timely and accessible way. This work is being supported by Sir Tim-Berners Lee & Professor Nigel Shadbolt. There are a number of technical partners involved in the project to date. These include the Comprehensive Knowledge Archive Network (CKAN): CKAN runs the catalogue at data.gov.uk/data as well as a growing number of open data registries around the world. It is a project created by the Open Knowledge Foundation to make it easy to find, share and reuse open content and data. The CKAN software provides a web interface, programmer's API, feeds notifying of changes, and a browsable history of all changes. The API is documented here: http://data.gov.uk/data/api.
  • data.gov.uk: Who is doing what? - This page lists the domains which publish and maintain linked data and short term projects developing the government use of linked data. Most sectors have one or more SPARQL endpoints, which enable you to perform searches across the data; you can access these interactively on this site.
  • London Datastore has been created by the Greater London Authority (GLA) as an innovation towards freeing London’s data. We want citizens to be able access the data that the GLA and other public sector organisations hold, and to use that data however they see fit – free of charge. The GLA is committed to influencing and cajoling other public sector organisations into releasing their data here too.
  • Open Data Communities - Open Access to Local Data. This site is the UK Department for Communities and Local Government's official Linked Open Data site. It provides a selection of statistics on a variety of themes including Local Government finance, housing and homelessness, wellbeing, deprivation, and the department's business plan as well as supporting geographical data. All of the data is available as fully browsable and queryable Linked Data, and the majority is free to re-use under the Open Government Licence.

Education

BBC

Other

Scotland

National

"Action 2.4 We will develop proposals with partners for releasing more government information and data for use by the public. Initial proposals to be developed and implementation to begin by end of July 2011. We invite suggestions for areas where the greater availability of public data could lead to new services or innovative applications " - March 3 2011

Health

  • ALISS stands for Access to Local Information to Support Self Management. It’s a wide-ranging project taking a number of approaches to making it easier to find local self management support.

Local

Ireland

Europe

USA

Gloal

UN

WordNet

Crowdsourced

DBpedia

WikiData

Geo

Other

Commercial

Development

JavaScript

Articles

APIs

See WebDev#API

Scraping

Tools

Retrieved from "https://wiki.thingsandstuff.org/index.php?title=Open_data&oldid=17693"