Infographics
This is a WIP resources page.
Really Useful Data project
With aims to provide insight on elements of data and design literacy, and describe patterns for using either for informational and learning purposes.
Really useful data, designed and engineered right.
Data, linked data, and how to visualise.
Related events
Roadmap
Quick to do list;
- Finish transfer of research notes
- Add additional info and data sources
- Focus on accessible methods of creating vis.
Key questions
- What data sets and sources are useful for our purposes?
- What types and examples of visualisation are available?
- Methods for display a relation between data and media?
- How to build and manage a system for visualising data?
Data
data, noun
- facts and statistics collected together for reference or analysis: there is very little data available
- the quantities, characters, or symbols on which operations are performed by a computer, which may be stored and transmitted in the form of electrical signals and recorded on magnetic, optical, or mechanical recording media.
- Philosophy things known or assumed as facts, making the basis of reasoning or calculation.
- http://en.wikipedia.org/wiki/Data_set
- http://en.wikipedia.org/wiki/Hyperdata
- http://en.wikipedia.org/wiki/Semi-structured_data
- http://en.wikipedia.org/wiki/Metadata_standards
See also Coding#Serialization_and_markup, Coding#Data_types_and_structures, Coding#Stats_and_big_data
- http://en.wikipedia.org/wiki/Data_wrangling
- http://en.wikipedia.org/wiki/Data_analysis
- http://en.wikipedia.org/wiki/Data_management
- http://en.wikipedia.org/wiki/Data_governance
- Data, Information, Knowledge, and Wisdom - some abstractions
Learning
Open Data
See also Open
"in 2009 and 9 the American Government launched data.gov - 'a comprehensive catalogue of data provided by federalo agencies and represents transparency and accountability in groups and officials... how the government spends tax dollars' - (YAU 2011)
Data.gov.uk - Opening up Government More than 9000 data sets
"As of April 2010 the following UK Government departments and agencies have provided data sets to data.gov.uk:BusinessLink, the Cabinet Office, the Department for Business, Innovation and Skills, the Department for Children, Schools and Families, the Department for Communities and Local Government, theDepartment for Culture, Media and Sport, the Department for Environment, Food and Rural Affairs, the Department for International Development, the Department for Transport, the Department for Work and Pensions, the Department of Energy and Climate Change, theDepartment of Health, the Foreign and Commonwealth Office, the Home Office, Her Majesty's Treasury, Lichfield District Council, the Ministry of Defence, the Ministry of Justice, the Northern Ireland Office, theOrdnance Survey, and the Society of Information Technology Management.
"All data included in data.gov.uk is covered either by Crown Copyright, the Crown Database Right or have been licensed to the Crown. In turn, all data available on data.gov.uk is available under a worldwide, royalty-free, perpetual, non-exclusive license which permits use of the data under the following conditions: the copyright and the source of the data should be acknowledged by including an attribution statement specified by data.gov.uk, which is 'name of data provider' data © Crown copyright and database right. the inclusion of the same acknowledgement is required in sub-licensing of the data, and further sub-licenses should require the same. The data should not be used in a way that suggests that the data provider endorses the use of the data. And the data or its source should not be misrepresented"
Semantic Web
- Semantic Web is a Web of Data — of dates and titles and part numbers and chemical properties and any other data one might conceive of. The collection of Semantic Web technologies (RDF, OWL, SKOS, SPARQL, etc.) provides an environment where application can query that data, draw inferences using vocabularies, etc.
- Agile Knowledge Engineering and Semantic Web (AKSW) is hosted by the Chair of Business Information Systems (BIS) of the Institute of Computer Science (IfI) / University of Leipzig as well as the Institute for Applied Informatics (InfAI). Goals: Development of methods, tools and applications for adaptive Knowledge Engineering in the context of the Semantic Web. Research of underlying Semantic Web technologies and development of fundamental Semantic Web tools and applications. Maturation of strategies for fruitfully combining the Social Web paradigms with semantic knowledge representation techniques.
- Sindice - Data Web Services. Millions of websites mark up their content using RDF, Microformats, Microdata, Schema.org, RDFa, Opengraph and more. Sindice helps you find, understand and integrate with their content.
RFD
- http://en.wikipedia.org/wiki/JSON-LD - RDF-like
Tools
- W3C: RDFImportersAndAdapters
- W3C: ConverterToRdf converts application data from an application-specific format into RDF for use with RDF tools and integration with other data. Converters may be part of a one-time migration effort, or part of a running system which provides a semantic web view of a given application.
- Redland is a set of free software C libraries that provide support for the Resource Description Framework (RDF).
- Raptor is a free software / Open Source C library that provides a set of parsers and serializers that generate Resource Description Framework (RDF) triples by parsing syntaxes or serialize the triples into a syntax. The supported parsing syntaxes are RDF/XML, N-Quads, N-Triples, TRiG, Turtle, RDFa 1.0 and 1.1, RSS tag soup including all versions of RSS, Atom 1.0 and 0.3, GRDDL and microformats for HTML, XHTML and XML. The serializing syntaxes are RDF/XML (regular, and abbreviated), Atom 1.0, GraphViz, JSON, N-Quads, N-Triples, RSS 1.0 and XMP.
- rdfstore-js is a pure Javascript implementation of a RDF graph store with support for the SPARQL query and data manipulation language. node.js
other
- http://www.slideshare.net/DLFCLIR/intro-to-lod
- http://www.w3.org/TR/microdata-rdf/
- http://www.thedigitalshift.com/2012/02/roy-tennant-digital-libraries/why-microdata-not-rdf-will-power-the-semantic-web/
OWL
SPARQL
- http://dbpedia.org/sparql
- http://data.gov.uk/sparql
- http://www.cambridgesemantics.com/semantic-university/sparql-by-example#%281%29
Linked Open Data
- W3C: Linking Open Data - The Open Data Movement aims at making data freely available to everyone. There are already various interesting open data sets available on the Web. Examples include Wikipedia, Wikibooks, Geonames, MusicBrainz, WordNet, the DBLP bibliography and many more which are published under Creative Commons or Talis licenses. The goal of the W3C SWEO Linking Open Data community project is to extend the Web with a data commons by publishing various open data sets as RDF on the Web and by setting RDF links between data items from different data sources. RDF links enable you to navigate from a data item within one data source to related data items within other sources using a Semantic Web browser. RDF links can also be followed by the crawlers of Semantic Web search engines, which may provide sophisticated search and query capabilities over crawled data. As query results are structured data and not just links to HTML pages, they can be used within other applications.
- Linked Data is about using the Web to connect related data that wasn't previously linked, or using the Web to lower the barriers to linking data currently linked using other methods. More specifically, Wikipedia defines Linked Data as "a term used to describe a recommended best practice for exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF." This site exists to provide a home for, or pointers to, resources from across the Linked Data community.
- datavisualization.ch: Introduction to Linked Open Data for Visualization Creators
- How to Publish Linked Data on the Web
- Learning Linked Data
- Learn Linked Data - Helping you get to grips with RDF, SPARQL & linked data.
- LOD cloud diagram shows datasets that have been published in Linked Data format, by contributors to the Linking Open Data community project and other individuals and organisations. It is based on metadata collected and curated by contributors to the Data Hub. Clicking the image will take you to an image map, where each dataset is a hyperlink to its homepage.
- LodLive project provides a demonstration of the use of Linked Data standards (RDF, SPARQL) to browse RDF resources. The application aims to spread linked data principles using a simple and friendly interface with reusable techniques.
LOV
VOAF
Python
JavaScript
- Sgvizler is a javascript which renders the result of SPARQL SELECT queries into charts or html elements. It is cool stuff (ivan_herman, timbl).
Validator
Search
Other
See also MediaWiki#Semantic
- http://semanticweb.com/ - business news
Data sources
Hubs / platforms
- CKAN is a fully-featured, mature, open source data portal and data management solution. CKAN provides a streamlined way to make your data discoverable and presentable. Each dataset is given its own page with a rich collection of metadata, making it a valuable and easily searchable resource.
UK
- http://www.data-archive.ac.uk/find/hasset-thesaurus/skos-hasset This new resource is an outcome of the Jisc-funded SKOS-HASSET project, led by staff at the UK Data Archive at the University of Essex, which owns and manages HASSET. Like dictionaries, thesauri describe the changing world around them; this is why the UK Data Archive continues work to ensure HASSET is up to date. Simple Knowledge Organisation System(SKOS) makes the thesaurus machine-readable. It is the version of Resource Description Framework (RDF) specific to classification resources. It encodes these products in a standardised way to make their structures comparable and to facilitate interaction.
Government
- Data.gov.uk is a key part of the Government's work on Transparency which is being lead by the Transparency Board. Data.gov.uk implementation is being led by the Transparency and Open Data team in the Cabinet Office, working across government departments to ensure that data is released in a timely and accessible way. This work is being supported by Sir Tim-Berners Lee & Professor Nigel Shadbolt. There are a number of technical partners involved in the project to date. These include the Comprehensive Knowledge Archive Network (CKAN): CKAN runs the catalogue at data.gov.uk/data as well as a growing number of open data registries around the world. It is a project created by the Open Knowledge Foundation to make it easy to find, share and reuse open content and data. The CKAN software provides a web interface, programmer's API, feeds notifying of changes, and a browsable history of all changes. The API is documented here: http://data.gov.uk/data/api.
- data.gov.uk: Who is doing what? - This page lists the domains which publish and maintain linked data and short term projects developing the government use of linked data. Most sectors have one or more SPARQL endpoints, which enable you to perform searches across the data; you can access these interactively on this site.
- London Datastore has been created by the Greater London Authority (GLA) as an innovation towards freeing London’s data. We want citizens to be able access the data that the GLA and other public sector organisations hold, and to use that data however they see fit – free of charge. The GLA is committed to influencing and cajoling other public sector organisations into releasing their data here too.
- Open Data Communities - Open Access to Local Data. This site is the UK Department for Communities and Local Government's official Linked Open Data site. It provides a selection of statistics on a variety of themes including Local Government finance, housing and homelessness, wellbeing, deprivation, and the department's business plan as well as supporting geographical data. All of the data is available as fully browsable and queryable Linked Data, and the majority is free to re-use under the Open Government Licence.
Education
BBC
Other
Scotland
National
- A Digital Ambition for Scotland - October 22 2010
- Scotland's Digital Future: A Strategy for Scotland - Strategy setting out how the Scottish Government will ensure Scotland takes full advantage of digital technology.
"Action 2.4 We will develop proposals with partners for releasing more government information and data for use by the public. Initial proposals to be developed and implementation to begin by end of July 2011. We invite suggestions for areas where the greater availability of public data could lead to new services or innovative applications " - March 3 2011
- http://www.scotland.gov.uk/Topics/Government/sustainabilityperformance/Data
- http://cofog01.data.scotland.gov.uk/ - linked data
- Scottish Index of Multiple Deprivation - Data Sources and Suitability
Health
- ALISS stands for Access to Local Information to Support Self Management. It’s a wide-ranging project taking a number of approaches to making it easier to find local self management support.
Local
Ireland
Europe
- http://open-data.europa.eu/
- http://ec.europa.eu/information_society/policy/psi/open_data_portal/index_en.htm
- http://publicdata.eu/
- http://lod2.okfn.org/eu-data-catalogues/
- http://latc-project.eu/datasets
- http://www.oecd.org/statistics/
- http://www.eea.europa.eu/data-and-maps
USA
Gloal
Crowdsourced
Geo
Commercial
- http://www.whoownsscotland.org.uk/ - has to cover land registry cost?
Development
JavaScript
Articles
Scraping
Visualisation
Media that represents data and other knowledge to help provice a mental context and scale
visualize, verb
- form a mental image of; imagine: it is not easy to visualize the future
- make (something) visible to the eye: the DNA was visualized by staining with ethidium bromide
See also Organisation#Knowledge, JS_scripts#Graphics
General
- http://en.wikipedia.org/wiki/Data_visualization
- http://en.wikipedia.org/wiki/Information_visualization
Quotes
From the Keller Paper;
Ware says “power of a visualization comes from the fact that it is possible to have a far more complex concept structure represented externally in a visual display than can be held in visual and verbal working memories". In this regard, visualizations are cognitive tools aiming at supporting the cognitive system of the user. Visualizations can make use of the automatically human process of pattern finding (Ware, 2004). They can draw both on the visual and the spatial working memory system (Baddeley, 1998; Logie, 1995).
COX;
Compared with an informationally-equivalent textual description of an information a diagram may allow users to avoid having to explicitly compute information because users can extract information ‘at a glance’ (p. 2). “Such representations work best when the spatial constraints obeyed by representations map into important constraints in the represented domain in such a way that they restrict (or enforce) the kinds of in- terpretations that can be made” (Rogers & Scaife, 1997, p. 2). They can help to ex- ploit the rapid processing capabilities of the human visual system and very easy per- ceptual judgements are substituted for more difficult logical ones (Paige & Simon, 1966). Chabris and Kosslyn (in this book) suggest the principle of ‘representational correspondence’ as a basic principle of effective diagram design. According to this principle visualizations work best if they depict information in the same way that our internal mental representation do.
- Get the above academic details for bibliography
Info and news
- http://spatialanalysis.co.uk
- http://www.oii.ox.ac.uk/vis
- http://chartporn.org
- http://flowingdata.com
- http://datavisualization.ch
- http://informationisbeautiful.net
Semantic Web and Linked Data
- http://notes.3kbo.com/visual
- http://taskman.eionet.europa.eu/projects/linkeddata/wiki/VisualizeLinkedData?version=8
- http://answers.semanticweb.com/questions/1071/visualisation-toolkits-for-rdf
- https://github.com/semsol/arc2/wiki - php libs
- EEA Daviz is a web tool developed by the European Environment Agency which helps creating interactive data visualizations easily through the web browser, no extra tools are necessary. It is free and open source. You can generate attractive and interactive charts and combine them in a dashboard with facets/filters which updates the charts simultaneously. Data can be uploaded as CSV/TSV or you can specify SPARQL to query online Linked open data servers (aka sparql endpoints). Daviz is the first Semantic web data visualisation tool for Plone CMS, entirely web-based! At the moment Simile Exhibit and Google Charts visualizations are supported. The architecture allows to extend Daviz with more visualisation libraries (visualisations plugins). - zope/plone
Systems
Karma
- Karma is an information integration tool that enables users to quickly and easily integrate data from a variety of data sources including databases, spreadsheets, delimited text files, XML, JSON, KML and Web APIs. Users integrate information by modeling it according to an ontology of their choice using a graphical user interface that automates much of the process. Karma learns to recognize the mapping of data to ontology classes and then uses the ontology to propose a model that ties together these classes. Users then interact with the system to adjust the automatically generated model. During this process, users can transform the data as needed to normalize data expressed in different formats and to restructure it. Once the model is complete, users can published the integrated data as RDF or store it in a database.
SIMILE
- http://simile-widgets.org/
- http://www.simile-widgets.org/wiki/Main_Page
- http://www.simile-widgets.org/wiki/Getting_Started_with_Simile_Widgets
- http://www.simile-widgets.org/wiki/Examples_of_Simile_Widgets
- http://www.simile-widgets.org/wiki/Exhibit
- http://www.simile-widgets.org/wiki/Getting_Started_with_Exhibit
- http://simile.mit.edu/wiki/Exhibit/How_to_import_RDF_on_the_fly
Other
Tutorials
Design, data and vis basics;
d3;
- http://blog.visual.ly/using-selections-in-d3-to-make-data-driven-visualizations/
- http://www.youtube.com/playlist?list=PL6il2r9i3BqH9PmbOf5wA5E1wOG3FT22p
- http://www.youtube.com/watch?v=C-_Lwm-2Hsg
- http://www.youtube.com/watch?v=ZEJR7ANcjMA
- D3 for Beginners Part I: Joe Golike on Thinking in D3
Learning
See also Learning
Journalism
Examples
- http://www.guardian.co.uk/news/datablog
- http://www.guardian.co.uk/news/datablog+politics/mps-expenses
- http://www.guardian.co.uk/news/datablog/2010/apr/25/uk-public-spending-1963
- http://www.guardian.co.uk/news/datablog/2013/apr/26/brie-bin-food-waste-value
People
- http://simonrogers.net/ - Data journalism and other curiosities
Creating
See JavaScript, JS_scripts#Visualisation
- Google I/O 2008 - Visualize your Data: Visualization API
- Google's Public Data Explorer: A New Tool for Visualizing Information