Bill Pascoe, 2019, DH Downunder
This course introduces beginners to using web mapping systems for humanities research with practical worked examples. Learn about the diverse techniques in humanities web mapping, such as detecting place names and mapping corpora of texts, layering datasets, geodata formats, handling multimedia, visualising quantitative geodata, interpreting metrics, change over time and mobility. We'll discuss common mapping terms and techniques, approaches to common problems facing humanities researchers, and an overview of what tools can be used for different purposes. No prior experience with coding, web development or GIS is needed, though an introductory coding course is recommended.
The following are good tools for a digital mapper to have. Download/sign-up while we talk. These usually offer a paid for version, so make sure you just get the free one. You can still follow the course if you don't manage to install them all, or don't want to sign up.
The word 'map' is used in different ways in different disciplines and contexts. We will be talking about geographical maps, but it is worth understanding this in the context of other related uses of the word 'map'.
Fairy Forts. Listen from 19:45 to 20:39. Not all maps are 2D pictures.
"This is the dreaming that belongs to Yajarlu..." Yarmurnturrngu and How I Came Back to Yajarlu told by Jacko Ross Jakamarra. This is one of my favourite stories because when I first read this story it made no sense so I knew there was a lot I couldn't understand. It seemed to just be a series of unrelated events. Why? The clue is in the first line.
Clifford Possom Tjapaltjarri's Warlugulong
“They awoke and the two of them travelled a long way. The sorceror’s fire woke at the same time, always present, always present. The fire chased them away from here, always further off it chased them, and they became more and more badly burnt. As they put it out, it consumed their feet. It ate their feet, their knees, their heads until their skin was covered in burns. At this point in the journey, the story belongs to the Pitjantjantjara.” – Nampijinpa, Uni (Warlpiri jukurrpa) ‘Warlukurlangu, What Happened at the Place of Fire’ in Napaljarri, Peggy Rockman & Cataldi, Lee (translators) Warlpiri Dreamings and Histories: Newly Recorded Stories from the Aboriginal Elders of Central Australia Newhaven: Yale University Press, 2011
“If one is not prepared to take the trouble to go to the place, then its story can only be given as a short version. If one expects, following Western traditions, that each person will be the sole source of a story, then one will be surprised to find a contrapuntal polyphonic style as a group of men or women gather together to collectively produce the text. Similarly, authority is deferred: “That’s about as far as the story I can give you… you might be able to get the other half from … Killer”, said a man in Looma as he saw the story was telling disappear over the horizon into another man’s country.” - Benterrak, Krim; Muecke, Stephen & Roe, Paddy Reading The Country: Introduction To Nomadology Fremantle Arts Centre Press: Fremantle, 1984
Traffic Island or springs serving as a health spa? ~20m from my student house. (source: Brisbane History Map)
1980s suburban sprawl and industrial wasteland or bora ring for ceremony? - at the end of the street parallel to one of the house I lived in as a kid. (source: Brisbane History Map)
The roundabout near No. 1 Sports Ground, or traditional burial ground? - a few blocks from where I live today. (Source: The Virtual Sourcebook for Aboriginal Studies in the Hunter Region and Hunter Living Histories)
I only learned these things in the last 2 years. One role for Digital Humanities mapping is to restore meaning to people about the places we live. It is there, but we are illiterate.
Werekata (work in progress)
Mapping software is in flux. Any time you wish to commence a mapping project it's worth re-checking what the state of play is. Check that:
Here are some examples but there are many tools and technology changes quickly, so it's worth Googling around.
|Quickly and easily create a map with points and pop-ups.||Google MyMaps|
|Desktop GIS for research||QGIS|
|Create map data and browse the world, import/export KML files.||Google Earth|
|Add an old map to a digital map and 'georeference' and 'georectify' it, to situate and warp it corresponding to more modern maps.||MapWarper|
|Advanced and varied mapping features.||ArcGIS CartoDB|
|National digital map research infrastructure, mapping toolbox, urban focus||AURIN|
|Deposit research geo data in a national repository.||National Map (Australian research data repository), ANDS, your Uni library.|
|Relate maps to and visualise statistical information.||Maps and stats: R and D3|
|Build websites integrated maps with multimedia, database, text, community, etc.||Web Archive/Publishing systems with WebMap software Eg: Recollect, Heurist, Omeka (these systems have varied capabilities, .|
|Image overlay 3D on your desktop, or take a screenshot to get a re-usable picture.||Google Earth, see tute|
|Easy image overlay to embed in website||Knight Labs Juxtapose|
|Build network visualisations||Gephi|
|Build network visualisations on the web||D3|
|Animating maps. This is not easy and requires a fair bit of work and expertise.||D3 on the web, and possibly QGIS on the desktop, but depends on needs.|
Add data to features:
Import and Export Data
Never put work into a system if you can't get it out again.
You can import as CSV but you can't export as CSV. This makes it tricky to map within MyMaps and maintain your data seperately and refresh one from the other without redoing things. There are some KML to CSV converters around: http://www.convertcsv.com/kml-to-csv.htm
Now you have made a map you want to do more don't you? Maybe it's tie it in with your database, or show stats for regions, or show change over time something.
Google MyMaps is great to make a quick and easy web map. A simple map can be very powerful. Sometimes the simpler the clearer the message is. If that is all you need, great. One advantage of starting a map project by putting some information in Google MyMaps, is that you very quickly learn it's limitations - that means you very quickly clarify what you really wanted your map to do. Rather than spend too long wondering about your specifications and requirements, give it a go - in play thought and action are one.
You can send the web address of your map to people, or you might have your own website and you want to put the map on a webpage. It might be your Uni content management system, your own Wordpress site, or your own web host. You can copy a snippet of code to embed the map on your own webpage.
At present I can't recommend any commercial DIY web site maker so for this demo just edit an HTML file on our computer.
Open your text editor and copy and paste this text into a new file. Save it as 'maptest.html'.
Double click the file you created and it should open in a web browser. If it doesn't it's probably because it saved as maptest.html.txt instead of maptest.html.txt.
<html> <head> <title> Experimental Map </title> </head> <body> <h1>An Experimental Map</h1> <p>This is a test. One choo.</p> </p> </body> </html>
It should look something like this:
<html> <head> <title> Fairy Forts </title> </head> <body> <h1>Fairy Forts</h1> <p>A map of fairy forts in south Armagh.</p> </p> <iframe src="https://www.google.com.au/maps/d/embed?mid=1MAO6J3XB8zzQCfFyx8IFuqmL7BEiZmyd" width="640" height="480"></iframe> </body> </html>
Refresh the page in your web browser and you should see the map.
Edit the snippet to change the width of the map to be 100%.
Coordinate Systems There are many different coordinate systems as scientists develop and improve on them. Some are based on spheres, some on the slightly distorted sphere shape of Earth, and some provide corrections to improve accuracy on previous systems. As digital humanists though the mathematical intricacies of their differences are usually not as important as the ability to use them to produce a map, often where high accuracy is not required, simply to indicate a town or region for example. You should know which coordinate system you are using though in case anyone wants to compare their map or data with yours. Which coordinate system you are using will be important if you do require a high degree of accuracy and precision - archaeologists may need to pin point digs, and good accuracy is needed for road navigation, for example. Latitude are parallel lines drawn around the globe, or the horizontal lines on a map. Longitude are lines drawn from each pole around the earth, somewhat like a quartered orange. Latitude and longitude provides a point. Google maps uses World Geodetic System (WGS) 1984. This is a decimal system, not one based on 'degree's, 'minutes' and 'seconds', which you may have seen.
Standard Data Format There are standard data formats for storing and/or communicating GIS data. Some common ones are KML (KMZ) and GeoJSON. Data might often also be stored in CSV files or relational databases. It's important that data can be removed from, or made available in a standard format, to ensure it's re-usable (even if it is only for yourself or to make a backup). A KML file is a kind of XML file. A KMZ file is a KML file that has been compressed. GeoJSON is a specific kind of JSON file.
Map projection The earth is curved, a map is flat. The 'projection' defines the way in which the 3D curved surface has been translated into 2D. This always involves distortion. The most familiar rectangular map of the world, the Mercator projection, for example increasingly stretches out proportions away from the equator such that the points at the poles are stretched out to be as broad as the equator. This is a popular topic in mapping which you can easily find out information about on the internet. Famously, the Mercator inflates the rich and powerful nations of the Northern Hemisphere and shrinks poorer equatorial and tropical countries, so that a more proportionate map, the Gall–Peters projection, was adopted by some. As humanists we must bear in mind that everything, even the 'objective' science of mapping technology, is political.
Layers The type of information on a map is typically called a 'layer'. We might add a layer of soil types, a layer of river systems, a layer of towns and cities, and a layer of the data we are particularly interested in. Maps are something we read. They are always something we write. The way we draw the map and what we draw on it is always chosen. These choices are influenced by cultural and personal assumptions and the particular purpose of the map. Marshall Islanders have maps of ocean currents, local councils have maps of water pipes, armies have maps of strategic positions, pirates have maps of treasure. Because maps are intended to convey knowledge, and this is typically for a purpose, they play a role in human agency and power. Some maps promise adventure and the chance to learn from the unfamiliar, other maps are more sinister when we understand what they were used for.
Basemap Usually we want to put information on a map. The map we put the information on, or the layers on, is called the 'basemap' or 'base layer'. Mapping systems typically come with a variety of options for the basemap of the world - terrain only, road maps, showing place names or not showing place names, and so on.
Points, lines, polygons When using mapping software, most information is displayed as either a point, a line, or a polygon. Lines may be irregular, such as the path a road takes, and shapes may be squares, circles or irregular drawings. Drawing polygons can be very time consuming, so normally you would not contemplate this unless there were only a few places you wished to outline, or if you can get the outlines from somewhere else. Another drawback in many systems is that lines and shapes can only be drawn without curves, making them somewhat clunky to look at. You can usually only simulate a curve or circle by adding lots of angles.
Image overlays This enables placing an image over the top of a map, by specifying where the corners of the image should go. In this way we can display old maps, or well drawn outlines, over a contemporary map for example, with 'show/hide' toggles and transparency. Some tools allow distorting the image so that old map features match exactly the current map.
Time and Space / History and Place In humanities research and critique, even outside history, we are often concerned with the historical development of our subject area. Time is often a crucial dimension to add to Humanities maps. Many systems have time widgets that enable showing data based on time spans.
Metadata Typically we are not only mapping coordinates, but sites that have a lot of other associated information. This metadata about a place is what you would often find in a pop up when you click on a point, or it may be part of the visualisation, for example being used to determine the colours or size of dots to convey intensity.
To simply find the coordinates of a point, the quickest way we have found is to go to Google Maps, right click a point and select 'What's Here?' If you are working within a system it probably has it's own user interface for placing markers, etc. There are also APIs to retrieve coordinates given addresses and so on, if you are a developer building software.
Common glitches include error messages, points not showing up on the map at all, or appearing off the West Coast of Africa in the Atlantic Ocean (ie: lat, long has defaulted to 0,0 in the event of a problem)
Some common causes of glitches:
It's useful to have some understanding of the inner workings of the tools you are using, so that when there is a technical glitch you can find and fix the problem.
See also Google's KML Tutorial
The following is a very simple KML file that would put a point on the map.
It begins with an XML declaration because KML is compliant with the widely adopted mark up language, XML.
It then declares a 'placemark' element with a name, description and point coordinates.
Copy and paste this into a plain text document and save it as 'someplace.kml'.
Double click the file and it should open in Google Earth (if you installed it).
<?xml version="1.0" encoding="UTF-8"?> <kml xmlns="http://www.opengis.net/kml/2.2"> <Placemark> <name>Simple placemark</name> <description>Attached to the ground. Intelligently places itself at the height of the underlying terrain.</description> <Point> <coordinates>151.770978,-32.927247,0</coordinates> </Point> </Placemark> </kml>
What does the following do?
<?xml version="1.0" encoding="UTF-8"?> <kml xmlns="http://www.opengis.net/kml/2.2"> <Placemark> <name>The Pentagon</name> <Polygon> <extrude>1</extrude> <altitudeMode>relativeToGround</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -77.05788457660967,38.87253259892824,100 -77.05465973756702,38.87291016281703,100 -77.05315536854791,38.87053267794386,100 -77.05552622493516,38.868757801256,100 -77.05844056290393,38.86996206506943,100 -77.05788457660967,38.87253259892824,100 </coordinates> </LinearRing> </outerBoundaryIs> <innerBoundaryIs> <LinearRing> <coordinates> -77.05668055019126,38.87154239798456,100 -77.05542625960818,38.87167890344077,100 -77.05485125901024,38.87076535397792,100 -77.05577677433152,38.87008686581446,100 -77.05691162017543,38.87054446963351,100 -77.05668055019126,38.87154239798456,100 </coordinates> </LinearRing> </innerBoundaryIs> </Polygon> </Placemark> </kml>
As in any XML document, 'CDATA' can be used to start and end a section that has textual content that can be anything you want, not necessarily valid XML. Often we use it to put in HTML, the web markup language, so we can, for example, display paragraphs and headings in a pop up.
<kml xmlns="http://www.opengis.net/kml/2.2"> <Document> <Placemark> <name>CDATA example</name> <description> <![CDATA[ <h1>CDATA Tags are useful!</h1> <p><font color="red">Text is <i>more readable</i> and <b>easier to write</b> and you can use any HTML you like here.</font></p> ]]> </description> <Point> <coordinates>151.770978,-32.927247</coordinates> </Point> </Placemark> </Document> </kml> </kml>
You might want the markers you draw on the map to look a certain way, either to make sure your map looks cool, or to convey information, such as different categories. You can do this first by defining a style, then by applying that style to the Placemark. Note that the Style element has an id. To apply this style to a placemark, you set the Placemark's styleUrl to be the same as the Style's id (preceded by a hash sign). In this case the id is transBluePoly, so under Placemark we set styleUrl to be #transBluePoly.
<?xml version="1.0" encoding="UTF-8"?> <kml xmlns="http://www.opengis.net/kml/2.2"> <Document> <Style id="transBluePoly"> <LineStyle> <width>1.5</width> </LineStyle> <PolyStyle> <color>7dff0000</color> </PolyStyle> </Style> <Placemark> <name>Building 41</name> <styleUrl>#transBluePoly</styleUrl> <Polygon> <extrude>1</extrude> <altitudeMode>relativeToGround</altitudeMode> <outerBoundaryIs> <LinearRing> <coordinates> -122.0857412771483,37.42227033155257,17 -122.0858169768481,37.42231408832346,17 -122.085852582875,37.42230337469744,17 -122.0858799945639,37.42225686138789,17 -122.0858860101409,37.4222311076138,17 -122.0858069157288,37.42220250173855,17 -122.0858379542653,37.42214027058678,17 -122.0856732640519,37.42208690214408,17 -122.0856022926407,37.42214885429042,17 -122.0855902778436,37.422128290487,17 -122.0855841672237,37.42208171967246,17 -122.0854852065741,37.42210455874995,17 -122.0855067264352,37.42214267949824,17 -122.0854430712915,37.42212783846172,17 -122.0850990714904,37.42251282407603,17 -122.0856769818632,37.42281815323651,17 -122.0860162273783,37.42244918858722,17 -122.0857260327004,37.42229239604253,17 -122.0857412771483,37.42227033155257,17 </coordinates> </LinearRing> </outerBoundaryIs> </Polygon> </Placemark> </Document> </kml>
Even if you are focused on web mapping, Google Earth is a very useful tool to figure things out, and to test and build KML files.
Here's a few KML files. Download some and double click them to open them in Google Earth.
To change anything you add to the map, right click it in the left hand pane, under 'place' and choose 'edit properties'.
Layers By adding many different pieces of information we layer up a bigger picture and start seeing bigger patterns, as in this map where we start to see contiguous indigenous travel networks connecting spanning the whole country across extraordinary distances Werekata (work in progress)
3D Zoom in close and use the little arrows under the 'N' in the controls to tilt the view. Alternatively, drag the person onto the map. Whether Irish fairy forts, or wayfinding across mountains on the east coast, using the 3D functionality helps us find places and routes. In these examples, using the peaks in Ireland was useful. Efforts have been made to identify routes taken from NSW to QLD for the bunya festival, using fragmented historical records and oral history. The 3D terrain helps see open plains, and where crossings would likely be made by rides, instead of gullies and so on.
Image Overlay You can overlay old maps or other images on the terrain. Try this image (right click and save as) 19 Counties of NSW or Cpt Cook's map of Newfoundland, celebrated for it's accuracy. Click the 'add image overlay' button at the top. Once you have found and added the image, use the green lines to rotate, resize and reshape it. You can change the 'opacity'.
Timeline The timeline can also be very useful. Have a look at the coal mines west of Singleton, NSW. Click the little clock icon for 'show historical imagery'. You should be able to see all the way back to 1984 (if you can only see back to 1995 you are zoomed in too close). Moving forwards in time see the expansion of coal mining over time. You might notice they seem to expand on one side but shrink on the other - that's because one side of the hole is refilled up with the dirt from the other side. Now take a look at some glaciers in Iceland.
Many of the considerations of a mapping project are similar to other DH projects. The following are all important considerations before starting a project. More importantly, don't let uncertainty about these questions stop you from starting a project. The sooner you start the sooner the answers to these questions become clear. Try to build an end-to-end working draft ('proof of concept', 'prototype' or 'feasibility test') as early as possible. All problems can be solved, but time and effort must be allocated.
We often need to deal with vague locations in source materials (eg: 3hrs north of the creek)
If you map from primary sources, such as journey narratives, do you map where they thought they were, or where your best estimate is of where they really were? Or both?
There is a difference between accuracy and precision. Eg: a coordinate may have 6 decimal places, but only be accurate to 3 decimal places.
There is a lot of material and software that we might have assumed to be available but isn't - but that is why this is an interesting research field - there are still important problems to solve, and those technical solutions will lead to discoveries that have not yet been made.
We might build a map in order to spot patterns but keep in mind all the factors that might influence patterns. For example, we might see a cluster of dots and think it significant, but if travel time is a factor, it may only be because those sites are in the mountains, and considering travel time, they are just as clustered as events spread out across a flat region.
If time is a dimension in our data, what if an event occured over an extended period? Or at multiple times? Or at the same time at several locations? (nothing is ever simple).
Statistics on coordinates are very hard. One does not simply average latitude and longitude. Points 10 degrees apart in longitude are much closer together than near the pole. This will skew statistics unless we do some complicated trigonometry.
Exceptions to the rule. Normally in software and STEM areas we are concerned with fitting everything to a consistent structure for ceteris parabus, repeatability of experiment or to meet the 80/20 rule in profit maximisation, as part of 6 sigma quality assurance etc. In Humanities it is often the exceptions to the rule, highly contingent instances, marginal cases, resistance to structure, or the changes in structure itself over time, that we are interested in. In humanities mapping, as in DH generally, we must be prepared to put in the extra effort to handle these exceptions sometimes in conflict with usual IT or commercial practice, or at least offer an explanation about how they relate to our generalised processing. It might be a matter of applying the 80/20 rule of applying a standard approach to as much as possible, but making sure we also cater for the other 20 instead of excluding them. This may require hacking, adding on to, or sometimes just surrounding explanatory and contextualising information.
Time Layered Cultural Map is an ARC funded, C21CH supported, infrastructure project to improve digital mapping for humanities researchers.
APOLOGY: as it unexpectedly takes a year for our peak research institutions to arrange an electronic funds transfer, some tools I was expecting to demonstrate are not yet available. Please check TLCMap for fresh mapping tools and techniques over 2020.
Some partner systems we will be developing with include Heurist, Recogito and HASS DeVL.
Temporal Earth You can add your own KML files to this map. This will soon handle KML with timespans.
Gazetteer of Historical Australian Placenames for the first time an aggregated gazetteer of places in Australia including historical places, from ANPS data, search and filterable by web interface or as web service.
Recogito allows you to annotate people and places in texts and create maps from them. It includes 'Named Entity Recognition' (NER). NER automatically processes text and uses natural language processing (NLP) techniques to identify words that appear to be names of people and places. The places are then matched to a Gazetteer (a big list of places and coordinats) to get the coordinates. Many places in the world have the same name so heuristic algorithms (ie: small computer programs that make a 'best guess') are used to pick one. This has a high error rate. In our experiments we find it is only about 50% accurate for historical Australian places. Still that will save you 50% of your time doing it manually. In recogito you can edit the results to correct them. TLCMap will be working with Recogito to improve performance, make better interfaces and add features.
Examples of Recogito applied
It's always good to browse through examples to see what's possible and get ideas. In a confusing world of different tools, an easy way to figure out how to do what you want to do, is to find a similar project and see what they used.
Mapping shipping in 2012 and 2016 and wind in 2019*. Like most things on the internet there is now too much activity to have a complete view of what everyone is doing. Nobody knows what the state of the art is anymore. Focus on the research question and what is needed for it, rather than finding a use for whatever is bleeding edge - we're doing humanities. Then we see if there is something that does that, and if not, hack, adapt, combine, customise, build it ourselves, bricolage.
Digital Humanities is a rapidly changing field. If you don't start now and finish soon, it will look old when you are done. How do you make sure the project is relevant and has lasting value? One approach is to not be too specific about what technology you say you are going to use so you can use whatever is most relevant near the launch time. The solution we use at C21CH is to adopt a Rapid development style - produce a working prototype early, and aim to continually enhance, build upon it and adapt. If there is a 3 year planning and development process with software as an outcome at the end, the software will probably be redundant when launched. Also, the launch must be considered as the beginning, not the end. The purpose of a DH development project is to reach the end of the beginning - then you move into the maintenance and upgrade cycle. This still needs resourcing, but not as much. A 'finished' development project without ongoing support is often wasted effort. Software is a living thing.
Web devs unfamiliar with mapping APIs may prefer to start here. The Google tools are always handy though, especially if you need to validate or tweak some KML.
If you don't, picking up a few beginner web development skills is surprisingly easy, though mastering the wide range of possibility is an endless journey. It's easy to find some tutes introducing topics, and then working on a specific project hones your skills as you figure out how to do what you want, learning from every bug and error on the way.
With this basic foundation do tutorials with web mapping APIs. Some popular ones include:
Do the GeoJSON with Leaflet tutorial.
Use geojson.io to draw a map and generate some GeoJSON.
Add it to your Leaflet map.
Some of the differences in the APIs are (note the differences change as these megacompanies compete aggressively, catching up and outdoing each other):
Google: faster; easy to use; more widely familiar to everyone.
ArcGIS: a bit slower but fast enough; more advanced and richer features if needed such as 3D terrain view, statistics displays etc.
TIP: a KMZ file is just a KML file that has been compressed. If you want to see what's inside it change the file extension to .zip instead of .kmz and extract the file by right clicking and choosing 'Extract all' or using compression software like 7zip.
Working with KML also makes it possible to add lots of different data from different sources to compare and contrast, to spot patterns and learn new things. It's also a good way to make your geocoded information available to others. Because it makes it so easy to share data, there should be free KML files with all kinds of info available everywhere, but it remains difficult to find sometimes. Infrastructure such as National Map is designed to address this problem. Geoscience Australia and other services provide useful data that we can compare with humanities information: http://services.ga.gov.au/. Lets just use the URL of our own KML file: http://hri.newcastle.edu.au/courses/mapcourse/EarlyEurasianPhilosophers.kml
Find some useful resources in our select bibliography for Digital Mapping in Humanities