 |
The records were pulled from Library Cloud using curl and saved as text files formatted as JSON.
Pulling data for "Joshua Chamberlain":
curl -v -o chamberlainandbowdoin0_end.json --basic -u xx:xx 'http://localhost:8080/solr/lccore/select?indent=on&version=2.2&q=collection:hollis_catalog+AND+keyword:chamberlain+AND+keyword:bowdoin&start=0&rows=2500&fl=call_num,creator,creator_exact,creator_keyword,data_source,dataset_tag,format,height,height_numeric,holding_libs,id,id_inst,id_isbn,id_lccn,id_oclc,keyword,language,lcsh,lcsh_exact,loc_call_num_sort_order,loc_call_num_subject,note,note_keyword,online_avail,pages,pages_numeric,pub_date,pub_date_numeric,pub_location,publisher,rsrc_key,rsrc_value,score_checkouts_undergrad,score_checkouts_grad,score_checkouts_fac,score_reserves,score_recalls,score_course_texts,score_holding_libs,score_extra_copies,score_total,shelfrank,sub_title,subject_keyword,title,title_exact,title_keyword,title_link_friendly,title_sort,toc,url,ut_count,ut_id,wp_categories,wp_categories_exact,collection,520a,lcsh_keyword,score&wt=json&omitHeader=true'The JSON output looks like this:
The JSON is then run through MetaCarta's geotagger:
geotag chamberlainandbowdoin0_end.json -d chamberlainandbowdoin0_end.tagged -e
The output is XML and looks like this:
From here we parse the data using a ruby script and generate both XML and .csv files. We match the geotagged data with it's original input IDs to reconstruct the original Library Cloud record, adding the geotagged data to it.
The data is imported to Solr (blog post to follow) and has the following format.
From here we use AJAX to retrieve data from Solr and push it through OpenLayers to create a heat map. The heat map code class used can be found here.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>OpenLayers Heatmap Layer (using <canvas>)</title>
<script src="http://openlayers.org/api/OpenLayers.js" type="text/javascript"></script>
<script src="HeatmapLayer.js" type="text/javascript"></script>
</head>
<body>
<h3>Solr Search - HEATMAP2-1</h3>
Query: <input id="query" />
<button id="search">Search</button>
<hr/>
<div id="map" style='height:400px;width=400' ></div>
<hr/>
<div id="results">
</div>
</body>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
<script>
function on_search(data) {
var query = $('#query').val();
if (query.length == 0) {
return;
}
var url = "http://sanger.hul.harvard.edu:8080/solr/ghcore/select?indent=on&version=2.2&q="+query+"&start=0&rows=9000&fl=Geocoded_Field,title,id_inst,Anchor,cLoc1,primLoc&wt=json";
var ajaxParams =
{
type: "GET",
url: url,
dataType: 'jsonp',
jsonp: 'json.wrf',
crossDomain: true,
success: function(data){
on_data(data);
},
error: function(arg){
errorFunction(arg);
}
};
$.ajax(ajaxParams);
}
function on_data(data) {
// Remove the heat2 layer if it already exists so that they do not overlap
if (this.map.getLayersByName("Heatmap2")[0] !== undefined)
this.map.removeLayer(this.map.getLayersByName("Heatmap2")[0]);
var heat2 = new Heatmap.Layer("Heatmap2");
console.log(query);
$('#results').empty();
var docs = data.response.docs;
var total = 'Found ' + docs.length + ' results';
$('#results').prepend('<div>' + total + '</div>');
var coordinates = [
[ 54.7408333333,9.43472222222 ],
[ 55.4700375,19.51416 ]
];
$.each(docs, function(j, item) {
//console.log(item.title+" "+item.primLoc);
if (item.primLoc != null) {
var junk = item.primLoc.split(",");
var x = parseFloat(junk[0].trim());
var y = parseFloat(junk[1].trim());
coordinates.push([x,y]);
}
});
for (var latlng in coordinates) {
var point = new Heatmap.Source(new OpenLayers.LonLat(coordinates[latlng][1], coordinates[latlng][0]));
heat2.addSource(point);
}
heat2.defaultIntensity = 0.1;
heat2.setOpacity(0.33);
map.addLayer(heat2);
map.zoomToExtent(heat2.getDataExtent());
}
function on_ready() {
$('#search').click(on_search);
/* Hook enter to search */
$('body').keypress(function(e) {
if (e.keyCode == '13') {
on_search();
}
});
map = new OpenLayers.Map('map', {
controls: [
new OpenLayers.Control.Navigation(),
new OpenLayers.Control.PanZoomBar(),
new OpenLayers.Control.LayerSwitcher({'ascending':false}),
new OpenLayers.Control.MousePosition(),
],
});
// MAP STUFF HERE - Populate an initial set to view.
heat.addSource(new Heatmap.Source(new OpenLayers.LonLat(9.434, 54.740)));
heat.addSource(new Heatmap.Source(new OpenLayers.LonLat(9.833, 54.219)));
var wms = new OpenLayers.Layer.WMS("OpenLayers WMS", "http://labs.metacarta.com/wms/vmap0", {layers: 'basic'});
layer = new OpenLayers.Layer.WMS( "OpenLayers WMS",
"http://vmap0.tiles.osgeo.org/wms/vmap0", {layers: 'basic'} );
map.addLayers([layer, heat]);
map.zoomTo(2);
}
var map;
var heat = new Heatmap.Layer("Heatmap");
$(document).ready(on_ready);
</script>
Formatting from HiLite.Me.
The result allows me to test the display of geotagged HOLLIS records based on a Solr query such as "title_key:chamberlain".
While not always available, you can try it yourself here. (http://sanger.hul.harvard.edu:8080/geohollis/heat2.html)
Next Steps
- Add the point data to Solr 5.1 as an RPT type using WKT
- Test the mapping against 1 million+ records
Live View:
No comments:
Post a Comment