Monday, July 30, 2018

Kepler.gl, geojson, census, QGIS, ArcGIS adventure day

I decided to take some time to try out Kepler.gl

As with almost any endeavor, it led to more discovery and re-discovery.  Particularly how to deal with geojson files, calculating latitude/longitude, converting shapefiles, dealing with census data.

I began with this awesome article as an introduction.  It had some datasets to download and try.  I also recommend trying the sample data within Kepler.


I decided to try some of my own data and use a smaller, more manageable data set.

The sources I used were:
The first thing I did was to create a "midwest" dataset from data downloaded from Natural Earth.  I ended up with a midwest_airports.shp, midwest_railroads.shp, and midwest_staets.shp.  I just used ArcGIS desktop to export out the selected items.  I really just wanted a point, line, and polygon.

Next, I had to figure out how to convert a shapefile into a geojson (or json) file.  I found I liked two options.  Just have to zip of the shapefile if you use ogre.

One is to use ogre
Here is a fact sheet I found on using it

The other was to use QGIS 3.2 (right click the shapfle, choose 'Export', 'Save Feature As', and choose GeoJSON as the format.  I pretty much stuck with EPSG:4326 - WGS84 all day.

Both methods of converting worked fine.

I was able to add the newly created geojson files into kepler.  I played with the symbology and that type of thing.

Next, I wanted to try out the hexbins, heatmap, that sort of things.  I figured I'd just try and use some census shapefiles and data.  It seems like census data is always more difficult than it should be to find and use.  I was finally able to download the correct data and got them joined. 

However, in order for things to work well, you must have a latitude/longitude associated with the feature in its table.  I used the lat/long of centroids.  You don't need to create a new point feature, just need to have lat/long within the table of the polygon feature you are using.
I was able to add the geojson file to kepler and was able to do some really cool graphic visuals. 

The next task was to try out the arcs.  This took a little thinking/viewing of existing data to figure out how to make them work correctly.  Essentially, you need a point layer that contains lat long for starting point and destination.  For example, if you wanted an arc to draw on kepler showing the front door of where you work to McDonalds you would have a csv that would have a latitude field for your work, longitude for your work, latitude for McDonalds, Latitude for Mcdonalds.  All this in one row.  I had the best luck using/creating a CSV.  If you wanted to add an arc from your work to your home, you would need another row containing the latitude for your work, longitude for your work, latitude for home, Latitude for home.

I think if you had field names like origin_Latitude, origin_Longitude, desination_Latitude, destination_Longitude it would work fine.

Of course, this led to revisiting how to calculate geometry (find lat/long) of shapefiles.  ArcGIS was fairly easy to remember.  Just open the table, add new fields (Double seemed to work). Right click on the new fields and choose 'Calculate Geometry'.  This worked for points and calculating the lat/long of the centroid of a polygon.

The QGIS method took a bit of memory digging.  I am using 3.2.  Basically, you add your point layer into QGIS.  Go to the 'Vector' menu, 'Geometry Tools', 'Add Geometry Attributes'.  This will create a new layer with the same data, but with additional attributes for showing geometric measurements (lat long).

Thats my quick summation of my first experiences using kepler.
 

Monday, July 9, 2018

Quick Notes - ESRI UC

JD Irving company

Living Atlas:
  • World Imagery Wayback
  • Sentinel 2 (bands)
  • Open Streetmap vector
  • Charted Territory
  • Earth systems Monitor

Density Based Clustering
Forest based (decision trees)

Web app builder
Appstation

ArcGIS Solutions (configurations)

ionsights apps

Loading textures - 3d buildings

ArcGIS indoors