Posted on

Ignite Locationtech Boston – March 25

The folks from AvidGeo, Boston’s #1 social spatial special interest group, are at it again.  Another awesome Meet-Up is scheduled for the end of this month.

From the AvidGeo Meet-Up Page:

 “In collaboration with the Eclipse Foundation’s LocationTech, this month we are doing another Ignite event around geospatial and open source technology.

 Space is limited, to register for this event please visit the eventbrite site here: http://ignitelocationtechboston1.eventbrite.com/

!!IMPORTANT!!
This event is not being managed through meetup, registering on this meetup page does not mean you have a ticket to attend, please visit the eventbrite site in the link above.”

The event is at Space with a Soul, 281 Summer Street in Boston (a short walk from South Station) and it starts at 7pm.

There are a few talks already in place and they look great – Hubway mapping, open-source multi-spectral imaging, Leaflet mapping, contributing to open source geo projects – what more could you ask for!

If you are in the Boston area and are into open geo you should need to come to this event.  I’m pumped that I can finally make an AvidGeo Meet-Up for the first time in months!

For more info check out Avid Geo’s website or hit them up on Twitter (@avidgeo).

See you there!

Posted on

Happy Presidents’ Day – Leaflet Edition!

I’ve been seeing a lot of Leaflet lately, whether it’s in my twitter stream, on Boston.com, or hearing others in the geo-community talk about enthusiastically.  So, on a snowy Sunday in Somerville, I decided to give a Leaflet a try.  To honor the 43 presidents (remember, Grover Cleveland was both the 22nd and 24th president) of the United States of America I put together a simple leaflet app of each president’s birthplace (according to Wikipedia).

Leaflet in Action
Leaflet in Action

I’ve built plenty of web mapping apps with Google and Esri APIs, but by no means am I an expert.  If you have ever built a web-map using either of those APIs you will be able to build and launch a web map with Leaflet, no problem.  More than likely, you will be able to create a web ready map more quickly with Leaflet as well (as was my experience).

The app I built adds a few markers with custom icons and modified popups.  I read in and customized a state boundary geojson file from a Leaflet tutorial to give the user some context of “where” at larger scales.  The background tiles are from CloudMade and are nice and fast.

I don’t think I configured the autopan for the popups correctly as it doesn’t work as I think it should.  More than likely I just don’t have the right settings configured.  If you see something in my jumbled (and undocumented) code leave a comment and I’ll make the necessary updates.  I would eventually like to add a drop shadow to the presidential seal icons and perhaps read the data directly from a PostGIS database, as opposed to creating static markers.

The Leaflet documentation was easy to understand and the samples provided enough guidance to get a map online that did what I wanted it to do.  I wish there were a few more samples available through the Leaflet tutorials section, but those will come as the user community grows.

Overall, my first experience with Leaflet was generally pleasant.  I know JavaScripting  but I am not an expert and I was able to get a map online pretty quickly.  I spent more time collecting and formatting the data than I did getting the map online.  That is a good thing.  I recommend to those who have programmed with the Google or Esri API to check Leaflet out.  Added bonus – free and open source.

The page I created is for demo purposes only.  I only tested this in Chrome and Firefox (sorry IE users).  If there is incorrect information in the map please let me know and I will update it accordingly.

Finally, here is the direct link to the app:

http://www.gisdoctor.com/presidents.html

Happy presidential mapping!

 

Posted on

Awkward Geo-Coding

Back in the “old” days (2004-2007) I used to do a lot of geo-data cleaning for municipalities and county governments across the northeast for the consulting firm I was working for.  I was mostly cleaning address ranges on centerline datasets, checking road directionality and working on parcel addressing.  It wasn’t glamorous work, but it served an important purpose for the clients.  If the work I was doing was wrong, the services within the municipality could suffer, and I didn’t want to be the guy who messed-up.

I used to take a lot of pride in getting addressing correct or generating datasets of important locations within local municipality. Whenever I see a bad geo-code or a misplaced placemark in an online mapping service I simultaneously cringe and chuckle.  It isn’t easy work to build these massive databases and for the most part they are  remarkably correct. However, every now and then, an error creeps through.  For example, my parents home address registers three miles from where it actually is located when using the Google Maps, however Bing, OSM, and the municipality’s local online mapping tool all place it in the correct location.

There is also the issue of bad placemarkers across the worldwide web of mapping.  I particularly enjoyed what I believe to be a bad placemarker that I found the other day. I work in downtown Boston and I’m often using Google Maps to search for directions to restaurants, bars, friends places, etc.  For some reason that escapes me, I was map-browsing around Fenway Park when I came across a strange placemarker a few dozen rows up from home plate on the third base side:

image credit - Google 2013
image credit – Google 2013

Huh?!?!

I’ve walked around Fenway many times in my life.  I was just over there a couple weeks ago and I didn’t notice this establishment (neither has Bing or OSM).  Maybe this is a new venture by the Fenway Sports Group to keep disgruntled Red Sox fans at the park during a prolonged losing streak? Perhaps this is a new type of viral marketing campaign? I don’t know if this placemark is correct or not, but I found it hilarious.  As of 1/27/2013 the map still had the unique placemark, but if it is wrong I believe it won’t be online much longer, especially if Larry Lucchino sees it.

Here is the full screen grab of the park taken on 1/27/2013 with the exciting new business on Yakwey Way:

image credit: Google 2013
image credit: Google 2013

When clicking on placemarker you get this interesting balloon:

image credit: Google 2013
image credit: Google 2013

As I have mentioned before, building and maintaining these massive address and placemarker datasets is tough work and if you have ever done it you know what I am talking about.  But this brings up a larger question.  Have you ever encountered an awkward geo-code?  Have you ever seen a placemarker that’s obviously in the wrong location, or tried searching for an address and taken somewhere you didn’t expect?  If so, post it to twitter and add the hashtag #awkwardgeocode or leave a comment. Let’s start capturing these map anomalies before they are corrected.  If you have a #awkwardgeocode feel free to share!

 

 

Posted on

Geo Ideas I Want to See More of in 2013

I recently wrote about the geo-terms I wanted to retire in 2013.  However, there are a few topics I want more of in 2013.  The term I want to see more of in 2013 is “open” and here is what I was thinking…

Open Source –  The open geo-software community had a great 2012.  I really see this momentum continuing to grow in 2013.  The key to continued adoption (beyond great software, easy to use platforms, and continued innovation)? Get open geo-software into academia.  The more undergrads who learn GIS on Quantum, GRASS, PostGIS and the rest, the more this movement will continue to expand.

OpenStreetmap – Recently Openstreetmap hit 1 million users.  As a somewhat semi-regular contributor I see great promise in OSM but OSM can’t end up like Wikipedia, which is losing editors and contributors.  OSM can never be completed, and the army of volunteers will hopefully see that.  I should probably do some mapping this weekend!

Open Analysis – I would love to see the the geo-community become more open with analysis. This could include sharing analysis techniques, working together to develop new analyses, or helping the world understand geospatial analysis.  A map is far more than the visualization of the abstraction of space. Let’s start promoting the science of the understanding of patterns in space!

Hoorary open geography! Hooray 2013!

Posted on

Geo Terms I Want to See Disappear in 2013

Geo/spatial/location/GIS is everywhere now-a-days.  That is awesome.  No longer is what we do a specific niche that is only found in a small set of industries.  Openstreetmap, location aware devices, dropping pins, Google maps – everywhere you look geo/spatial/location/GIS matters.  Heck, even Gizmodo has a “Maps” tag.

This ubiquity is both a blessing and a curse.  Thanks to the every growing understanding of what we collectively do there has been some abuse of a few keywords throughout the geo-world (some of which I am guilty of).  For 2013 I think we should retire a few terms to keep us (me) sane.  Here is my short list.

1. Heatmap -Heatmaps are one of my least favorite cartographic representations of data across space.  I love density maps, choropleth maps, and interpolated surfaces, but misrepresenting data for the sake a cool map is a giant pet peeve of mine.  Let’s make a conscience effort in 2013 to stop people from using the term heatmap to describe any map with bright colors on it.

2. Cloud – We get it. Enough already.  To the Cloud!

3. Analytics – Overused and abused term #3.  What are you actually analyzing? I love identifying and understanding patterns across space, it’s what I do everyday, but let’s lay off using the term analytics to represent any type of math or stats done on a spatial dataset. To me analytics involve higher level operations, whereas I think people often use the term to represent basic stats.

4. Big Data – You have big data, I have big data, we all have big data.  Question.  What is big data?

These opinions are mine and mine alone.  Are there any geo related terms that you think have been overused in 2012 and need to be retired in 2013?  Leave a comment.  This could be fun.

 

Posted on

The Worst “Heat Map” I Have Ever Seen

I am no fan of geographic heatmaps, especially those that are really just density maps or poorly interpolated surfaces.  To me the term “heatmap” is just an overused marketing term.  So, when I came across this “heatmap” today I had to laugh:

“Rental Heatmap” from Boston.Curbed.com

And here is a screen grab (in case they edit their page!).

There is no heatmap here, at all (unless I am totally using this page incorrectly, which isn’t out of the realm of possibility).  No doubt, it’s a slick web-map application that really aids the story, but let’s call a spade a spade.

Side note:  Is the term heat map so popular because choropleth doesn’t sound cool?

Posted on

GeoAwesome: Ignite Spatial Boston 2012

One word: GeoAwesome.  That is how I would describe tonight’s (11/14/2012, GIS Day!) Ignite Spatial Boston 4, which was organized by Avid Geo, hosted by the Center for Geographic Analysis at Harvard University and sponsored by AppGeo, Axis Maps, NBT Solutions, Azavea, and CDM Smith.  The twelve, five minute talks covered a variety of geo-topics from analysis to workflow, apps and technology.  It was a really well-rounded night of topics.  I thought all the speakers did an excellent job.  Here is a real quick rundown of the talks:

Crowdsourcing Boston’s Neighborhood Boundaries – Andy Woodruff, Axis Maps.  Great idea, great application, great participation, and great execution.  I can’t wait to see what they do next.  Check out Bostonography for more.

Save your Mouse: Automate your Spatial Workflow with PostGIS – Jesse Bishop, Research Associate, Woods Hole Research Center. You know you’re onto something when you are saving yourself literally millions of mouse clicks. Great project and great technique. Really opens up your schedule!

Visualizing the Urban Metabolism of Neighborhoods – Dr. David Quinn, urbmet.  This was one of several talks that discussed a geospatial analysis that was data intensive, innovative, and had great visualization component.  Dr. Quinn did a great job of breaking it all down in five minutes.

Trafficked by Sea – Stacy Bogan, Center for Geographic Analysis, Harvard University.  Presented a unique approach to modeling ocean traffic networks.  During her talk she proved that projections do matter!

Geeky Boating Elk, Ryan Westphal, Lead Developer, jQuery Geo Project.  Great presentation, taking a totally wild and creative subject, and using it to demonstrate the power or jQuery Geo.

GeoHOLLIS: Mapping the ILibrary Catalog – Bonnie Burns, Harvard Map Collection.  I used to work in a map library so I can really appreciate this project.  So far they have geo-cataloged 12.5% of their 12 million records, allowing patrons to search their catalog geographically.

Fontly: Mapping the World of Vintage Typography – Brendan Ciecko, Founder, Fontly.  I was really impressed with this talk.  Check out Fontly and contribute to the project.

Estimating Sandy – Ben Spaulding and Boyd Zapatka, AIR Worldwide.  My co-worker and I gave this talk about the work we have been doing the past couple weeks.  It’s amazing how much geoscience you can jam into five minutes.

Python as an ETL – Mark Zito, GIS Specialist, CDM Smith.  Mark demonstrated a few workflows using Python as a geospatial extract,transform and load tool.  I need to ask him about getting one of his code samples.

Partly Cloudy: Real World Tales of Geo Migration to AWS – Michael Terner, EVP, AppGeo.  Michael shared AppGeo’s experiences about moving to the cloud.  Really focused on the benefits, which I thought were right on the money (he talked about cost savings, if you didn’t get my pun).

Making Sense of 500 Million Location Requests Per Day – Richard Sutton, Geospatial Lead, Skyhook.  I think everyone who was in the room (100+ people) wanted his data and databases after he finished his talk.  I can’t describe how incredible the data and analysis he described are.  The potential for this data is unbelievable.

Cartographic Ingredients from the Eye Candy Kitchen – Jeff Blossom, Center for Geographic Analysis, Harvard University.  We often forget the basics that make a map a pleasure to gaze upon.  Jeff did a great job of breaking down the basic ingredients all cartographers need to remember when creating a map.

A big thanks to Guido Stein and the Avid Geo crew for bringing us this event.  There is a strong geo-community in Boston and events like this really bring it to the forefront!  

Posted on

More Spatial SQL – Calculating Lines between Points

Recently I posted some tips and example script to measure distance between points using SQL Server.  The obvious extension of this script would be to generate the lines between two points. I wanted to create a script that took two points from a table and using STLineFromText to create the line. The vast majority of the script is the same as the previous post, with the exception of the line to create the lines.  So, here it is:

select 
t1.Id as ID, 
t1.NAMEASCII as t1_Name, 
t1.SOV0NAME as t1_SOV0NAME, 
t1.Latitude as t1_Latitude, 
t1.Longitude as t1_Longitude, 
t2.ID as t2_ID, t2.NAMEASCII as t2_Name, 
t2.SOV0NAME as t2_SOV0NAME, 
t2.LATITUDE as t2_Latitide, 
t2.LONGITUDE as t2_Longitude,
Geography::Point(t1.Latitude, t1.Longitude, 4326).STDistance(Geography::Point(t2.LATITUDE, t2.LONGITUDE, 4326)) as Distance_Meters,
Geography::STLineFromText('LINESTRING('+cast(cast(t1.longitude as float)as varchar)+' '+cast(cast(t1.latitude as float)as varchar)+','+cast(cast(t2.longitude as float)as varchar)+' '+cast(cast(t2.latitude as float)as varchar)+')', 4326) as line
from dbo.Populated_Places t1 cross join dbo.Populated_Places t2
where t1.SOV0NAME = 'United States' and  t2.SOV0NAME = 'United States'  and t1.ID != t2.ID and t1.NAMEASCII = 'Boston'

A few notes about this script:

  • I am using SQL Server 2008 R2.
  • Unfortunately, you cannot pass values directly into the STLineFromText method like you can other Spatial SQL methods.  When using the STLineFromText the user needs to convert the coordinate pairs into a LineString.  This isn’t that big of a deal since the LineString allows a users to string together a series of coordinate pairs to create complex line objects, which is a big plus (in this example I am creating lines between two points), I had to use a cast a couple times to get the coordinate data in the correct format.  I experimented with a couple different ideas and this one worked.  This probably isn’t the best way to do this so if you have a better idea please post a comment!
  • The line that is being generated is a geodesic line.  Why is that?  I am using the Geography data type and I am calling an SRID of 4326 when STLineFromText is executed.  The combination of the two will create “near” geodesic lines as opposed to planar lines.
  • Like in the previous post, a self cross join is being used to create the location pairs.
  • There are a couple items in the where statement that need to be mentioned.  The query above will not work if the coordinate pairs represent the same location, meaning STLineFromText cannot create a line where the two points represent the same space.  To prevent this error I simply set the query to not select IDs that are equal.  The results of this query return the distance and generates a line between all populated places in the United States from Boston.  To prove this works, here is a nice little image:

Lines between Points
Ok, that’s it.  Pretty simple and pretty fast (the example query takes two seconds to run). Perhaps I’ll put together a simple script to string together multiple points into a line or multi-line object next!

Posted on

Hurricane Sandy! Data and Maps

I noticed earlier that my post about last year’s major New England storm was getting a ton of action today.  It’s probably time to get some info up about Hurricane Sandy.

I’ll keep updating this page when I come across data sets or mapping sites with pertinent information about the event.  If you have a good link post it in the comments section!

Last updated 11/4/2012

Storm Mapping Sites

Hurricane Sandy Data

Twitter/Social Media Mapping


Power Outage Sites

I’m in Somerville, Massachusetts and the wind has been whipping all day.  If the power stays on I’ll try to update this page if I come across some more data and maps.

 

Posted on

Spatial SQL for the Geographer – STDistance Example

When I put together the Spatial SQL for the Geographer posts I never wrote anything about measuring distance.  Geographers often need to know the distance between locations, as distance decay is one of those things we find important.  Fortunately for us, measuring distance between points is pretty easy using Spatial SQL (and pretty fast).  The following query demonstrates an example of measuring distance between a set of defined points from the populated places data available in the sample data.

select 
t1.Id as ID, 
t1.NAMEASCII as t1_Name, 
t1.SOV0NAME as t1_SOV0NAME, 
t1.Latitude as t1_Latitude, 
t1.Longitude as t1_Longitude, 
t2.ID as t2_ID, t2.NAMEASCII as t2_Name, 
t2.SOV0NAME as t2_SOV0NAME, 
t2.LATITUDE as t2_Latitide, 
t2.LONGITUDE as t2_Longitude,
Geography::Point(t1.Latitude, t1.Longitude, 4326).STDistance(Geography::Point(t2.LATITUDE, t2.LONGITUDE, 4326)) as Distance_Meters
from dbo.Populated_Places t1 cross join dbo.Populated_Places t2
where t1.SOV0NAME = 'United States' and  t2.SOV0NAME = 'United States'

You will notice that in the query a few different things are happening:

  • A self cross join is used to take each record and join it to every other record.  This is done to get the lat/long combo for each city and matched up with every other city.  There is probably a better way to do this, but for this example it works.
  • In order to calculate distance two points are generated on the fly from the columns by calculating the point with Geography::Point(LAT, LONG, SRID).  Setting the SRID is important, as that will impact the measurement.  More often than not the user will be using an SRID of 4326 (WGS84), unless they are using projected data.  For more info on SRIDs check out this handy Wikipedia article.
  • You could replace Geography with Geometry when generating the points.  Doing so would lead to vastly different measurement results because of the way distance is calculated on the plane or on the spheroid.  When using the Geography data type the distance results are returned in meters while with the Geometry data type the results are reported in decimal degrees.
  • Once the points are created on either side of the STDistance method the query can be run.  The documentation provides a couple notes about using STDistance. The SRIDs need to match between the points and there is some error involved when measuring distance.  The distance measurement error is important to understand if one has tight tolerance they need to adhere to.

The results of this query will return a set of columns with the measurement in meters from every record to every other record for US cities.  With some additional coding you can generate a distance matrix, find the nearest locations, or create line features between points pretty quickly.  This query can also go pretty fast, if the proper indexes are generated and called in the query it should take less than a minute.  Not to bad when returning over 600k records.