It’s 2016 and I Want to be Better at What I Do

Welcome to the future, 2016. It’s nice to meet you.

2015 was a great year.   I did a lot of cool stuff, and almost all of it wasn’t geo-related.   Seriously, like 95% of the things I did this year had nothing to do with geo or GIS.  That’s all fine and good, but I really love geo. I went to school for a long time to just kinda give up on it in the past year.

During the last few days of 2015 I made a list of things I want to do in 2016.  The list is fairly short. I want to start to learn Mandarin Chinese, become a better runner, read more, remember my passwords to stuff, and get better at what I do.  Geo is “what I do” and I really want to get better at it in 2016.

I know the path to meeting a couple of my 2016 goals, but how do I get better at geo? I’ve been working in geo for almost 10 years.  I have a job that allows me to challenge myself on a regular basis, and do a lot of geo work.

In this case, I think the path to getting better is by doing more. Practice makes perfect. In 2016 I will strive to create more maps, tools, algorithms, datasets or anything else.  This, I hope, will force me to learn new skills, hone my existing skills, and inspire me to try new things and test new ideas. I won’t just leave ideas written down on a post-it note on my desk.

I will make some maps from those ideas I have written down, post tools and code I create to my deserted github account, write more technical and research/analysis GISDoctor.com posts (even if they aren’t any good), contribute to my local geo-community, and make a real effort to add to OSM more than I have in the past.

By doing more, I think I can keep challenging myself to expand but also refine and tighten my geo-skills. More importantly, I will keep myself motivated and interested in geo! I’ve definitely already put the first 10,000 hours in.  Maybe the next 10,000 will really define who I am as a professional geographer and member of the geo-community.

Here’s to what is hopefully the most productive and inspiring geo-year of my life!

 

 

 

In 2017 I will stop over using the prefix Geo.

Spatial SQL – Multi-Point to Line Example

I have been using Spatial SQL for a while now.  I like it.  A few lines of code can do a lot of analysis or data processing.  I’ve covered a number of basic topics but there are always more to do.  Here is a script to take a series of points and convert them into a single line.   This script will use a few SQL commands, including using a cursor, developing a linestring, and STLineFromText.

The sample data comes from NOAA and the National Hurricane Center. The points represent some sample tropical cyclone forecast points for Hurricane Sandy.

The basic idea of the script is to link a set of points together using a common attribute using a cursor, combine the coordinate pairs into a line string, and use the  STLineFromText method to convert the coordinate pairs into a single line.  The script works pretty well, but since I am using a cursor it can slow down with larger (a few hundred thousand rows) datasets. A good SQL programmer probably wouldn’t use a cursor here since they can be slow and cumbersome to use.  In fact, I’m sure a good SQL programmer wouldn’t use a cursor.  I am investigating ways to not use a cursor, so if you have a suggestion let me know!

The sample script includes a sample dataset that is available here.  Take a look at the script.  If you have any suggestions to make this script faster or more efficient let me know.

/*#################################################

Script Name: multi_points_to_Line.sql
Purpose: This script will take pairs of coordinates
 of the same line and convert them into a LineString 
that can be used to generate lines of the geometry 
data type. User will need to update the database, 
tables, and column names relevant to their own analysis.

Sample Sandy data tracks represent five day
models from the National Hurricane Center.

Prepping the data - The user will need to download 
the following file and load into their SQL database:

http://www.gisdoctor.com/downloads/Line_Parts.txt

Here is a quick script to take the text file and load 
it into a table generated for this exercise:

create table Spatial_Database.dbo.Distinct_Points
([ADVISNUM] varchar(3), [lat] float, [lon] float, 
[MaxWind] int)

BULK INSERT Spatial_Database.dbo.Distinct_Points
FROM 'Path to Line_Parts.txt file'
WITH (FIELDTERMINATOR =',',FirstRow = 2);

###################################################*/
 
--Set the database to process in
use Spatial_Database
--Drop temporary table
drop table #Sandy_hur_tracks
--Create new temporary table
create table #Sandy_hur_tracks
([EventID] int, [line] geometry)

--Declare cursor variable
DECLARE @eventID varchar(10)
--Declare text string that will store coordinate pairs to
--populate the LineString
DECLARE @coordString VARCHAR(MAX)

--Initialize the cursor using the ADVISNUM column
DECLARE db_cursor CURSOR FOR
select distinct ADVISNUM
from Spatial_Database.dbo.Distinct_Points
order by ADVISNUM asc

OPEN db_cursor
FETCH NEXT FROM db_cursor INTO @eventID

WHILE @@FETCH_STATUS = 0
BEGIN

-- Clear the coordinate string with each iteration of the 
-- cursor - otherwise the coordinate string will 
-- append itself each time
set @coordString = ''
--collect all coordinate pairs and add them to a single row. 
-- Coordinate pairs are separated by a comma.
select @coordString = (COALESCE(@coordString + ', ', ' ') + 
(cast(Lon as varchar) +' ' + CAST(lat as varchar)))
FROM Spatial_Database.dbo.Distinct_Points
WHERE ADVISNUM = @EventID

--Insert the eventId and coordinate pairs into the table.         
--Coordinate pairs string is used to build the LineString to      
--create the line geometry
insert into #Sandy_hur_tracks
select @eventID as EventID,
Geometry::STLineFromText('Linestring 
(' + right(@coordString,LEN(@coordString)-1) + ')' 
, 4326) as line

FETCH NEXT FROM db_cursor INTO @eventID
END
--Close and delete the cursor
CLOSE db_cursor
DEALLOCATE db_cursor

--Select results from the temp table.
use Spatial_Database
select * from #Sandy_hur_tracks

Geo Ideas I Want to See More of in 2013

I recently wrote about the geo-terms I wanted to retire in 2013.  However, there are a few topics I want more of in 2013.  The term I want to see more of in 2013 is “open” and here is what I was thinking…

Open Source –  The open geo-software community had a great 2012.  I really see this momentum continuing to grow in 2013.  The key to continued adoption (beyond great software, easy to use platforms, and continued innovation)? Get open geo-software into academia.  The more undergrads who learn GIS on Quantum, GRASS, PostGIS and the rest, the more this movement will continue to expand.

OpenStreetmap – Recently Openstreetmap hit 1 million users.  As a somewhat semi-regular contributor I see great promise in OSM but OSM can’t end up like Wikipedia, which is losing editors and contributors.  OSM can never be completed, and the army of volunteers will hopefully see that.  I should probably do some mapping this weekend!

Open Analysis – I would love to see the the geo-community become more open with analysis. This could include sharing analysis techniques, working together to develop new analyses, or helping the world understand geospatial analysis.  A map is far more than the visualization of the abstraction of space. Let’s start promoting the science of the understanding of patterns in space!

Hoorary open geography! Hooray 2013!

Geo Terms I Want to See Disappear in 2013

Geo/spatial/location/GIS is everywhere now-a-days.  That is awesome.  No longer is what we do a specific niche that is only found in a small set of industries.  Openstreetmap, location aware devices, dropping pins, Google maps – everywhere you look geo/spatial/location/GIS matters.  Heck, even Gizmodo has a “Maps” tag.

This ubiquity is both a blessing and a curse.  Thanks to the every growing understanding of what we collectively do there has been some abuse of a few keywords throughout the geo-world (some of which I am guilty of).  For 2013 I think we should retire a few terms to keep us (me) sane.  Here is my short list.

1. Heatmap -Heatmaps are one of my least favorite cartographic representations of data across space.  I love density maps, choropleth maps, and interpolated surfaces, but misrepresenting data for the sake a cool map is a giant pet peeve of mine.  Let’s make a conscience effort in 2013 to stop people from using the term heatmap to describe any map with bright colors on it.

2. Cloud – We get it. Enough already.  To the Cloud!

3. Analytics – Overused and abused term #3.  What are you actually analyzing? I love identifying and understanding patterns across space, it’s what I do everyday, but let’s lay off using the term analytics to represent any type of math or stats done on a spatial dataset. To me analytics involve higher level operations, whereas I think people often use the term to represent basic stats.

4. Big Data – You have big data, I have big data, we all have big data.  Question.  What is big data?

These opinions are mine and mine alone.  Are there any geo related terms that you think have been overused in 2012 and need to be retired in 2013?  Leave a comment.  This could be fun.

 

Sunday Morning Geo-Fun

Why is it that the only time I have to blog is on Sunday mornings?  Here are a few quick geo-items that tickled my fancy from the previous week.

  • Check out the article, The New Cartographers: OpenStreetMap’s World Takeover, from Carl Franzen at Talking Points Memo.  The first two parts of this story have been tweeted a lot this last week and I can see why.  The article provides a fairly good overview of OSM, including some background on the project, the nuances of licensing OSM data, and adoption in the tech industry.  Part three of the article comes out on Sunday.  Makes me feel good about calling 2012 the year of OSM.
  • Years ago I used to pump out Google Map Mash-Ups on a regular basis, some of which were developed during my time at the Map and Geographic Center at the University of Connecticut.  Well, after nearly two years one of those mash-ups got some press!  Check out the article in the Atlantic, Pre-Sprawl Aerial Images:’Next Best Thing to a Time Machine. The article discusses the dual-map mash-up that I developed for the On the Line Project that is used to compare the drastic changes in Connecticut’s landscape using current and historical aerial photography.  Pretty cool.
  • The guys at Google’s NC data center, which just got the indoors street view treatment, definitely Rickrolled streetview (Check out the image on the screens, also, why didn’t they blur out Rick Astley’s face too?).
  • Brian Flood has been doing a lot of great things for the online mapping and spatial data communities for a while now.  This video and post on the MapBox blog is the latest example.  Using Arc2Earth Sync to integrate with MapBox and ArcGIS appears smooth and simple.  Awesome.  There is a lot of great work happening in “spatial” and it’s only going to make what we do as geo-professionals better.
  • Speaking of MapBox, when does Esri try to scoop them up (if they haven’t already), like they just did with GeoLoqi?
  • Avid Geo Boston had their October meet-up this past week.  The video is here. Since I am a horrible member and missed the meet-up for the third straight month I cannot comment on the talks, but I’m sure everyone had a good time.
  • Avid Geo will be hosting their wildly successful annual Ignite Spatial event on November 14th at the Center for Geographic Analysis at Harvard.  Tickets are available here, and they are currently looking for presenters.
  • Don’t forget to take the totally unscientific GISDoctor.com ArcGIS 10.1 survey!
  • Finally, I’ll be updating some pages on my site this week, including the blogs page and some of the mash-ups.
  • As always, follow me on twitter @GISDoctor, and hopefully I’ll blog more this week.  I have tons of ideas!

 

Whaddya think of 10.1?

ArcGIS 10.1 has been out for a few months now and I am curious as to what the GIS public thinks of the newest release.  I haven’t upgraded any of my Esri products to the newest release yet, and probably won’t for a while, but I want to gauge the public’s reaction to the most recent “dot” release.

If you have any other thoughts or comments on ArcGIS for Desktop (ArcMap), ArcGIS Online, Esri Maps for Excel, or anything else spatial leave a message!

I’ll post the results in a couple weeks.

Here is the awesome, non-scientific, one question survey.  Take it!

Create your free online surveys with SurveyMonkey, the world’s leading questionnaire tool.

Report Bugs! Do it!

Like all of you, I run into the famous (infamous?) Esri error report, more often than I like.  Ever wonder what happens when you actually report the error (which I do when I have the patience)?  Check this out this blog post from Esri’s Resource Center.  One item that I took away from the article is that I need to add more information to my error report than just my frustrated babbling, as Esri’s error reporting algorithms evaluate the available information before it is passed on to the engineers.

If you are a heavy Esri user and run into the error report often you should take the five minutes to read the post.

Open Job – Senior Software Engineer – GIS / SQL Spatial Application / C# – Boston, MA

AIR Worldwide, based in the Back Bay section of Boston, Massachusetts is advertising an open position for a GIS/SQL spatial software engineer.  The position requires 3-5 years experience in software development and strong base of GIS and GIScience skills.  AIR develops natural catastrophe modeling software (a lot of spatial analysis) that is used by a variety of insurance companies, financial institutions and governments to help them understand their risk from natural catastrophes.

Are you a developer with some serious GIS chops, or a GIS pro with some serious programming chops?  Looking for a job in Boston?  Would you like to work with a scientists, engineers, programmers, analysts and others on  a variety of interesting projects? Then I would recommend you check this job out!

FYI, this is a shameless plug

Blogs for Wannabe GIS Programmers (like me)

Anyone who knows me and works with me on a regular basis knows that I am not a developer.  I am a geographer who develops code for visualization and analysis applications that will hopefully work.  In our line of work knowing how to write and understand code is critical and on my quest to become a better programmer I am continuously searching for the next resource to add to my collection of how-to guides and programming resources.  Some of my favorite resources are spatially focused programming blogs.  Usually the bloggers are facing the same problems I am dealing with and they are using jargon that I understand.  These two factors make it much easier to follow their examples and ideas.

Here are three blogs (from people who know what they are doing) that I follow (do people still follow blogs, or is that so 2007?) and have referenced in my quest to improve my marginal programming skills(clear overuse of () in a sentence).  Check these blogs out when you get the chance:

GeoChalkboard – A number of Esri javascripting posts, which is great for me, since I am doing a lot of that type of work lately.  Also, professional courses are made available through the site.

Guerilla GIS – I’ve referenced this blog before, mostly because I like two things about it.  The variety of code examples and the GIS snark.

odoenet – A number of GIS programming examples, along with a number of tips a tricks.  For those of you new to GIS programming  should check out the two blog posts about simplifying GIS development in ArcGIS.  Not a bad read.  Also, this post is very true.

I know there are many more blogs like this out on the interwebs.  If you know of one or have a favorite spatial programming resource post it in the comments section!