Posted on

A Few Geo-Things I Learned in 2015

Let’s wrap-up the year with a few things I learned in 2015.

  • 95% of all the maps you see on the internet, or that you make for your own purposes, could probably be replaced with a nicely formatted table.  The exception – those cool web maps that show animation with some sort of time value. Those are rad.
  • QGIS, R, python, and PostGIS keep getting better and better.
  • ArcGIS Pro, at least for me, totally missed the spot.
  • Formatting and sharing data is easy.  Knowing how to properly analyze it when you have it takes skill.
  • Everyone has big data and no one has big data.
  • I really don’t care about online mapping.  It’s cool, but I am a technical analyst, and numbers and stats are more important to me in my line of work. However, I see the incredible value of what CartoDB, MapBox and others are doing to push the geo-tech agenda forward.
  • Don’t believe anything you read in blogs or on twitter.   Especially mine 🙂
  • Using “Geo” as a prefix automatically makes you look like you know what you are doing.
  • All the cool kids are doing geospatial analysis (see what I did there…).
  • Spatial isn’t special. If it really was, we would all be making a lot more money.
  • If you are young, in college, and interested in GIS, I recommend you go into a natural/earth science or engineering field. Learn some stats, and get some programming. You will either learn how to use GIS on the side in a required course or in the job you got with your degree in your technical field where you learned how to do analysis. I don’t think majoring in “GIS” is a good idea anymore.
    • side note – If you want major in geography, make sure you go the quantitative/technical route.

I hope 2015 was great for everyone.  I am glad I started blogging again and I am glad people are still reading my posts. I have big plans for 2016! Now, let’s see if I actually implement those plans…

Posted on

Some Thanksgiving Geo-Thoughts

Happy Thanksgiving!  I’ve been running a lot recently, and when I run, I think. Here are a few things that have been bouncing around my brain while dodging traffic.

  • Is it just me, or is every “viral” map that hits the internet lately powered by either CartoDB or Mapbox?  It’s pretty impressive to see these two companies gain traction and grow, especially in the non-traditional geo-crowd.  If anyone one sees a “viral” map from Esri, that wasn’t made by Esri, let me know.
  • Recently at work I’ve been building a custom interpolation algorithm to analyze some model output.   The out-of-the-box tools in the standard GIS suites usually aren’t flexible or fast enough for my work, so I have been using scipy and numpy to code some stuff up.  I hadn’t used the spatial component in scipy until this project. Scipy spatial, in combination with numpy, has been a powerful alternative for coding the analysis. If you haven’t, you should check it out. Since I am working in a modeled spatial environment (sparse and standard grids), I’ve been using Jupyter Notebook for an IDE and matplotlib for on-the-fly visualization, both of which are awesome.
  • I am excited that Boston has been chosen as the host city of FOSS4G 2017!  I haven’t been to a FOSS4G conference yet, but I am 100% going to FOSS4G in 2017. Big thumbs up to Avid Geo, the folks at AppGeo, and others in the Boston geo-sphere who put the proposal together.  FOSS4G is the future of geo and GIS. There is no ifs, ands or buts about it.
  • Rant time – We, as an industry, need to get colleges and universities to stop providing (or drastically improve) GIS certificate programs (also, we need to end “technical certifications”, but that’s a whole other rant).  For the most part, many don’t do enough to really help someone trying to become a geo-professional or improve their skills if they are already working in the field. In many cases, I believe they can railroad students into a skillset that is really only viable in the very narrow role of a GIS technician.  Those departments that teach GIS and geospatial analysis should work more on developing and integrating cross discipline geospatial curricula that provides technological depth, solid analysis skills and strong problem solving skills.  All too often I have seen the “Open ArcMap, push some buttons, use some shapefiles, and make a crappy map project route” approach to fulfill requirements. This is a horrible disservice to students, many of whom are competing for jobs that are becoming increasingly tougher and tougher to land.  Geospatial higher education has to be better. Colleges and universities don’t need to be career training centers, but they should teach the skills necessary to for their students to be able to compete in today’s environment.

That’s it for now!  Have a good holiday season.

Posted on

Why OpenStreetMap is the Most Important thing in Geo

Not too long ago, Bill Morris, posted the following tweet:

Go read the blog post that Bill links to from Mapzen and then come back here.

I love OpenStreetMap. It’s the best. In my opinion, it’s the most important thing to have happened to the geo community since Dr. Tomlinson started working with map overlays in the 1960s*.

Geo-tech has rapidly grown and matured over the past two decades. Not long ago, most things GIS and geo-tech were reserved for governments (local to national) and academic institutions. Thankfully, that isn’t the case in today’s world. Geo-thinking and geo-tech is everywhere and OpenStreetMap is a major reason why.

But how has OpenStreetMap helped make location and geo so prevalent to a growing variety of industries and institutions outside of government and academia? Why is that story that Bill linked to so important?  I think there are a few reason.

  • OpenStreetMap has greatly expanded access to geo data worldwide – When OpenStreetMap started over ten years ago it was created as a response to limited geo data access, primarily in the UK. From those early days to now OpenStreetMap, and its many partners, have made access to both consuming and contributing data increasingly more easy and efficient – for anywhere in the world. Here in the US of A we have always had pretty great access to some level of geo-data, but that hasn’t always been the case in many other countries.  Beyond opening up data resources, OpenStreetMap has also helped introduce spatial data to a wide audience, including many with very little knowledge of geo as perhaps you or I know it.  That’s awesome.
  • It is open (and flexible) – Some early detractors (including myself) were once worried about OpenStreetMap undercutting the “authoritative” and “established” (and for cost) data providers in the marketplace.  I think there was an early worry about data quality and data trust. I believe that almost all of those worries are now gone. The fact that anyone with a web connection can view or edit the data makes it that much more powerful. And because it is open, I believe the people who contribute (many of whom are “pros”) take their adds and edits very seriously. I know I do. OpenStreetMap is also incredibly flexible.  Even though the commercial data vendors have gotten much faster at incorporating updates (in general, governments are still slow), OpenStreetMap is known for its ultra-fast updates, primarily due to its dedicated community of contributors. That, to me, is one of its greatest advantages. The data is never old on OpenStreetMap.
  • It is for the greater good –  One of my favorite things about OpenStreetMap is its large and dedicated community. One of the best parts of this community has been HOT OSM, whose members help coordinate updates to OpenStreetMap to areas impacted by natural or humanitarian disasters. The data they develop in the hours and days after a major event (for a great example, look at their work following the 2010 Haiti earthquake ), especially in developing regions, is critical to many, including those on the ground aiding relief efforts. See also @TheMissingMaps project.
  • It has enabled an emerging industry to grow – OpenStreetMap has forever changed the way we look at geo-data and the technology surrounding it. This change in perspective has helped foster a growing number of geo-enabled companies.  Would companies like Boundless, CartoDB, Mapzen,  Mapbox, or Mapkin be as successful if OpenStreetMap wasn’t around? Maybe. Maybe not. What about the success of the Leaflet API, or the growth of FOSS4G? Would they have been as important in the geo and tech worlds if it wasn’t for OpenStreetMap? Probably, as they are both awesome. However, I believe that because of OpenStreetMap and all that it provides and all that it stands for, that these types of organizations, ideas, and companies will continue start-up and contribute positively to our profession.

So, let’s go back to the beginning of the post.  Why did I link the tweet from Bill about Mapzen’s latest efforts regarding Mapzen Search?  Well, I think their work, along with many others, is proof of how valuable OpenStreetMap really is and will continue to be. The OpenStreetMap community is maturing and the industry surrounding it is expanding. We have only seen the tip of the proverbial iceberg when it comes to the potential of OpenStreetMap and what it can do.  I look forward to being apart of what’s next.

You should too.

* sidenote, GPS came in a very close second

Honorable mentions for “Most Important thing in Geo” – GPS, QGIS, TIGER, ArcView 3.2, Landsat, Smartphones

Posted on

Long = X, Lat = Y – Free Illustration Included

latlong_final
Feel free to use this everywhere. Print it out and hang it on your wall.

My biggest geospatial pet peeve is when people confuse the relationship between x/y and lat/long. I believe this is a fundamental concept in geography and I am here to reintroduce you to math you probably learned in elementary school (middle school maybe?).  If you already know the proper x/y-long/lat relationship then you can skip the rest.

First, let’s review the Cartesian coordinate system.  The X axis is the horizontal plane and the Y axis is the vertical plane. Two dimensional.  Pretty simple.

Now, let’s look at latitude and longitude. Latitude measures angular distance from the equator to a point north or south of the equator. While longitude is an angular measure of east/west from the Prime Meridian.

Now this is the important part.

  • Latitude values increase or decrease along the vertical axis, the Y axis.
  • Longitude changes value along the horizontal access, the X axis.

Easy.  X = Longitude, Y = Latitude.  Now you will no longer be confused when calculating centroids or setting webmap coordinates.

Happy mapping!

 

Posted on

I’m Back!

It has been a long time since I put GISDoctor.com into sleep mode. I haven’t posted anything here since 2014 and I have actually culled material from the site.

I did this for a couple reasons.  I was busy with life and I wasn’t all that excited about GIS. Things got a little stale for me. I was bored. I was having a little bit of a early-career crisis in my early 30s.

Professionally, my job has been excellent.  I am challenged with new problems everyday. But at the same time, the field of GIS wasn’t as awe inspiring for me as it was five or ten years ago (yes, I am old enough to say I have been doing this for over 10 years). So, I spent the past 18 months or so doing other stuff and not GIS blogging.

It was a great 18 months.

I’ve spent over a year being a content consumer.  I want to be a creator and contributor again.  I plan on posting with some relative frequency, tackling various technical and non-technical geo-topics – some short form and some long form.  I am not going to write exclusively about GIS, but rather the larger, more exciting geo-space.  There are so many amazing things happening right now across the geo-spectrum.  There are more people who consume maps, geospatial data and analysis, and location technology than ever before.  But, I see a big gap in the our space.  There are a relatively small number of people like me, the hardcore technical geo-nerd, but there countless casual geo-enthusiasts.  I hope to write about topics that the hardcore technical geo-nerds will appreciate and that casual geo-enthusiasts will understand and find interesting.   It’s probably a lofty goal, but I am up for the challenge.

It feels good to be back.

 

 

Posted on

Boston: America’s Most Active Geo-City!

That’s probably an overstatement.

But…

The Boston geo-calendar is jammed packed with awesome stuff right now.  If you are in the area over the next few weeks, check out some of these upcoming events:

Now, I am sure I missed a few events.  If you know of an event that I missed tweet it out to @GISDoctor and I will make sure to add it to this list and retweet to my followers!

Posted on

Not the end, yet…

As with almost all blogs at some point in time the writer(s) eventually lose interest, become busy with other projects, or just fade into the digital sunset.

I am awfully close to meeting all three of these categories.

It’s time for something different.

When I started GISDoctor.com I never had any real expectations of what I would do with the site, and that shows with the topics I have written about over the past several years. For better or worse, I haven’t written much over the past year.  Between my job, renovating a multi-family home, and literally running a 1,000 miles I haven’t had the time, energy or desire to keep the site up-to-date.  Twitter has been great for keeping my ideas out in the social geo-sphere, unfortunately, I just haven’t had the inspiration or time to put many ideas or projects into this medium.

But, it’s not truly the end.  GISDoctor.com isn’t being turned off, but the focus is going to change.

I still have a lot of ideas I want to share with the community I love.  Even though I have a great passion for geo-tech I will avoid writing about it.  Over the past couple years I’ve been observing the great GIS-troll war.  No matter what project you are interested in, what technology you use, or how you use a given technology there is someone out there who will have no problem trashing you, without any sense of context or content.  We are all guilty of this.  Hell, I provided one of the most popular outlets for this very practice.  But for me, this has run its course and I am really tired of it.  Geo as grown by leaps and bounds over the past 10 years.  We should be working harder to do more as a community to solve problems and develop ideas, regardless of our personal “stack” choices.

So, where does this leave me and the site at this point in time?  Well, I will be cleaning up GISDoctor.com, turning off the comments (if you have anything to say, just twitter it), retiring some old code example pages (I still see some of my Google Maps examples still being used, which is neat), and updating some of the more actively viewed pages on the site. I also hope to change the feel of the site.  When I do post, it will be about the G(eography) in GIS. Tech trends come and go, but the geographic problems we are interested in solving, and desperately need to solve, are here to stay.

Thanks for all the clicks.

 

Posted on

The Tool Belt Approach

Firstly, it’s been a while since I’ve blogged.

I’ve been busy.

My wife and I bought a two family home in South Medford, Massachusetts a couple months ago. It’s a nice little place, in a walkable neighborhood with access to transit, and it’s only 4.5 miles from downtown Beantown (no one here calls it Beantown). The home, as they say, is a “fixer-upper” and both units need lots of work. My wife and I have been spending every waking moment doing yard work, rebuilding our first floor unit’s kitchen and bathroom, painting walls, doing demolition in our basement, working with our plumber and electrician as they rewire and replumb the entire home, negotiating the city hall permitting process, Et cetera, Et cetera, Et cetera…

During the past couple months not only have I learned a ton of new homeowners stuff, but I have acquired a ton of new tools (consignment tool shops are the best place ever). Thankfully, I grew-up in a very handy family so I’m not totally in the dark when it comes to home improvement and these tools come in handy. I’ve learned that not every project needs every tool. Before I start a project I scope out what I need to get done, load up my tool belt and get to work. I don’t haul the entire toolbox (or toolboxes) to the project each time.

My tool belt is a wonderful thing. It is lightweight, I only load up what I need for the specific project, and it and forces me to think about my project and make the right planning decisions.

I see so many parallels between my tool belt project approach and what I try to do as a geo-professional.

In the spatial world we often get tied to the idea of the toolbox(es) when working on analysis projects. Toolboxes, whether geo-toolboxes or regular toolboxes, are often full of tools one doesn’t need for a specific project, and sometimes they can be full of tools we use improperly (how many of us have actually used Kriging in the right context or tried to use a flat head screw driver as a chisel?). Without proper planning – planning out a project before you even start – may cause one to use tools in their toolbox incorrectly, perhaps coming to less than a correct conclusion.

We, as geo-professionals, will be much better at what we do if we learn how to solve the problems and answer the questions related the projects we work on first, instead of trying to know how to use every tool in our toolbox. Yes, there will always be the plumbers, contractors, and electricians who have every tool that there could ever be related to their job, just as there will be those all-knowing GIS gurus. However, the vast majority of geo-professionals are those who do other things and not “all GIS, all the time.” I really believe that by using the tool belt approach we can develop a better class of geo-professionals. Understand your problem, do the research to solve it, and then load your tool belt with the proper tools to solve it. And, good, detailed geospatial analysis like good, detailed home improvement never goes as fast as it does on HGTV.

Now, where did I put my hammer?

Posted on

Finally on GitHub

I’ve had a GitHub account for a few months but I never posted anything to it, mostly because I am a busy guy, and I ‘m not very confident in my code (I am not a developer, but I know how to cause some trouble).

Well, I finally got over my fears and I posted three repos on my GitHub page for three simple Leaflet apps: a random map generator, a icon toggle app, and an extent tracker.  I don’t know if these particular apps have any use to anyone, but I figure it’s better putting them up on GitHub than burying them on my website.  Speaking of burying them on my website, I’ve updated the code examples page to include the three working copies of the aforementioned pages:

There are plenty of areas of improvement in my repos (I didn’t comment anything and my JavaScripting is probably not in the best form), so I’ll hopefully be making some new commits over the next couple weeks.  Now I just need to get the hang of checking code in to GitHub on a regular basis.

 

Posted on

2013 Esri UC Wrap-Up

Now that I am back in Boston it is time for my annual Esri UC wrap-up.  There were between 10,000 to a million people at the conference.  The final count all depends on who you were talking to.  Here is my brief rundown.

Biggest Themes – from my perspective…

Esri made a big push to make this a geo-tech conference, not a ArcThis or ArcThat conference.  I think many of the participants appreciated the change.

Web  GIS – or is it WebGIS -Big expansion of the analysis capabilities of Esri’s online products.  It seems to me that this was GeoIQ‘s big unveiling.  Thankfully the word “cloud” was used sparingly during the plenary. ArcGIS Online was pushed hard again, which makes sense.

JavaScript – Did you know that Esri has Flex and Silverlight web APIs?  If you came to this conference and didn’t know that you probably weren’t going to find out very easily. JavaScript was a major focus of Esri’s web products at this conference, and understandably so. It’s what everyone is using or going to be using very soon.  Also a lot of node.js, CSS3, and HTML5.

Big Data – There were a number of big data sessions.  I hate the term big data, since many have no idea what big data is.  However, the folks from Esri did a good job breaking down the terminology, it’s value to the geospatial community, and how to get started using big data, through Hadoop.  The presenters from Esri were very clear that big data systems and analysis aren’t for the faint of heart and not every GIS procedure or dataset requires a big data approach.  It was good seeing Esri pay attention to an emerging area in geospatial and data analysis.

Analysis/Data/Imagery – Esri will be providing a number of new data, analysis and imagery services in the coming months through their online services.  Some free, some at cost, some requiring an AGOL account, some not.

Other stuff

  • ArcGIS Professional – Most were confused as to what ArcGIS Professional was, myself included.   Was it the next version of ArcMap?  Does it replace ArcGIS Desktop? When is it going to be released? Being a confused geo-user, I went and talked to the ArcGIS Professional development team in the tech center.  I had a nice talk with a developer who explained that ArcGIS Professional, which is a totally new application (fully 64 bit, ribbon interface, streamlined tools), is meant to sit side-by-side with ArcMap as a stand alone application.  I am sure you will see more and more of ArcGIS professional over the course of the next year as the product moves forward.
  • STEM – Esri has pushed geo and GIS in the STEM fields for a while now.  During the plenary there was plenty of STEM education talk, including a (somewhat awkward, somewhat great) appearance by Will.i.am.  As I have said before, geography and GIScience fits in very well with STEM, and I appreciate any effort by any organization to push the geo fields in STEM.
  • Big new 10.2 features – GeoEvent Processor, easier access to ArcGIS Online, faster geoprocessing tools, and a number of quality improvements.
  • More Esri on GitHub.  Good.
  • Leaflet!
  • A rumor on the street was that there were a lot fewer federal users at the UC this year. Since I’m not in the federal space I couldn’t say if this was true, but there were fewer people in the defense section of the exhibit hall and there was a lot less talk of GIS for drones.

The fun stuff

I set a PR in the Esri 5k – 21st overall, 19:32!  Here is proof!

5kFinish

Big thanks to Mike Olkin for pushing me early and helping me set my pace!

Half priced Padres tickets = great seats

IMG_1100

And the Evening in the Park in the park was great again.  The key – get there early and get in line at the good food spots first.

IMG_1137