Posted on

GISDoctor.com has gone @social

GISDoctor.com is now up to 2009 standards.  I have started a Twitter account for the site to spread the message, keep up with what others are doing, and to follow the trends.

Follow GISDoctor.com at @GISDoctor on Twitter or check out Twitter.com/GISDoctor.

Now, I’m a big fan of the real world so I may not tweet fifteen times a day, but I’ll keep the blog and twitter feed in sync.  As of today I have one follower and guess who it is…@Esri!  They have probably read the comment sections of this post and this post…

Happy GIS tweeting!

Posted on

ArcGIS Viewer for Silverlight – That was easy

This past Wednesday, on the ArcGIS Server Blog, Esri informed the geo-nerd world that their ArcGIS Viewer for Silverlight v1.0 is now available for download to the world.  The viewer is essentially a wizard for creating Silverlight based ArcGIS mapping sites with a number of easy to use features.

Side note: They also released a updated version of ArcGIS for SharePoint, but since I don’t use that I’m not going to write about it.

I downloaded the ArcGIS Viewer for Silverlight and gave it a spin.  Now, I haven’t built a Silverlight app in a while as I have been focusing my attention on building Javascript applications, you know, because Silverlight is dead (jk, not really).

I had seen demonstrations of the demo version a couple times, but I had never tried it out.  Also, I had never used the ArcGIS Viewer for Flex so this was a brand new experience for me.

Here is what I thought:

  • Installation was pretty quick and I had the application up and running in a couple minutes.
  • I followed the quick start guide until I got into the application.  Once I was in the application I used the guide that was available within the tool.  I thought this was a great feature, as it can walk anyone through how to create a mapping application.
  • Following the in-program guide, I selected my basemap, easily established the “look and feel” of the site and started to add data.  I even added a logo and a couple links to the menu bar.  This literally took two minutes.  Real easy and really fast.
  • At this point I could have launched the application, but I decided to play around with the layers in the app.  For my test application I added a US counties layer, a US states layer and a few thousands points from my company’s ArcGIS Server.  I was easily able to set a number of options for the polygon data, including the altering the symbology, the attribute table options, and pop-up info.  However, when I was working with the point data the options to configure the attribute table and symbology were not available.  Now, I’m sure there is a reason for this and if I had read more the of documentation I’m sure I would have been able to figure it out.
  • One of the nice features of this application is that the page updates while you are working on it.  This is a great feature if you are not a developer or GIS expert and you just want to get a map to the web.
  • Once I was happy with my test page I simply clicked the deploy button and the application was launched on my local machine.  I sent the link to a couple coworkers and they were able to view the map and data and asked how they could create their own mapping apps.  They thought the ArcGIS Silverlight Viewer was super awesome.
Two Minutes, Pretty Easy

All this took about 12 minutes while I was waiting for a geoprocessing task to complete. I didn’t even scratch the surface of the options available in the application.  When I get a few more minutes I’ll test out other features including adding tools and geoprocessing tasks, data from ArcGIS online, and working with the layout and display options.

Traditionally I have written the code for these types of applications, which at times can be cumbersome, especially if you are managing a variety of sites.  If a user needs a quick, light weight, Silverlight mapping app and has access to well developed ArcGIS Server I would recommend this tool.

That was easy... Is this a SOPA violation?

I have to say that using tool was refreshing, especially after a couple of tough GIS software days.  You know what I am talking about.

Good work Esri!  When does the ArcGIS Viewer for JavaScript with HTML5 support come out?

Posted on

Tip of the Day – Check the Code

Some of the most popular pages on GISDoctor.com (shameless self-plug) are those that provide examples of how to read WMS data into Google Map applications from a number of sources including ArcGIS Server, OpenLayers, and MapServer.  People take the code from these pages, embed the scripts into their own applications, make some modifications, and publish their own mash-ups.  I think this is awesome and I encourage it on the pages themselves.  I’ve seen a few good applications of this code and I hope more come in the future.

However, I need you, the mash-up artist, to make sure you do one little thing when copying the code into your own applications. Do not copy the Google Analytics code into your own page.  This is the last two blocks of script at the end of each page.  To aid the user I have included a comment to remind them to not to include the Google Analytics script in their application.

Thanks!

A few times a week I notice pages that have used the code from GISDoctor.com (second plug of the post) that still include my Google Analytics information.  Most of the time the user will eventually remove the code or embed their own tracking info, but there are hold-outs.  I’m talking about you, person in Spain.

Thanks, and happy GIS-ing!

Posted on

2012 – The GISDoctor.com Year of Spatial Analysis

With GISDoctor.com entering its second year in a blog format I have some plans for the site that will hopefully keep readers interested and bring in new readers.  Here is what I am planning for 2012!

  • Spatial Analysis – As a geographer by training, and professional geographer by occupation, I do a lot of spatial analysis and spatial statistics.  I will be developing a series of spatial analysis and spatial statistics posts over the year.  I will start with the basics and move into more complicated subjects.  Like the Intro to Spatial SQL guides, I will include test data and examples.  The tutorials will be targeted towards the geographer and GIS user, as that is my area of expertise.  I really want to focus this site towards the technical GIS professional and technical geographer during 2012.  Too many GIS blogs review the news (including this blog).  I want GISDoctor.com to become a technical resource for people who have questions like I do.
  • Spatial SQL – During January and February I will be adding some more posts on Spatial SQL and its use in geographical analysis.  I will be adding another series of more technical Spatial SQL how-to guides later in the year.
  • Online Mapping – I will hopefully add a few how-to guides on developing map-mash ups using a variety of APIs.  I’m interested in learning more about Bing Maps as well as Open Layers and Map Server.  I hope to get a few examples out sometime this spring.
  • Software Reviews – In 2012 ArcGIS 10.1 will be released (Hooray! or shucks. It depends on your point of view).  I’ll post a detailed review a few weeks after I get to use the software intensely.  Also, when I get SQL Server 2012 (Denali) I’ll review the improved spatial components.
  • Conference Reviews – I’ll make sure to review the 2012 Esri UC and any other conferences I make it to.
  • Finally, when applicable to the readers of this site, I’ll post news stories and items that I find interesting.

I’m sure there will be a number of other topics that I write about.  So make sure you subscribe to the feed to get the latest updates.

Thanks for being a reader.  The site has done a lot better than I could have imagined and I hope 2012 goes just as well.

Happy GIS New Years!

 

Posted on

Maps in the News!

There was an article today on Slate.com in regards to David Imus‘ CaGIS’s “Best in Show” award-winning map of the United States.  The story was also picked-up by Uber-nerd tech blog Gizmodo!

Even though this map has been out for a while (I think the map had to be published in 2010 to be eligible for the 2011 award) there are a couple items to take away from the Slate article.   As the readers of this blog know cartography is both a science and an art.  The Slate article discusses how Mr. Imus spent a tremendous amount of time placing labels and focusing on the details.  For anyone who has every been involved with a large cartography project knows the best GIS label placement algorithms will not place all the labels in the best locations from a design perspective.  I’ve personally seen drastic differences in how ArcGIS places labels compared to Quantum, or how Quantum places labels compared to CadCorp.  I’ve personally spent hours fixing the location of labels and graphics, but no where near the time Mr. Imus dedicated to this project!.

Beyond proper cartographic construction (the article discusses the use of a conic projection, label placement theory, feature identification, color usage, etc.), the author brings up what I believe is the key to creating a beautiful and effective map.  Design.  Without having an eye for design a cartographer will often create a map that is often boring or hard to understand.  Combine a well designed map with proper cartographic techniques and you get a map like Mr. Imus’.  One that is elegant and effective.  The map reader should be both interested in the content of the map and the “look and feel” of the map.  The design will pull the reader in and the content will keep them interested.

From a technical perspective I’d be interested in getting more information on a couple things, which may be answered when Mr. Imus’ website is back-up and running:

  • What set of programs did he use to create the map.  We all know there was some component of GIS in this effort, but how much?  At what point was this project moved into Adobe Illustrator or a similar graphics program?
  • What were the data sources?  Did he do any verification?
  • How long did this image take to draw on screen?  I know that when I have a ton of labels and graphics in a map the drawing time can be horrendous.
  • Do you think he has generated tiles of this map and will make them available as a map service?  Hey, Esri, call this guy up and add this map to ArcGIS.com!

I believe we need more high quality cartography like this in the world.  There is a lot of really good cartography happening in the world today, and people are now more familiar with maps, cartography, and GIS than ever before.  Unfortunately, their point of reference of what is a “good map” is the most bland and cartographically boring map I can think of.

Currently, the CaGIS site has the 2010 awards posted.  Go and check out some of the past winners.  Some are truly impressive maps.  Also, Mr. Imus’ website is down (*Update – It was back up as of 1/3/2012).  I think he got a lot of traffic today and his hosting company didn’t like it.

 

Posted on

Map Geek Gifts!

Well, many of you may be like myself and have done little-to-no holiday shopping for your map enthusiast friends and family members.  Don’t worry, there still is time and there are plenty of great gifts out there to be had!  Here are a few recommendations for those easy-to-buy-for map geeks.

–Visit a Map Store! If you are in the Boston area, check out WardMaps in Cambridge’s Porter Square.  They have an awesome selection of historical maps, map souvenirs, map coasters (my favorite) and T memorabilia.  They also have framing services as well for when you buy that historical map.

–Giant Wall Maps are awesome – Whenever I fly this is always my favorite item in SkyMall.  I like to think of these as map FatHeads.  Oh wait, they have map FatHeads!

-Need some flexibility in buying a giant wall map?  Well, check out MuralsYouWay.com.  Lots of cool maps are available and the buyer can select the size.  Take note, there is some installation required and they are not cheap.

–Atlases are old school cool – Don’t have space for a giant wall map?  Tired of online mapping?  How about a huge atlas or a nerd atlas?

–Globes are classy because they have no distortion- Not in the market for a giant map?  How about some mini-globes that would look great on a desk or bookcase?

–Books – Does your map geek like to read (probably)?  How to Lie with Maps is on the shelves of many cartographers, geographers, and map enthusiasts.  Why Geography Matters is a great read, and for the collector, they would love an early edition of Arthur Robinson’s Elements of Cartography.

–Strange Maps, the Book!  A collection of maps from the blog, Strange Maps.

–Specialty maps, my favorite type of art – Does your map friendly friend have some empty space on their walls?  How about a unique wall map?  Check out my friend Jeff’s post on Very Spatial on some really awesome typographic maps from Axis Maps.

–Puzzles!  L.L. Bean offers a hometown puzzle based on topo maps.  If you live in a topographically boring area this might not be a great puzzle, but the map geek would appreciate it.

–Wear your maps – Ties, tshirts, socks.  All the latest fashions!

Well, those ideas should get you started.  There are plenty of other map gifts out there, so if you know of any others post them in the comments section!  Happy holidays!

Posted on

Site Updates

For those of you who visit my site regularly (who aren’t my mom) you will notice a few changes were made over the weekend:

  • Some of the appearances have changed, including new header images!  Exciting!
  • I fixed a couple broken data links on the mash–up pages.  I moved data to my server, which might be slow at times.  I’ll rework the examples at some point to improve performance.
  • The GIS help page has been updated to include links to help pages that are on the site.
  • I removed a few mash-up example from the mash-up examples page  for a couple of reasons explained on the page.  I have a few new examples that will be published soon!
Be on the watch for more updates.  With the holidays coming up and a number of days off I’ll have some time to add new content!
Posted on

Spatial SQL for the Geographer – Part 2 – Basic Spatial SQL Scripts

The tutorial continues…

If you haven’t already read the first two parts of this guide please read:

The first two parts of the guide give some background information on SQL Server, covers some basic concepts, provides some links to download some software and data.  Once the user has down completed the first two part of this guide they will be able to understand this section of the guide, which will provide some basic spatial SQL scripts.

Tutorial

Here the user will learn about a few basic Spatial SQL scripts.  Nothing too complicated, but enough to get the user started.  Hopefully the user will start to think about ways to expand the scripts into their own ideas.

This tutorial will cover the MakeValid, calculating centroids, and buffering a polygon.

Checking the Validity of Shapes

Before the user starts to calculate buffers, generate centroids, union or intersect tables or perform any other Spatial SQL methods the user should first run the MakeValid command. MakeValid will check the shape to see if the data set breaks any of SQL’s rules for storing the geomerty of the shape. As mentioned in the first part of this guide the geometry datatype is very flexible, but the user should run MakeValid first before doing any analysis because there still might be self intersecting polygons, bad ring order or any other number of topoligical errors within the dataset.

Many GIS programs won’t worry about these problems on the surface, but SQL will and users should perform these queries on all spatial datasets, especially polygons data, as it is usually full of errors.

This first example will update the State_Provinces table, checking the geometry of each record in the table.  This query will update the spatial data type column in a table so that the shape is valid, meaning that the shape meets SQL’s requirements for the datatype

In this example the State_Provinces table is being checked. The user will need to
run the MakeValid command against the column that stores the spatial data type, in this case, the geom column.

use Spatial_Database
go 
update dbo.States_Provinces
set geom = geom.MakeValid()

This query make not catch all errors in the data set.  The following scripts will check the data for validity as well.  The third script will check to see if any of the records still contain any invalid geometries.  These scripts were modified for the geometry datatype from Beginningspatial.com, a great resource the user should check out.

--Fix invalid shapes
update dbo.States_Provinces
set geom = geometry::STGeomFromWKB(geom.MakeValid().STAsBinary()
,geom.STSrid)

--Correct the order of the points in the polygon ring
update dbo.States_Provinces
set geom = geometry::STGeomFromWKB
(geom.STUnion(geom.STStartPoint()).STAsBinary(),geom.STSrid)

--Check to see if any records still have any invalid geometries. 
--The results should return an empty table.
select  ID, geom.STIsValid() as ValidCheck,
geom, geom.ToString() as Spatial_Text_String
from dbo.States_Provinces
where geom.STIsValid() != 1

When these queries are run the geometry in the table be updated.  From there the user will be able to repeat this process for each of the tables loaded in the first tutorial.  If the user does not run the MakeValid query some of the examples in this tutorial may not work correctly.

Calculating Centroids

The next set of queries will take a polygon layer, calculate the centroids, and convert the centroids into latitude and longitude.  Using the world_countries table in the Spatial_Database database the user can run this query and return the name of the feature and the X and Y of the polygon’s centroid.  This query uses a subquery where the centroid is first calculated.  Once this is done the outer query converts the centroid into latitude (Centroid.STX) and longitude (Centroid.STY).

select U.Region_Name, Centroid.STY as Longitude,
Centroid.STX as Latidude
 from
(	 select geom.STCentroid() as Centroid, NAME as Region_Name
	 from dbo.world_countries
)U

The following will expand on the previous query.  In the next example the user will calculate the centroid for a specific polygon and then buffer the centroid.  In the subquery a where statement is set to select a specific record that meets a certain requirement. In this example the user will select records from the name_1 column that equal ‘Massachusetts’.  Also, when the STCentroid() method is called no parameters are needed, unlike other spatial methods which can require several methods.

In the outer query the centroid is buffered by 0.05 decimal degrees, set in the STBuffer method. Why are decimal degrees used here?  Well, it is because the data were loaded in as WGS84.   There will be a future tutorial on how to tackle this problem.

The user can run this query and return a buffer around the centroid for the state of Massachusetts.

select U.Centroid, Centroid.STBuffer(0.05) as buffer, U.StateBound
from
(	select geom.STCentroid() as Centroid, geom as StateBound
	from dbo.states_Provinces
	where name_1 =  'Massachusetts'
) U

Calculating a Buffer

The last query that will be demonstrated will introduce the buffer, even though I just gave an example. The buffer operation is a common operation within GIS tools. The following script will give a basic example of how to generate a buffer in Spatial SQL.

The user will notice that the shape column, geom, is set and the buffer method, STBuffer, is called. STBuffer accepts a parameter, the size of the buffer.  In this example the buffer size will be 0.05 decimal degrees.  Also, in this example a where statement is set to query a specific record from the table. When this is done only that record would be buffered. Again, the States_Provinces table is used.

select geom.STBuffer(0.05) as buffer
from dbo.States_Provinces
where name_1 =  'Massachusetts'

What’s Next

That is it for this section of the tutorial. As I said earlier, pretty basic stuff. In the next tutorial I will get into some other topics including unions, intersects, spatial indexes, joins and other methods and operations will be covered. And, it looks like I may end up creating several more tutorials than I originally planned!

Unitl next time…

Posted on

Spatial SQL for the Geographer – Part 1 – Welcome to Spatial SQL

Before we begin I would like to make it clear that learning all the capabilities and limitations of SQL Server and Spatial SQL cannot be covered in three short lessons.  If you believe you might need some more training please visit the Microsoft SQL Transact reference guide.

What is Microsoft SQL Server?

Microsoft SQL Server is a relational database system that stores data and provides several mechanisms to retrieve stored data.  One of the beauties of relational database systems is that they allow you to effectively store a variety of data sets and they are both scalable and securable, and in my experience, much more trustworthy than any flat file.

There are a number of good references out there explaining SQL Server in much more detail than I would attempt.  I recommend that you check a couple of them out:

Explanation of Spatial Data Types

As you probably know by now a column in a table or view can be assigned one of many different data types.  Most are familiar with the traditional data types such as varchars, integers, bits, floats, or decimals.  With SQL Server 2008 two new spatial data types were introduced.  These data types support the storage of spatial data, and are broken up into two different types.  Their characteristics are important to understand:

  • Geometry:  Primary characteristic – spatial data stored in a planar system.
  • Geography: Primary characteristic – spatial data stored using the ellipsoid.
  • Here is a great blog post that describes the spatial data types in more detail.

Tutorial

Step One: Getting Started

The user first needs to download some software and data to their local machine.

Downloading Microsoft SQL Server 2008 R2 may take a few minutes.  The user needs to make sure that they read the download and installation instructions before they proceed. Also, when installing the software the users needs to pay particular attention to the installation instructions.

Note:  This tutorial was completed on a 64 bit machine running Windows 7.  I cannot guarantee that any of this will work on anything different.

Notes about the Data.  The populated places, countries, and states and provinces data sets were downloaded from Natural Earth and the roads data set came from Data.gov. The data represent the three different vector data types, points, lines and polygons. They also provide a variety of query options to work with during for the tutorials. Also, I renamed the data from the original names so for clarity during the tutorials.

Once you download the data I recommend first viewing it in a GIS program to make sure you downloaded it correctly.  Here the user will see the four data sets in Quantum:

Tutorial Data

Step Two: Create a Database

Now that the software has been downloaded the user can open Microsoft SQL Server Management Studio.  To do this the user will select to Start–> All Programs–> Microsoft SQL Server 2008 R2–>SQL Server Management Studio.  Since this will probably be the first time the user opens MSSQL it may take a couple minutes to configure and load.

Next, the user will be prompted to connect to a server.  In this case the server is the local machine.   In this example I connect to my local machine instance and use Windows Authentication.  Unless you configured your SQL instance differently you will connect the same way.  Once these options are selected click connect.

connect to server

Now that the user is connected to the server they can create a new database.  First, the user will expand the database instance.  Then the user will right click on the database folder and select New Database from the menu.

Create Database

The new database window will now open.  Here the user can configure a number of settings before the database is created.  In this tutorial we want to name the database Spatial_Database which is set in the Database Name text box.  Another option I like to configure is the location of the data and log files.  This is done by clicking the … button under the Path column.  I recommend storing the database in a location that has plenty of room to grow, as the default location on the C drive may not have space to expand on your local machine.

Change Location

Once the settings are all set the user will click OK to create the database.  The user can refresh the Databases folder to see the new, empty database.

Spatial Database

The user is the system admin on this sever and this database be default.  This means the user has permissions to create databases, delete databases, create tables and load data into them.  With these privileges already in place the admin user can load data into the database without any further configuration.

Step Three: Load Data into the Database using Shape2SQL

Using Shape2SQL is really easy.  The user will first open Shape2SQL and add a shapefile that will be loaded into the database as an individual table.  To do this the user will first navigate to shapefile stored on the local machine by clicking on the Select Button.

Select Shapefile

The next step is to select the server and database where the data will live.  The user will click on the Configure Button to do this.  In the Configure window the user enter the name of the server in the Server Name window.  The user will then select Use Windows Authentication and then select the appropriate database.  In this tutorial the database has been named Spatial_Database.  Once this information has been configured the user will click OK.

Configure Server

Now, back in the uploader window the user has several additional options to configure before loading the data into the database.  First, the user needs to select the data type. The user will select either the geometry or geography data type.  In this tutorial we load the data as the geometry data, as it is more flexible, in terms of what it will accept, than the geography data type.

The second setting that needs to be set is the name of the geometry column.  The default for this setting is ‘geom’, but the user can change this if desired.  Remember, all the tutorials will use the ‘geom’ column.

The user will then make sure the Create Spatial Index option is selected.  This will ensure that a spatial index is built, which will improve the performance of the table in SQL Server.  Finally the user will set the name of the table in the Table Name window.

There are a few settings that do not need to be selected or altered.  Since this tutorial is using the geometry data type the user does not need to set the SRID.  The user can also change which columns are loaded into the table.  By default all columns are taken from the shapefile and loaded into the table. If a user does not want certain columns they will simply uncheck the columns they don’t want.  Once all the settings are selected the user will select Upload to Database.  Again, I may use all the columns in future tutorials, so you are best off keeping all of them.

Once the user clicks Upload to Database the shapefile will be loaded into the database as a table and the spatial index will be created.  Depending on the size of the shapefile and the power of the user’s machine it may take several minutes to load the table into the database.

The user will repeat these steps and load all four shapefiles into their database.  Once that is completed the user can proceed to the next section of the tutorial.

Step Four: Perform a Basic Query

Now that the shapefiles have been loaded into SQL Server the user can start to perform some queries.  Now, if the user does not have any SQL experience at all it may make sense to read through this tutorial from Microsoft.

The first example query is probably the most simple SQL query one could perform. In this first example I will use the dbo.States_Provinces table.  The user will open SQL Server, connect to their server, and then click the New Query button.  The user will then paste the following into the query window:

use spatial_database
go

select top 100 * from dbo.States_Provinces

The results will return the first 100 records from the dbo.States_Provinces table.  In SQL Server the user will notice there is a Results tab and a Spatial results tab.  The results tab will display the tabular data and the spatial results will display the spatial data.  The Spatial results tab will only appear if there is spatial data returned in the query.

Step Five: Set a Where Statement to Query Specific Data

The user can narrow down what is selected by incorporating some basic SQL statements, such as a where statement.  The basic premise of a where clause is to limit the data returned from a query. The user is setting a condition in regards to what they want returned.  For example, let’s say we only want the return records from the States_Provinces table that are for the United States.  If the user explores the table they will set that this information is stored in the Name_0 column. However, we are unsure of the exact syntax of “United States” so we can use a like operator in the where clause. For example:

use spatial_database
go

select * from dbo.States_Provinces
where NAME_0 like '%United States%'

The user will see only those records that have ‘United States’ in the Name_0 column. If the user clicks on the spatial results tab they will see the spatial representation of those records.

Spatial Results

Alright, that is it! We have completed the first part of this tutorial. The user has downloaded the appropriate software and data, installed the software, loaded a number of shapefiles using Shape2SQL into SQL Server, and have completed two basic queries.

Before the next tutorial I recommend the user, if they are light in their SQL experience, read up on some basic queries from either the W3 Schools or from Microsoft.

The next part to this tutorial will cover some more Spatial SQL queries.  I’ll go into more detail about some spatial operations and provide a number of examples.  If there is anything the you want to see let me know!