Posted on

One Million Points

I am working on a couple projects and I need to generate some random points across defined bounding boxes.  I have the basic code worked out in python and I am testing the results.  Just for kicks here is a million points generated in Python to a csv file (in 10 seconds) and drawn (rather quickly) in Quantum.  Sweet.

One Million Points

I’ll share the code on Thursday.

 

Posted on

2012 – The GISDoctor.com Year of Spatial Analysis

With GISDoctor.com entering its second year in a blog format I have some plans for the site that will hopefully keep readers interested and bring in new readers.  Here is what I am planning for 2012!

  • Spatial Analysis – As a geographer by training, and professional geographer by occupation, I do a lot of spatial analysis and spatial statistics.  I will be developing a series of spatial analysis and spatial statistics posts over the year.  I will start with the basics and move into more complicated subjects.  Like the Intro to Spatial SQL guides, I will include test data and examples.  The tutorials will be targeted towards the geographer and GIS user, as that is my area of expertise.  I really want to focus this site towards the technical GIS professional and technical geographer during 2012.  Too many GIS blogs review the news (including this blog).  I want GISDoctor.com to become a technical resource for people who have questions like I do.
  • Spatial SQL – During January and February I will be adding some more posts on Spatial SQL and its use in geographical analysis.  I will be adding another series of more technical Spatial SQL how-to guides later in the year.
  • Online Mapping – I will hopefully add a few how-to guides on developing map-mash ups using a variety of APIs.  I’m interested in learning more about Bing Maps as well as Open Layers and Map Server.  I hope to get a few examples out sometime this spring.
  • Software Reviews – In 2012 ArcGIS 10.1 will be released (Hooray! or shucks. It depends on your point of view).  I’ll post a detailed review a few weeks after I get to use the software intensely.  Also, when I get SQL Server 2012 (Denali) I’ll review the improved spatial components.
  • Conference Reviews – I’ll make sure to review the 2012 Esri UC and any other conferences I make it to.
  • Finally, when applicable to the readers of this site, I’ll post news stories and items that I find interesting.

I’m sure there will be a number of other topics that I write about.  So make sure you subscribe to the feed to get the latest updates.

Thanks for being a reader.  The site has done a lot better than I could have imagined and I hope 2012 goes just as well.

Happy GIS New Years!

 

Posted on

GIS Doctor Mailbag

This blog has been up and running for a while now and I have gotten a number of email questions, responses to blog posts, and search engine keyword questions.  Let’s answer a few of them!

How do geographers collect spatial data?

  • By many means.  There are the arm chair geographers (like me!) who collect, develop and analyze spatial data from a variety of published data sources and there are those geographers who go out into the field.  These rugged souls create and administer surveys, collect data in the field in all conditions, and develop scientific studies to derive their own data.  Depending on the nature of the study a geographer may use a mix of data collection methodologies.

How do I use more memory with ArcGIS 10?

  • I hear this question more and more, especially as more users have 64 bit machines and automatically assume that everything will be faster.  ArcGIS 10 will use all the memory available in a 32 bit environment, which is 4 gigabytes. However, not all the tools and processes in ArcGIS will use all the memory.  To work around this problem users can chunk their analysis to try to speed things up, or they can review their analysis and try to develop a set of processes that make the analysis smaller, therefore faster.  In the past I have split up large input data sets (20 million plus points) and used python and ArcGIS tools to have the analysis run effectively.

How much ArcGIS Server 10?

  • Not cheap.  If you want free check out MapServer.

How can I get Bing Maps in my GIS?

  • If you are not behind a firewall a user can click on the Add Data button in ArcGIS and select Add Base Map.  The Bing data should then be a base map option. Since my instance of ArcMap is behind a firewall I cannot add the Bing Service. This is a bummer, since I am a big fan of Bing Maps.

How can I add Google Maps into ArcMap?

  • There isn’t a quick tool to load Google Maps into ArcMap.  However, there are a few applications out there that allow a user to port in the imagery from Google into ArcMap.  One well known tool is provided by Arc2Earth.  Now, if the user is running ArcServer Esri provides a number of tools to publish data in the Google Maps interface.

How do I kill a geoprocess ArcGIS 10?

How do I speed up ArcGIS 10 on 64 bit machine?

  • Wait until Esri releases a 64 bit version of ArcGIS, or start using GIS software that is native 64 bit.

Is it possible to use embed a WMS data source into a Google Maps mash-up?  

  • Why, yes it is.  Check some examples here…

Is ArcGIS difficult to learn?

  • I’ve been in the field of GIS for close to 10 years and I’m learning something new about ArcGIS everyday. GIS is complicated and the theory behind it is deep.  Many want GIS software to be simple, but users need to understand the details to properly frame an analysis and understand the results.  If GIS were very easy I believe you would get many people, more so than now, making poor decision from poor analysis.

Why did Esri skip version numbers?

  • They did?

What is wrong with ArcGIS 10?

  • Unfortunately, plenty.  Check out the bug fixes for SP1 and SP2.

Why do ArcGIS tools take so long?

  • Another question I hear all the time,  “why is process XYZ so slow?”.  The answer is never as direct as the person asking the question wants it to be.  There are a number of reasons why a particular process or processes may be slow.  To try to understand the issue I always work through a series of questions.  First, what are the specs of the machine running the process?  What other processes are running on the machine?  What are the dimensions of the data being processed?  Are the data sets in the analysis in the same projection? How large is the data, both spatially and in memory?  How intensive is the process itself?  How detailed is the data being processed?  The point is that there isn’t always a smoking gun to solve all processing questions.  Sometimes the process will be slow.

OK, that is it for now.  If you have any additional questions drop me a line!  Until next time.

Posted on

More on ArcGIS “Sucks”

On February 24, 2011 I posted a blog asking for opinions on ArcGIS 10, mainly due to all the bad reviews I see and hear.  Since that post the hits on this site have spiked, and not because of my awesome content regarding information about earning an advanced degree in GIS or learning about different uses of geographic information on the web.  Most  people have found their way to my site through  “Googling” something related to their feelings about ArcGIS 10.  Thanks to Google Analytics I have extracted the following keywords that have driven traffic to my site over the past two weeks.

  • arcgis 10 sucks
  • arcgis 10 comments
  • “arcgis 10 sucks”
  • arcgis 10 is awful
  • arcgis 10 issues
  • arcgis 10 never opens
  • arcgis 10 problem
  • arcgis 10 reviews
  • arcgis stinks
  • arcgis sucks
  • arcmap 10 sucks
  • arcgis10 review 2011
  • does arcgis10 suck
  • esri arc 10 ratings
  • i hate gis – (haha, I love this one.  I once had a friend in grad school who lived by this mantra)
  • problems with arcgis 10
  • problems with arcmap 10
  • reviews arcgis 10

This is crazy!  What drives people to go to Google and type in one of the previous keywords?  I want to know!  Yes, there are problems with ArcGIS 10, but I have found much more success than frustration since I installed it this past June.  For example, I have built a number large and complex models and python scripts that handle millions (yes, millions) of points, that perform analysis on tens of thousands of polygons, and create multi-gigabyte output files that run perfectly ever time.  I have a model running right now and the results will be ready when I get to work.  I’ve had no major problems with raster analysis, map creation, or data sharing.  Are there bumps in the road? Sure, but a comparable number of bumps to the other software that I use…

Now, do I have some kick ass machines that run the processes?  Yes.  Do I have an ArcInfo level licenses?  No, I’m using ArcView. But the point is that I am able to do everything I want to do with the software.  Are people having issues with software configurations, hardware limitations, or user error (it’s never user error by the way…).

I want to know what problems others are running into so that I can avoid those mistakes, because I hate downtime and I love results.

Have you had a problem?  Leave a comment.  I’d love to know what problems people are having.  You never know, someone else may have had the same problem, or they know of a solution.

 

 

 

Posted on

Does everybody hate ArcGIS 10?

We have all been there.  You are in the middle of some process using ArcGIS 10, trying to finish a project for a deadline, and one of two things happen: the hour glass starts to run and never stops, or your ArcGIS  screen just disappears and you get the “oh no, something broke” window to report an error (which you never do).

AAAAHHHH!

Since ArcGIS 10’s debut back in June I’ve read and heard a number of opinions (many times unsolicited) about the latest release and subsequent service pack.  As you can imagine some reviews are glowing, and some are not so great.  When I do a quick Google search for “ArcGIS 10 Review” very few results appear that aren’t from Esri themselves.

Now when I do a Google search for “esri arcgis 10 stinks” or “esri arcgis 10 sucks” the results are much more, let’s say, robust.  Three message boards really stuck out and can be found here, here, and here.

When reading though these message boards you must read them with a grain of salt. First of all, people will complain about anything and everything.  Second, many complaints have nothing to do with the software itself, but with user error or machine problems.

I also believe that many users are intimidated by GIS software and when something goes wrong they may get the urge to post insane comments with poor grammar because they are frustrated with something that may be out of their control.  I can understand where some of these people are coming from, but ArcGIS really isn’t a beginner’s software.  I’ve been using Ersi’s software for over eight years and I learn something new all the time.

Yes, ArcGIS 10 has its share of problems.  Yes, ArcGIS 10 probably isn’t the best piece of software for beginners or light users of GIS. I do believe if you are a high end user working with large datasets or you need to  perform some serious GIS analysis ArcGIS is a strong tool that I would recommend.

It could be worse for Esri, they could be SQL Server, Oracle, or Visual Studio.

And for the record, I like ArcGIS 10, especially ArcGIS Server 10.

Well, that is my two cents.  What has your experience with ArcGIS 10 been like?  Did you have to revert back to 9.3.1?  Do you post insane comments on GIS message boards when you are having a tough GIS day?  If so let me know.