Scott Gregory & Eric Weisbender
Advanced Integrated Management Services, Inc., Contracted to Western Area Power Administration
New technologies can be seductive, but are you ready to switch from something that already does that job? Maybe you want to take some time and allow the programmers to fix the bugs in the new stuff. Maybe you don't want to retrain your staff. Maybe you just like Workstation ArcInfo and you don't want to change. We'll show you how to integrate ArcInfo with some simple user written Intranet applications, documents, and images that can bring information and clients together in a low-cost high availability solution.
Well, sometimes necessity is the mother of invention. Here's where we were. After years of doing small GIS projects we suddenly acquired a big GIS project. We had to map over 5,000 miles of high-voltage electric transmission line - in eight months! We needed a tool that could allow our managers to track the progress of the project since noticeable progress occurred daily. None of our managers had any GIS experience or training. Not even ArcView could help us here. So, in order to avoid the unpleasant reality of spending valuable time continuously producing plots for managers we had to come up with a free, easy to use solution.
The solution: a self-developed web-based GIS using ArcPlot as the map engine.
Several components were necessary. We needed a web server. We needed some PERL scripts. We needed an RPC client. And finally, we needed a piece of software to convert Sun rasterfiles to something that web browsers could display - like .gifs. The new UNIX server we purchased for the project came with a web server. We wrote some simple PERL scripts. The RPC client was provided by Esri (see /Esri/arcexexx/aiclient). The final piece, converting the Sun rasterfiles to .gif was available on SGI's Freeware website.
The result: a high-availability user oriented web-based GIS with the cartographic power of ArcPlot.
Okay, to be honest, it took a couple of years to put all the pieces together. First, we had to develop a scheme to store and retrieve data for the Western United States. We began with a library tile structure and used Librarian to setup these data. The library tiles were the extent of USGS 100K quadrangles (898 of them) and a lot of our data were from USGS Digital Line Graphs. We had several field collectors using mapping grade GPS to collect the locations of transmission line points of intersection and roads. All this stuff, after some processing, was placed into the library tiles.
If you've ever used Librarian, well, you know. So we developed a way to locate the necessary tiles for display using the library's index coverage. This stuff was packaged into an AML program that drew the data for the selected geographic extent in an ArcPlot graphic window. Part one, the data and initial drawing commands were complete.
Next we had to understand enough about Inter-Application Communications (IAC) and Remote Procedure Call (RPC) to figure out how to connect a web server to an ArcPlot session running in IAC mode. Okay, to be frank, we never really understood how the thing worked. Esri documented much of what we needed to know within ArcDoc (see Using Inter-Application Communications) and provided the shell program "arcclient". A little playing around yielded a working model of "C" code. Now we needed a PERL script (or some sort of CGI program) that could call the program that called the sleeping giant (ArcPlot). A little book about PERL helped, and soon we had a working CGI program.
If you're using UNIX you're probably already okay with IAC and RPC. If you're using Windowsâ you'll need to locate and install RPC software. No problem, you can find something free on the Internet. We run our map server on UNIX and we tested a model on Windowsâ . Both worked fine, but we prefer UNIX for stability.
Okay, so what is Inter-Application Communications (IAC)?
IAC enables software applications to communicate with each other. These applications can run on local or remote hosts. IAC provides a way for application programs to access ArcInfo. We're using a PERL script to call a "C" program to call ArcPlot running as an IAC server. Oh yeah, the PERL script is called by the web server when a client browser sends a map request. You might think this is a big hassle, but if you look into the bundled applications that you may already be running you'll find similar things going on. Also, your methodology may be a little different based on your own programming capabilities. Again, by the title, we built our application with what we had.
What does the CGI program have to do?
The CGI program responds to the client's HTML form requests. The form request should contain a bunch of information that your mapping application needs to know to draw the map. Many of the items needed by the CGI program can be embedded in the HTML document as "HIDDEN" values. We've recently become aware that there are actually Web Mapping Standards (WMS x.x) that describe all the things you need to make your application comply. Typical elements include the coordinates of the current map extent and the point where the user "clicked" on the map. Our CGI program actually computes the map extent for the user request based on current map extent, scale (zoom) factor, and interpolated mouse click coordinates. After a little processing, the CGI program calls the arcclient "C" program. The call to the "C" program looks like this (in PERL):
system `arc_w_client3 $x1,$y1,$x2,$y2, $file_name $size_x $size_y $identify $xc $yc`
The CGI program launches a series of events meant to connect to your web server, make a map or do a query, and return the map or query to your user. Anything less leads your user to staring at an empty browser window watching a spinning globe. We don't like the "Retrieving Data" thing. Our application typically returns a dynamically created GIS map within six seconds.
What does the "C" program have to do?
The "C" program contains the stuff necessary to connect to the ArcPlot IAC server. Esri does most of the heavy lifting here. We started with the arcclient.c program contained in /Esri/arcexe72/aiclient. This program contained all the key elements needed to connect to IAC servers. Esri also included a "Make" file for compiling the program after we made some modifications. Basically, all our "C" program does is setup a call string with parameters to pass to the big AML that handles the map and query requests from our users.
What does the AML have to do?
Okay, some of you may not know what AML is. For those of you who don't, AML is Arc Macro Language. If you didn't already know this we probably can't help you in this little paper. For those of you who do, please read on.
The AML you'll write is a huge piece in this application. Here's where all the drawing commands, queries, and ArcPlot I/O happen. This code needs to be pretty tight, since an error here will affect whether or not your user gets his or her map. Our application draws maps for several states in the Western United States. We allow infinite zoom and pan capabilities, a feature which some of our users have exploited for fun.
ArcPlot: The Forgotten Language
Okay, so if you don't know what ArcPlot is we probably can't help you either. We recently interviewed a handful of applicants for an open position. None of them had any experience in ArcPlot other than printing simple maps. ArcPlot is an extremely powerful mapping and query program. The nice thing about using ArcPlot for our map engine is that we can access its full cartographic power. Pretty cool for an application that doesn't cost us any additional money.
Oh yeah, one big idea is how to get the map image created by your ArcPlot IAC server running your big AML back to the user. One of the necessary arguments to pass to your AML is the name of the image you want to save. This image must be written to disk to a directory that your CGI program can access.
And finally…
The final piece was the one that kept us looking and waiting. We knew we could set map extents and issue draw commands based on map scale, and we knew we could save the on-screen image as a Sun rasterfile. Somehow we needed to convert this blasted Sun rasterfile into something usable by web browsers. After months of poking around the answer appeared on the Silicon Graphics, Inc (SGI) web site. There it was - but in two parts. Part one, convert the Sun rasterfile to an SGI image. Part two, convert the SGI image to a .gif. Okay, so it's not an entirely clean deal, but it works. And it works pretty darn good.
So, one quiet afternoon in Lakewood, Colorado, the whole thing came together. Here's a block diagram...
We call our application "IMAP". IMAP runs in ArcPlot on a UNIX server with a display on a UNIX workstation. IMAP is extremely stable. One particular instance ran for 180 days without error. Typically the only reasons IMAP is taken off-line are related to system administration, power failure, or to move equipment. Since we're dealing with AML we can change, update, or reconfigure IMAP on-the-fly. Also, to add more processing capability we can hook up additional map servers using the "Dispatch" option with the AML function [IACCONNECT].
We've been running IMAP for over two years now. We provide a very simple and easy to use interface all written in HTML. IMAP is totally cross-platform and can be run on any ArcInfo compatible system with RPC capability. IMAP's user interface is HTML, so we can support users running just about anything.
Another interesting possibility is getting data from within feature attribute tables and turning them into hypertext links. This can be done within your big AML that takes care of plotting and query for your map server. We input the filenames of engineering drawings in attribute tables and supply a link to the relevant engineering drawing when a user requests a feature query of a system component.
You can, after doing a little research and reading a few "white papers", construct an effective web-based map server using ArcPlot. This is a nice way to get your company or clients used to mapping with a web browser using the data you create in-house. The advent of the Geography Network and cool stuff like ArcIMS is great. For us, at the time we developed IMAP, these items were nowhere close to reality. We had no money and no management support for additional software. IMAP has been the best marketing tool we could have ever developed. We showed that leveraging existing technology with a little creativity and some good UNIX systems could yield some easy to use and productive tools for our clients. Additional web programming has allowed us to integrate many disparate elements within our organization. Who knows, perhaps we even created a tool that could help someone. And that, in the end, is what this GIS thing is all about.