BC Environment (BCE) has developed a data visualisation tool called the "GOAT", which provides non-technical staff with access to BCE's Data Warehouse. This tool has formed a key part of BCE's finalist standing in the Smithsonian Innovations in Technology award for 1995. The authors developed this tool in-house because no commercially available tools met the requirements that it:
The development took place first in the Genasys scripting language, later in ArcInfo AML, and finally in ArcView Avenue, in response to changing standards and technology.
In mid-1992, BCE initiated a GIS pilot project at its Smithers regional office, which undertook to
The user interface which the Smithers Pilot started became the first version of the GOAT. BCE's System Services Branch, including Bruce Mackenzie and consultants, made a valiant effort to make the GOAT usable and useful, which resulted in a second version in 1993. However, regional staff continued to shun the GOAT, despite the extreme need for access to spatial data - which is perhaps the clearest indication of its failure at that time.
The early GOAT has no scope, so it attempts to do anything and everything: provide hard-coded analyses for specific business needs, perform data conversion and data maintenance for GIS staff, and produce cartographic output - in addition to general viewing and querying. Each time staff or a consultant wrote a useful Genamap script, they added it to the interface.
It also lacks an integrated design. The early GOAT has an abundance of functions, almost none of which can work together (e.g., a user may not be able to view a spatial analysis over a background dataset of their choosing, or can not print a view of their data).
An important factor contributing to the lack of integration is the inability for one module to know what other modules are doing. The Genasys command line interface (like that of ArcInfo) maintains only limited 'state' information (e.g., it does not know what data has been drawn on the screen). In the early GOAT, state set in one module is unavailable to any other modules.
The early GOAT makes many assumptions about data content and use: for example, it expects to find a specific layer of road information and lakes information in order to perform a hard-coded lake accessibility analysis. This proved awkward, because data in BCE was rapidly changing, and because staff was readjusting their ideas about how they wanted to use their data.
Finally, project management underestimated the expertise required for the software development. One of the summary reports from the pilot obliquely states, "The pilot has shown that all applications developers should have strong GIS and software development skills. The team should include expertise in Genamap, Genius, Oracle, SQL and Unix." (DMR, 1993)
In late 1993, Bruce Mackenzie and a consultant spent several days attempting to 'fix' GOAT v2, but it was difficult to modify or extend.
The authors proposed to discard the user interface, and design and write a completely new viewer themselves, with a well defined scope. They bravely estimated five man-weeks development time, and to their surprise, management agreed.
The authors began with a half day meeting with representative users within BCE. They then formulated requirements, including a statement of scope that the viewer would be a tool for "view, query, simple analysis, and limited plotting of graphic and attribute data by people with no special training in Unix, GIS, cartography, or ORACLE. It is NOT intended for data management, data translation, complex analysis or applications, or cartographic quality output." (Mackenzie & Glover, 1994)
An important goal was to relieve GIS staff of some of their tasks (plotting, queries, and reports) by enabling users to do them.
Data scope: The authors felt that the viewer must not make assumptions about the thematic content or use of spatial data. The viewer must display and query any standard dataset or topology (e.g. point, line, or polygon) without customisation.
Seamless view: Wherever possible, BCE's spatial data is managed seamlessly, not cut up into mapsheets. The viewer must work with any combination of datasets on any area of the earth.
Configuration: Any staff member with Unix and Oracle accounts and access to X-Windows software must be able to run GOAT without any configuration overhead.
Reporting: Oracle attributes linked to spatial data must be easily accessible with the viewer.
Their next step was to design a coherent software system. The design started small, then grew for the first four days or so, until it seemed unworkable. The authors continued working with paper and a white board, resisting the impulse to start coding to 'get something done', while management became increasingly nervous about a lack of 'product'. Finally, the design began to shrink as the authors found ways to integrate it.
Themes: Users of the v1 and v2 GOAT had great difficulty finding and understanding the complex GIS data stored in Unix directory structures. For example, Guide Outfitter Territory polygons were in /usr4/data/gisdata/skeena/gots/ZF10. Expecting casual users to navigate this guaranteed frustration.
The authors decided to present data views with a title in the user's language, such as 'Guide Outfitter Territories, 1:250,000 (multicolour shaded)'. These evolved into 'themes', which the GIS staff define in a Unix text file, using a syntax the authors developed for the purpose. Each entry in the file links the title of a theme to a specific view of a dataset, detailing how it will be drawn, queried, and linked to Oracle.
One spatial dataset can be presented in a variety of ways. For example, one trapline theme might colour traplines based on the number of animals trapped; another might display only those where a certain species of animal was trapped. GOAT provides a means for a local expert to simplify the complexity of spatial data, so that users can concentrate on data content, not structure.
Reselect / Redraw scripts: When a theme is selected for viewing, GOAT transforms the theme definition into a pair of script files in a temporary directory. One of these scripts selects features for the theme, the other draws the selected features. When GOAT needs to draw a theme, it simply executes its draw script. When the user changes the geographic extent (the area of the earth represented on the screen), GOAT simply executes the select script, without needing to know what it is doing. This simplifies the design, because GOAT does not have to use internal variables to track details of theme composition, and also makes it possible for themes to represent data in ways not anticipated by the authors.
Save / Restore session: GOAT can save its current state, (viewing window, visible themes, title text, attribute labels, etc.) in a session file. This is also a script file, which contains calls to GOAT software managers (see below) to restore their settings.
In mid-1994, BCE began a switch to ArcInfo as its corporate GIS. The authors re-wrote GOAT in ArcInfo ArcPlot based on the same requirements and enhancement requests from GOAT v3.
Consultants with ArcInfo experience helped during the first four days of design, after which development took three weeks. The authors started with a clean design slate, but the ArcPlot GOAT design again evolved into the same basic structure.
Although ArcView 2 seemed a natural choice for a development environment, the authors ruled it out because:
A GOAT v4 theme definition has up to 12 keyword-tagged data blocks (some are quite complex) in a single text file, providing much better control and readability.
Drawtype: The drawtype specifies how a theme is drawn, separating the internal structure of an ArcInfo dataset from its representations on the screen. It is very common for one coverage to be used in several themes with different drawtypes.
Symbology: Theme definition options are included to facilitate use of standard symbology, driven from feature codes in the data.
Trackextent: If a theme definition includes the 'trackextent' option, then GOAT will restrict processing to only those features in the current viewing window. This is typically enabled for themes with high feature density over a wide area. This option is invisible to the user, but makes a conscious trade-off between processing time when the window changes, and display time when the theme is drawn.
ArcPlot has many more commands for custom cartographic output, so the GOAT uses these to make a much nicer output window.
By the time of GOAT v4 design, BCE had developed additional data standards so GOAT is able to make more assumptions about its data.
Common projection: ArcInfo cannot do analysis between datasets stored in different projections, so all ArcInfo data is stored in the BCE Albers province-wide projection.
Metadata: A menu function displays meta-information (source, history, warnings, accuracy, etc.) stored within each BCE ArcInfo dataset. Metadata warnings are displayed automatically when a theme is opened.
Feature codes: Every dataset contains feature codes which link to standard symbology.
Links from ArcPlot to Oracle are more flexible than from Genasys, so the theme definition allows a theme to be linked to more than one Oracle table or view, possibly using different columns as join keys.
The select and draw scripts in the Genasys GOAT had worked well, so that concept was extended to reporting. GOAT v4 reporting panels select features spatially, and then call external scripts to perform all but the simplest of attribute reports. These scripts can run any combination of ArcPlot, Oracle, and Unix commands.
In mid-1996, BCE hired an outside consultant to re-write GOAT in Esri's PC ArcView 3/Avenue programming language, again based on the same requirements. At the time of writing (April, 1997), it is in alpha testing, and too early to report on its acceptance or use. ArcView implements a 'theme' concept, but GOAT themes contain much more data definition information than ArcView themes.
Open Map dialog and standard queries: This image shows the buttons to perform standard scripted queries, and the 'Select Map' dialog, with its bewildering array of cryptic map names.
Translators menu: This image shows the buttons to perform adhoc queries, and the 'Translators' menu, with its optimistic pushbutton data translations.
Water menu and HTML help: This image shows the buttons to perform water queries, and the help in HTML.
Open Theme selector: This image shows the output of a report location, and the Theme selector.
Report panel and HTML help: This image shows the File/Open menu, the theme-on-theme report panel, and the help in HTML.
Theme-on-theme report: This image shows the theme-on-theme report panel, and the attributes associated with the 359 traplines within the 6 Management Units selected for query.
Go to Named Area panel: This image shows the Go to Named Area panel, listing named areas defined by the local data manager. The data window shows all of North America reprojected to BC Albers Equal Area projection, which is used for all ArcInfo data.
View menu: This image shows the various options for setting the viewing area, starting with a view of all North America. Where do you want to go to today ?
Legend panel: This image shows the Map Legend panel. The user chooses legend lines associated with the currently open themes, and a location on the map to draw them. The standard title bar is also enabled.
Attribute Label panel: This image shows the Attribute Label panel. A Route feature-class layer is labelled with street names.
metadata report: This image shows the output of the 'Report Metadata' menu. It prints metadata stored in each open theme (coverage or ArcStorm layer) to the screen.
Layout view: This image shows a Layout view with a long/lat graticule draped on the plot. GOAT generates the scale bar, North arrow, and legend.
The authors divided the tasks to be performed by the GOAT, including the state that had to be maintained, among a number of "managers". The state information each manager required is considered that manager's "secret", and other routines must make calls to that manager to access the information. This functional breakdown is effectively unchanged across both the Genasys GOAT (v3) and the ArcInfo GOAT (v4).
Manager Description
control Calls all the other managers for high level functions. For
example, a call to ctl_mgr ini at system startup results in
appropriate initialisation calls to the other managers.
theme Provides functions to open, close, draw and list themes. When
the theme manager opens a theme, it builds select, drawing
and reporting AML scripts based on its makeup. To draw a
theme, the theme manager executes these scripts.
extents Tracks and controls the relationship between the earth and
the display window. It provides zooming, panning, and similar
functions.
projection Controls the projection in which the data is displayed. The
authors implemented this only in GOAT v3, because of the
superb support for on the fly reprojection in Genasys.
display Controls graphics windows (their size and location; creating
and destroying them), and the output device (screen vs. file
vs. printer). In GOAT v4, the functions associated with the
display surround (title, etc.) were enhanced and added to
this manager.
oracle/ Provides functions to connect to Oracle, build links between
relates spatial features and Oracle tables, and to query the
characteristics of Oracle objects.
report Selects features based on user input, makes calls into
Oracle, and presents report results. This was significantly
enhanced in GOAT v4.
label Provides functions to draw attribute values on the display
window (e.g., lake depth), based on GIS or Oracle attributes.
annotation In GOAT v3, manages the display surround: title, scale bar,
north arrow, etc.
options Provides services to control user options and change Unix and
Oracle passwords.
error Provides error and informational messages to the screen and
to a log file. The authors wrote this manager first.
debug Implemented in GOAT v4, this provides a debug trace.
help Help for the GOAT is on the World Wide Web, at the Ministry's
GIS Web site. The help manager starts a web browser, opened
to the appropriate page in the help documentation.
This section provides a cursory look at some of the differences between the language environments provided by Genamap (Genasys Scripting Language, here abbreviated GSL) and ArcInfo (ARC Macro Language, or AML).
Both GSL and AML provide a scripting environment for a vector-based, command-line GIS platform: this is probably their only point of similarity. For example, AML provides access to a much richer suite of functions to control data and plotting, as well as to display and state primitives.
GSL stat <<-HERE > temp.tmp
y
y
y
HERE
aspect_ratio="$(sys -p awk \'/INCHES/{print\ \$NF/\$(NF-1)}\'
\$IOPATH/temp.tmp)"
AML &sv winsize = [show windows ARCPLOT size]
&sv aspect_ratio = [extract 2 %winsize%] / [extract 1 %winsize%]
GSL is based on the Unix Bourne shell, and shares its flexible syntax, near-native system interaction, and easy command redirection. AML, perhaps reflecting its origins in Command Procedure Language, lacks these advantages. Esri gives a significant insight into AML when they remark on its "ability to execute such 'high-level language' actions as branching, variable manipulation and argument transfer" (Esri, 1991). The authors suggest that these characteristics do not qualify AML as a 'high-level' language.
GOAT v4.1 contains about twice as many lines of code as v3 (approximately 15,000:7,500). While some of the added size can be attributed to increased functionality, the authors feel that several factors contribute to making AML a bulkier language than GSL:
GSL read -l "$(getv wind)" minx maxx miny maxy junk
AML &sv extents = [show mapextent map]
&sv minx = [extract 1 %extents%]
&sv miny = [extract 2 %extents%]
&sv maxx = [extract 3 %extents%]
&sv maxy = [extract 4 %extents%]
GSL distance > temp.tmp
sys -p "awk '(NR>1)' $IOPATH/temp.tmp"
AML UNITS MAP
LENGTH
UNITS PAGE
&sv length = [show length] /* This is the length in
meters
/* Convert to a more meaningful number
&if %length% >= 30000 &then &do
&sv length = %length% / 1000
&sv length = [round %length%] km
&end
&else &do
&sv length = [round %length%] m
&end
GSL term_type = "$(sys -p echo \$TERM)"
AML &sv tmpfile = tmp/tmp.lis
&sys [quote echo $TERM > %tmpfile%]
&sv lun = [open %tmpfile% openstat -read]
&sv term_type = [read %lun% readstat]
&sv junk = [close %lun%]
&sv junk = [delete %tmpfile% -file]
GSL term_type = "$(sys -p echo \$TERM 2>&1)"
if [ $? -ne 0 ]
then
err_mgr add "$0: error checking environment: $term_type"
exit 1
fi
AML &sv tmpfile = tmp/tmp.lis
&sys [quote echo $TERM > %tmpfile%]
&sv lun = [open %tmpfile% openstat -read]
&if %openstat% <> 0 &then &do
&run err_mgr add Error: unable to read command redirection
file
&call bailout
&end
&sv term_type = [read %lun% readstat]
&sv junk = [close %lun%]
&sv junk = [delete %tmpfile% -file]
/* If read failed, return.
&if %readstat% <> 0 &then &do
&return
&end
The authors see three significant indicators of the success of the GOAT. First, general purpose spatial data browsers have a reputation for being difficult for non-technical users - even an Esri Product Manager acknowledged that ArcView is too complicated for an untrained user. By limiting and clearly defining its scope, the GOAT has become a software tool that BCE staff use with fifteen minutes or less of orientation.
Second, when a user is first presented with a new data viewer application, they will be absorbed in how it works, and how to use it. The length of time until they stop noticing the software and begin to notice their data instead is a crude measure of the software's ease of use. First-time GOAT users often reach this transition in 10 minutes.
Finally, a clear measure of the failure of the early GOAT is the fact regional staff would not use it. Similarly, the fact that regional staff heavily use the current GOAT and clamour for additional functionality is a measure of its success.
AML ARC Macro Language, the scripting language used in
ArcInfo
BCE The BC Environment division of the Ministry of
Environment, Lands and Parks
Esri Environmental Sciences Research Institute, the producer of
ArcInfo and ArcView
GSL Genasys Scripting Language, the scripting language used in
Genamap
HP Hewlett Packard
ksh Korn Shell (a Unix command interpreter and scripting
language)
RDBMS Relational Database Management System
SQL Structured Query Language, the common data query and
manipulation language used by Oracle RDBMS
Mackenzie, Bruce, and Todd Glover. 1995. About the BCE GOAT, WWW document, BC Environment.
Mackenzie, Bruce, and Todd Glover. 1993. About BCEGIS, WWW document, BC Environment.
Glover, Todd, Bruce Mackenzie and Dianne Allen. May 1995. "In Search of a Unified Projection for GIS Analysis", GIS World Report/Canada.
Mackenzie, Bruce. 1995. GIS Implementation at BC Environment. Proceedings of the Tenth Annual Symposium on Geographic Information Systems in Natural Resources Management. Vancouver: GIS World, Inc.
Esri. 1991. AML User's Guide. Redlands, California: Environmental Sciences Research Institute.
DMR Group. June, 1992. Regional GIS Pilot Implementation, Volumes I - IV. Unpublished report, BC Environment.
DMR Group. June 1993. Skeena Regional GIS Pilot: Summary of Experience. Unpublished report, BC Environment.
CGI Group. April 1993. GIS Action Plan Audit for BC Ministry of Environment. Unpublished report, BC Environment.
Version 4 of the user interface was the first to be called the "GOAT", though for convenience, that term has been used throughout this document.
Glover, Todd
Spatial Data Manager
Skeena Region, BC Ministry of Environment, Lands, and Parks
Bag 5000
Smithers, BC, Canada
V0J 2N0
250-847-7283 Fax: 250-847-7728
tglover@smithers.env.gov.bc.ca
Todd Glover is the Regional Spatial Data Manager for BC Environment's Skeena Region in Smithers, a position to which he was appointed as a reward for surviving the Smithers Pilot.
Mackenzie, Bruce
GIS Technical Analyst
Information Services Branch, BC Ministry of Environment, Lands, and Parks
3rd Floor, 737 Courtney St
Victoria, BC, Canada
V8V 1X4
250-387-4192 Fax: 250-356-7297
bmackenz@ssbpost.env.gov.bc.ca
http://www.env.gov.bc.ca/gis/
Bruce Mackenzie has a Computing Science degree from the University of Victoria. He developed software for a GIS vendor for several years, and has supported GIS implementation at BC Environment since 1993.