Robert L. Gage and Raymond L. Gates

Alternative Techniques for Development of Complex ArcView/Avenue Applications

Development of complex ArcView/Avenue applications in a multi-developer environment presents unique challenges and opportunities. In the development of the Environmental ToolBox, a set of extensions for aircraft noise analysis, developers were confronted with a variety of challenges, which through innovative design and implementation, were transformed into working solutions that may be applied to similar development efforts. This paper will describe those challenges and share solutions with the user community. Topics to be discussed include logically partitioning a large application into extensions; configuration management strategies; debugging tools; applying object oriented techniques to Avenue; and overcoming limitations inherent in ArcView/Avenue application design/development.


Historical Background

In 1998, the GIS Team at NASA/Langley Research Center was tasked to develop a Common, Off-the-Shelf (COTS)-based replacement for the Air Combat Command’s (ACC) Assessment System for Aircraft Noise (ASAN). The decision to base the new system on Esri’s ArcView product was predicated on two factors. First, ACC personnel had a level of familiarity with the ArcView system through prior GIS activities. Secondly, ArcView, through its powerful extension mechanism, allows the Avenue programmer flexibility in extending and supplementing applications.

Over the two years that preceded the release of the 1.0 Beta release, the developers of the Environmental ToolBox faced many challenges in the design and implementation of the system. In the process of developing and implementing the Environmental ToolBox, many lessons were learned. We hope to share some of those lessons with the user community to aid others in the development of complex applications.

Challenges

One of the difficulties encountered in any medium to large-scale project is the issue of coordinating the activities of multiple developers. As with any software project, clearly defining the requirements and overall architecture is critical. Beyond this, there are several techniques and tools that can be applied to an Avenue development project to provide assistance including sensible use of the extension mechanism, applying some object oriented design techniques from other traditionally non-OO languages, and the use of configuration management software tools.

Another area that presents some unique challenges in Avenue is that of user interface design. Integrating your application’s needs with the extensive functionality inherent in ArcView can present some difficulties including the need to anticipate a large number of possible user interactions and how that effects the application’s state. In addition, the limited set of control types and the large number of scripts required to handle the various events can make an application difficult to manage. In tackling these issues, we have discovered several techniques and design guidelines that may provide some benefit.

Additionally, testing and debugging extensions can be a difficult and time consuming task because there are many instances where the standard ArcView error handling does not return sufficient information to locate the offending code and often the built-in debugging tools are not available when testing the extension. In response, we have developed some testing/debugging guidelines and a tool to provide a detailed trace of the execution.

Using Extensions as a Component Framework

As mentioned earlier, clearly defining the requirements and architecture of the application is the most important first step because all the developers must be aiming for the same goal. ArcView’s extension mechanism can be leveraged here to provide a logical means for defining the components that make up the overall application. In the development of the Environmental Toolbox, seven extensions were developed.

Environmental Toolbox Architecture
Figure 1. Environmental Toolbox Architecture

The dependencies of six of these extensions are shown in Figure 1. Some of these extensions are purely computational. They present no user interface components to the user and thus, are used only by other extensions. An example here is the Data Manager extension that provides a common set of utilities needed by several of the other higher level extensions to perform common tasks such as data import and route polygon generation. The extensions at the top most level in the diagram are those which provide user interface components. One of the key design features in the Environmental Toolbox was the desire to provide a common framework for doing any type of impact analysis such as noise, air pollution, etc. The assessment manager extension provides this common framework by presenting the user with a new document type called a scenario which allows the user to perform common operations, such as selection of a population and definition of the annual set of operations in the area. The interface provided by this extension is itself then extended by the Noise Analysis and Noise Reports extensions to provide functionality particular to environmental noise analysis. The Noise Analysis extension was in turn designed to accept multiple underlying computational extensions to handle the actual noise prediction calculation. The MRNMAP extension implements the standard interface which is required by the Noise Analysis extension and acts as a wrapper for the text file driven application software. ArcView manages all the dependencies at load time. For instance, if the user selected the Noise Analysis extension to be loaded, ArcView would automatically load the Assessment Manager, Data Manager, and MRNMAP extensions as well. There are several advantages to this approach versus a monolithic extension. The first is the flexibility in the design to easily incorporate new functionality in the future. The second is the ability to have developers working on each of these components in parallel. To accomplish this, it is important to clearly identify each script as public or private and to exactly specify the input parameter and the return value. This is essentially the contract between the extension developer and any other developer who might use it. Another advantage that will be addressed later is that it is much easier to test these extensions in isolation rather than having to debug the whole application at once.

Managing Changes in Source Code

While the use of a hierarchical extension framework works well during initial development, as the project progresses and developers begin to work on each other’s code, the need for a configuration management tool becomes more apparent. The problem is that typical Avenue extension development involves doing all the editing in a single ArcView project file and then building the extension using a "make" script. Most CM tools will simply control access to this one .apr file, making simultaneous editing almost impossible if it uses pessimistic locking. Additionally, viewing the differences between revisions is almost useless because of the cryptic looking way the source code is embedded in the .apr file. To make matters worse, ArcView will automatically renumber object ids in the .apr file, creating a tremendous amount of irrelevant output. When confronted with this situation, we did a comparison of various CM packages and decided on using the open source alternative known as CVS or Concurrent Versions Systems. CVS has a command line driven interface as well as a plethora of GUI interfaces for a wide range of systems. It implements an optimistic locking scheme in which simultaneous edits are allowed and conflicts are resolved at check-in time.

AVCVS
Figure 2. AVCVS

To ease its use for ArcView development, we developed an extension called AVCVS that provides the functionality of CVS from within an ArcView project. It stores each Script, Dialog, DocGUI, or other supporting file independently in the CVS repository. This makes looking at source code differences much more useful and allows finer grain tracking of who edited what object and when. A browser provides a quick view of all the objects in the project with current version number and status information. Controls are provided in the browser and within the individual script and dialog editing interfaces to check-in changes, merge updates from the repository, get change logs and version differences, and add and remove objects from the repository. In addition, CVS supports a tag mechanism that can be used to identify the particular set of object versions which went into a particular build of the extension. This allows you to reconstruct any prior release of the extension or use a particular release number as a reference for looking at differences. This has proven to be extremely useful in tracking down the bugs that are inevitably introduced into the code as features are added between revisions.

Using Object Oriented Techniques When You Can’t Define a New Class

While Avenue’s syntax appears to be Object Oriented (OO) in nature, it lacks many of the features required to call it an OO language. The most apparent of these is its inability to subclass existing classes or even define new classes at all. However, there are many techniques that you can use to achieve similar results. The first technique is the use of ArcView’s object tag functionality to store extended state information about particular objects. By doing this we can effectively subclass a built-in ArcView class. In the case of the Environmental Toolbox, a new document type called a Scenario is introduced into the project. This new document type is conceptually a subclass of ArcView’s native View class. This is accomplished in the Assessment Manager extension’s install script where the View DocGUI is cloned and modified. The modifications include changing the event scripts for various controls to point to a wrapper version that will usually call the original event script and perform some additional work as well. In effect, the new event handler is overloading that of the built-in View class. This can prove useful by itself in many simple cases, but in other cases we need to modify not only the methods of a class, but its state information as well. The object tags provide a good mechanism for maintaining this additional state data. To allow an object to be extended or subclassed multiple times (similar to multiple inheritance), the preferred method we use is to attach a Dictionary object as the tag and then add string referenced objects as needed. For example, the Scenario document’s object tag dictionary contains, among other things, an entry referenced by the name "MissionSet", which is itself a list structure containing all the FTab and VTab objects needed to completely define the operational data. In addition, each theme in the Scenario is tagged in a similar manner to record additional data related to what type of information is in the theme. This information is used in the overloaded update event handler from the Scenario’s "Delete Themes" menu choice to decide which themes can be safely removed.

The technique outlined above works well for extending many object classes which have a visual aspect and which don’t require a great deal of modification. Another technique that can be applied is taken from other traditionally non-OO languages. Originally, perl was not designed as an object oriented language, but was later extended. Essentially perl uses a normal data structure, which maintains the state information, to reference an object. The data structure is "blessed" into a class to allow the new OO operators to access the methods. While this is close to the right idea, it isn’t possible to extend ArcView’s syntax in the same way. A better example is the technique used in many GUI toolkits such as the Gimp Tool Kit (GTK) which are accessed from the C language. In a similar manner, a data structure is used to maintain the state of a particular object instance. The key is in the standard way in which the methods are invoked. A method is simply implemented as a function named using its class name followed by its method name. The function is always called with the first parameter set to the data structure containing the state of that particular object instance. This technique was successfully used in Avenue to implement a Union class that allows multiple VTabs to be accessed as if they were a single VTab. The class was implemented as a new extension that consists of a set of scripts whose names all begin with the word "Union" followed by the standard method name from the VTab class. The input parameter is always a list containing a Dictionary object that maintains all the state information for the instance, followed by the other standard parameters which are required by the method. The return value is the same as that normally returned by the similarly named VTab method. The one exception to this rule is the "UnionMake" script, which accepts information about the VTabs that are to be used and constructs and returns a new Dictionary object for the new instance. The Dictionary structure maintains handles to the individual VTabs, a list of the common field names, the VTab to be used when adding new records, etc. While it is obviously not possible to use this Dictionary data structure as a parameter for other Avenue requests which require a VTab, it does provide a useful construct in many circumstances. Because the method’s names, input parameters, and return values are all identical to a normal VTab, it requires minimal communication between the developer of this extension and any developers who need to use it in their work. One particular use of this extension was in the selection of the population points from two independent shapefiles, one of which contains the initial set of locations and the other containing additional user defined locations. The selection Bitmap used in the new methods is a concatenation of the individual Bitmap objects from the underlying VTabs. This Bitmap can be used to perform a query on the union, loop through the records, and perform updates to the underlying VTabs for instance.

Don’t Try This At Home

ArcView can exhibit some very strange behavior at times. As an example, it’s fairly easy to end up with a zero length Bitmap object, which can cause all sorts of problems when you actually try to use it. For this reason, and many others, it’s a good idea to never test an extension in the project where you are developing the code. Always load and test the extension in another fresh project. In many circumstances it’s all too easy to trash a project to the point where your original code is almost irretrievable. It’s also a good idea to construct a "test suite" project for each extension. This project should be designed to test the features of the extension in isolation of the other extensions. This can help identify problems before they are discovered in the complete application where they will likely be much harder to track down.

While running your extension outside the development project can avoid the loss of hard work, it also limits the use of the built-in debugging tools. Additionally there are many circumstances where ArcView’s error reporting does not provide sufficient information to locate the offending code. To resolve these limitations, an extension was developed to trace the execution of a running Avenue application. When loaded, the extension shadows all the existing system and extension scripts and provides a detailed log of the execution.

Trace Log
Figure 3. Trace Log

The log identifies the name of the script being executed, the input parameter, and return value. Each entry is indented to show the chain of execution and the input and return values are list expanded and identify the class and value of each item. This tool has proven extremely useful in understanding the execution of event scripts and delayed runs and has helped identify many problems which result in a segmentation fault or "AVARRAY" error, for instance.

Dialog Designer

The graphical user interfaces needed to support the requirements of the Environmental ToolBox are complex and require the use of many of the available Dialog Designer controls as well as integrating many of them to form "roll your own" dialogs. Consequently, layout of the windows that the user uses for input to the program was tedious, but paled in comparison to the underlying code that was needed to effect the actions of the user interface. A representative example is that of the Mission List Dialog shown below (Figure 4).

Mission List Dialog
Figure 4. Mission List Dialog

As one can see, there are a variety of Dialog Designer controls in use for this dialog. An interesting feature of this dialog and its underlying code is seen in the "Operations" control. This user interface component mimics a simple spreadsheet for the input of annual mission operations. Through Avenue scripts, the user can select either a single entry or an entire column for input. Although it would have been nice for Dialog Designer to have included a spreadsheet widget as part of the supplied controls, we have demonstrated that at least some rudimentary functionality can be achieved through concerted Avenue programming.

As far as advice to users of Dialog Designer in the development of more complex user interfaces, we would offer these points:

One other area to investigate is the implementation of event scripts. If you discover that you have a large number of event scripts and/or these event scripts have similar code, you can look at consolidating these into a single script. This can be done by embedding code in the event script to check for the name of the control that initiated it. Avoid taking this to extremes by consolidating every event into one script for instance. There is some additional overhead in the script to determine what control invoked it and one giant event script can be even more difficult to interpret than the sea of event scripts with which you started.

At this point, we may ask, "How could Dialog Designer be better?" We offer a few suggestions to make Dialog Designer a more complete product:

  1. Improve (actually, develop) debugging tools
  2. Give the user the ability to build their own controls, or alternatively, pay the Dialog Designer code gurus to develop more controls
  3. Give the user the ability to set/modify font properties (size, color, style)
  4. Allow the user to associate controls with a control panel
  5. Improve/develop low level graphics API
  6. Allow access to event structures
  7. Allow initialization of controls other than by scripting

To DocGUI or not DocGUI

When adding a new document type to ArcView, there are several ways of providing the new associated DocGUI. One way is to create the DocGUI during the install script by cloning and modifying an existing one for instance. Another technique is to store the DocGUI in the extension and simply add it to the project during the install. AVCVS provides support for this by allowing DocGUIs to be managed as objects in the repository.

The second method is more appropriate when the new document type should provide a restrictive environment to the user. In the Environmental Toolbox, the Mission Sets document type was implemented in this manner. A Mission Set document manages all the operational data for a particular region. Because of its focused usage, many of the tools provided by ArcView for a View document are unnecessary and it is in fact difficult to anticipate the complex impact the use of these controls will have on the state of the application.

In contrast, the first method was used in the implementation of the Scenario document type. Because Scenarios are intended to provide an open mechanism for the analyst to evaluate a variety of environmental impacts, it is desirable for the user to have access to the full functionality of ArcView and its other extensions. For these reasons, the Scenario DocGUI is constructed in the installation script. This retains all the built-in functionality of a View and any functionality that has been added by any additional extensions.

There are clear advantages and disadvantages to both approaches and they must be weighed independently for each application.

Conclusions

We hope that our discussion of some of the challenges we encountered in our development project and the techniques and tools we employed to meet those needs will be of benefit to other ArcView/Avenue developers.

Acknowledgments

All work on the Environmental ToolBox was performed under NAS1-20431.

Reference

ArcView Dialog Designer, Environmental Systems Research Institute, 1997.


Robert L. Gage
Senior Member of the Technical Staff, Computer Sciences Corporation
NASA/LaRC GIS Team
MS 445
NASA Langley Research Center
Hampton VA 23681
Tel: 757-864-6867
Fax: 757-864-8463
Email: r.l.gage@larc.nasa.gov

Raymond L. Gates
Computer Scientist, Computer Sciences Corporation
NASA/LaRC GIS Team
MS 445
NASA Langley Research Center
Hampton VA 23681
Tel: 757-864-6993
Fax: 757-864-8463
Email: raymond.l.gates