The VPF Viewer Extension for ArcView

David Fong, Satish Malipeddi, Joe Metzidis
Esri
380 New York Street
Redlands, CA 92373
dfong@Esri.com, smalipeddi@Esri.com, jmetzidis@Esri.com

Vector Product Format (VPF) is a standard U.S. Department of Defense (DOD) format for vector-based digital map products. The National Imagery and Mapping Agency (NIMA), the primary mapping organization for the DOD, developed this standard and is producing an increasing number of digital products in this format. Esri's initial response to handling VPF data was to develop conversion tools to import and export the data into and out of ArcInfo coverages. This solution was satisfactory if one had ArcInfo, but it was inefficient if one merely wanted to query a VPF database. Esri has now developed and released an extension to ArcView that will directly read VPF data without requiring the conversion of the data into another format. With this extension, VPF data can be treated like any other ArcView-compatible data source.


Introduction

Vector Product Format (VPF) is a standard format, structure, and organization for large geographic databases that is based on a georelational data model and is intended for direct access by application software (MIL-STD-2407, 1996). The National Imagery and Mapping Agency (NIMA) produces its vector digital map products in this format. Because data in this format are becoming more common, Esri has developed an ArcView extension that directly reads and displays VPF data. This paper provides a brief summary of VPF, discusses Esri's involvement in its development, and then explains how to use the VPF Viewer extension.

What Is VPF?

NIMA is responsible for maintaining imagery and geospatial information and providing it to the United States Department of Defense (DOD) and other national policy makers. It is one of the leading producers of mapping products in the world. In 1992 the agency's predecessor, the Defense Mapping Agency (DMA), developed VPF as a standard for its vector-based digital map products. The primary reasons for developing VPF were to

  1. Create a flexible standard that can be applied to the many different digital products produced by NIMA.
  2. Create a format that could be accessed directly by GIS and other application software.
  3. Create a format that was compatible with DIGEST standards. (Final VPF Design Manual, 1992)

In 1992 DMA released the first data product in VPF, the Digital Chart of the World (DCW), a digital map covering the entire world at a scale of 1:1,000,000. Since then, DMA/NIMA has released or plans to release a wide variety of digital data products in this format. The Digital Nautical Chart (DNC), Vector Smart Map (VMap) at four different scale levels, the World Vector Shoreline (WVS), Vector Interim Terrain Data (VITD), and Digital Topographic Data (DTOP) are examples of some of the products generated in VPF. Their applications vary from marine and aeronautical navigation to terrain analysis to targeting and site location; thus, the content of these data sets varies greatly between products. However, these different products all use the same georelational data model. In the model, data are organized into a five-level hierarchy. At the top level is the database, a logical collection of data managed as a unit. A database will contain one or more libraries. A library is usually organized along some type of geographic categorization (political units or lat-long quadrangles, for example). Within a library, data are organized thematically into coverages. A coverage is the basic unit of data storage in the VPF data model. It holds a collection of geographic features that are both thematically and topologically related to each other. The structure and concept of a VPF coverage is similar to that of an ArcInfo coverage. Coverage features can be subdivided into classes. A feature class is a group of features that shares a homogeneous set of attributes. At the bottom of the hierarchy is the feature. Features are represented in two ways. First, they are represented geometrically by topologically linked sets of spatial primitives called nodes, edges, and faces (points, lines, and polygons). Second, features are represented thematically by a set of descriptive attributes. In the data model, feature primitives and feature attributes are linked together through relational tables.

VPF was designed to permit the direct access of data by software applications. It was not intended to be merely a data exchange format. VPF data were meant to be stored on and read directly from CD-ROM to take advantage of that media's high storage capacity and interactive capabilities. It was not supposed to be necessary to translate the data before they could be used by an application. Some of the uses of the various NIMA products are potentially time sensitive and translation could become a hindrance.

VPF was designed to be compatible with the Digital Geographic Information Exchange Standard (DIGEST). DIGEST is a series of standards for all types of digital geographic data that have been agreed upon by the Digital Geographic Information Working Group (DGIWG), a group of defense mapping organizations from eleven NATO countries in North America and Europe. One of the standards is Vector Relational Format (VRF), which is for vector data in a relational data structure (DIGEST, Annex C, 1994). VRF can be considered to be equivalent to VPF because both share the same georelational data model and structures (there are minor differences in some of the metadata table column definitions). Currently, there are numerous DGIWG and several non-DGIWG nations producing VMap level one data for their countries in a coproduction agreement with NIMA.

Early Esri Involvement

Back in 1991, Esri was one of several commercial firms that helped DMA develop the VPF standard. This is evident in the VPF georelational data model, which is similar to ArcInfo software's. Esri personnel also developed the first version of VPFVIEW, DMA/NIMA's query software for VPF data. Esri's initial adaptation of its software products to VPF was to create conversion tools to import and export VPF data into and out of ArcInfo coverage format. Conversion conflicted with the original intent of VPF as a direct access format, but the decision to concentrate on development on a translator rather than a direct reader was made for several reasons. First of all, Esri's Professional Services Division has been and continues to be a major producer of VPF data. It has won numerous Basic Ordering Agreement (BOA) contracts to produce DNC, VMap, and Urban Vector Smart Map (UVMap) databases for DMA/NIMA over the past five years. Our primary methodology was to compile the source data in ArcInfo coverage format and then convert the ArcInfo coverages into VPF. In order to facilitate the production of VPF data, in 1993 Esri developed the VPFEXPORT and VPFIMPORT commands for ArcInfo. Second, other contractors were also winning VPF BOA contracts and needed these conversion tools, so there was a demand for an ArcInfo-to-VPF and VPF-to-ArcInfo translator. Third, since VPF was still new in the early 90s, there was not a lot of data available, and thus, there was little demand for GIS software to use VPF data. The VPFVIEW query software that came with the VPF data on CD-ROM seemed adequate at the time.

VPF Data Become Available

As more digital products were designed in VPF and more data were converted to VPF, the GIS community gradually became more aware of these data and started to use them. Until recently, in order to utilize VPF data, the data had to be converted into the native format of the GIS software or queried by using VPFVIEW. This was an adequate solution in some cases, but not in all.

VPFVIEW would only run on Sun UNIX or MS-DOS platforms (recently, VPFVIEW has been made available for Windows). A user who did not have one of these platforms was out of luck. A user with a VPFVIEW compatible machine could query the database, but he/she could not do much else since VPFVIEW was never designed to perform sophisticated GIS functions. Even if the GIS software included a VPF translator like the ArcInfo VPFIMPORT command, the user could run into trouble. Translation is often time-consuming. The conversion of some VPF databases could take hours or even days, depending on the amount of data to be converted and the processing capacity of the computer being used. Some applications are time critical and would not allow for real-time translation. Also, in the case of ArcInfo software, conversion from VPF to ArcInfo has proven to be very sensitive to topological errors in the VPF database. If any intersections are found in the VPF data (i.e., where lines cross and no node is present), the conversion process aborts and sometimes tedious sessions in ARCEDIT are required to rebuild the topology.

The biggest problem, however, was that VPF was not being used for the pupose for which it was designed. It was being used as an exchange format, rather than a format that was directly accessed by a GIS application. It was an ironic situation: Esri would convert ArcInfo data into VPF products for DMA/NIMA, DMA/NIMA would ship the VPF data on CD-ROM to an end user, and the end user would then convert the data back into ArcInfo coverages so the data could be used in a GIS application. With the development of the direct VPF Viewer extension for ArcView, this inefficient process can now be bypassed.

Development of the VPF Viewer Extension

In 1996 Esri released ArcView GIS, Version 3.0, which introduced the concept of extensions to this software product. Extensions allow user/developers to add applications to ArcView, but also allow Esri's Software Development team to modularize ArcView software's functionality. Also with this release, Esri wanted to emphasize ArcView as a "universal" tool to access all types of geographic data (both in raster and vector formats). In keeping with this philosophy and because VPF was becoming a common format for military geographic data, the decision was made to add a VPF direct reader extension to ArcView.

The development of the VPF Viewer extension began in the spring of 1996. The ArcView Software Development team first defined the functionality for the extension. Because they had extensive experience in working with VPF data and NIMA's VPFVIEW, Esri's Professional Services Division provided valuable input in determining what the extension should do. The main goals for the VPF Viewer extension product were to

  1. Read VPF data directly without translation to another format.
  2. Minimize changes to ArcView software's graphic user interface (GUI).
  3. Adapt ArcView software's Area of Interest tool for ArcInfo Map Libraries to VPF's library tiling scheme.
  4. Support VPF specific Avenue requests to customize the ArcView product.

Software Development had to decide the best way to implement the desired VPF functionality. ArcView software's design and architecture are rooted in basic principles of object-oriented design and analysis. ArcView supports an object-oriented scripting language called Avenue. Description of public classes can be found using ArcView software's on-line help. The basic building blocks for thematic mapping and table query are the feature theme and feature table. The ArcView VPF Viewer is essentially a subclass implementation of the feature theme and feature table. Special support was added to make the extension behave somewhat like ArcView software's Map Librarian to add support for tiling and area of interest.

Once a workable prototype was developed, Professional Services personnel provided testing and feedback to the Software Development team in order to improve the product. A small number of preliminary beta sites were also established at this time to supplement Esri's in-house testing. These sites were established ArcInfo and ArcView users who also had experience in working with VPF databases. By the fall of 1996, the software was ready for formal beta testing with a larger group of test sites. In the Winter of 1997, after supplementing ArcView software's on-line help files and several more rounds of testing were completed, the VPF Viewer extension was ready for release. The extension is available on our ftp site, and it has been included in the release of ArcView GIS Version 3.0a.

How To Work with VPF Data in ArcView

The VPF Viewer extension was designed so that VPF data can be treated like any other ArcView-compatible data source. ArcView software's GUI is not altered by the implementation of the extension. Themes derived from VPF source data can be displayed, queried, and used in conjunction with themes derived from other sources (both raster and vector).

The VPF Viewer extension is loaded into ArcView like any other extension. The Extension Dialog box is brought up by selecting the Extension option in the Project menu. By checking the box for the VPF Viewer extension and then hitting the "Okay" button, VPF-based themes can be added to a View document, or a Table document can be created from a VPF table. Adding VPF themes to a view is similar to adding themes from an ArcInfo coverage. By selecting the Add Theme button from the View document button bar, the user brings up the ArcView source browser GUI. When the user navigates down to the VPF library directory, the browser displays the available VPF coverages for that library. Each coverage contains one or more feature classes, and each feature class can be selected and added as a theme to the View document. The VPF Military Standard allows for complex feature classes (a feature class made up of two or more other feature classes or primitive types), but no current VPF product specification contains complex feature classes. Therefore, the VPF Viewer does not support complex feature classes at this time.

If the VPF library is made up of tiled coverages (and most are), the Area of Interest must be set in order to let the software know which tiles need to be displayed or queried in the View. By default, the Area of Interest is set to the center tile. By using the Area of Interest tool in the View document tool bar or the Area of Interest form in the View Properties dialog box, the user can control the spatial extent of the tiled VPF-based themes. Typically, a VPF library contains a Tile Reference coverage that delineates the extent of all the tiles in a library. If this coverage is added as a theme to the View document, then it can be used to help determine the Area of Interest. The Area of Interest for ArcInfo LIBRARIAN layers is determined in the same way.

Once the VPF themes have been added, they behave like any other feature theme. They can be displayed, queried, used as a part of a theme-on-theme selection, and so on. They are read only, and therefore cannot be edited, although a user can convert a VPF theme into an editable shapefile. Shapefiles, however, cannot be converted into VPF data. The one unique aspect of VPF themes is that when their attribute tables are displayed, a descriptive column for every coded attribute column is attached to the virtual table. This was done because most VPF products are attributed in the Feature and Attribute Coding Catalog (FACC) scheme. This means that with the exception of attribute columns that store proper names or interval measurements, feature attributes are stored as code values. Since most end users would not know offhand what these codes mean, it was decided that adding these code descriptions to the attribute tables (virtually) would be a useful feature.

The Future

Currently, VPF themes, like other feature themes, are symbolized arbitrarily from ArcView software's default symbol sets. The user can customize the display of the themes by using the Legend Editor to choose symbols from the default symbol sets. However, the default symbol sets do not always meet the requirements of military users. There is a lack of good ArcView symbology to use with the VPF data sets that are being produced. Esri has done some preliminary work in developing an ArcView-compatible symbolset for use with DNC data, but more needs to be done, especially on some of the other products.

References

Department of Defense. Interface Standard for Vector Product Format. MIL-STD-2407. 1996.

Digital Geographic Information Working Group. The Digital Geographic Information Exchange Standard (DIGEST). 1994.

Environmental Systems Research Institute, Inc. Final VPF Design Manual. CDRL B027.2. Redlands, California. 1992.