Arthur Miller
ABSTRACT.
The Interior Columbia Basin Ecosystem Management Project (ICBEMP) is a multiple agency project charged with creating a broad scale environmental impact statement for an area covering approximately 75 million acres of federal lands. The ICBEMP has generated thousands of maps since its inception over six years ago. These maps were generated by a distributed and ever changing staff and were created to meet a variety of needs from internal discussions to official publication and web distribution.
The challenge for the ICBEMP was creating a tracking system for cartographic products that was accessible by all staff members and did not interfere with often demanding production time lines. This was eventually achieved by developing a run-time Access database tool, the Graphics Tracking System (GTS), that contains just the key elements necessary to identify, and locate any particular map. The GTS is not a fully automated system and has no actual ties to the GIS work environment. It relies on the map creator to enter relevant information and provides tools for updating location information as well as report and query tools for retrieving information.
The GTS is an evolving tool that has been flexible enough to allow the tracking of ever changing map producing methods, work environments, and personnel changes. Though the GTS was created specifically for the ICBEMP, the basic concepts of the system could be used to track map production in any GIS environment that does not have resources to build a permanent or an administered tracking system.
INTRODUCTION.
Most people using Geographic Information Systems (GIS) are familure with the concept of metadata. In the United States, Federal government agencies are now mandated to use the Federal Geographic Data Committee standards (FGDC) for capturing metadata. The FGDC are designed to document metadata for data themes. While thematic metadata is essential to a GIS it does not contain all information pertinent to GIS work. In particular the FGDC does not specifically tie thematic metadata to outputs such as maps or reports that present combinations of thematic elements in one product. This paper addresses the attempts of the Interior Columbia Basin Ecosystem Management Project (ICBEMP) to track metadata for various graphic products and the relationship of those products to thematic data.
THE INTERIOR COLUMBIA BASIN ECOSYSTEM MANAGEMENT PROJECT.
The United States Department of Agriculture Forest Service (FS) and the United States Department of Interior Bureau of Land Management (BLM), upon the direction of President Clinton, initiated the ICBEMP in July 1993. The ICBEMP is chartered with providing a broadscale ecosystem based management strategy, based on an integrated scientific assessment, for all FS and BLM lands within the Interior Columbia River Basin.
The ICBEMP covers some 145 million acres within the United States portion of the Columbia River basin and portions of the Klamath and Great basins in Oregon. The ICBEMP spans an area from the crest of the Cascade Mountains in the west to the crest of the Rocky Mountains in the east. The FS and BLM manage over 75 million acres of this area (see figure 1). The ICBEMP area includes portions of 7 states and 100 counties.
Figure 1 - Major Ownership in the ICBEMP
GIS work for the ICBEMP has been performed at several locations within and outside the project area, and by a large number of analysts. This work has led to the release of hundreds of data themes and to the generation of thousands of output products such as maps, charts and reports. Of the maps created by the ICBEMP, only a fraction has appeared in official publications. The majority of maps have been for internal review, and for presentation in forums such as public briefings. Formats of project maps have ranged from thumbnail size images for electronic distribution to wall size posters.
These factors, from the variety of map formats and purposes to the number and location of creators necessitated a method for keeping track of the ICBEMP graphic output products.
THE AUTOMATED TRACKING MODULE, A PROCESS TRACKING SYSTEM.
At the outset of the ICBEMP, tracking system requirements were much more ambitious than simply keeping track of graphic products. The goal was to build a process tracking system that could not only track data through initial capture, analysis and output products, but would be able to re-run analysis and re-create output products when source data was updated.
The goal of process tracking led to the creation of the Automated Tracking Module (ATM). The design of the ATM was greatly influenced by the technology that was available. In 1994, the ICBEMP was running ArcInfo 6.1 (with Grid) on Unix workstations. The ATM was built with ArcInfo 6.1, using INFO, AMLs, Arc Tools and some C code.
Several key decisions were made that allowed the ATM to function within this computer environment.
DATA LINEAGE.
The ATM met the stated requirements of tracking data from inception to output products, and was based on the concept of tracking data lineage, or parent/child relationships of datasets.
All datasets tracked in the ATM were classified as either master data or derived data. A master dataset was defined as any dataset not dependent on other datasets within the ATM. A derived dataset was defined as being dependent on data already entered in the ATM as master data or generated within the ATM as derived data. Any data being referenced as parent data to a process has to already exist in the ATM.
Dependent datasets were created within the ATM by entering an AML that generated the dataset and information linking them to their parent datasets. This technique allowed the tracking of processes that generated new datasets as well as processes designed solely for creating cartographic products or reports.
ATM FEATURES AND TOOLS.
The ATM contained numerous administrative tools. The most powerful tool allowed the re-generation of any dependent dataset if it was determined to be older than data anywhere in the chain of parent data. The ATM also contained various report functions such as a location verification tool that would audit all data location information and report invalid pathnames. Along with system wide tools, the ATM contained report tools for generating individual dataset lineage reports as well.
GONE BUT NOT FORGOTTEN.
Although the ATM functioned as it was intended, it ultimately failed to meet the needs of the ICBEMP. Although all ICBEMP GIS analysts had access to the same ArcInfo software, at the time it was not possible to link all of the distributed GIS platforms into a central system containing the ATM. This meant that using the ATM for all processing would require re-creation of much of the distributed analysis within the ATM.
Due to the automated nature of the re-creation tools within the ATM it was also possible to overwrite important datasets that needed to be maintained as a snapshot of analysis at a particular point in time. Given the amount of analysis being performed in a short time frame, reviewing the impacts of these updates became an administrative bottleneck.
The final and fatal flaw of the ATM was its inability to track non-GIS processes. This was not initially a requirement of the system, but became essential as the ICBEMP distributed much of its analysis out of the GIS and into PC Access databases. This change occurred in response to the inability of INFO to effectively handle the scope of database processes being performed.
By 1995 these problems reduced the use of the ATM primarily to tracking the creation of graphic output products. In many ways the ATM proved inadequate to this task without major revisions to query and reporting tools. This need arose at the same time that the ICBEMP was upgrading from ArcInfo 6.1 to ArcInfo 7.0, and was porting its thematic metadata (pre-FGDC format) from Oracle on a DataGeneral platform to Microsoft Access on PCs.
The decision was made to build a new tracking system specific to graphics and metadata rather than upgrading the ATM. Data was eventually ported out of the ATM, and the ATM was never upgraded.
THE GRAPHICS TRACKING SYSTEM (GTS).
To create a tool for tracking graphics you must first define what is considered a graphic. The ICBEMP has defined a graphic as a map, poster, or chart created in GIS. Initially all ICBEMP graphics were kept in ArcInfo graphics meta file (GRA) format. This was revised over time to include PostScript (PS) files generated in ArcView. At the time the GTS was being designed, the ATM already contained over 300 graphics.
WHAT WAS GIVEN UP.
While the concept of lineage and reproducible processes was the key feature of the ATM, it was not as important to graphic products as to other datasets since graphic products tracked in the ATM were not used as parent datasets for further processing. The ICBEMP already used AMLs to generate graphic products, so the information within these AMLs could be used as the record of source data for graphics. With information necessary to create a graphic already existing within AMLs, the emphasis for the GTS became identifying the specific AML for creating a graphic. Any re-creation of graphics would rely on manual re-running of the appropriate AML.
WHAT IS TRACKED IN THE GRAPHICS TRACKING SYSTEM (GTS).
The Graphics Tracking System was created with the ability to track the following metadata for a graphic product. Some of the terminology used in the GTS is very specific to the ICBEMP, but the type of information being tracked is fairly universal to map creation work. The "*" indicates fields that are required to save a record the GTS.
HOW THE GTS WAS BUILT.
The GTS was originally created in Microsoft Access 2.0 and could be used as a runtime executable or use an installed version of Access. The GTS has been updated several times since its creation, and is currently being updated to be an Access 2000 application. The application links to two separate databases and a security file. One database contains the data entry forms and reports, the second database contains the shared data for the system. This design allows the application to run locally and access shared data and security files located on another PC.
The GTS allows multiple users to simultaneously access the system. The Access Database handles record locking. While the ICBEMP uses the GTS extensively, there are rarely more than 2 or 3 people accessing it concurrently. In this environment multiple access has not been a problem.
HOW DATA IS ENTERED INTO THE GTS.
Data entry into the GTS is a manual process. There has been no attempt to directly tie the GTS to the GIS. Originally the GIS existing on Unix and the GTS on PCs necessitated this separation of tracking from the GIS.
While this separation has meant relying on the accuracy of keyboard entry of data, it has also kept the GTS flexible enough to use regardless of changes to GIS environments.
Metadata for new graphics are entered through the graphic file metadata editor form (see Figure 2). This form contains three sections. The top section has fields related specifically to a graphics, the middle section has fields related to AMLs used to generate the graphic, and the bottom section has fields for information on documents where the graphic is used.
Since all of the GTS forms are built in Access, they have all of standard Access keyboard shortcuts available. The fields with down arrows to their right all have pick-lists. Even with the pick-lists and shortcuts it is still requires a great deal of work to enter information for large numbers of maps. The Create Clone Records tool was added to the GTS to provide a shortcut for some data entry.
Figure 2 - Graphic File Metadata Editor Form
The Create Clone Records button displays a pop up form (see Figure 3). This form assists entering a graphic file with the same name and similar metadata to a previously entered graphic file. A new location must be entered for each record the user wants copied. The rest of the metadata must be edited from the main form.
Figure 3 - Clone Edit Form
TRACKING AMLS.
New AMLs can be entered either in the Graphic File Metadata Editor Form (Figure 2) or in the AML Metadata Form (see Figure 4).
In the Graphic File Metadata Editor Form, if an AML has not been entered before, a new entry should be made in the blank AML Filename and AML Location text boxes on the form. A new AML Id is automatically assigned.
A new AML entry must be entered manually. Using the pull-down and editing will change the existing AML metadata. Doing so will not create a new record but will change the data for all other references to this AML.
Additional metadata for an AML, such as Creator, Version, Creation Date, Revision Date and Coloration is added using the AML Metadata Pop Up form. This form is accessed from the AML Metadata button on the Graphic File Metadata Editor Form .
Figure 4 - AML Metadata Form
For published graphics, source information is entered through the Source Themes Editor (see Figure5). The specific graphic is located by using the Select Document and Order By fields. For each graphic referenced to a document, the Doc, Fig, Title, and Subtitle fields are populated from data already entered in the Graphic File Metadata Editor Form (Figure 2). The Source Themes Editor is accessed from the top level of the GTS (the top level form is not included in this paper).
Three types of data are entered per published graphic:
Figure 5 - Source Themes Editor
QUERIES AND REPORTS.
The GTS Graphic File Reports Form (see Figure6) allows access to a multitude of reports based on different organizational principles and filtered by various selection criteria. Each report has a report title, date generated, the name of the user requesting the report and the restrictions used to generate the report listed in the report header.
The upper portion of the GTS Graphic File Reports form selects the organization of the report. The lower portion of the form allows the reports to be restricted to selected items. Pull down selection lists are available to allow selection of existing values. The values entered into the selection fields also recognize wildcards. The restrictions are treated as if a wildcard is appended to the end of the restriction. After a report organization is selected, restrictions that are not valid for that report are grayed out.
Figure 6 - Graphic File Report Generator
The report format and restrictions features generate results showing all graphics within the GTS that meet the specified restrictions. This feature makes the reporting function extremely useful as a query tool when trying to find graphics with particular themes or of particular types (see Figure7 and Figure8 for report examples).
Figure 7 - First Page of Report by Filename.
Figure 8 - First Page of Report by Document.
DATA LOCATION MAINTENANCE:
The GTS contains a tool for maintaining location path information for graphics and AMLs entered in the system. The Path Editor (see Figure 9) is designed to allow updating file location metadata when files are moved to different directories. This change of directories is sometimes necessary for efficient disk management.
Only pathnames that currently exist may be entered into the Existing Pathname text box. After the pathname is entered the Existing Path File List window on the left will display all the files whose metadata indicates they are currently located there.
When the New Pathname is entered the right file list box will display all the files whose location metadata indicates they are located there.
Figure 9 - Path Editor Form
If the "Update AML and Graphic Paths" button is activated, all files displayed in the left window will have their location metadata changed to the New Path Name. The "Update AML Paths only" and "Update Graphic Paths only" perform as described by their labels.
NON-GRAPHIC DATA IN THE GTS.
There are other features of the GTS that have not been described in this paper. Many of those features relate to ICBEMP methods for tracking work requests.
THE USEFULNESS OF A GRAPHICS TRACKING SYSTEM.
In the four years since the GTS was created over 3500 graphics created by the ICBEMP have been entered into the system. Staff members that have long since left the project created many of these graphics. Many of the graphics were created for audiences that have changed dramatically.
When the ICBEMP receives a request for a map, it is possible for even the newest employees to quickly determine if such a map already exists or if a similar map exists that can be modified for the desired purpose.
Even with the need to manually enter data into the GTS, the savings in time spent searching for graphics and in re-creating past work have been significant.
INTEGRATING GRAPHICS METADATA WITH FGDC METADATA.
With the exception of graphics that appear in published documents, the ICBEMP has not made an effort to integrate their thematic metadata with their graphics metadata. The relationship between published document graphics metadata and their source themes is simply a reference. There is no direct link to the thematic metadata. Although it has not been a priority to tie the graphics and thematic metadata to each other, having the information within the GTS in a database format will allow such integration in the future.
CONCLUSIONS.
The ICBEMP started with lofty goals of tracking every GIS process and product with the ability to automatically recreate any process. This was revised to the tracking of graphic products with enough information to reproduce them when necessary. While this down sizing of ambitions was necessary for expediency sake and proved to be successful for the ICBEMP, the original goal of tracking graphic products all the way back to the metadata of their source themes is a valid one. If metadata is necessary to accurately use and interpret thematic data, then it is equally essential to correctly using and interpreting multiple theme graphic outputs such as maps.
The loftier concept of a GIS environment that automatically tracks parent/child data relationships and integrates graphics and thematic metadata into the workflow may still be a ways off.
ACKNOWLEDGMENTS:
Terry Locke was responsible for the original Microsoft Access code behind the GTS as well as designing the data structure and relationships. Terry also co-wrote the original users manual for the GTS, which has been borrowed from liberally in this paper. Previous to work on the GTS Terry, helped create the data structure and many of the functions in the ATM.
Dan Powers has inherited the job of supporting the GTS and is working on major updates to interface and structural code.
Bart Sunseri and John Bagdanof both provided general GTS maintenance and software support over a span of several years.
Markus Matt-Wandel somehow managed to make the ideas behind the ATM functional in a very challenging software environment. This successful code work eventually led to the GTS.
I also must acknowledge the enormous amount of GTS data entry work performed by ICBEMP GIS analysts over the years.
AUTHOR INFORMATION:
Arthur Miller
Averstar, Inc.
On contract to Bureau of Land Management
c/o Interior Columbia Basin Ecosystem Management Project
U.S. Forest Service, Region 6 Headquarters
333 S.W. First Avenue, 4th Floor
Portland, OR 97204
Tel.: (503) 808-2877
Fax: (503) 808-2622
e-mail: amiller01@fs.fed.us
web site www.icbemp.gov