Developing Solutions for Integrating Field Data into a Centralized GIS

Authors

Abstract

Efficient, accurate field acquisition can be key to any coherent plan for data collection, analysis, high quality output generation and improved customer service. Many sectors currently use hard copy forms to gather field data, which are brought back to the office for manual data entry - the backlog of paper can be extensive. However, by incorporating various mobile computing tools, such as GPS receivers, digital cameras, wireless connectivity, PDA's and PDA compatible GIS software, the potential exists to acquire qualitative, cost-effective, timely digital data and thus diminish the use of standard hard copy forms and manual mapping methods.

The presentation provides an example of a digital field acquisition system, which implements ArcPad 6.0 software, PDA's, GPS and wireless connectivity options. It will outline methods for developing effective field GIS applications, and subsequent in-office integration strategies. The paper provides insight into the steps taken to complete the presented field acquisition system and centralized GIS integration solution.

Introduction

Having the ability to digitally collect accurate, timely field data, and then promptly posting these field edits back to a centralized database, holds the potential to effectively streamline a corporation's geographic data acquisition network, and thus their resource management goals.

In many sectors, emphasis has always been placed on increasing production and quality, while reducing operational cost, notably in the realm of GIS, where decision makers are dependent on GIS results. Finding a means to increase the accuracy and speed at which data is acquired while reducing the paper trail caused by field acquisition, is key to any coherent plan for data collection, analysis and high quality output generation. Time and time again, in government, industry and business, field inventory is collected on paper, brought back to the office for database entry - the backlog of paper can be extensive. This causes time efficiency and data accuracy concerns, as months may go by before manual data entry can commence. Meanwhile, data analysis gets put on hold. In addition, hard copy field data forms and maps can contribute to a loss in data quality, as they may be out of date, left incomplete, ineligible or entered incorrectly when transferred from paper to digital format. Implementing mobile computers, compatible software, such as Esri's Field GIS product ArcPad 6.0, and developing field data/Geodatabase synchronization strategies may provide a means of reducing these concerns. The technology diminishes the use of standard hard copy forms and manual mapping methods, and enables acquisition of qualitative, cost-effective, timely data, and offering the ability to promptly post these edits back to an in-office database.

Thus, the underlying context of this paper is to provide insight into a method, in the form of an application, which integrates GIS tools to effectively improve the quality of acquired field data, and the speed at which it can be incorporated into a centralized GIS.

The Application: EcoLoGGer

The purpose of the following application, called EcoLoGGer, is to provide insight into the ease in which collecting field data can be integrated into a GIS network. Current advancements in computing technology have provided easy access to software customization capabilities, such as ArcPad's current Application Builder. Thus, applications can be developed efficiently, improving the overall performance of field acquisition techniques and accuracy of the resulting spatial data.

EcoLoGGer has been specifically designed to facilitate the natural resource sector, more specifically those interested in a digital wildlife identification guide and observation recording system. However, the application may serve useful to other sectors, as it provides insight into how integrating different GIS tools can be used to collect and update an in-office database.

The complete application was developed with the following software:

• Esri's ArcGIS product ArcMap 8.1, ArcObjects and Visual Basic for Applications (VBA)
• Esri's Field GIS product ArcPad 6.0, and ArcPad Application Builder 6.0
• Microsoft Access

The application uses Microsoft Access to house most of the data, including wildlife identification data, such as names, observed locations, photos and audio. It should be noted however, that any Windows compatible workspace environment is suitable. Microsoft Access was selected as the main data warehouse, as it enabled easy storage, organization and management of all required data formats (spatial and non-spatial) in one project.

Although this application is specialized in terms of the feature classes, datasets and attribute fields that are used, there are many fundamentals that can be applied to any field data integration application. The purpose of this paper is to highlight some of these fundamentals, in hopes that it may provide insight into one of the many ways to synchronize an in-office GIS database with a field acquisition system.

How EcoLoGGer Works

A database contains a point feature class of observed wildlife sightings for a large district. A user can extract a subset of the feature class for reference and editing purposes when they are in the field. For example, the user may want to edit an existing feature, add a new sight location to the map or simply make reference to the observations made at a sight. For this application, the information collected about a wildlife observation at a particular sight is not directly attached to the point feature, as there may be more than one wildlife observation recorded at a site. Therefore, a non-spatial dataset contains the recorded information for all observations collected at a particular point. A location identifier field (named Loc_ID) is the link between the spatial point data and the attribute data. This relationship is illustrated in Figure 1.

Figure 1. Illustrates the relationship between the Observation spatial and non-spatial data.

There are three main components to the application:

1. Preparation of spatial and non-spatial data for field reference and editing purposes
2. Handling the acquisition and management of spatial and non-spatial data while in the field
3. Subsequent posting of edits from field acquisition to the original database

Preparing Data for Field Use

In ArcMap, the ArcPad Tools for ArcGIS extension, in conjunction with a custom developed macro can be used to quickly and easily export required field data from a parent GeoDatabase to a mobile device. In addition, the macro has been developed to enable the import of all field edits back into the parent GeoDatabase when returning from the field.

The first task is to use the ArcPad Tools for ArcGIS extension to create an ArcPad map project (*.apm) of the background map data and desired Observation point features.

An ArcObjects custom macro developed in ArcMap can then be activated to export the non-spatial data into a format ArcPad can read. Once the macro is activated, the form, shown in Figure 2, will prompt for the path and name of the GeoDatabase and the exported ArcPad map project. In addition, this form is used to specify whether data will be exported for field use, or if field data will be imported for GeoDatabase update.

Once the form is completed, a series of steps are taken to prepare the spatial and non-spatial datasets.

The first step is to develop a means of flagging whether a point feature had been added or modified. This will facilitate the automation of posting these field edits back to the parent feature class, when returning from the field. The high level steps involved are:

• Add a new field to the exported Observations point shapefile. The new field will be used to flag those point features that have been added or changed.
  
• Create a custom designed ArcPad Edit/Identify form for the Observations shapefile to hide the new field.
  
• Create a VBScript to be activated when a point feature has been added or changed in ArcPad. The script will assign the value 'New' or 'Edited' to the flagged field, depending on whether the selected feature is a new feature being added to the map or an existing feature being modified.

The second step involves exporting the necessary records from the original non-spatial datasets. The user may export all of the records contained in a dataset or a subset of records. Figure 3 illustrates the form used to prompt for the records required for export. If a subset of records is selected, a Query Builder form, similar to that used by ArcMap will be presented. The Query Builder form enables a SQL statement to be created to select the desired records to be used for field identification.

Once the user has completed the SQL statement, the records within the SQL query filter can be exported.

The data are now ready for transport to the PDA and use within ArcPad.

Acquisition of Spatial and Non-Spatial Field Data

ArcPad 6.0 and the development environment, ArcPad Application Builder 6.0, can be used to create applications, such as digital forms for acquiring field inventory. The advantage of this functionality is, it enables customization of spatial and attribute data. Much of today's field data is inventoried via paper forms, maps, and aerial photography. This increases the potential for poor quality service to clients, low productivity and redundancy. PDA's along with compatible software can substantially increase the acquisition of reliable data, as a digital application can provide validation for fields and dropdown menus can prevent attribute discrepancy and entry errors. Digital applications also offer direct download of data into a database system. This application, EcoLoGGer, takes advantage of ArcPad's customization capabilities to build an efficient wildlife observation system.

Using the ArcPad Application Builder, the EcoLoGGer application offers forms to enable wildlife sightings, event and weather information to be recorded for a currently defined location, and in addition, enables identification of wildlife species through the use of a digital reference guide.

Tab One: Species Guide

• Location ID: This is the value assigned to the observation point location found on the map.
• Observation ID: Every database entry is assigned a unique identifier. This is a read only field and cannot be modified.
• Date/Time: The computers date and time is recorded when the form is loaded, however this may be modified.
• Observer: Contains the name of the user recording the observation.
• New User: If the desired name is not included in the Observer list, one can be added by clicking on this button and typing in a new name.
• Species: This dropdown box is populated with the wildlife records selected during export.
• Code: A read-only field containing a code value for the selected wildlife species.
• Song: If an audio file exists for the selected species, it can be heard by clicking on the Song button.
• Photo/Clear: Clicking the Photo/Clear button will display a picture of the species that is selected (see Figure 5). To remove the photo from the display, click the button again.

Tab Two: Get File

The 'Get File' tab (see Figure 6) enables the name of a photo or audio file acquired during the observation to be entered, and stored as part of the observation entry. This can be done by clicking on one of the check boxes and navigating to desired file. If a photo is selected, it will be displayed on the form, if an audio file is selected, it will be played.

Tabs Three and Four: More Data Entry Fields

Tabs three and four provide more entry fields for collecting information on observed wildlife, and in addition a Save button is found on the last page. Clicking 'Save' updates the dataset with the specified field entries.

Once field acquisition is completed for the day, the edits may be quickly posted to their parent datasets.

Posting Edits to the Parent Database

Updating the original GeoDatabase with changes and additions made during field acquisition becomes fairly simple for the user and the developer, as the following four steps illustrate. This is due to the attribute field that was added during the export of datasets, to flag those features and records that were modified. The main task is to update the parent Observation feature class and the related Observation datasets found in the GeoDatabase. The four high level steps to perform the updates are as follows:

1. Select the features that have a value of 'Edited' or 'New' stored in the flagged field.
2. Those features, whose attribute values were edited, can be updated by referencing the existing point in the parent GeoDatabase using the Loc_ID field (as this is a unique value field), and updating the feature's attribute values with that of the edits made during field acquisition.
3. If the point feature is a new point, the macro posts the point to the original GeoDatabase if the point is not within a specified radius of an existing point.
4. Updating the related Observation tables follows a similar procedure. The records that are flagged as 'Edited' are updated in the original dataset, and those records flagged as 'New' are simply added to the end of the dataset.

These are the high level steps used to update the in-office GIS database. However, other methods are available and may be more appropriate depending on the intended application. For example, if wireless connectivity is available for the study area, one could explore direct map and database updates to a central server while in the field. A VBScript could be activated when a feature is added or changed in ArcPad. The script could open a text file that appends the features spatial coordinates and the attribute values. The file could then be posted to an Internet or Intranet site. Another script on the server side could then extract the text file from the site, and activate an ArcObjects program to update the in-office database with the changes found in the text file. This would eliminate having to return to the office at the end of the day to post these field changes. Time, costs and storage space requirements may then be reduced.

Another concern for many organizations is multi-user editing capabilities. In the application discussed above, a copy of the parent dataset is loaded into ArcPad for editing purposes. At the same time, another user may want to extract a copy of the same dataset for editing. Thus, when posting these edits back to the parent dataset, conflicts may arise if the same feature has been modified. There are ways around this issue however, by simply adjusting the macro being used here, or implementing ArcSDE and the versioning capabilities that it offers.

The macro could be modified to accommodate multi user editing by keeping track of the user names, time, date and selected records. For example, the names, date and time of all users that activate the macro to export a subset of records, could be appended to a file. When a user returns, the file could be updated to include the features that were modified while in the field. Thus, when a user posts their edits to the parent database, the file could be examined to determine who else has extracted data and returned to post their edits during the same time frame. If a feature has been modified during that time, a form could be loaded to inform the user of the conflict, and prompt the user to decide whether the modifications should be overwritten.

Conclusion

The intent of describing the application, EcoLoGGer, is to highlight some of the steps involved in exporting and importing field data, and to illustrate the ease and benefit of building a system that enables a smooth and easy process for integrating accurate and timely field data into a centralized network. This is simply one of the many methods to efficiently inventory field data. Determining the most appropriate implementation design for efficient collection and management of field data may be the most complicated part of the process. However, with automated tools becoming more readily available, the job is becoming much easier.

For complete documentation of the application and associated code, please visit the Applied Geomatics Research website at: http://142.227.25.130:8787/agrg/index.php.

Authors: