The U.S. Environmental Protection Agency (EPA) has previously developed desktop ArcView geographic information system (GIS) tools to georeference, or reach index, surface water data to the National Hydrography Dataset (NHD). Using desktop tools, EPA is assisting states that are currently linking surface water information to the NHD. To expand the use and increase the ease of development and deployment of these georeferencing tools, EPA is developing web-based ArcIMS applications to duplicate some of the same functionality as that of the desktop georeferencing applications. RTI, under contract to EPA, evaluated various ArcIMS options to meet EPA requirements. This paper describes the process of moving from ArcView to an ArcIMS implementation.
Georeferencing is the process of locating an entity in "real world" coordinates. An example would be obtaining latitude and longitude coordinates for a person's home. Reach indexing is the process of georeferencing surface water entities, such as Clean Water Act Section 303(d) waterbodies, to a spatial database, such as the NHD.
The NHD is a spatially enabled database that contains geometries and attributes representing and describing streams, lakes, and other surface water features. A combination of the U.S. Geological Survey (USGS) Digital Line Graph (DLG) hydrography data and EPA's Reach File Version 3 (RF3), the NHD contains "measured" features that allow users to georeference to them using Environmental Systems Research Institute's (Esri's) dynamic segmentation model.
EPA has previously developed two tools that enable users to georeference to the NHD. These tools are the National Hydrography Dataset Reach Indexing Tool (NHD-RIT) and the Web-based Reach Indexing Tool for the Clean Water Act Section 319 Grants Reporting Tracking System (WebRIT-GRTS).
The NHD-RIT was developed as an ArcView system to facilitate semi-automated assignment and capture of NHD reach addresses and associated metadata. It is a standalone application that requires the local installation of ArcView and additional components. To expand the use of the NHD and to better meet the needs of regions, states, and other stakeholders, EPA developed an alternative, lightweight, web-based application to provide easier access to georeferencing tools. This application was designed for the Grants Reporting Tracking System and was named WebRIT-GRTS. WebRIT-GRTS offers some of the functionality of the original NHD-RIT, primarily the ability to capture reach addresses for whole reaches. Much of the advanced functionality found in the ArcView NHD-RIT is not yet available in the WebRIT-GRTS, and the two applications tend to serve different user groups. Groups with few users and large amounts of data to georeference tend to use the NHD-RIT, whereas groups with many users, each with small amounts of data, would prefer to use a less-costly web-based solution.
EPA has recently identified additional needs for an Internet-based tool to enable regions, states, and other stakeholders to submit, verify, and update locational data for programs such as Clean Water Act Section 303(d), 305(b), and others. The new application should be able to work with locational data not georeferenced to the NHD. To meet these requirements, the new application will build from the previous tools. It will be built under the name WebRIT-WATERS, which stands for "Web-based Reach Indexing Tool for the Watershed Assessment, Tracking, and Environmental Results (WATERS)."
WebRIT-WATERS will incorporate many of the features of the original ArcView based NHD-RIT combined with the WebRIT-GRTS functionality and additional features to enter, validate, and update EPA locational data. In order for WebRIT-WATERS to incorporate the NHD-RIT functionality to georeference to the NHD the application must be able to perform dynamic segmentation. ArcIMS 3.1 does not perform dynamic segmentation. WebRIT-WATERS will follow a design strategy utilizing ArcIMS and ArcSDE as the primary display components and Oracle and Oracle Spatial for specialized server-side processing functions.
This paper discusses the current design chosen for EPA's WebRIT-WATERS tool. For additional information on georeferencing and the NHD-RIT browse to http://www.epa.gov/owow/monitoring/georef/. For additional information on WebRIT-GRTS, browse to http://www.epa.gov/waters/webrit/. For additional information on NHD, browse to http://nhd.usgs.gov/.
WebRIT-WATERS is designed to meet the following general requirements:
In order for WebRIT-WATERS to incorporate the NHD-RIT functionality to georeference to the NHD, the application must be able to work with measures. ArcIMS 3.1 does not provide access to the measures of a feature, thus WebRIT-WATERS relies on Oracle and Oracle Spatial for this.
The process of georeferencing requires access to the measures associated with the spatial data in the NHD. Georeferencing utilizes Esri's dynamic segmentation data model, which allows attributes to be related to spatial database components (NHD reaches) without any editing of the spatial objects themselves. The dynamic segmentation model is based on the idea that attributes of a feature can be displayed by simply specifying the start and endpoints of the feature along an arc instead of joining the attributes directly to the arc attribute table. It can be used to georeference point or linear features to a routed/measured linear spatial data set. Each record is stored in an "event table" and displays as an "event" along a reach. The use of events allows users to index multiple entities to the same NHD reach or set of reaches. It also allows the user to index an entity to a portion of a reach without having to modify the spatial extent of the reach itself. The start and endpoints of these dynamic segments created with WebRIT-WATERS are stored in the F_meas and T_meas fields in the event table. The linear events are displayed as linear features along the reach. Point event features have a single measured position (P_meas) instead of a "From" and "To" measurement and are displayed as a single point on a reach. WebRIT-WATERS relies on Oracle Spatial's Linear Referencing System (LRS) functions and procedures to perform the required dynamic segmentation tasks.
Figure 1 is a screen capture of the WebRIT-WATERS application currently in development.
Figure 1. Screen Capture of WebRIT-WATERS Interface.
The architecture of the WebRIT-WATERS application can be broken into three tiers: a client-side interface tier, a server-side middle tier, and a server-side database tier. For the interface tier, WebRIT-WATERS uses a modified version of the default HTML viewer that can be generated from Esri's ArcIMS Designer. In general, the interface consists of a combination of HTML frames and JavaScript. JavaScript code is used behind the tools either to generate Arc Extensible Markup Language (ArcXML) request strings for ArcIMS to process or to populate and submit hidden HTML forms to Microsoft's Active Server Pages (ASP) code to run some server-side middle tier or database tier code. The server-side middle tier is coded with a combination of ASP and Visual Basic (VB) COM+ Dynamic Link Libraries (DLLs). The middle tier code provides reusable functions via COM+ DLLs that wrap the database connection code, provide server-side data validation, and provide quick, reusable access to Oracle PL/SQL procedures in the Oracle database. The database tier contains the Oracle database, Oracle PL/SQL procedures, and Oracle Spatial functionality. Each of the three tiers and their functions are described in this paper. Figure 2 depicts how the three tiers work together.
Figure 2. WebRIT-WATERS Application Design Overview.
The WebRIT-WATERS interface provides all of the basic tools and mapping features that GIS users are used to. These include zoom tools, a pan tool, an identify tool, a measure tool, a legend, a scale bar, and an overview map. Much of the default ArcIMS HTML application's code was used to provide these basic tools. Modifications were made to some of the default selection tools, though. One of the early requirements of the WebRIT-GRTS and WebRIT-WATERS applications was the inclusion of a selection tool similar to what GIS users were used to working with in ArcView. The problem with the unmodified ArcIMS HTML viewer selection code is that new selections replace current selections. The WebRIT-GRTS and WebRIT-WATERS applications required the ability to add to, subtract from, or replace the current selection set.
To meet these requirements, client-side JavaScript arrays were added to the interface code to hold lists of feature database identifiers. Three arrays were added - one for the current selected set, one for the previous selected set, and one for the new selected set. When a new selection is made, the new selection set is populated with the identifiers of the selected features. The current set is then copied into the previous set, and the new selected set is combined with the current through addition, subtraction, both addition and subtraction, or replacement. By setting different flags, the default selection type can be changed. By default, the "both" selection type is used in WebRIT-WATERS. This method adds features that are not currently selected and removes those that are, basically inverting the selection state of features in the selection area. This type of selecting was added to the Select by Rectangle, Select by Line/Polygon, and the Select by Attribute query tools.
The selection tools were modified to select on multiple layers of different types. For example, it is possible to select on a stream (linear) layer and a waterbody (polygonal) layer with a single selection rectangle. The selection sets contain the feature identifiers as well as key words identifying the layers that the features are on. Also, to minimize the need to learn the active layer concept, separate tools were provided where it made sense, or drop down lists were provided in the text frame.
In the interface code, WebRIT-WATERS makes extensive use of the acetate layer. The identifiers from the current set are used to query the different layers, and the geometries are displayed on the acetate layer. In addition, other markers are drawn on the acetate layer, such as feature end points, points along a feature, or other line and polygon geometries. This is all done in the client-side JavaScript by customizing the refresh map ArcXML request.
The middle tier code is initiated from the interface when the user uses certain tools. JavaScript in the application interface code fills in forms contained in hidden frames, and the forms are automatically submitted to ASP code for processing. The ASP pages connect to and call procedures stored in the Oracle database. The ASP pages also perform server-side error capture and recovery. In addition, in the event of an error, ASP is used to write embedded client-side JavaScript to alert the user to the cause of the error. To complete the loop from HTML to ASP and back again, the ASP processing code generates an HTML page with an onload event that takes the ASP results and updates variables in the client JavaScript or calls additional client JavaScript functions, which may or may not submit another form to ASP for further processing. Many of the georeferencing tools require this type of multi-step approach, which is easy to accomplish with a desktop application like ArcView but more complicated with a web-based application like WebRIT-WATERS.
Connection to the WebRIT-WATERS database is accomplished with ASP and custom COM+ DLLs. The custom DLL uses Oracle Objects for OLE (OO4O) for communication with the WebRIT-WATERS database's procedures and other EPA Oracle program databases. OO4O is an Oracle middleware component that allows native access to Oracle databases from client applications using Microsoft's OLE standard. It was chosen over Active Data Objects (ADO) for performance and compatibility reasons. Both OO4O and ADO can also be used to call the Oracle procedures directly in the ASP code, or they can be used to send direct SQL calls. Direct SQL calls are occasionally done when very simple queries are needed. Generally, all database updates or inserts are done through the custom DLLs because they usually require more than a simple line or two of SQL or PL/SQL to complete. All of the WebRIT-WATERS database's procedures are written using PL/SQL and call functions made available through Oracle Spatial.
Oracle Spatial is an add-on to Oracle that provides the user with functions to work with geometries stored in Oracle's Spatial Data Option (SDO) format. Oracle Spatial data can be registered with ArcSDE and displayed in ArcIMS applications. WebRIT-WATERS uses Oracle Spatial functions to create new Oracle Spatial records, which are then automatically displayed back to the user when the map is refreshed. In addition, Oracle Spatial is used to access the measures along NHD geometries for creation of events using Esri's dynamic segmentation model.
WebRIT-WATERS uses an Oracle database for centralized storage of georeferencing submittals. An event is the result of georeferencing data to the NHD; physically it is the Oracle database record with a geometry field and corresponding attributes and metadata. Event data for many EPA programs are stored in the Reach Address Database (RAD). The RAD is an Oracle database that serves as the central EPA repository for the NHD and the associated information related to reach addresses and events. The event tables in the WebRIT-WATERS database are used to temporarily store the spatial and attribute information entered by the WebRIT-WATERS user. The events stored in the WebRIT-WATERS database will periodically be moved to the RAD after they have gone through a verification and approval process.
ArcSDE will be used to display spatial data. The spatial data stored in the WebRIT-WATERS database will be stored in SDO format. ArcIMS will be used to produce the map image and perform additional queries on the data stored in ArcSDE using ArcXML request and response XML strings.
Moving from desktop georeferencing applications such as the NHD-RIT to a web-based environment requires the combination of many technologies. WebRIT-WATERS is EPA's most recent web-based georeferencing tool, which combines ArcIMS, ASP, Oracle Spatial, and ArcSDE. WebRIT-WATERS uses ArcIMS primarily for the mapping interface and Oracle Spatial to perform the dynamic segmentation functions required in a georeferencing application. Microsoft's ASP is the glue that holds these two technologies together. This combination, with the addition of ArcSDE, provides all of the functionality necessary to move a georeferencing application from the desktop to the Web.
Using Oracle Spatial to perform the dynamic segmentation functions, create new spatial data, and perform some spatial analyses has proven to be an efficient and robust solution. Some of the benefits of using Oracle Spatial for the implementation of WebRIT-WATERS are
1) Moving the processing closer to the databaseThe WebRIT-WATERS design takes advantage of the power of ArcIMS and ArcSDE in combination with Oracle and Oracle Spatial to provide georeferencing tools to a large audience with varying needs and successfully enhances the functionality found in the WebRIT-GRTS by modifying the ArcIMS default HTML viewer and adding additional Oracle server-side processing. In addition, designing the application with multiple tiers greatly increases the reuse value and eases the effort to update it in the future. With the basic ArcIMS HTML viewer model and the ArcXML language, WebRIT-WATERS combines many technologies into an efficient, easy-to-use application.
The work described in this paper was funded by the U.S. Environmental Protection Agency under Contract 68-C-01-007 with Research Triangle Institute (RTI). RTI gratefully acknowledges this support.
Disclaimer: Although the research described has been funded wholly or in part by the U.S. Environmental Protection Agency Contract 68-C-01-007 to Research Triangle Institute, it has not been subject to the Agency's review and therefore does not necessarily reflect the views of the Agency, and no official endorsement should be inferred. Mention of trade names or commercial products does not constitute endorsement or recommendation for use.
RTI International. September 2000. Reach Indexing Tool for the National Hydrography Dataset (NHD-RIT) Design Document.
RTI International. April 2001. Web-Based Reach Indexing Tool for the Grants Reporting Tracking System (WebRIT-GRTS) Requirements Document.
RTI International. September 2001. NHD Reach Indexing Tool (NHD-RIT) Data Structure for Events, Metadata, and Transaction Tables.
RTI International. September 2001. Web-Based Reach Indexing Tool for the Grants Reporting Tracking System (WebRIT-GRTS) Design Document.
RTI International. January 2002. NHD Reach Indexing Tool (NHD-RIT) Data Structure for Events, Metadata, and Transaction Tables: Updated to Include Values for WebRIT-BEACHES.
RTI International. February 2002. WATERS Locational Tool (WALT) Requirements.
RIT International. May 2002. WATERS Locational Tool (WALT), Version 1, Preliminary Design Document.
Author Information
James Rineer, PE
GIS Specialist
RTI International
3040 Cornwallis Road
RTP, NC 27709
Anne Marie Miller
GIS Specialist
RTI International
3040 Cornwallis Road
RTP, NC 27709
Thomas Dewald
US EPA Office of Water
1200 Pennsylvania Ave NW (4503T)
Washington, DC 20460
Michael Plastino
US EPA Office of Water, Immediate Office
EPA East
1200 Pennsylvania Avenue NW (4102M)
Washington, DC 20460