As part of Boston Water and Sewer Commission’s (BWSC) Citywide Catch Basin Inspection, Cleaning and Preventative Maintenance Program, Camp Dresser & McKee (CDM) has developed an application and data management program to support field-collection and data integration with the BWSC's existing Geographical Information System (GIS) and Computerized Maintenance Management System (CMMS), CASSWORKS by The RJN Group. The project requires mobilizing crews to capture spatial and attribute information on approximately 33,300 catch basins, and ultimately integrating this data through a front-end GIS maintenance/editor and into a tiled GIS. In order to maximize efficiency and data quality, CDM developed a mobile computing solution that resulted in a total increase in efficiency of approximately 58% -- with an approximately 9% increase in productivity with catch basins inspections, 100% increase in productivity with data collection, and 100% increase in productivity with the catch basin mapping. This paper will discuss the customized ArcView application that has been developed to assist the field crews as well as the integration strategy to transfer data to BWSC’s GIS and Oracle systems.
1. Background
Under the 1987 Clean Water Act (CWA) amendments and subsequently issued regulations, the Environmental Protection Agency (EPA) was directed to issue National Pollutant Discharge Elimination System (NPDES) permits to municipalities with separate stormwater drainage systems serving 100,000 or more people. Because the Boston Water and Sewer Commission's (BWSC) separate stormwater drainage system serves approximately 107,000 people, it is subject to these regulations.
In September 1999, the EPA issued a NPDES permit to BWSC that required implementation of a stormwater management program, including catch basin cleaning and maintenance. In order for BWSC to meet its obligation in using best management practices to ensure that pollutants are not discharged into public waterways, it became necessary to develop a comprehensive catch basin preventative maintenance plan.
To meet the BWSC's needs, CDM developed a customized ArcView application and an integration strategy to assist the Commission in meeting its permit requirements. The first phase of the project is currently underway, and CDM is using the customized application to conduct a survey and inspection of all of BWSC's catch basins. The actual number of catch basins in BWSC's stormwater drainage system is not generally known. Although there are approximately 26,544 catch basins identified in BWSC's Geographical Information System (GIS), estimates place the actual number of catch basins at approximately 33,300. Presently, catch basins are modeled as both nodes and points within the sewer coverage. Of the 26,544 catch basins represented in BWSC's GIS, approximately 7,000 catch basins are modeled as points. These catch basins do not have any connectivity defined between the catch basin and the drainage/sewer system it is associated with.
BWSC is currently using Esri's ArcINFO version 7.2.1 and ArcView version 3.2, running on Windows NT workstations. BWSC's GIS consists of digital orthophotos of the City of Boston, compiled in April 1995, landbase data (planimetric and topographic), and facilities data (water and sewer), which were converted to ArcINFO. BWSC uses coverages as its spatial data format, which are stored in librarian. The librarian database consists of 277 tiles. All of BWSC's spatial data is linked to an external Oracle database (version 7.3.3.6) through a unique feature identification number generated by Oracle.
The link between spatial data and attributes tables stored in Oracle is maintained by software developed by BaySys, Inc. BWSC uses BaySys' Librarian Interface Module (LIM) to maintain coverage data within a tiled environment. LIM has functions that extract data from a tiled library into a single coverage for updating by the Water Network Management (WNM) editor. The WNM editor assists with updating facilities water and sewer data in ArcINFO, and is used to maintain the link between the spatial data and the Oracle attribute database.
1.1 The Need for Mobile Computing
Historically, projects similar in magnitude and scope have relied on traditional documentation methods (e.g., paper data sheets, field books, etc.) to record field inspections. Once these inspections were completed and documented, the paper documents were typically given to a data entry technician, who was responsible for transferring the information from the hardcopy documents to a database. In addition to transferring the catch basins' attributes to a database, the spatial information documented on the paper records also needed to be transferred to an electronic medium. Traditional methods for mapping the catch basins involved technicians creating and placing scaled polygons on a base map. The catch basin's scaled measurements were obtained from the inspection field notes/sheets.
Although these traditional methods were steadfast, they were inefficient and sometimes unreliable. The process of documentation, transcription, and mapping was extremely inefficient, particularly because it centered on replicating data from a paper medium to an electronic medium. The inefficiency associated with transferring the data from paper to a database also resulted in accuracy issues. Data entry errors and illegible field notes/sheets were typically the culprit for erroneous results. As a result, these projects had to dedicate significant resources towards quality assurance and quality control.
CDM used results from a recently completed catch basin inspection in New York City as a benchmark for the efficiency of traditional field-inspection methods. During this paper-driven catch basin inspection project, 137,000 catch basins were inspected with the following efficiency:
In order to improve on the catch basin inspection statistics realized on the New York City project, CDM developed a mobile computing solution that would optimize efficiency and minimize data errors. The result was a customized ArcView application that facilitated the electronic and immediate capture of the catch basin inspection data and mapping of both existing and new catch basins.
2. Data Collection
2.1 Field Inspections using the Catch Basin Application
Five BWSC field crews deployed throughout the city of Boston are conducting the catch basin inspections. Each field crew is equipped with a ruggedized laptop carrying a copy of an ArcView version 3.2 software application developed by CDM. The Catch Basin Application (CBA) consists of two components, a geographic information system (GIS) interface and a database. Field crews use the GIS interface to assist in locating the catch basin, as well as determining if its GIS-plotted location is accurate. The field crews can add new catch basins, move existing catch basins, and delete missing catch basins directly in the software application. The field application also links each catch basin plotted in the GIS to a database, where the field crews use drop-down fields to describe the catch basin and its attributes. The field database instrument integrates the inspected catch basin's spatial and attribute data and BWSC's existing information.
2.1.1 Software Description
BWSC has provided their ArcInfo GIS data for catch basins and manholes for use in the CBA. This data has been converted to an ArcView shapefile for the CBA. The specific data provided is listed in Table 2-1. The application uses the same unique identifiers as BWSC, and uses BWSC's street file as the basis for the street pick list. In addition, the map interface for the application is based upon the GIS data provided by BWSC.
Table 2-1: Data provided by BWSC and used in the CBA.
| Data Provided | Format | Use in Application |
| Catch Basin Data | ArcInfo coverage | Map interface used by inspection crews to select specific Catch Basins for data entry. |
| Manhole Data | ArcInfo coverage | Map interface used by inspection crews to select specific Drop Inlet manholes. |
| Street Names | Access database | Pick list of street names associated with a catch basin. |
| Unique Identifiers | Facility ID Feature ID Access database | Unique identification/tracking of catch basins. |
To use the application, a crewmember selects the specific catch basin or drop inlet manhole being inspected from a map view. Figure 2.1 presents an example of the map interface used by inspection crews.
Figure 2.1: CBA Interface.
Users are then presented with a form on the screen for entering attribute data for that structure. The majority of data is entered by selecting values from a pick list. This preserves data quality by preventing the entry of invalid data. Other data fields, such as depth measurements and comments, require a crewmember to enter data using the keyboard. This data collection form is shown in Figure 2.2.
Figure 2.2: Data entry form in the CBA.
2.1.2 Hardware Description
Each field crew conducting catch basin inspections have a field laptop with the catch basin application loaded onto the machine. Five Panasonic CF-17 Toughbook laptops were purchased for this project for data collection. Each laptop comes installed with Windows 98 and has 128 megabytes of RAM, a four-gigabyte hard drive, a 56K modem and an 8.4" antiglare touch screen. The touch screen capability enables crews to use a finger or other pointing device to interact with the software application on the machine, eliminating the need for a mouse or other peripheral hardware that may get lost or be difficult to use in the field.
Additional hardware devices associated with the laptops include five additional batteries, five adapters, two vehicle port replicators, one external floppy drive, one 10x CD-ROM, two D-Link USB 10/100 Ethernet Adapters, three battery chargers and five carrying cases with harness.
2.2 Data Collection
The mobile solution developed by CDM maximizes the efficiency of the catch basin inspection and data collection tasks because it merges the inspection process with the data collection and mapping processes. There is no data replication between paper documents and database population because the CBA fuses the two steps into a single data collection interface. This differs significantly from the traditional inspection methods, which require the data collection process to start in the field and end with a data entry technician transferring the data to a database. The data collection process defined by the CBA is both started and finished in the field.
2.2.1 Attributes
The following attributes are being collected in the CBA to support the Program, and the inspection tracking and data integration needs:
The accuracy of data collection is maximized through the inherent design of the CBA. The CBA utilizes drop-down menus to populate the majority of the attributes. Drop-down menus restrict the responses captured during the inspection, thus facilitating quality assurance and quality control directly in the field. In addition to the drop-down menus, the direct and electronic capture of the inspection results eliminates the errors produced from interpreting hand-written field notes/inspection sheets.
2.2.2 Mapping Catch Basins
Because the CBA contains BWSC's current ArcInfo GIS data for catch basins, field crews conducting the inspections are not required to re-map the inspected catch basins or manually document the spatial coordinates of the inspected catch basins. As a result, the efficiency associated with mapping the catch basins is maximized because it is relying upon existing information. The CBA is also equipped to accommodate correcting spatial data discrepancies between the GIS data and field observations.
2.2.2.1 Spatial Data for New Catch Basins
While in the field, crews may locate catch basins that do not exist in the CBA. Since the CBA was developed from BWSC's GIS data, such basins would not be included in BWSC's GIS. To locate these new points the field crew must first identify nearby known points that exist in the GIS. The location of the new basin is determined by scaling off the measurement from the known point to the new catch basin, with an accuracy of +/- 2 feet. The crew can then use a tool in the CBA to place the new catch basin.
For new basins, attribute data is entered into the application's data entry form in the same was as existing basins. One such attribute is rotation angle. New basins can be placed in the CBA by first clicking the new basin location, and then dragging a line to define the rotation angle of the catch basin symbol, parallel to the sidewalk.
Since new basins do not have a predefined unique facility and feature identifier, a new unique facility identifier will be assigned by the application. This identifier will be a combination of the Locator map ID (neighborhood grid map) and a unique consecutive number. The identifier assigned by the application is designed to properly track new basins in the application. During the integration into the GIS, BWSC-defined FACILITY and FEATURE identifiers will automatically be assigned for new basins.
2.2.2.2 Spatial Data for Moved Catch Basins
While in the field crews may locate catch basins that are not in the same location as indicated on the field locator maps or in the CBA. A basin depicted on the sidewalk but actually is located in the gutter, or a basin depicted in the gutter but actually is located on the sidewalk, will be relocated and properly aligned with the curb line using the CBA.
3. Data Integration
To ensure that data collected and used for this project can be utilized by BWSC to support business objectives beyond the catch basin inspections, it was necessary to define the technical procedures for how data will be integrated into existing BWSC databases. CDM developed a Data Management & Integration Plan to integrate the data collected in the CBA with BWSC's existing information systems.
The data contained in the CBA will be used to populate the spatial coordinates of all new and moved catch basins, the physical attributes of all inspected catch basins, and the inspection results for each catch basin. The spatial coordinates of the catch basin will be stored in BWSC's GIS, the physical attributes will be stored in BWSC's Oracle business tables, and the inspection results will be contained in BWSC's new CMMS, CASSWORKS.
3.1 Data Flows
Figure 3-1 outlines the general flow of data associated with this project. It begins with BWSC's existing catch basin data and progresses through the use of this data in the Catch Basin Application (CBA), and eventual transfer of updated data to BWSC. As illustrated in the figure, the data will ultimately reside in the GIS, the Oracle attribute tables, and BWSC's new CMMS, CASSWORKS.
Figure 3-1: Proposed data flow.
3.2 Integration Routine
As illustrated in Figure 3-1, the data needs to be routed through the BaySys LIM module before it can update the GIS. Typically, when BWSC needs to change or add a feature to the GIS, a GIS Technician will use the LIM module and the WNM editor to interactively process the feature's modifications. Although these modules work exceedingly well for a single asset, they are not designed for batch processing of several assets.
In order to efficiently integrate the information on the inspected catch basins, CDM partnered with Geographic Information Technologies, Inc. (formerly BaySys Inc.) to develop an integration routine that would automatically incorporate the catch basin records contained in the DBF neighborhood deliverables, thus eliminating manual data processing/editing.
When appointing features to the sewer coverage, the WNM editor automatically assigns the appropriate FACILITY_ID and FEATURE_ID to the feature and uses these identifiers to open a new record in the Oracle attribute table. Thus, the key element of the integration routine was centered around the unique identifiers associated with the data.
3.2.1 Unique Identifiers
To perform the data integration, a procedure for maintaining and tracking the unique identifiers was required. The unique identifiers being used by the CBA are primarily the BWSC-defined FACILITY_ID and the FEATURE_ID. BWSC's FACILITY_ID and FEATURE_ID are both assigned by the WNM editor, based upon the tile number the feature is located in, the type of facility the feature is, and the sequential number of that feature in the current tile. However for new features, there are no FEATURE_ID or FACILILTY_ID defined in BWSC's databases. Although the CBA automatically generates a field-defined Facility_ID, the logic used to generate this identifier is not the same as that used to assign BWSC's FACILITY_ID and FEATURE_ID. The field-defined Facility_ID is assigned based upon the neighborhood grid the feature is located in, and the sequential number of that new feature with respect to other features found in that grid.
The integration approach developed for this project centered on both the BWSC-defined and CBA-assigned unique identifiers and type of location update associated with the inspected catch basin.
3.2.2 Location Updates
The purpose of supplying data from this catch basin inspection project is to enable BWSC to update and supplement its existing catch basin GIS and attribute data. During field inspections, the location of all existing catch basins are assessed and corrections to the locations of existing catch basins, as well as the location of new catch basins are documented. Location information is included in the CBA database as x and y coordinates based upon the NAD83 State plane feet projection system used by BWSC.
In addition to the actual location of the catch basin, the type of location update required must be tracked in order to automate the corrections. The Location Status (LOCSTATUS) field stores this information. There are four categories of location updates assigned in this field:
The type of location update indicates the type of processing required during the integration.
3.2.2.1 Mapped/Found Catch Basins
All catch basins that are currently identified in BWSC's GIS and clearly observed in the field will be assigned a value of "Mapped/Found" in the Location Status field. Given that these catch basins already have coordinates defined in the GIS, there is no requirement to transfer the spatial coordinates of these catch basins through the WNM editor. The integration routine for these catch basins focuses on transferring the attribute data from the CBA to the Oracle attribute tables (both GIS and CASSWORKS) using the existing unique identifiers as the link between the new information and the existing data.
3.2.2.2 Not Found Catch Basins
Catch basins that are not observed in the field but are mapped in the GIS are assigned a value of "Not Found" in the Location Status field. Before assigning a value of "Not Found" in the Location Status field, crews confirm that the catch basin does not exist as opposed to being paved over.
All features assigned "Not Found" are not processed in the integration routing because they will eventually be deleted from BWSC's GIS. Before deleting these features from its GIS, BWSC deploys its own field crews to verify that these features do not exist. After verification, BWSC manually deletes these features from its GIS.
3.2.2.3 New Catch Basins
Any catch basins that are observed in the field, but are not mapped in BWSC's GIS are assigned a value of "New" in the Location Status field. Catch Basins found by the field inventory that do not exist in the GIS are added as LABEL POINTS in the GIS via their coordinates.
For new catch basins, the unique identifier (Facility ID) is automatically assigned by the CBA. In order to preserve the relationship between the inspected features and the attribute information gathered by the CBA, the integration routine needed to create a new field in the Oracle attribute table, FIELD_ID, which would store the identifier assigned by the CBA. In order for the attribute information collected for new features to be accessible to the Oracle attribute table, the attribute table must also allow for a field that stores the CBA-assigned identifier as well as the new BWSC-defined FACILITY_ID and FEATURE_ID. As the data for new catch basins is processed through the integration routine, the routines assign the catch basin with a BWSC-defined FACILITY_ID and FEATURE_ID.
The integration routine extracts the CBA-assigned Facility ID and stores its value in a new field in the Oracle attribute table (FIELD_ID). This establishes a link between the CBA data and the Oracle table. Assigning the CBA-designated ID to the new field FIELD_ID preserves the link to the CBA data via a unique identifier, without interfering with the BWSC-assigned identifiers. The newly generated BWSC FEATURE_ID and FACILITY_ID can then be stored in their respective fields in the Oracle attribute table.
3.2.2.4 Moved Catch Basins
Any catch basins that are observed in the field, but differ in their location from that mapped in BWSC's GIS are assigned a value of "Moved" in the Location Status field and are mapped or relocated and properly aligned with the curb line using the CBA.
Moved catch basins can also pose a problem with respect to unique identification and integration. As previously discussed, the FEATURE_ID and FACILITY_ID associated with each existing BWSC feature is inherently linked to the tile that the feature is located on. If a catch basin is identified in the field, but is moved to a new location that is outside its originally designated tile, then this feature must be assigned a new FEATURE_ID and FACILITY_ID. The same procedure for tracking the original identifier associated with new catch basins is used for moved catch basins. The only difference is that the identifier that is stored in the FIELD_ID is not the CBA assigned, but rather represents the original BWSC-defined FACILITY_ID. This preserves the link between the CBA data and the Oracle table.
As moved catch basins are processed through the integration routine, the routine assigns new identifiers that populate the FACILITY_ID and FEATURE_ID in the Oracle attribute table, reflecting the correct tile location and sequential number for the moved catch basin. In the instances where the catch basin is not moved outside its originally designated tile, then new BWSC identifiers are not generated and the FIELD_ID and FACILITY_ID are equivalent.
As discussed previously, BWSC models some of its catch basins as nodes and others as points. The batch function for moving catch basins that are points in BWSC's network coverage is much simpler than the function associated with moving catch basins that are nodes. Catch basin points do not have connectivity within the sewer coverage, therefore moving them to a new location merely requires the new coordinate. Yet, when moving nodes in a dataset with ArcINFO commands, any attached pipes are also moved. Several topology issues can arise with moving a node:
BWSC has 18,393 total catch basin pipes associated with its catch basins and 10,592 have multiple vertices (ranging from 1 to 9 vertices in addition to the end points). It is presumed that the majority of these extraneous vertices were generated during the creation of the data, and do not represent actual bends in the pipe. Therefore, as part of the batch function, catch basin pipes are "cleaned" (remove extraneous vertices) before moving the catch basin nodes. As a result, the new catch basin pipe extends from its original connection at the storm drain or combined sewer to the new field-defined coordinate. This solution maintains connectivity without corrupting the existing topology.
4. Results and Conclusion
As discussed previously, the foremost intent of creating a mobile computer solution for accommodating the catch basin inspections was to optimize efficiency and minimize data errors. Equipping the field crews with the CBA, has resulted in an average inspection rate of approximately 38 catch basin inspections per day. This rate includes catch basin inspection, data collection, database population, catch basin mapping, and preliminary data verification.
Comparing this rate of 38 inspections per day to the rates observed with the paper-driven New York City Catch Basin inspection project, CDM's mobile solution has resulted in an increase in productivity of approximately 9%. Although this is a valuable gain in productivity and efficiency, the largest gains can be seen with the efficiency improvements associated with the database population and catch basin mapping.
Providing a single data entry point to capture both the inspection data and the spatial information has eliminated the need for additional data entry steps. As a result, the crews have realized an improved efficiency rate of 100% for data collection/database population and 100% for catch basin mapping. After an inspection is completed, the data collection process is complete and the data is ready for analysis. No additional data transcription, mapping, or interpretation of the field notes/sheets is required.
Overall, the mobile solution developed by CDM has yielded an increase in productivity of approximately 58% over that of traditional, paper-driven inspection projects. Ultimately, the mobile solution and integration strategy developed for this project optimized data collection, maximized efficiency, and resolved complex data exchange problems such as maintaining data integrity in a tiled GIS, maintaining connectivity after moving features modeled as nodes, and providing a solution for bypassing the limitations imposed by GIS maintenance/editor to add/modify spatial data automatically. CDM's approach ensures that the Citywide Catch Basin Inspection, Cleaning and Preventative Maintenance Program is performed with maximum efficiency and that all data collected as part of this project will be successfully integrated with BWSC's information systems and will provide value for future projects.
Joseph McEachern
GIS Director
Boston Water and Sewer Commission
Stephen Keen
GIS Specialist
Camp Dresser & McKee
Amymarie R. Corriveau
Information Management Systems Specialist
Camp Dresser & McKee