From Pipe Dream to Reality, New York City’s GIS Experience

From Pipe Dream to Reality, New York City’s GIS Experience

Vincent Soriano

Clifford Konnerth

The City of New York supplies water to 7.5 million people through a network of over 6,000 miles of pipe. Having recently completed conversion of archives to their legacy system, initial goals of improved update and access have been met. Their legacy system (GDS) has however, discontinued further development. The 1997 purchase of a privately serviced area necessitated a platform change. Careful research led to the selection of Esri’s Arc Geodatabase. Together with Esri and URS, the Department has created a complete Arc data model for their distribution system. This paper will describe the system and development of this model.

New York City’s Water Main Mapping Project is complete. Six thousand miles of water mains have been entered into a digital database. The database has become a fundamental building block for a geographic information system (GIS) that includes 180,000 valves, 105,000 fire hydrants plus appurtenances such as reducers, roundabouts and pressure-reducing valves. Update procedures and access and distribution of water main information have been facilitated for a 322 square mile area of the city.

To begin meeting infrastructure management challenges systematically, the City of New York had dedicated precious resources to lay the framework for a utility-based GIS. In the mid-1980s, the city’s Bureau of Water and Sewer Operations (BWSO) in the Department of Environmental Protection (DEP) and URS Corporation, Paramus, NJ, developed computer-assisted mapping techniques and a database collection methodology, as well as comprehensive, computer-based sewer system maps for a heavily populated area in the borough of Queens. Based on this experience, in 1990, DEP contracted URS to digitally reconfigure over 6,000 water main maps and compile a relational database.

The existing water distribution system was at that time recorded on three series of maps: trunk main maps (TMs), distribution maps (DMs) and detailed distribution maps (DDMs). These were the primary tools that the city’s Bureau of Water Supply and Wastewater Collection used for planning, system maintenance, and routine and emergency field operations. Of the 6,000 DDMs, 150 DMs and six TMs, a good number had a backlog of updates some extending back as much as four decades. About 4.5 million feet of low- and high-pressure water mains had not even been compiled to DDMs but existed in the form of notes and individual field cards. There was no established digital database.

Bureau personnel manually updated maps from field cards; there were approximately 31,000 known cards on backlog when the project began. As discrepancies were uncovered and resolved this number grew to 50,000 over the course of the project. Fire hydrants, valves and appurtenances are shown on both DMs and DDMs necessitating new updates to both series with selected mains and appurtenances also shown on TMs. Maps, followed some very general engineering specifications in each borough but varied greatly in scale, level of detail, date of compilation and sheet size. Despite the enormous inventory of water main maps on hand, citywide coverage on DDMs was incomplete – 700 new maps were required to record the entire city-owned water main distribution system. A significant challenge at the start of the project was that no detailed, large-scale base map of New York City existed. Each borough has its own "map base," related grid system and unique data.

As the project began, the project team reviewed the scope of work in great detail, then chose two DM areas in Manhattan for the initial pilot study. Each subsequent borough also went through a pilot process due to their differences. Data sets had to be completed and overlaid from three different sources: topographic maps, low-pressure DDMs and high-pressure DDMs. The consultants began by inventorying existing maps and evaluating their levels of detail, scale, currency, and types and quantities. They also sought other source materials, interviewed the users and incorporated their ideas into the final map design and related database.

Procedures were refined and revised throughout this pilot study leading to a user guide and standards manual and full-scale production techniques. For each borough start-up, URS engineers determined both its organization and its quality assurance program, then began selective field verification and spot checks of fire hydrants. Full production also included publication of a project management manual, follow-up training, and several update and database maintenance tasks. Throughout the project, institutional knowledge gathered from the city users was incorporated into production procedures.

The project team developed cartographic specifications and digital drafting standards to meet several guidelines, starting with those from the bureau. They had to be GDS compatible, meet available industry standards and be "best-fit" per map scale. Accuracy and uniform appearance, utility and aesthetic qualities were also specified.

In some cases, the original manuscripts are manually digitized into the database, because either the maps were of such poor reproduction quality or the density of line work did not justify the cost of vector-raster-vector conversion. A controlled network of coordinate pairs (derived from New York City planning maps and U.S. Geological Survey 1:24,000 topographic maps, and adjusted to the project’s drawing origin) was used to reposition, scale or rotate the scanned vector files to align with each other in its own geographic coordinate system.

Though of limited topology due to software constraints at the time, vision plus careful and complete planning was fundamental to this successful GIS implementation. This process was complex, often frustrating, always challenging and usually long. The key to the development (and later, maintenance) of a GIS is realizing that the "S" in GIS is more than hardware and software – database design, organizational structure/interaction, personnel, time, financial resources, management support and training are also very important components of the system.

The City-Wide Water Main Mapping now resides on a minicomputer, primarily because of the city’s large and complex infrastructure database. As work was being completed similar GIS projects were being accommodated on PC-based systems, particularly for thematic, land-use mapping applications. In fact, desktop mapping systems, coupled with relational database management software, have now become the choice of municipalities and other local government agencies. Because of aggressive competition and an awareness of the benefits of spatially relating geographic, statistical and utility data in a common framework, PC-GIS hardware and supporting software are dramatically improving in quality while becoming much more affordable.

Most cartographers agree that regardless of automation, the editing process accounts for almost two-thirds of map-making labor costs. However, the cost savings realized from GIS is found in the speed with which one can access, manipulate and update data, and the unique ability to spatially relate datasets in real time. These same datasets can be output, selectively in tabular or graphic form, in a matter of minutes or hours, as compared with the weeks or months needed to perform these operations in the pre-computer era.

Infrastructure maintenance, rehabilitation and expansion in the 21^st Century will require an abundance of detailed, current information that is accessible and easy to understand. The New York City Water Main Mapping Project has been realized as an essential "next step" in the evolution of GIS in the City and as an essential tool for municipal decision-makers.

In 1997, as the GDS mapping effort neared completion, an area whose water distribution system had been privately owned was purchased by the City. The Jamaica water service area (JWS) covered approximately 27 square miles in the southern half of Queens County. This area was logically proposed to be added to the existing database. As a change order was being negotiated Graphic Data Systems Corporation announced they were discontinuing development of their GDS product. At this point the pending conversion of the Jamaica Water System data seemed an excellent opportunity to migrate a pilot area to a different software platform. Over the next 12 months the NYCDEP’s Central Mapping and Records Group (CMR) evaluated the several leading GIS platforms, taking into account experience, expandability, pricing, customer support as well as ability to perform the required functions of their group. This also provided an eye toward the future of an enterprise-wide GIS for NYCDEP. After an exhaustive software evaluation procedure DEP selected Esri’s ArcInfo product.

DEP’s mapping and database standards had been heavily customized within GDS. To the greatest extent possible Arc standards were to emulate those within GDS. This included both graphic and attribute capture as well as existing functionality. From the beginning the true GIS nature of ArcInfo versus the customized AM/FM nature of the GDS database promised much greater analysis and query abilities. Esri’s technical assistance was added to the project team, and together with URS and DEP an ArcInfo database design for NYC water was begun.

Just prior to this, when four of the five boroughs had been submitted and were successfully in use, NYCDEP contracted for the acquisition of aerial photography for the purpose of building a geodetically correct planimetric base map. A flyover was conducted in the spring of 1996 at sufficient altitude to develop 100’ scale mapping. ASI Landbase division, now part of Sanborn Map Company, was contracted to develop the planimetric basemap at an accuracy of +/- 2 feet. DEP management requested the Jamaica area be compiled first to serve as a pilot for water facilities to be registered to the basemap in ArcInfo foramt. Basemap production coordinated seemlessly with the development of ArcInfo standards.The first planimetric files were available as the project team was ready to begin conversion of the JWS area. All graphic input was implemented in ArcInfo 7.2. Esri technical services assisted in developing production menus to facilitate drafting. As the project progressed, Esri was beta testing their Arc 8 product. The project team registered as a beta tester and was able to contribute to product development during the course of production and development of the city’s water data model. This period, though educational, was difficult. As is often the case users discover bugs that software developers would never even consider. As a result of these difficulties the project team decided to complete production in Arc 7.2 and capture attributes in an accompanying Access database. It was decided to await release of ArcGIS version 8.1, a product with more stable features, and then port the version 7 data along with the Access data to 8.1.

The team received an advanced copy of ArcGIS 8.1 in April of this year. Features necessary for the CMR group are available and working and all data have now been successfully converted. While improvements continue (and are welcomed) the topology of the ArcGIS has already proved itself valuable in the operations of CMR. New York is such a densely developed city that drafting water features in their true location often resulted in crowded intersections with feature symbols overlapping and difficult to read. In ArcGIS details can be easily created that automatically revise if a change is made to the original database. This reduces cartographic license required to view the maps at several different scales. Large-scale plots can be produced with a true representation of feature locations, most suitable for field investigation, service and design purposes. At the same time small-scale large area plots are produced from the same database that allow a more system-wide look with details available as needed. Production of details had been avoided in the GDS product due to the difficulty in creating them and the need to update the separate datasets. As of this writing the project team is completing the final quality assurance procedures. All edits and map submittals scheduled for completion at the end of July 2001.

Harvey Moutal, David R. Bowen and Wendy Dorf. GIS: New York’s Pipe Dream

Civil Engineering magazine, February 1992

Authors:

Mr. Vincent Soriano, Chief Bureau of Water and Sewer Operations, Central Mapping and Records,

New York City Department of Environmental Protection

59-17 Junction Boulevard

Corona, NY 11368-5107

E-mail vsoriano@nysnet.net

Voice 718.595.5755

Fax 718.595.5781

Department Chief responsible for GIS project coordination for the New York City Water Main Mapping GIS project and technical manager for other Department GIS initiatives.

Mr. Clifford Konnerth, Project Manager GIS Services

URS Corporation

Mack-Cali Centre II, One Mack Centre Drive

Paramus, NJ 07652-3909

E-mail cliff_konnerth@urscorp.com

Voice 201.262.7000

Fax 201.262.9199

Senior Project Manager with over 17 years of professional experience in designing, operating and managing conventional and digital mapping systems and activities to include GIS.