Newton, a relatively affluent City of approximately 83,000 people and 27,000 parcels, began its GIS program in 1991 with an initial GIS development focus within the Assessing Office. By 1994 this system expanded, both in data and applications, to include the Assessing, Engineering, Public Works, Planning, Inspectional Services, Schools and Management Information Systems departments. This paper will focus on the technical and administrative issues that were faced by the City in broadening the reach of this UNIX-based GIS program. Specific reference will be made to: use of the City's cable TV system as a network conduit for distributing GIS functions; development of user interfaces to support departmental activities; the importance of hard copy generation; GIS integration with Assessing's ORACLE-based CAMA system and DPW's ORACLE-based AM/FM system; and political and funding issues that were addressed during system development to insure continued system operation. The paper will also discuss current successes as well as ongoing challenges and problems. The thrust of the presentation will be on the transition from system development to sustained, operational implementation.
1. INTRODUCTION: The City of Newton, Massachusetts is a relatively affluent suburb located directly west of Boston. Newton is home to approximately 83,000 people and covers approximately 18 square miles. In 1991 the City began a GIS development effort. The catalyst for GIS development was the Assessor's office, however, development took place under the guidance of an inter-departmental GIS Executive Committee. Importantly, the GIS Executive Committee was comprised of senior staff and department heads. The majority of GIS development has been funded through "capital improvement" moneys authorized by the Board of Alderman. The GIS Executive Committee created a 5 year capital improvement budget. The capital improvement budget outlined the funding requirements for all aspects of GIS development: hardware, software, data development, and staffing. Due to continued progress and success with the GIS implementation the GIS Executive Committee has been successful in securing the full funding outlined in the plan for each of the last 4 years. To date, approximately $1,000,000 has been authorized for GIS development. Applied Geographics, Inc. (AGI) became involved with the City of Newton in 1992 when it won a competitive bid to automate the parcel data layer. In spite of the Board of Alderman's strong commitment to funding GIS, they were reluctant to create a new full-time staff position. Consequently, it was decided to out-source the high-level GIS work. Subsequently, AGI won competitive bids to provide system administration consulting services to the city. AGI has provided from 2 - 3 days a week of on-site system administration and project development services for the past two years. Based on the success of GIS implementation to date, the maturity of the GIS data base and user community, and the need for more than 2 days a week of support, the GIS Executive Committee was successful in lobbying the Board of Alderman to create a new full-time position of GIS Administrator. A new slot was not created, however, the Board of Alderman allowed the Management Information Systems (MIS) Department to upgrade and fill a slot that was currently vacant. This position was filled on April 18th, 1995. AGI will remain on contract through September 1995 to assist in the transition to in-house support and to complete system documentation. 2. THE NEWTON GIS NETWORK Newton has been able to build and maintain high quality computer system through the MIS department for a number of years. City Hall is networked with a series of Novelle file server and a combination of 386, 486 and Pentium based PC's. Several other buildings such as the DPW garages, Police Headquarters and the School Department have their own internal LAN's as well. The City is in the midst of implementing a WAN based on fiber telecommunications that will eventually link all city buildings (including all schools and libraries) into a high performance WAN. Currently, the City's cable TV operator provides channels to the city that can be used for wide area communications. These channels are provided free of charge as part of the basic agreement between the city and its cable TV operator. While many communities do not take advantage of this "fine print" opportunity, it is widely available to communities with cable TV franchises. Newton has implemented technology and currently one of the DPW garages is connected to City hall using what is referred to as the "I-NET". Over time, the I-NET will be replaced by the new fiber net. No special networking was introduced to support GIS. The GIS merely became another application available on the existing network infrastructure. Initially, the ArcInfo GIS software was run on a Sun SPARC2 workstation (later upgraded to SPARC10) located in the Assessor's office. Subsequently, the Assessor upgraded the platform on which the Assessor's CAMA data base (described in detail below) ran to a high performance SPARC1000 server. ArcInfo and ORACLE were moved onto this machine as well. Several of the PC were configured with high end graphics capabilities, and Hummingbird's EXceed/W X-windows terminal emulation software was loaded. These PC's access the SPARC1000 over the existing LAN to gain GIS access. The city currently maintains 7 UNIX ArcInfo licenses. Several large format and small format plotters are available on the network. To date the following departments maintain the following connections to the GIS, and GIS equipment: Assessor: SPARC1000, SPARC10, Tektronix X-terminal, backlit table digitizer, 3 X-windows emulation seats, HP650c e-size plotter Engineering:2 X-windows emulation seats, 2 digitizers, HP650c e-size plotter Inspectional Services:1 X-windows emulation seat MIS: 3 X-windows emulation seats, HP650c e-size plotter Public Works: 1 X-windows emulation seat Public Works Garage:1 X windows emulation across the I-NET. Planning: 2 X-windows emulation seats, HP XL300 b-size color plotter The school department and police department will be adding GIS seats this fiscal year. Currently, the prototypical GIS workstation configuration is a Pentium PC with a 19" or larger monitor capable of running at 1280 x 1024 resolution that will run X-windows emulation software. The SPARC1000 will be augmented by additional UNIX processing capabilities within the MIS department this year. 3. THE GIS DATA BASE: 3.1 The Core Data Base The Newton GIS data base has grown steadily over the last 3 years. The principal base map for the system is derived from the Assessor's maps which originally dated from 1980, and were based on 1979 photography. These maps were produced using high quality photogrammetric techniques and were delivered at the metric scale of 1":1,000" (1":83.33'). These maps contained parcel lines, rights-of-way, building footprints, and several types of ancillary text (street names, lot dimensions, etc.). Fortunately, these data were developed on an early CAD system and it was possible for the City to obtain DXF files of the original linework. These CAD files were turned over to AGI prior to parcel data layer development. AGI took the 168 individual files and completed the following steps to create a seamless ArcInfo digital data layer, with appropriate pointers to the existing Assessor's data base: 1. Conversion from DXF to appropriate ArcInfo coverages 2. Edgematching, polygon closure 3. Creation of topology 4. Assignment of Assessor's section block and lot number 5. Update of parcel lines to reflect changes from 1980 - 1991 6. Check plotting Building footprints and rights-of-way were processed into ArcInfo coverages, however, they were not linked to any attribute data bases. 3.2 Additional Data Sources Other data layers that have been developed and/or integrated into the GIS include street centerlines, and several layers of annotation. In addition, new data layers have been developed covering a range of administrative boundaries within Newton, including: 1. School districts 2. Wards and precincts 3. Community development block grant neighborhoods 4. Police patrol areas 5. Snow plow routes These layers are typically developed using existing parcel lines and/or street centerlines as templates. Development of these types of layers as projects will be discussed in greater detail below. Another important source of data for the growing GIS was incorporation of data obtained by the local electrical utility: Boston Edison, Company (BECO). BECO funded a comprehensive flyover and base mapping (1":100' scale) project of their entire service area. Part of the deliverables of that project were planimetric data layers in ArcInfo format. Through ongoing communication with BECO, Newton secured copies of the BECO data for Newton. Three data layers were of particular interest to Newton: 1) the building footprints, 2) the edge of pavement, and 3) hydrography. Newton's existing building footprints layer was based on 1979 aerial photography. The BECO footprints were based on 1991 aerials. Thus, it was possible to receive a 12 year footprint update at very low cost. Similarly, the existing Newton parcel data base contains parcel lines. Typically, parcel lines conform to the "back of sidewalk". By obtaining the photogrammetrically interpreted "edge of pavement" it is possible to build the width of the sidewalk and enhance the base map. In addition, other structures, such as traffic islands, show up in the edge of pavement layer that are absent from the parcels. The BECO data also allowed Newton to obtain a high quality hydrography layer. This layer included the shore of the Charles River and ponds ranging from hazards on golf courses to the several hundred acre Crystal Lake. In addition, Newton received a copy of BECO's utility pole data layer. Since, BECO provides street lighting to Newton this data layer will have multiple uses within the DPW and Engineering departments. These data were easily integrated into Newton's GIS. In spite of the 12 year difference in the technology of data collection, and a datum shift, the two disparate data source overlaid almost perfectly. In fact, the back of side walk (parcel) and edge of pavement create a convincing graphic sidewalk. While there are slight differences in the interpretation of building footprints, the overlay is excellent and offsets range from 0 - 3 feet. New structures, and additions to existing structures (the new wing on the hospital), are easily discernible. Researching developments in the local GIS community has paid dividends for Newton. The ongoing challenge will be to work out an ongoing relationship with BECO so that they can obtain digital data updates from Newton and so that Newton has ready access to BECO's changing infrastructure data. 3.3 New Graphic Data Under Development Once a system becomes operational there is an almost insatiable appetite for new and better data. Built into Newton's five year capital improvement budget was funding for additional data development. The following will briefly outline major data development projects currently under way. During the 1979 flyover a 2 foot contour map was developed. These data were not available in digital format and digitization was planned. An early needs assessment completed by another consultant estimated the cost of digitization at between $100,000 and $150,000. Consequently, these funding estimates were built into the capital improvement budget. By 1993 it was observed that this level of funding could perhaps fund a new flyover that would result in the digital recalculation of 2 foot contours but would also allow for the development of a digital orthophoto back drop for the existing planimetric data. The cost of this project was estimted, and it was decided that a request for proposals to develop the contour data via a new flyover would be pursued. At the same time, the DPW was pursuing plans to develop data layers for the water, sewer and drain system. It was envisioned that the new flyover could be used to capture the manhole, catch basin, water valve and hydrant structures photogrammetrically. The DPW agreed to pool resources with the contour development project and the result was a comprehensive flyover project that included the following: 1. New ground control to support the flyover 2. Color aerial photography 3. Signalization (painting) of infrastructure features 4. Development of a 2 ft. digital contour data layer 5. Development of 1":100' color digital orthophotographs 6. Planimetric development of manholes, catch basins, water valves and hydrants. This project was bid out in the Winter of 1993-94 and James Sewall Company of Old Towne Maine was selected as vendor. Photography was completed in the Spring of 1994 and evaluation of all products for a pilot area within the city is underway. In tandem with the photogrammetric project is a contract to automate a schematic data layer for the linear water, sewer and drain pipe networks. Funding was not available to support digital data development from the very accurate "as-built" drawings. However, it was decided that a compromise level of accuracy could be attained by automating the existing 1":400' schematic plans of these pipe networks and then fitting the schematic data to match the photogrammetrically collected manhole data. Key attributes of the pipe networks would include the as-built drawing reference number as well as key information compiled from the as-builts, such as invert elevation of the manholes. This contract was won by Applied Geographics and is scheduled for initiation this Spring. The last significant graphic data development project now underway involves scanning the house connection records for the water and sewer system. The house connection records are sketches of how each house is connected to the system and show both where on the house the connection is and measured distances to valves. Due to the large number of house connections, 27,000+ for both water and sewer, it was anticipated to be too expensive to accurately capture this information in vector format. Instead, Newton has a contract to scan the existing records (stored in field notebooks) and then link the scanned images to a vector rendition of the parcel. In this system, it will be possible to "point at a parcel" with the mouse and have the appropriate scanned image appear. This allows Newton to maintain the integrity of the original data collected in the field without having to undergo the expensive process of having it reconciled with a photogrammetrically collected base map. The original field notebooks date from the 1890's to the present. An important added benefit of the scanning project is the creation of a permanent, reproducible copy of a unique and valuable city archive. In addition, the currently cumbersome card file access to the connection cards will be replaced by both cartographic access via ArcInfo and data base access (keyed on parcel address) via a PC based image viewer. 3.4 Attribute Data Due to limitations with the INFO software (to put it mildly), Newton attempts to minimize the amount of data stored in INFO. The main attribute data base is attached to the parcel data layer and is called the Assessor's Computer Assisted Mass Appraisal (CAMA) system. This system is responsible for storing data on parcels, structures on parcels and owners of parcels and creates the property tax bills for the city. Literally hundreds of columns of attribute data are available on each parcel through CAMA. CAMA system are created by a number of vendors. Traditionally CAMA systems have been COBOL applications running on some kind of mainframe or mini computer. Newton uses a CAMA system written by SIGMA. In part due to interest by communities implementing GIS, SIGMA made a decision to port its CAMA application from COBOL to the ORACLE relational data base management system. Newton agreed to purchase the ORACLE software prior to the official availability of SIGMA's CAMA application. The SIGMA ORACLE software is still not yet available. However, the ORACLE table structures are available. In the interim, Newton has written a number of scripts that take the COBOL CAMA data and load it into ORACLE. These scripts are run on a weekly basis and provide the Newton GIS community with relatively up-to-date, comprehensive parcel attributes. Examples of attribute fields found in CAMA include: 1. Owner information 2. Property value information 3. Structure value information 4. Information on date of last sale 5. Parcel details (frontage, traffic zone, address, dozens more) 6. Structure details (number bedrooms, bathrooms, roof type, kitchen quality, air conditioning, fireplaces, dozens more) Currently, the ArcInfo parcel coverages are linked via the relational data base interface (RDBI) to the ORACLE tables storing the CAMA data. Thus, the only attribute stored in INFO is the parcel ID number. When SIGMA's fully ORACLE CAMA system becomes operational, the GIS coverages will be linked into the transaction data base in a read-only fashion. Currently, the casual GIS user in Newton is offered a user interface which facilitates both access and query of the CAMA data base. Similar to the Assessor's office the DPW has decided to purchase an ORACLE based third party data base application for facilities management. This application will facilitate the storage of information on the infrastructure (roads, sewer, drain, water) that DPW maintains and manages as well as the creation and management of products such as work orders. The Newton DPW selected the Hansen software running on ORACLE on Novelle file servers for these functions. Just as with the parcels, the plan is to store a unique ID for all infrastructure elements in ArcInfo and then use that number to link directly into Hansen for access to a richer universe of attribute data. The Hansen data base is currently under development by the DPW and thus a fully operational system has not yet been achieved. As with the parcels it is anticipated that a user interface will guide a user through the relates that are necessary for easy access and query of the Hansen data base in association with ArcInfo. 4. TYPICAL GIS PROJECTS: To date, the Newton GIS has been used for dozens of GIS projects ranging from one-time only specialty maps to applications that are used on a weekly basis. Rather than catalog every application, this paper will attempt to categorize a couple of the typical types of GIS project activities undertaken by Newton. 4.1 User Interface Development Due to the conflict between a desire for wide, easy usage to GIS and the reality of a steep and long learning curve to gain GIS proficiency, many user interfaces have been developed. These user interfaces have been developed by Applied Geographics in response to specific requirements put forward by the GIS user community. In addition, the user interfaces have been supplemented with many features that show the possibilities of GIS to users who may be unfamiliar with the full range of capabilities. For example, the orthophoto and scanned connection card information projects described above were prototyped and introduced to the GIS Executive Committee via the user interface prior to the decision to move ahead with a citywide project. Examples of the features found on the Newton GIS user interface include: Standard Arcplot Navigation: Ability for users to identify their study area (mapextent) in a variety of methods including, address, graphically with the mouse, parcel ID, zoom-in, pan, et al. Also, the ability for the users to select the layers, and symbolization of layers, they want from menus. Access to Attribute Data: A number of standard features for accessing attribute data have been built-in including, establishing relates to ORACLE, highlight parcels via a query builder to ORACLE attributes, identify attributes in various data layer by pointing at features with the mouse. Hard Copy: Many users want to create maps to be used at meetings and as working documents for planning sessions. The user interface gives Newton users a variety of hard copy formats for both large format and 8.5 x 11 printouts. Maps can be created for the currently viewed study area on the screen or for a previously assigned sheet layout. On a monthly basis maps highlighting a parcel and showing adjacent land use and/or zoning are created by the Planning Department to support zoning review decisions. Analysis: Over time various analytic functions have been grafted into the user interface. A good example is an application that allows the user to identify abutters of a parcel. The user can select the radius for their selection and the system will automatically identify abutters within this radius and generate an address listing. Optionally, the user can create a file which can be converted directly to mailing labels for notifications (e.g. changing a street to one-way, changing zoning). 4.2 Citywide Hard Copy Mapping While experienced GIS user know that the major benefits of GIS come via on-line access and maintenance of GIS data, many people within municipal government will initially think of GIS as a way of making hard copy maps. In order to gain the interest of these people, and foster a deeper understanding of GIS, it is important to insure that the GIS can effectively create high-quality maps. One of the easiest means of gaining interest is to create a map that no one has ever seen before. To this end the GIS was used to create a template for citywide mapping at 1":1000'. At this scale every parcel and every building footprint in the city is legible and every street name can also be displayed. Seeing this amount of data crammed onto a single sheet while maintaining cartographic integrity and good style helped tangiblize the development of the GIS with a modest amount of effort. Once the template was created it was a simple matter to adapt it to create a large variety of captivating thematic maps. An early test used the Assessor's CAMA data base to generate shading for all parcels reflect the "home style" attribute column. In this manner, a map showing the location of all "colonials" vs. "ranches" vs. "Victorians" was created. This was a map that would be impossible to create without GIS and had wide interest to people who are very familiar with a city. It both confirmed their knowledge (there is the cluster of post-war ranches in the southeast part of town) and highlighted new information (what is that cluster of Tudor homes in the center?). The same kind of mapping has since been completed for several other Assessor's attributes ranging from obscure (parcels owned by people who live out of town) to vital (zoning or property value). The captivating nature of new, information-rich maps, on a familiar hard copy medium, has been key for generating interest in GIS, which over time has been translated into support. 4.3 "Code Centerline" Projects For a growing GIS, it is just as important to develop new attribute data as it is to develop new GIS data layers. In fact, many of the most important GIS projects undertaken in Newton have been relatively simple and have followed a repetitive format. One of these "classes" of projects has been to develop attribute data that can be associated with the street centerline data layer. To a new GIS user requiring to develop a "snow plow route" data layer, the temptation is to "digitize in" the routes. In a maturing GIS with an existing centerline file, the proper approach is to add a new column to the attribute data base that can store the "snow plow route number" of each segment of the street network. This new column, or in some cases multiple columns, can be physically attached to the arc attribute table (.AAT), however, it is better to develop this attribute information in an external data base table that can be related to the .AAT. In this manner, multiple departments can share a single street centerline coverage, yet they can still have access to department-specific attribute data. Having only a single centerline coverage is critical because as changes are made, they only need to be made once (as opposed to once in the centerline coverage, once in the snow plow route coverage, etc.). In Newton, a series of application template AMLs have been written that allow the system manager to easily set up a project that fits into this class. A new table must be defined (either in INFO or ORACLE) and then the macros can be tuned to allow the user to input data into the table based on a form menu. The form menus give the user access to all appropriate ARCPLOT or ARCEDIT graphic selection tools. By developing these AML templates it is very quick and easy to set up new projects. Also, the menus simplify the use of ArcInfo to the point where non-GIS professionals in the city departments can be effectively trained to input the data. Beyond snow plow routes, other examples of code centerline projects include: pavement surface type, water line replacement history, one-way streets, public transportation routes, bicycle routes and walking routes for historic properties. When complete, each of these new attributes can in many ways function as a "new data layer". 4.4 "Code Parcels" Projects The "code parcels" project class is very similar to the "code centerlines". In this case, the user is identifying certain parcels for which additional attributes can be stored. Again, a series of macros assist the user in creating new data base tables that can be related to the parcel coverages for display of additional attributes. In some cases, the user actually identifies the parcels graphically and then uses a form menu to load in the new attributes. More frequently, however, the user already has a data base (typically stored on PC) which has the additional attributes referenced by street address. In these cases, the parcel ID number must be added to allow the data base relate to the parcel coverages. To facilitate this process a series of macros have been developed which allow the user to "addressmatch" their external data base table to the parcel coverages. The parcel coverages have address topology based on the address stored in the Assessor's CAMA data base. Once the addressmatching has taken place, the macros use ArcInfo overlay techniques to automatically associate the parcel ID to a successfully matched address point. The related table is then generated from the result of this overlay process. In general, the addressmatching success rate is in excess of 85%. The parcel ID's of non-matching records are then manually identified. The result of this kind of project is a broader availability of data with new thematic content. Many of these types of projects have resulted in data sets which act a "new data layers" even though they are really only sub-sets of the parcel data layer. Examples of the results of "code parcels projects" include: protected open space, children attending schools, recipients of Newton Housing and Rehabilitation Fund grants and loans, water meter reader routes, and national and state registered historic properties, 5. ISSUES & LESSONS LEARNED During the past 2 years of GIS implementation, the city of Newton has had to address a wide variety of issues and has learned a great number of broadly applicable lessons. The following describes some of these as a means of providing fuel for realistic planning by other communities. 5.1 Staffing & Training As described earlier Newton decided to out-source the GIS administration functions to a consulting firm. One of the roles of this firm was to provide training and technical assistance to city personnel as they became involved in GIS. AGI developed and taught a full 10 day, hands-on ArcInfo course for 6 city staff. This course was taught on-site and all lessons were based on the actual city of Newton GIS coverages so that the trainees could gain experience in both the network and data base that they would be working on. The trainees were selected by the departments and were told in advance of the time commitments that were thought to be necessary to develop GIS skills to proficiency. Nevertheless, less than half of the trainees found time to pursue GIS to any significant degree after the training was concluded. Existing responsibilities within the departments precluded this. In spite of growing needs and expectations within the departments, staff time was not freed to allow for in-house development of these products. Instead, AGI, acting a system administrator, had to meet these departmental needs. The user interface was expanded greatly so that GIS trained personnel could access the GIS data base, but the majority of departmental needs involved ad hoc access to the data that required some level of proficiency with the software. A second training class was also given, with similar results. It is the author's belief that there is a strong need for dedicated GIS staff within municipal government. Existing staff are too busy to dedicate the time that is necessary to develop robust GIS skills. It is realistic to expect departmental access to a GIS data base through a user interface, but the limitations of user interface access for real world planning and problem solving must be understood. If a department is to develop a strong internal GIS capability, they must allocated dedicated staff time in excess of .4 full time equivalents in order to achieve proficiency and success. Alternatively, a multi-departmental GIS service bureau acting on behalf of several groups could be developed. Departments could channel requests through this group for specific project needs. The appropriate sizing of this group must be carefully planned and budgeted for if the range of activities from ad hoc map making to system planning to data maintenance to user interface development is to be handled effectively. 5.2 Ongoing Update of Data Simply put: it is more fun to make maps then it is to fix defective data. However, defective data can undermine the credibility of a map. Keeping a GIS data base up to date is critical. In Newton the hard copy maps have been historically maintained by the Engineering Departments. While the Engineering Department has been supportive of the GIS effort, they have not had the luxury of dedicating a staff person to GIS. At the same time, the Engineering Department has been automating its design and construction work to use CAD technology. In this environment of new technology and static staffing it has been challenging for Engineering to efficiently make the transition from hard copy data maintenance to ArcInfo coverage maintenance. This challenge is currently being addressed and among other activities a user interface to facilitate parcel coverage updating has been developed for Engineering. In older bureaucracies it is unrealistic to expect overnight changes. In Newton these changes have unfolded over years. Years in which the GIS data base has accrued a number of backlogged changes. Strong communication between all GIS departments is key for identifying the need for data maintenance. Cooperative work in helping departments automate their functions and adjust to new technology is also critical. Finally, a commitment by all users to identify and catalog errors that are located in the data base is necessary. Even if the mechanisms are not yet in place to correct data errors in "real time", a means for cataloging these errors in a systematic fashion is necessary. A backlog of work to be addressed over time, is better than the false belief that the data base is perfect. 5.3 GIS Technology is Dynamic GIS technology is constantly changing. ArcInfo 6.0 is replaced by ArcInfo 7.0. The PRIME platform is replaced by UNIX. Each of these types of changes can have radical impacts on an operational GIS installation. ArcInfo has changed dramatically with the advent of 7.0 and the improvements to ARCVIEW and ARCCAD. Newton will be working to assess the need for additional software tools and the degree to which it will adopt Esri's new offerings (or those of other companies). Newton would strongly prefer to not have to buy any further, and expensive, ArcInfo workstation licenses. Consequently, ARCVIEW is very attractive. However, Newton has hundreds of hours of development time invested in a robust AML user interface. Moving to ARCVIEW would require reinvesting those hours in AVENUE programming. This is plainly both a challenge and an opportunity. Similarly, the Newton Engineering Department is well versed in AutoCAD. Thus, the use of ARCCAD to maintain the core ArcInfo coverages is very attractive. However, given ARCCAD's current inability to access and edit UNIX ArcInfo coverages the overhead in moving data from UNIX to DOS and then back again is prohibitive. When the software changes to allow this, the protocols for data maintenance will have to change accordingly. These are only the current changes. What will happen to a public sector, operational GIS like Newton, when new object oriented GIS data models and programming languages become available (next year, or next century)? Re-investments will need to be made, or conscious decisions to fall behind will have to be swallowed. This is the reality of the GIS business, and these eventualities must be planned for to insure that a GIS installation remains viable, efficient and successful.