SDE Implementation Issues with a Mature GIS in Local Government
Angela Mills, GIS Database Administrator
This paper presents the challenges and victories of implementing ArcSDE in a local government environment that has an established GIS of 14 years. The presentation will address issues such as annotation created in ArcInfo 5.0 and it's migration to SDE, distributing data to multi-platformed clients and integration with Internet mapping tools. Additional focal points will be the impact of transitioning from a tile-based data storage format to Countywide format for daily data maintenance and the integration of a parcel solution.
HISTORY
Evolution of GIS in Prince William County
The history of Prince William County’s GIS (Geographic Information Systems) can be traced back to 1984 when a committee at the executive level was formed to study land information systems (LIS). One of two key recommendations that were made was to establishment an Office of Mapping. The newly created department would unite all of the existing mapping resources throughout the County. The Office of Mapping was established in July 1985. Immediately the County began the process of creating a base map that would support all other mapping themes that would be created in the GIS. Because the existing tax maps were created at a scale of 1"= 400’, recompilation was necessary to 1" = 200' to reconcile discrepancies that would be found between the tax maps and the digital data that was to be delivered. In 1985, a mapping staff of four persons completed the three-year process of converting from 400 scale to 200 scale.
The establishment of the geodetic monuments was one of the first major steps taken. Although the County had 17 existing monuments, a professional surveyor was hired and placed 83 new control monuments throughout the County in approximately 2-mile intervals. Using GPS (Global Positioning System), coordinate values were collected and all computations were submitted to NGS for review, comment and approval.
Prince William County then began the process of selecting an aerial photography and photogrammetric mapping firm to create the digital land base. In March 1987, the County was flown and aerial photographs were taken at a scale of 1:14,000. The maps were delivered in both traditional hardcopy and digital files. During this same time period, Prince William County chose the GIS Software package ARC/INFO. They selected Esri (Environmental Systems Research Institute) based in Redlands, California.
Prince William County's first GIS server was a PRIME 6450 running ARC/INFO version 3.0. User terminals were two IBM AT's with graphic emulation, one Tektronics 4111 graphic terminal, two Calcomp digitizers and a Calcomp 1040GT Line Printer. The user base included Mapping staff, Public Works and Planning. With the hardware and software installed, Prince William County began to march forward with digitizing the parcel map layer. In-house GIS Specialists performed this work by manually digitizing the 1" = 200" compiled mylars into the new digital taxmap series. This task took the same 4-person team 2 years to complete. Soon afterwards other map layers began to be created and GIS Staff increased to 26.
The County then developed an anchor layer, the street centerline network known as CENNET. This vital layer contains address ranges for street segments. It can be linked for demographic information, serving as well, as boundaries for school districts, political districts and planned land uses. Over 100 additional layers have been added throughout the 14 year history of the County’s GIS, all building from the initial data delivery in 1988. The early 1990's focused on the development of AML programs to produce standard mapping products. Focus was also on the standardization of procedures to acquire and incorporate land oriented data into the GIS, and to provide services to other County Agencies as a means of integrating GIS in County business practices. PWC GIS ended the decade by utilizing technology to begin publishing data on the Internet. It became apparent that this was the most effective and cost efficient way to distribute accurate GIS products and services. Since 1997, Prince William County has deployed two MapObjectsIMS Intra/Internet applications, two ArcIMS intra/internet applications, one external digital data wizard, and one internal map creation wizard.
INVENTORY
Hardware and Software
ArcSDE was first rolled out on an HP9000/800 Series Model K210 running OS version 11.x, 32-bit kernal. The server was 120MHz with 1GB RAM, 2 active processors and 100GB Disk Storage. In October 2001, Prince William County upgraded their GIS server to an HP9000/800 Series N4000-55 that was 550MHz with 2 active processors and doubled the RAM to 2GB We are is still running OS version 11.x. Connected plotters include six HP DesignJet 755ps, one HP DesignJet 3500cp and two HP DesignJet 5000ps models. Users digitized on GTCO Accutab boards and accessed the GIS via WindowsNT 4.0 workstations through WRQ's Reflection X window emulation software or ArcGIS Software. Network configuration standards are TCP/IP connectivity through a T-1 Microwave and 10/100Base-T CAT5 UTP Ethernet switch.
Prince William County utilizes a wide variety of Esri products. Data layers are created and maintained using ArcInfo 8.1 Workstation on both WindowsNT 4.0 sp6a and Unix Server via Reflections X software. ArcInfo 8.1 Desktop is best utilized for the creation of custom thematic maps and map analysis. NovaLIS ParcelEditor software is the current parcel solution used by Prince William County. Clients access the enterprise database and SDE database through ArcInfo 8.1 Desktop, Workstation or ArcView versions 3.2x. ArcIMS 3.1 is also connecting to ArcSDE from our internal web map server.
Users
GIS Prince William County's user base encompasses over 75 internal GIS Users. The GIS Staff consists of 18 professionals: one GIS Manager, one GIS Database Administrator, one GIS Supervisor, one Demographer, five GIS Programmer Analyst, six GIS Analyst and three GIS Specialist II. Thirteen different County Agencies have GIS embedded into their organization, each employing an analyst to perform work strictly for that department. GIS centralized staff provides support to these remote user locations. PWC GIS provides support to 57 remoteusers throughout Prince William County government.
GIS Database
The GIS database is roughly 118GB and contains 128 geo-spatial data layers stored in a tiled library and 51 data layers are loaded in SDE. Over 920,000 features are stored in coverages and 700,000 features are stored in SDE. Prince William County has over 113,000 parcels to be maintained on a daily basis by the GIS Division and numerous additional layers by our remoteusers. On an average, Prince William County records 5000 new parcels per year.
HOW GIS IS USED IN PRINCE WILLIAM COUNTY
Prince William County's GIS Division provides GIS data and GIS support to several county agencies and is centralized in the Office of Information Technology. Each agency either has direct connectivity or uses a disk transfer system to gain access to the core data located on the enterprise database. We have approximately 126 layers in the GIS at this time and are anticipating many more in the future. The parcel layer is maintained daily because the origination for all land business transacted within the county begins with the Geographic Parcel Identification Number (GPIN). The GPIN number is used to generate a record for the County’s code enforcement database. All proffers and permits for that property are tied to this record. Base information, or land detail, is maintained through the sources such as building blueprints or aerial photography. From these sources, we can enter fence lines, buildings, parking areas and even individual trees. The street centerline network is also maintained daily. This is important because other county agencies such as Fire and Rescue, Police and many others use this information as an analysis tool when preparing models for their departments. The Census areas and COG Areas are used for demographic forecasting and analysis and are able to allow the Demographer to provide accurate information for inquiries from the public as well as other county agencies.
GIS is used as application services in a variety of County Agencies. Service level applications that utilize GIS are as follows:
Public Safety Communications
Police
Fire & Rescue
Economic Development
Agency on Aging
Schools
Park Authority
County Attorney
Registrar
Health Department
Planning
Public Works
Finance
Service Authority
County Executive & Board of Supervisors
GIS
One of the reasons that Prince William County considered moving to SDE was because our users are spread throughout the entire county. The GIS is the central department that controls the enterprise database. County Offices are located on both the Western and Eastern ends of Prince William and the network issues that existed made it difficult for users in these areas to access GIS information quickly. Data would have to put cut to CD-ROM and distributed quarterly to these departments. Additionally, these departments were creating data layers that were beneficial to other departments and there needed to be a method of sharing that data to the users without having to put them on CD-ROM.
The GIS that was operational when the decision was made to move to SDE had to be upgraded to our current server assignment. The original HPUX K210 Server had storage and processor speed issues that needed to be addressed. Prince William County performed a network upgrade over a period of 12 months that included County buildings to migrate from 10BaseT to 100BaseT network connections. Once these upgrades were completed, we were able to proceed with installing the proper GIS hardware. The conversion to the N5000 series server took 2 months to complete.
Annotation
When annotation was first created in PWC in late 80's, it was Esri's recommendation that annotation be stored as a secondary feature within coverages. When SDE documentation became available, we found that ArcSDE only loads annotation as subclass definitions. The decision was made that the first step in the migration process was to begin by converting existing annotation to subclass annotation. Using AML, annotation was moved into subclasses over a period of three weeks. Once this phase was completed, test data was loaded into ArcSDE and viewed on each type of client that would be used to view the data.
Immediately we found that annotation was not handled as expected with ArcSDE. When viewing the annotation in ArcView 3.2a, the annotation appeared in the proper location but a single line appeared underneath each piece of anno. It was discovered that this was because the annotation was added as line annotation and SDE stored the value of two or more points entered when adding the actual annotation. We could not alter or hide the feature in any manner. Viewing the same data in ArcMap gave us a completely different result. The lines under the annotation disappeared; however, the annotation then appeared upside down and backwards. After many discussions with Esri technical support, this was assigned a bug number and we were told that it would be resolved by version 8.3. An internal decision was then made to load data layers without annotation for the testing phase.
Multi-Platform Data Distribution (Entity Types)
Several of Prince William County data layers were comprised of multiple entity types. When these layers were loaded into ArcSDE, ArcView 3.2a users were able to choose the desired folder and feature with the familiar drop down choice. However, those same layers were not visible in ArcMap or ArcIMS. After researching the user forums, we became aware that others were having these same issues. The recommended workaround was to load each individual feature type as a separate layer. Since ArcSDE organizes layers alphabetically instead of a tree structure like ArcView, users would have to scroll through hundreds of layer names to get to one entity type for their layer. Additionally, it was estimated that this would increase our database size by as much as 20%. Major concerns arose because our users access the GIS through a variety of different platform clients. They need to continue using the same software to access the data to avoid any workflow interruption or cost impact. Forcing these departments to switch software and be re-trained was not a desirable option.
We began to assess which data layers our users were accessing most frequently and what was the most common GIS software utilized. Each agency was reviewed to determine these factors. The results showed that the ArcView users and ArcGIS users were very close in percentage and that single entity layers were the only viable option for visibility in both software packages. Furthermore, we identified 51 layers that were high priorities for storage in ArcSDE and would be the first data layers loaded.
Integration of a Parcel Solution (NovaLIS Parcel Editor)
Parcel Editor is also currently Esri’s recommended parcel solution. Several things have impeded or kept Prince William County (PWC) from fully implementing both SDE and Parcel Editor. These obstacles range from cost of software to basic SDE implementation in Parcel Editor (PE 3.4.1).
The first is the cost of a customized version of Parcel Editor and NovaLIS’, GATE software. PE 3.4.1 is the cogoing solution and GATE is NovaLIS way of communicating to SDE. Both pieces of software are highly customizable by NovaLIS to meet your agency workflow. However, the cost of these products is based on the amount of customization needed for your agency. This option of customization was not an avenue PWC could look towards.
The personal customization of the product to change with our needs has been cumbersome. Since PWC took the option of customizing Parcel Editor ourselves, many challenges have occurred. Currently at PWC our data support historic and parent relationships in our parcel data. This relationship is critical for both county and public research and information. The GATE product developed by NovaLIS allows and creates this relationship. However, PWC has had difficulty in implementing this aspect of SDE and automating this aspect of Parcel Editor alone. Our solution to this date has been manual exporting and coding this layer as an ArcInfo coverage.
Transition issues from tiled library
Since the GIS' inception in 1988, we have stored our data in 10k tile formats. The data was originally created in single precision and cleaning the coverages was an acceptable procedure at that time to rebuild topology. Unfortunately, fuzzy creep has taken its toll on our data and we now have gaps and overlaps at tile boundaries. In 1996, we moved our data to double precision and began edgematching, however, when dissolving between tile boundaries we still get 2 and 3 foot gaps between tiles that contain the universal polygon. Because of this, we are unable to remove completely dissolve the entire county to make it seamless. Tiles checked out up to 23 times a year.
SUCCESSES
Layers loaded successfully
Once the decision was made to eliminate loading annotation and coverages with multiple entities were resolved, data layers were loaded successfully into ArcSDE. Layers are visible from ArcView, ArcMap and ArcIMS. Most data layers were loaded as countywide layers with a grid size of 19354; smaller grids were used in specific areas. Setting the envelope limits for data growth was not an issue since any annexation that Prince William County was to participate in would still fall within the original envelope boundary.
Connection Speed Increases
ArcSDE is used successfully used by the Prince William County GIS community. Users have responded positively and report an increase in access speed over WRQ's NFS. Because they are able to have access to the most current data, many have implemented ArcSDE into their routine projects.
ArcIMS using ArcSDE
Prince William County launched the QuickInfo internet/intranet application in the summer of 2001. This application took advantage of connecting to the ArcSDE database and returning spatial data directly from the Oracle tables. The speed of connection and returned data was substantially higher than connecting to an Access database. Additionally, a prototype application for the Police Department connects to ArcSDE and returns spatial layers and performs queries directly from ArcSDE.
Data is updated weekly
Without the help of NovaLIS Gate product, parcels and other layers must still be maintained in ArcInfo Coverages. ArcSDE data layers are updated weekly through a series of AMLs. These AMLs regenerate the countywide data layers from the tiled libraries and perform several quality control checks. The SDE layers are then dropped and reapplied, rebuilding indexes and permissions as they are added. Dynamic layers, such as parcels, zoning and road centerline networks, are updated weekly. More static layers, for example vegetation or hydrology, are updated monthly or quarterly.
FUTURE PLANS
Migrate all to enterprise geodatabase
Prince William County plans to implement an enterprise wide geodatabase by 2004. Plans for Phase I of this implementation are in effect. Use case models are being built for each layer based on who must maintain that data. We are working with each department individually to assess how their data layers are being used and design the geodatabase layer accordingly. A design team has been established with people who have expertise in GIS, Management Skills, Programming and Subject Matter Expertise. This team works together to first identify the needs for a layer, build a conceptual model and finally, will program the physical model. Depending on the complexity of the data layer and its use, this process can have a repeated life cycle before implementation occurs.
Migrating applications to ArcGIS Desktop
Because true geodatabase functionality is only available from ArcGIS Desktop, we must first convert all standard operating procedures for data layer maintenance from ArcInfo Workstation to the Desktop environment. This will require conversion of over 70 applications. We expect this process to take approximately 10-14 months to complete. Additionally, all custom and standard map products available at our Customer Service Counter will have to be altered to access SDE instead of the current enterprise database.
Angela Mills
GIS Database Administrator
Prince William County
Office of Information Technology
Geographic Information Systems Division
4379 Ridgewood Center Drive #201
Woodbridge, VA 22193-5308
703-792-6868
703-792-7192 (fax)
amills@pwcgov.org
www.pwcgov.org/oit/gis