Xiaodong Hong
GIS Manager
InfoTech Enterprises, Inc.
An ISO 9001 Registered Company
4900 Seminary Road, Alexandria, VA 22311
To improve the efficiency of existing emergency response and facility management systems, many utility companies choose to implement the ArcFM GIS. Based on my experience designing and implementing an ArcFM GIS for the Potomac Electric Power Company (PEPCO), Washington D.C., this paper provides a road map through implementation of an enterprise-wide GIS using technologies built-on ArcInfo 8 platform (Geodatabase, ArcFM and ArcSDE). The paper summarizes the implementation process in four phases: Data Model Design, Building a Geodatabase into a Multi-user GIS, Migrating CAD Data to a Geodatabase, and GIS Deployment and Data Maintenance Strategies. Major roadblocks related to each phase are identified, and alternative solutions are provided.
As GIS evolves from a single user editing system to multi-user database driven system, many utility companies choose to implement GIS to improve the efficiency of facility management, emergency response, and outage management. ArcGIS 8 takes advantage of a relational database to store spatial and non-spatial information, allows multiple users editing the data, and manages the editing in version system through ArcSDE. Unlike its predecessors, ArcInfo 8, for the first time completely, runs on a geo-relational database that is called a Geodatabase in the ArcGIS family. The underlying geo-relational database structure in a Geodatabase that enables multi-user editing and version management is the foundation of an enterprise-wide GIS. It is clear that developing a Geodatabase and managing it through ArcSDE is the key to successful implementation of an enterprise-wide GIS.
PEPCO selected InfoTech Enterprises, Inc., Alexandria, Virginia to implement an enterprise GIS in ArcGIS and ArcFM 8 platform. The advice here is based on my experience as a project manager in designing an Oracle Geodatabase for PEPCO, as well as managing the data migration from PEPCO’s legacy CAD data to the Geodatabase. The discussion not only addresses issues in designing a data model and building a Geodatabase in an Oracle database, but also provides insights on data migration from CAD to GIS and discusses future data maintenance and system integration strategies. Although this paper focuses on implementing GIS for an electric utility company, the issues discussed here can be applied to other utilities companies that want to implement an enterprise-wide GIS. In addition, this paper documents some valuable lessons learned during the course of our GIS development and technical limitations of current ArcGIS software that can be of help to other organizations when developing a GIS.
A typical GIS implementation using the Esri ArcInfo Coverage data format (supported by ArcInfo version 7.2 and older) consists of three phases: data conversion specification, data migration, and application development. When designing a data conversion specification, one needs to first assess the functional requirements of the GIS system and collect requirements from various departments in an organization and then transform the requirements into database structure. For example, if the users say that a transformer needs to have Nominal Voltage, Operating Voltage and Rated KVA, then the point attribute table (PAT table) in the device coverage should contain these items. A data conversion specification typically contains table and item definitions as well as business rules that need to be followed during the data conversion. In addition, the specification should also contain symbol requirements for each feature. Due to limitations of the Esri Coverage data format, relationships cannot be implemented at the data structure level. The relationship rules are typically enforced at the application level. In other words, the GIS will function properly only with a set of applications. It is recommended to design the GIS applications and data conversion specification simultaneously so that the data structure will support GIS applications.
Before implementing GIS, many organizations have already converted their data migrated from paper source to CAD formats, such as AutoCAD DWG and Microstation DGN. The target GIS data format is most likely to be ArcInfo coverage that stores spatial and non-spatial data. External database tables can be used to store additional attributes. For example, a transformer in the device coverage contains a link to an external Customer Information System (CIS) database. ArcInfo provides some useful data importing tools that make data migration to GIS relatively straightforward.
After accepting the first GIS data delivery, an organization should start developing GIS applications. Depending on levels of automation and complexity of the data structure, GIS application development can be a lengthy and labor-intensive process. Major considerations in this phase are:
·Support the GIS operations, such as data query and data analysis
·Support data maintenance and enforce business rules
ArcInfo and ArcView have many built-in functions to support data query and data analysis. It is relatively straightforward to tailor these functions to meet the needs of the organization. However, the development of data maintenance tools is a complicated and time-consuming process. GIS maintains topology and the geographical locations of each feature. Retaining data accuracy during data editing in GIS is a difficult task. Applications not only need to enforce business rules (such as valid values, relationships to other features, or links to data records in an external database), but also maintain topological integrity. One important limitation of a traditional GIS is that the Esri coverage data format only allows single user to access and edit the data. Many efforts have been made to overcome this limitation, resulting in products such as Library and ArcStorm. While these two modules allow users to checkout and check-in data, they do not have features like versioning and rollback that are commonly used in a multi-user information system. The Geodatabase data structure introduced in ArcInfo 8 provided a solution that supports an enterprise-wide GIS. For the first time, both spatial and non-spatial data are stored in a relational database. Most importantly, the Geodatabase, through ArcSDE, supports multi-user access and editing, versioning and rollback. Relying on a relational database structure, Geodatabase allows users to define objects and behaviors (Zeiler, 1999). No longer are “point”, “line” and “polygon” the only feature classes supported by ArcInfo. True feature classes (such as “Switch”, “Fuse”, “Primary Electric Line”, etc.) and their behavior can be defined at the database level. The new features offered in the Geodatabase introduce new elements to the GIS implementation. The traditional GIS implementation approach is no longer sufficient and a new paradigm exists for implementation plan development. Based on the author’s experience in developing a GIS based on the Esri Geodatabase model for PEPCO, the following summary of an enterprise GIS implementation plan is presented.
The implementation plan proposed here is based on the following assumptions:
The enterprise GIS development is organized into four phases: Data Model Design, Build a Geodatabase into a multi-user GIS, Migrate CAD data to a Geodatabase, and GIS deployment and Data Maintenances Strategies.
The data model is the heart of enterprise GIS. A data model design must meet the operational and functional requirements of an organization. The data model must also support the requirements of ArcFM software and shall be extendable to integrate with other information systems, such as the Customer Information System (CIS) and Outage Management System (OMS). The data model shall also support symbol display and the data maintenance process. There are six steps in this phase:
There are four types of GIS requirements:
Daily operations and facility management is the first requirement and basic function of GIS. To fully identify operational and functional requirements and transfer the requirements into GIS database structure, cross-functional GIS team members must include the GIS data modeler, GIS manager, and representatives from departments that may directly use or maintain the GIS. A GIS data model workshop is an effective way to collect all user requirements. Members of GIS team should participate the workshops. The more user departments that are involved in the workshops, the more comprehensive the requirement list will be.
Second, the requirement list should include needs to integrate with other information systems, such as the Customer Information System (CIS). If the needs exist, the GIS team should identify database linkage between GIS and the other information systems. If the linkage does not exist, the discussion shall focus on how and when to generate the linkages. Issues related to system integration need to be addressed in the data-modeling phase to avoid data structure incompatibility between GIS and other systems.
Third, the requirements of the GIS software functions and applications shall be considered. For instance, PEPCO has chosen ArcFM as their primary interface to GIS database. Feeder management and tracing tools offered by ArcFM require certain items existing in the Geodatabase. Therefore, the data model is required to include some system items to support functionality of ArcFM. Similarly, the requirement from other GIS products shall be considered, such as ArcIMS.
The GIS shall also support symbol display, data migration and the data maintenance process. Each organization has its own data maintenance procedure and data approval process. To support revision management, the data model may include item(s) to record the revision stage of a feature. For example, PEPCO has a business process to record a feature from the “proposed” to the “approved” stage. To facilitate the same process in GIS, an item called “Construction Status” was added to the PEPCO’s data model. The valid values for this item are “Proposed”, “Build not energized”, and “Approved/As-Built”. The data migration process must also be considered in the data model. Many legacy CAD systems contain feature attributes in various forms. To ensure that CAD attributes are migrated correctly, the data model must include some temporary items to hold CAD information during the migrations process. These temporary items should be deleted after the data are fully migrated and validated in GIS. To avoid project delays due to the future data model changes, other considerations should also be addressed in the data-modeling phase. For instance, the GIS team needs to determine whether the data model supports the displaying of symbols and how the data model supports coincident features.
The underlying data model in the ArcInfo 8 Geodatabase provides an object-oriented approach to GIS design. To fully take advantage of this technology, the four types of requirements should be identified and implemented in the data-modeling phase. The more comprehensive the data model is, the better the GIS implementation will be. Data model design is the most critical phase in the GIS implementation. The PEPCO data model was built on ArcFM 8 data model developed by Miner & Miner within the ArcGIS 8 framework. ArcFM 8 provides a customized data model and a series of editing and tracing tools to meet specific needs of data maintenance and analysis in utility network.
Tip: To design a GIS data model, ArcGIS data models provide a good starting point.
In ArcInfo 8 environment, there are two approaches to develop a data model. One is to generate a data model in ArcCatalog. The other is to develop an UML data model in Visio. Although both approaches are valid, the UML data model has more advantages over using ArcCatalog. The UML model can be used to generate a complete Geodatabase while one has to generate the Geodatabase one feature at time in ArcCatalog. Moreover, the UML model provides a graphical representation of the data model and therefore is more intuitive to the users. However, there are trade-offs in using the UML model. Developing and generating the database schema from a data model in UML can be a lengthy process. ArcInfo 8.1 provides some new tools to detect schematic errors in a repository and improve the database generation process. In addition, Esri and Miner & Miner offer well-design GIS UML models. It is more efficient to develop new data models on the ArcGIS data model. This will shorten learning curve and development time and ensure compatibility of the data model to ArcGIS and ArcFM software.
Tip: It is a good idea to have technical support from Esri or Miner & Miner when developing a data model.
Developing and testing a data model are two interrelated processes. Before the data model can be tested, a Geodatabase schema needs to be generated. Once the schema is generated, the following inspections/tests need to be performed to ensure it meets all the requirements:
Using test data set to validate the data model is very important because it is the first time users can directly see and “feel” the data without understanding abstract terms in the UML data model. Thus, the test data set should contain all possible features and data configurations. This allows users to fully test the behavior of all possible features. It is typical to have multiple iterations of data revisions and loading before having a data model that is satisfactory to the users.
Tip: Use Schematic Checker to validate the repository database before using the Case Tool wizard to generate the data model.
As the data model development is an iterative process, configuration management should not be over looked. Documenting changes to the data model is crucial in data model design. For instance, I conducted more than two dozens data model workshops before the data model was finalized. Documenting every workshop helps keep track of all data model changes. The documentation also provides a clear trail of data model development, which will help to secure the data model approval. It also helps in generating the data migration specification.
Tip: It is a good idea to obtain data model certification from a third party, such as Esri or Miner & Miner, once the data model is finalized.
In the ArcGIS framework, there are two types of Geodatabase, a Personal Geodatabase running in MS Access and an Enterprise Geodatabase running under ArcSDE in a relational database such as Oracle, MS SQL, DB2, etc. While the Personal Geodatabase supports single user editing, Enterprise Geodatabase supports multiple-user editing through versioning. It is obvious that an enterprise Geodatabase heavily relies on relational database technology. Database configuration and tuning become an important part of GIS development. In order to have a functional GIS, the Geodatabase needs to be built with proper database and ArcSDE configuration. In-depth knowledge of ArcSDE and Oracle are the keys in building an enterprise Geodatabase. The technical details of ArcSDE and oracle configuration are out of the scope of this paper.
Tips: Esri provides white papers on how to configure Geodatabase into a Multi-user GIS.
Data migration is a labor-intensive process and often is the most expensive step in the entire GIS implementation. The key to data migration is the data accuracy. In order to produce accurate data, the data conversion vendor should be provided a data conversion specification generated for the data model before starting full production. Data validations in both CAD and GIS are critical to the success of data migration. Data validation routines need to be developed before starting full production. These routines are typically developed in CAD using AutoLisp, MDL, or VBA and in GIS using AML.
There are many ways to migrate CAD data to a Geodatabase. In the PEPCO project, the ArcInfo Coverage format was used as a bridge to migrate from CAD to the Geodatabase. The CAD data is first converted into an ArcInfo Coverage. The ArcInfo Coverage is processed according to the data structure in Geodatabase. Finally, the Coverage is loaded into a Personal Geodatabase because the Personal Geodatabase is portable and easy to manage whereas the Geodatabase in Oracle or MS SQL Server is less portable but is consistent with the target system. The decision on whether to receive deliverables in a Personal Geodatabase or Enterprise Geodatabase in Oracle/SQL Server has impacts on the data acceptance procedure. Refer to Phase III in the Figure - “An Enterprise GIS Implementation Plan”.
Like traditional GIS implementation, an organization needs to develop data validation routines in the GIS. It is beneficial to the GIS project to share the validation routines with the conversion vendor during the data migration because data errors can be detected and corrected in the early stages. During data migration and acceptance, large amounts of data have to be transferred between Geodatabases. Incremental data loading in ArcInfo 8 is a time-consuming process and deserves appropriate attention. Incremental data loading is addressed in greater detail in the Road Block section. After all, a successful data migration requires detail resource planning and process planning.
Tip: Appending data to a Geodatabase is a time-consuming process. GeoDBLoader will help to resolve the problem.
An enterprise GIS is built for data access and data sharing through an organization. Issues related to GIS deployment need to be addressed in the early stages of GIS implementation. In a utility company, there are two groups of users: users with a stable LAN connection, and mobile users. ArcSDE 8 provides many utilities to manage access to the GIS database through LAN connections. Internet technology such as ArcIMS can be used to rapidly deploy GIS data to the entire organization with relatively low cost. Because ArcGIS software requires a high-end hardware configuration and provides more functions than a mobile user needs, the needs of mobile users to view data on a mobile computer may require a different solution. There are two approaches to resolve this problem. One is to use a wireless connection to retrieve data from a central GIS database. This approach requires a large and dedicated bandwidth and the performance is questionable in the current wireless technology. The other approach is to use a lightweight data-viewing tool on mobile computers. Such a viewing tool requires less system resources and provides basic functions to a mobile user. The performance of the tool is satisfactory on a mobile computer. This approach however requires daily download of GIS data from the server to the mobile unit. PEPCO chose a viewing product called VuGIS developed by InfoTech to view GIS data. The product can run on a 266 MHz and 64 MB RAM machine. It provides viewing, query and redline functions.
After building the GIS, data maintenance is another major issue an organization needs to address. The questions that need to be asked are who, when and how to maintain the GIS data. Many organizations have existing data maintenance procedures for the CAD database. These procedures are good starting points. Because most CAD systems only allow single-user editing, one department in the organization normally handles all data maintenance. Because GIS provides multi-user access and has complex relationship rules, it is more efficient to decentralize the data maintenance responsibilities to multiple departments and maintain a centralized approval process.
Beside GIS deployment and data maintenance, other issues also need to be addressed to ensure a thorough GIS implementation. These issues include: system integration, user training, data backup, future upgrades strategies, etc.
Tip: Although many data editing functions have been provided in ArcMap and ArcFM, further customization is needed to facilitate data queries and data maintenance.
Data model changes are inevitable during GIS implementation. However, frequent changes of a data model could lead to project delays and even unsuccessful implementations. Therefore, first major milestone of the project is to freeze data model.
Prior to the release of ArcInfo 8, symbols were generated from a specific font file. In order to display customized symbols on different features, each feature has to carry an item in which the symbol index in the font file is stored. An application needs to be developed to maintain the symbol index when a feature type is changed. In ArcInfo 8, symbols can be assigned to a feature based on combination of up-to three attributes. This effectively eliminates the need for a symbol item in the database and the application to reassign symbol values. It is important to validate whether the data model has suitable attributes from which distinct symbols can be generated for each feature class.
The data review and acceptance process can happen only after the data has been loaded into an enterprise Geodatabase. Incremental data loading to a Geodatabase in ArcInfo 8 is a cumbersome process. Currently, there are two tools to load data to an existing Geodatabase. One loads one subtype of a feature class at a time. The other loads one feature class at a time but requires all composite relationships be removed. To facilitate data migration, InfoTech developed a data-loading tool – GeoDBLoader – to transfer data from one Geodatabase to another Geodatabase in a batch process. The Geodatabase can be a Personal Geodatabase or Enterprise Geodatabase in Oracle or MS SQL Server. This tool will expedite the data loading process, providing significant efficiencies and substantial reductions in labor.
>> Users can select a source Geodatabase and a target Geodatabase.
>> Once the user selects “Load”, the data will be transferred from the source Geodatabase to the target Geodatabase.
>> Status of the transferring process is shown in status bars.
>> Users can stop the process any time by clicking the “Stop”. The user interface is simple and informative.
Viewing GIS data on a mobile computer requires a lightweight-viewing tool because a mobile computer typically has less computing resources. Current viewing tools on the market require excessive computing resources and provide too much functionality, and therefore are not cost-effective. To facilitate data viewing on mobile computers, PEPCO has selected VuGIS, a product developed by InfoTech. VuGIS provides basic query, viewing and redlining functions and can run on a 266 mhz and 64M RAM machine.
Figure: Querying Attributes
Figure: Creating Feature Linked Redlines
Oracle and ArcSDE configuration and tuning are two important aspects of an enterprise GIS. Poor system configuration and tuning can lead to an inoperable system. To have a functional GIS, the GIS database should be configured properly and efficiently. The following questions need to be answered:
Once the GIS database is established, the database has to be tuned to maintain its performance. The GIS database administrator plays an important role in this task. The GIS database administrator should have in-depth knowledge of the Oracle database and receive in-depth training in ArcSDE.
As GIS evolves into an enterprise multi-user access system, the traditional GIS implementation methodology is not longer suitable to large-scale GIS implementation. This paper presents a road map to the implementation of enterprise GIS. On this road map, I defined five phases: Data Model Design, Build a Geodatabase into a Multi-user GIS, Migrate CAD Data to a Geodatabase, and GIS Deployment and Data Maintenances Strategies. While each of the five phases is critical to the process, the data model design phase is most important because it lays the foundation for GIS development. Furthermore, I have identified five roadblocks along the path to building an enterprise GIS. Solutions to these roadblocks are also presented.
While the paper has focused mainly on the implementation process, there is no doubt that understanding the new technologies, such as ArcSDE and the Enterprise Geodatabase structure play an important role in the success of GIS implementation. Due to the scope of this paper, technical details of ArcSDE and the Enterprise Geodatabase structure are not discussed.
Although the new road map to the GIS implementation is derived from my experience on developing an enterprise GIS for PEPCO, which is a utility company, I believe that the approach summarized in this paper provides an understanding of enterprise GIS and can be beneficial to GIS implementations in other organizations.
Zeiler, Michael, 1999: Modeling Our World – The Esri Guide to Geodatabase Design, Esri Press
I would like to thank Mr. Akhlesh Kaushiva, manager of PEPCO IT projects, and the entire PEPCO GIS team for their dedication, teamwork, and ability to work through difficult and complex issues. Without their in-depth knowledge of electric systems, the electric data model would not have been developed. I would also like to thank Esri and Miner & Miner for their continuous and generous support on resolving software, data modeling issues. Finally, I would like to thank Mr. Karlu Rambhala and Mr. Pat Callahan in InfoTech Enterprises for their guidance and encouragements throughout the PEPCO GIS project.