Charles P. Butler
One of the downsides of implementing a GIS is the huge cost of the data which it will contain. This fact is particularly true when it involves a custom data product which can only be reproduced from it's source. Huge database conversion costs however can be greatly reduced and made more predictable by simplifying the process through documentation and an effective AML interface. Costs are lowered, made more predictable and product quality is maintained by allowing lesser skilled individuals to perform the "nuts and bolts" of the conversion work by building programmatic intelligence into the interface based on orderly documentation. This paper presents an innovative project approach which was developed for a major metropolitan water utility to largely automate and simplify the conversion of their water and sewer facility maps.
Introduction This paper seeks to present an approach for simplifying the data conversion task so that lower skilled and less costly individuals can perform the bulk of the work. In this respect, labor costs can be reduced and made more predictable while maintaining the desired product quality. Admittedly, the approach was developed to tackle a large scale conversion project and may therefore may not be applicable to some smaller projects. The approach of conversion simplification relies heavily on two major aspects: 1) Process and specification documentation, 2) The development of an Intelligent Interface. It is this authors experience that both are necessary to achieve the benefits that this paper describes. The ensuing discussion is organized into the following four parts so that the concepts presented are clear, and understood to be sperate activities: -Process Documentation and Specification Development -Project Design -Intelligent Interface Development -Project Prototype and Implementation Background The approach discussed herein draws much of its practical experience from a large scale conversion project with one of the Nation's largest Water and Wastewater utilities. The setting for this company was one in which a completely manual mapping method concerning water and sewer maps was to be automated to improve efficiency and customer service. At the inception of the project, the company used and maintained many different types of databases in a mainframe environment and had performed a corporate-wide GIS pilot project some three years earlier. Also, it was involved with other organizations which had executed an agreement to share digital data and GIS experiences. I. PROCESS DOCUMENTATION AND SPECIFICATION DEVELOPMENT This activity involves the investigation and documentation of the project's who, what, when, where, why and how. An activity often dreaded by clients and consultants alike, it is however the cornerstone by which a data conversion project can be successfully simplified. The following paragraphs briefly describe the major tasks undertaken concerning the subject company. Define And Document Manual Specifications The objective here is to find out EXACTLY what is currently being done and by what guideline, and to record that information clearly. Unfortunately, all to often in the conversion process, only a part of the needed specifications are completely defined, or they may exist as generalized facts, especially if multiple special conditions apply. In either of these situations, the actual documentation may or may not exist. Chances are if they do, they are not clear and complete with respect to their application. Areas of ambiguity are recipes for problems when tying to simplify the process. Since this approach involves programmatic development, the complete specification must be known so that it can be logically addressed by computers and humans who are otherwise unfamiliar with what to do and why. In the case of complex operations, poor specification definition and documentation will definitely make the conversion process more costly and burdensome through changes in scope or work orders, with little guarantee that the actual product delivered is correct and functional as needed by an organization. In the case of the water utility company, this investigation uncovered conflicting specifications which resulted in individuals in the manual process not uniformly applying the specifications. It was discovered that this problem had developed slowly over a series years in which different mapping changes were instituted, and some dropped. Without complete documentation, this inconsistency would have carried over into the conversion process where problems would have surfaced. Define And Document Manual Procedures The objective here is to find out EXACTLY what is currently being done; step by step and to document it clearly. These steps may not be completely replicated when automated, but procedures usually exist for an important reason. The goal is to collect and clearly document procedural information so that what is applicable to the conversion process is not missed. In general, it is easier to eliminate or combine a procedure then it is to incorporate a new one later. Having a full disclosure of the existing procedures can provide valuable insight to the conversion process and will aid in addressing maintenance issues. When performing this task for the water company, it was clear from the start that there was no such formal documentation available. Unfortunately, this meant that an understanding of the steps to be taken when performing the current mapping tasks were only acquired over a period of time and by relying on experienced individuals. This situation led to areas of ambiguity even among experienced technicians. Consequently, a clear understanding of what the current process dynamics were was needed before the conversion process could proceed. Define And Document User Community Often products are made to address a cross section of users otherwise known as the User Community. The objective here is to identify and record who uses the product and how it is used. The investigation and subsequent documentation of these two items provides insight into the "end users" requirements. In addition, the information may reveal valuable information which otherwise would have been unknown, and is important for planning the automation of the conversion process. To illustrate, the water company had two major user communities with different data needs and cartographic requirements, but they all used one product per the current method of operation. An investigation documented the difference in data requirements and presentation form for later consideration in design of the conversion process. Consequently, it became apparent to the company that the complexity of converting the original product could be both simplified and made more effective by producing two separate products (from the same database of course). This outcome was then a premiere point in designing the automation of the conversion process. Define And Document Special Circumstances The issue of converting existing data into digital form usually involves some type of special circumstances which may appear to be inconsequential at first glance. These circumstances, which are usually uncovered during the definition and documentation stages may also offer hidden simplification opportunities as opposed to just being another project hurdle to deal with. For this reason, its a good idea to document the circumstance so that all facts are available when planning the conversion automation. Once again revisiting the water company example, it was discovered that the Maintenance Division of the water company collected specific information from the water maps whenever changes were made. Further investigation revealed that the division maintained a full digital inventory model of the water maps (without coordinates). Suspecting an opportunity, this activity was documented for consideration in the design of automating the conversion process. In the end, the digital model, which was judiciously maintained, was incorporated into the automation design to provide a form of digital quality control of the newly converted data. Furthermore, an investigation in to the Sewer Division of the company revealed that a like database also existed for the sewer maps. Although different in structure and content, this database contained data which was also incorporated into the automation of the conversion process to automatically generate digital manuscripts of the sewer maps. II. PROJECT DESIGN Conceptualize An Automated Conversion Process. As with any GIS conversion project, the process should be designed to be the most efficient possible which meets the project specifications and the intended uses of the product. The twist here is to approach the project design from an automation perspective so that individuals who are unfamiliar with the project and whom possess lower skill levels can perform the work. This of course means attention to programmatic feasibility and the required development times involved so involving a programmer is advisable. If the proper documentation as described in Part I has been performed, the conceptual layout of this task will be far less painless since the likelihood of unknown variables surfacing is virtually eliminated. The water company for example had a particular desire to simplify the conversion process for themselves since they were unsure if the actual conversion work would be performed in-house, contracted out or be completed using some combination of the two. Consequently, armed with a plethora of information about what was going on manually and the products required, the involved programmers and analysts in the conceptual design of the conversion process. Almost unbelievably, within a matter of a few hours the conceptual design was completed with a comfortable feeling that it would actually work when designed in detail. Perform A Detailed Design Of The Conceptualized Process This task involved spelling out the specific tasks to be performed, their estimated level of effort and subsequently documenting all of it into a planned course of action. The design effort is expected to identify and remedy any shortcomings of the conceptual plan, provide the means by which revised time estimates for performing the actual conversion can be made, and create a blueprint by which the project will be performed. Once again using the water company as an example, the required funding and the actual method of project execution had yet to be fully determined. In addition, since the project was slated to be a multi-year process there was a concern about its survivability over time. Consequently, the documentation which was compiled resulted in two documents: 1) A Specifications document which detailed all the applicable GIS requirements such as database structure, contents, tolerances and accuracy, 2) A Procedures Manual which effectively was a step by step "how to" document. Finally, new time estimates for completing the project were easily developed in preparation for funding requests. III. PROJECT AUTOMATION Automate The Conversion Process Using An Intelligent Interface With a detailed blueprint of what and how to create a particular product in hand, especially a complicated product, the tasks which were identified to be automated now need to be addressed. Essentially this part consists of preparing conceptual program designs and then writing the code. Since the goal is to simplify the conversion process, programmatic automation can be extensive dependent on the particular needs of the project. Controlling the project with programs, especially one that incorporates a user interface, offers tremendous flexibility. Given the enormous power of AML to customize ArcInfo functionality, the process can be made to be largely "point and click" with built-in error checking and correction. In addition, if identified in one of the previous tasks, other digital data within an organization is now more accessible and can be used to aid the conversion process. The development of conceptual program designs is recommended. The conceptual designs are useful for previewing friendliness and functionality, and managing the progress of the individuals responsible for developing the code. Also, these design serve as the basis by which the programmers will develop their code. It is advisable that if at all possible, coding standards are employed and documentation headers are used. In inevitability, the code will require maintenance throughout the course of the project and may even be usable for subsequent maintenance. Admittedly, the front-end investment in programming can be costly, dependent on the program sophistication needed and the programmatic approach taken. However, the potential ease of project execution, accuracy and flexibility gained is unmatched by traditional conversion techniques. In addition, it is a good idea to consider the development of a User Manual for the interface. Otherwise, as turn-over during the project occurs or if the project is scaled-up, new individuals will have to be given at least some cursory training by someone as to it's proper use. Continuing the example of the water company, before any programs were written, conceptual designs were performed and submitted for approval. Once approved, a full set of programs and interfaces were developed using AML to put the majority of the conversion process "on-line" and intelligible to the technician level user . Also, as previously discussed, digital databases from other parts of the organization were incorporated into the interface and its programs. Now, individuals could perform sophisticated operations with a little guidance and training. IV. PROTOTYPE AND IMPLEMENTATION Perform A Pilot As almost all GIS based projects do, a pilot should be performed to prove the process and techniques developed before full production begins. This affords the opportunity to assess problems and the need for any final modifications. A pilot also offers the opportunity to record production times and largely refine schedules and budgeting needs. In the case of the water company, a brief pilot was executed to help measure the intended results by using a new individual unfamiliar with the conversion process and to record the times spent in executing various tasks. As expected, some changes were performed based on the users comments and some based on minor bugs. Implementation Of The Project The implementation of a conversion project at this point is really quite simple. In general, the only things needed is the equipment by which to perform the conversion and the manpower to actually do and/or supervise the project. Yes, this is an over-simplification of the implementation phase, but keep in mind everything else has already been laid out and was geared to be virtually "plug and play". In reference to the water company, they decided to implement the conversion process by initially using an on-site consultant to use their (water company) automated process and equipment. This implementation allowed the water company the means by which to negotiate a favorable labor rate, and to closely supervise the consultant and project progress. Furthermore, the company plans to reassign the equipment supplied to the consultant to in-house staff upon the project's completion. The Benefits -The specifications and production process is completely and clearly defined Both client and consultant alike know exactly what products to expect and have a good feel for the time required. In addition, complete documentation of the manual and conversion specifications and process now exists thereby acting as both a "paper trail" and answer index for the project. -The production process is completely or largely automated. Both client and consultant can use multiple and different individuals with minimal training time. The automation introduces a standard mode of operation. -The project possesses ease of scalability Since all the known processes and tasks are known, the project can accommodate production increases and decreases and their side effects more easily addressed. -Flexibility through a non-proprietary solution In the case of the client, this approach offers both the method and means by which individuals from within the organization or other consultants can quickly and accurately be productive. The client is not tied to a particular consultants operation. Consequently, the project could be completed through a number of different implementations which include: in-hose, out-sourced, or some combination of these. -Project survivability As a benefit to both the client and consultant, the emphasis on documentation and automation help to insure that the success of the project can last over time and through personnel changes. This approach greatly decreases the dependency on certain "key" individuals being around throughout the life of the conversion. -Budgeting Protection As a benefit to both the client and consultant, this approach helps offer budgeting protection through dependable estimates. The well defined process, specifications and task automation provide the means for real measurable estimates which help avoid budget "shortfalls" from mid-project changes resultant of "unforeseen" problems or changes in scope. The Downside -Expertise, time and funding Dependent on the project site, there is the potential to spend a lot of time and money before the process actually begins. Therefore, it is understandable that many organizations may not possess the time and/or proper manpower to fully document, design, create and fund the approach presented herein.