Using the
Geodatabase with Pipeline Data Models
By Ron Brush, New Century Software, Inc.
ABSTRACT
PODS (Pipeline Open Data Standard) is an emerging standard data model in the pipeline industry that can be tightly integrated with the ArcSDE geodatabase. This technical presentation will describe ways that PODS linear referencing can be used with ArcGIS route events in a geodatabase. This presentation will discuss challenges unique to the pipeline industry such as centerline maintenance, handling station equations, and managing event tables. In addition, this presentation will discuss geodatabase versioning and the applications of geometric networks. This presentation will include examples from current pipeline GIS projects.
Keywords: Pipeline Open Data Standard, PODS™, database, pipeline, geodatabase
Web Site: www.pods.org
By Ron Brush, President, New Century Software, Inc.
In the pipeline industry, many companies are using Geographic Information Technology (GIT) or Geographic Information Systems (GIS) to help manage pipeline facility and operational data. Some companies have independently developed proprietary GIS databases and software applications at significant cost and at significant risk. However several failures of early pipeline GIS implementations continue to make some companies leery of GIS in general.
Early adopters are starting to look at the Esri Geodatabase to store their pipeline attribute data and make it accessible on an enterprise level. While a few companies are making forward steps, most operators are not familiar with the geodatabase and the benefits that using it offers. So what is a geodatabase as opposed to a conventional database and what are the advantages? How can pipeline operators take advantage of the ArcGIS geodatabase?
This paper will address implications for pipeline operators wishing to implement a geodatabase. This discussion will focus primarily on the PODS (Pipeline Open Data Standard) data model.
PODS (Pipeline Open Data Standard)
In 1998, GRI (Gas Research Institute) led the formation of a new initiative for a standard pipeline data model. This model was developed out of earlier work developed under the GRI ISAT (Integrated Spatial Analysis Techniques) project. Several goals were identified in the formative meetings:
·
Development of a pipeline industry association to
oversee model development
·
GIS independence – it should not be developed for any
specific GIS software
·
Enterprise-wide solution – going beyond GIS
·
Improved documentation and training workshops
·
Improved performance and data dictionary depth over ISAT
·
Development of a true industry standard with many vendor
applications
·
Improved communications – web site and published CD
·
Improved versioning scheme for future releases
From these goals, the Gas Research Institute led pipeline industry operators and vendors to voluntarily develop a new database model referred to as the Pipeline Open Database Standard (PODS). The PODS database is designed to be an open data model in that PODS Assocation members each contribute to model development and approve changes.
The PODS Association, Inc was incorporated in August 2000 and currently has 25 member companies. The data model and direction is driven entirely by the volunteer contributions of members. Annual membership cost is US$1000 per company, regardless of size. Operating companies, service providers and data providers are encouraged to join and contribute. Association members and officers can be found on the PODS web site www.pods.org.
PODS is a standard normalized relational database schema. It provides the ability to store geographic location information in regular database tables. It is GIS neutral and can be used with an Esri geodatabase. Before discussing how PODS can be used with ArcGIS, it is important to know what a geodatabase is.
Two Geodatabase Types
An ArcGIS Personal geodatabase is stored in Microsoft Access 2000. A simplistic description is that a personal geodatabase stores shape files in an Access database. Using ArcCatalog, shape files can be easily imported into a personal geodatabase. It is perfect for one user or a very small group of users. A personal geodatabase can be accessed and maintained by the less expensive ArcView version of ArcMap, and everything but versioning is supported. So, for pipelines up to about 500 or 1000 miles, a personal geodatabase may be something to consider for a small mapping group.
The ArcSDE (Enterprise) geodatabase is stored in an enterprise
database such as Oracle or SQL Server. Larger pipelines around 5000 miles and
up, who want multi-user or web access to data should consider using ArcSDE. Two
important reasons for this are performance and scalability. ArcSDE has the
ability to serve graphics to many users at a time. It can also grow with the
addition of new layers without a significant performance hit. Shape files and
other graphic files are stored in ArcSDE, but ArcSDE goes well beyond the
ability to just store graphics.
Using ArcSDE vs. using a
Geodatabase.
Prior to ArcSDE 8, SDE 3.x was commonly used as an enterprise repository for graphic files – shape files, and coverages. The main advantage of SDE v.3 was performance since SDE is optimized to find spatial data very quickly. It is important to note that some companies still use ArcSDE 8 to only store graphics and do not take full advantage of the geodatabase features of ArcSDE. Thus, just because a company is using ArcSDE 8, does not mean that they are using the geodatabase features.
Should I use a geodatabase?
There are several advantages to the geodatabase that pipeline companies should consider.
ArcSDE geodatabases support the ability to edit different versions of the database. An example of this is creating a new ‘version’ of the data when entering a new project or job. First the user creates a new version and gives it a name. The version name might be a job or work order number, or it could simply be a user name. Then in the edit session, the user edits reference that named version so that all data edits are linked together. Versions can be public private—only the owner can access private versions. Normally other database users would not ever see these versioned edits until they are posted back to the main (default) data set. The major benefit is that these versions can span days or weeks. This would allow a new construction project to be added to the database over a course of days or weeks. Only the person making the edits would see the new pipeline until it is posted. Versioning is sometimes referred to as a long transaction.
Another advantage of the geodatabase is the ability to build a connected network. Most GIS systems store graphics so that pipelines appear to be connected, but there is no logical relationship or link between them. A geometric network allows a user to perform a trace and see which pipelines are connected to each other, or which areas are under the same pressure regulation, or even to simplistically simulate product flow. Valves, taps and tees are represented as nodes and can be opened and closed. As the network is edited connectivity is maintained.
Similar to CAD, text can be stored with in the geodatabase. Annotation can be linked to graphics or independent as in CAD. Annotation layers are great for labeling pipeline groups in close proximity such as you would find in a gathering or distribution system.
In order to use a geodatabase with PODS it is important to understand how PODS linear referencing works.
Since stationing is integral to most pipelines, PODS supports linear referencing as part of the core data structure. PODS uses the Esri routes and measures scheme which allows
stationing to be equated to measures on the pipeline. Two tables in PODS store this data. The Route table stores a continuous section of pipe – called a route. The Station_Point table stores pipeline stations and their equivalent measure.
Thus each point on the pipeline network has a surveyed station and a calculated measure—together the route and the measure uniquely identify any point on the pipeline network. The measure is managed by the database itself. This is important, as a pipeline geodatabase implementation must support linear referencing.
Geometric networks also support linear referencing if the underlying centerline (polyline) geometry is measure aware. Complex edges work best since the pipe route does not need to be broken at pipeline tie-ins.
Making it easy for the user to do a reroute, to add or remove pipe from the system, and to update the centerline based on GPS data are some of the challenges facing pipeline operators. A reroute is an example of where versioning may be useful. A user could make a version of the database that represents the pipeline state after the completion of the reroute. This may include cutting a capping pipe, laying the new centerline and adding station equations. A versioned database would allow these many edits to be performed without affecting on-line users. When the reroute changes are ready, the named version can be posted to the main (default) database.
One challenge associated with versioning is that geodatabase delta tables are used to store the additions and changes. Delta tables store edits until they are reconciled and posted back to the default version. This makes it difficult to use database triggers to automatically manipulate data. Centerline and lifecycle state management are important issues for pipeline GIS users.
In PODS the centerline coordinates are stored in the Coordinate table. When using a geodatabase, it is important to decide if the Coordinate table will continue to be populated, because it may redundantly store the centerline geometry. In addition, PODS provides for the potential use of multiple geometries (locations) for a centerline. This becomes more challenging in a geodatabase, as one would normally represent multiple geometries with multiple centerline feature classes.
When station equations are introduced during a reroute, the underlying measures of the (measured) polyline need to be adjusted. It is important that the measures in the centerline layer correspond with the measures stored in the PODS database. This referential integrity cannot be easily enforced by the database, so it must be handled by the centerline maintenance application.
One advantage of PODS is that all events are handled the same way. Point and linear event tables (business tables) are all linked to the Event_Range table. Example event tables include the Pipe_Segment, Valve, Casing, External_Coating, Foreign_Line_Crossing. In addition to centerline maintenance, another important application is simply the ability to easily and quickly add, edit and delete data from these attribute tables. This application ensures that only valid data can be entered into the system, according to prescribed business logic.
Building conventional referential integrity into the data model is difficult if each of these event tables is a versioned geodatabase layer. This is because of the aforementioned delta tables. This problem is potentially solved through relationship classes in the geodatabase.
One area where many companies are reticent about the geodatabase is in the area of application development and integration. Because only Esri software can be used to manage the geodatabase, integration with conventional client-server applications or third-party applications can be more difficult. For example, an application developer must use either ArcObjects or the SDE C or Java API to insert records or change data in a versioned layer. Also querying the geodatabase becomes more challenging with the geodatabase delta tables. Thus, not only are ArcSDE licenses required to perform these operations, it is important to make the necessary investment in developer training and possible application migration.
Everything that it takes to have a successful enterprise GIS prior to the geodatabase applies to a geodatabase implementation as well: high quality data, a good support and I.T. infrastructure, training, good workflows and so on. In addition, there will likely be additional application migration/development costs to support ArcObjects and get developers up to speed. The geodatabase modeling itself will likely be an iterative process. Because it can be harder to modify the geodatabase structure once in place, it is wise to be sure of the model before launching into production mode.
The present goal for the PODS Association technical committee is completion of PODS release 3, due sometime this summer or fall. It is expected that a geodatabase version of PODS will begin sometime soon. The scheduling of this is dependent upon the interest of Assocation members and the willingness of volunteers.
Building and maintaining a pipeline GIS is an investment. It is an investment in gathering and maintaining quality data, creating manageable workflows, building and supporting software and user training and support.
The ArcGIS geodatabase is a compelling approach to enterprise-level GIT. When this technology becomes commonly used, it will be hard to imagine how pipeline operators managed data without it. Yet, early adopters will face many challenges getting to a production environment as they face a steep initial learning curve.
The PODS Association exists to help develop standards that can help operators mitigate risk and improve GIS capabilities.