Seamless Integration of GIS and Transportation Modeling Software: MWCOG's Approach

Charlene Howard

This paper explains how the Metropolitan Washington Council of Governments (MWCOG) Department of Transportation Planning is using the traditional Esri coverage model and workstation version of ArcInfo to improve the relationship between its GIS and transportation planning software. Ultimately, this multi-part project will provide MWCOG with a comprehensive spatial database for transportation applications. Most importantly, the master transportation network, used for several transportation planning activities, will be wholly maintained in a GIS environment. Customized menus and tools will allow network users to easily edit networks, select a network by year, and import the selected network into COG’s transportation planning software package.

Introduction and Background

The Washington Metropolitan Council of Governments (COG) Department of Transportation Planning (DTP) provides staff support to the National Capital Region Transportation Planning Board (TPB), the designated metropolitan planning organization for the Washington, DC region. The primary responsibilities of the TPB, and hence the DTP staff, are to develop a Transportation Improvement Program (covering six years) and a Long-Range Plan of at least 20 years. The COG master transportation network, a collection of spatial and attribute data, provides the necessary base for the creation of these two products. The master network includes both highway and transit links that represent the historical, present, and future transportation infrastructure for the region.

COG uses both GIS and transportation planning software packages to edit and maintain the master network. Over the years staff has progressively worked on improving the relationship between these two related yet quite different software types, striving to eventually create a single relational database containing all relevant data for transportation planning activities. Prior to the start of this project, the most notable progress has been made on the challenging process of extracting transportation planning data from the MINUTP travel demand software package and utilizing it in a spatially accurate ArcInfo GIS environment.

The master network, base year 1994, contains all possible links (A-B node combination)— historical, present, and future. The links, however, were edited to represent the link in its most disaggregate form, meaning that if an original link was split, that node and the new A-B node pair ID would be present. If a user created a network year prior to the split, the link would still be split, but the two links created from the original link would contain the attributes for the original link. The master network coverage itself contains base year attributes and future features, with changes to the links over the years kept in a large change file, historically in Excel spreadsheet format. To create an ‘any year network,’ a series of AML routines join the selected year’s link attributes to the links in the master network to produce a new coverage for the specified year. A program written in SAS then transforms the new coverage into an ASCII file formatted to input into MINUTP.

Project Description

The overall goal of this project was to allow users to perform all of their spatial and attribute editing in one place. The former practice requires users to complete spatial editing in one environment, ArcInfo, and attribute editing in another, either MS Excel or MINUTP transportation modeling software, depending on the type of editing being performed. The newly developed processes make the following improvements to the system of master network maintenance:

- eliminate duplication: coordinate spatial and attribute editing
- creation of true 'any-year' networks
- creation of transit network link data in GIS

The current workspace version of ArcInfo was chosen to develop these improvements for a number of reasons. With timeliness a consideration, it was more efficient to use a familiar software environment to get the work done, and the coverage data model provided a comfortable data structure for both the GIS staff and the intended application users, COG transportation engineering staff. Instead of totally recreating the GIS- travel modeling integration process in an entirely new data model, this project proposal sought to incrementally provide enhancements to the current process. After careful consideration, it was decided that this could be better achieved by utilizing the more familiar ArcInfo work environment. In addition, at the time this project began, the coverage data model provided greater support for transportation network and route building. Lastly, the multiple-platform nature of the COG computer network does not permit a system-wide upgrade to ArcGIS at this time.

The improved relationships between GIS and transportation planning activities have also been enhanced by COG’s upgrade from the MINUTP transportation planning package to TP+. Compared with MINUTP, the TP+ software has more robust memory capabilities (for transportation analysis zone processing) and a direct format import/export utility for Esri shapefiles.

Simply put, the initial project proposal had three main objectives—eliminate duplicate work effort and coordinate spatial and attribute data, create more accurate network representations, and integrate transit link data into the master transportation network. These objectives were accomplished through the development of simple editing tools, spatial data reorganization, attribute data reorganization, and routines to facilitate data transfer from a GIS to transportation planning environment.

Before the tools could be developed, however, some spatial and attribute file cleanup was necessary. The original master network lacks a route system configuration; duplicate arcs represent each travel direction in the transportation network. Therefore, the first step in preparing a new version of the master network was to delete the duplicate arcs, and build a route system to represent directionality. The deleted links (reverse links) were also maintained in a related INFO file. In order to effectively implement all desired changes, selecting a new baseline year for the master network was necessary, and thus year 2000 now represents the base year. Staff also reorganized the coverage’s attribute table, discarding unused fields and updating outdated fields. The change file, a large list documenting every change made to a link over its lifetime, was also reorganized, and all records pertaining to pre-2000 links were discarded (since 2000 had become the new base year). Furthermore, the change file attributes were synchronized with the master network attributes to ensure all necessary fields were present. To reflect the change from a double-arc coverage to a single-arc coverage, staff also separated the change file records into base and reverse link attribute records. Doing so provided a straightforward way to query the change files against the master network coverage.

Project Components

Eliminate duplication of effort- coordinate spatial and attribute editing

To achieve the goal of editing all relevant data at the same time and in one software package, we created a series of Formedit menus and AMLs to allow for the simultaneous editing of the GIS coverage and the change file. This allows network attributes to be updated at the same time as the links themselves. Before this improvement, staff completed attribute changes (edits to the change file) independently of spatial editing. Normally the attribute edits performed in both systems are identical. However, one person/team make changes to the change file/MINUTP network and one person makes changes to the GIS coverage. Under the new plan, the same person/team can make all relevant changes. The disjointed nature of the editing process- aside from being quite labor intensive, also produced notable user-introduced error (misspelling, inconsistency in naming conventions, etc.) Additionally, the new process renders the current lengthy AML routines for checking consistency between the change file and the GIS coverage unnecessary.

Essentially, the editing that currently occurs in the ‘change file’ now instead takes place behind the scenes (transparent to the user) in a related file tied to the GIS coverage. A series of AMLs and Formedit menus allow the user to query the master network coverage to locate the appropriate records in the appropriate change file (base or reverse). Then, depending on the operation selected, the change file, represented as an INFO table, is updated using both user-supplied attributes (via a form) and script-generated values. It should be noted that these forms and programs run from the Arcedit module. Each of the editing options on the menu represent necessary edits to the change file(s) and/or the master network coverage. The first three options, change existing record, create new record from existing link, and delete a link, only pertain to the change file update. When adding or splitting a network link, both the change file(s) and the master network coverage must be updated. The menus allow the user to query the GIS coverage to find the link to update, and use the menu-driven process to change necessary attributes, add a new year for the same link, or delete a link at a specified period in time. The figure below depicts the ‘change existing record’ scenario, where a link’s attributes need to change for one reason or another (project year moved forward or backward, new capacity added, etc.). In the past, these changes would occur in MINUTP and Excel, and later be reflected in the ArcInfo coverage after a series of AMLs were run to link the change file and coverage AAT.

The improvements to the master network editing process can perhaps best be seen in the coordination of spatial and attribute data for adding and splitting network links. This next figure shows both the old and new process components for adding a link to the network. Before we developed these menus, staff had to make changes to the same master network via three different software systems- MINUTP, Excel, and ArcInfo. The new process utilizes ArcInfo for all necessary editing tasks. Now, when a user adds a link, related INFO files allow operations transparent to the user. Automating these processes reduces the chance of user-introduced error, which can affect results as the network is run through transportation modeling software.

Create true 'any year' networks

COG’s master transportation network contains all possible links for past, present, and future incarnations of the regional transportation system. In the past, creating a network year previous to the current year produced coverages that essentially have extra nodes. For example, if a road is due to split in 2005 and we want to see a 2000 version of that road, it would not be possible to view the road as one intact link, since the road has already been split to plan for the 2005 network. Therefore, to create what we call ‘true’ any year networks, the staff developed a methodology that allows networks to be created only containing the features relevant for the given network year. To achieve this, we built a base route system on the network, something that did not exist previously.

The basic methodology consists of the following steps:

- Creating a route system for a target year, by creating a route system for all arcs in the network, and selecting out the links that have had changes over the years, and deleting those changed links; the remaining features represent all arcs that have never been altered (thus have no records in the change file)

- Querying the change file to find link records to build the route system for the given year; links with a year less than or equal to the target year are selected

- Joining the base link attribute file—containing original the original configuration for each link—to the selected link records from the above step; the resulting file becomes the temporary change file upon which further operations occur

- Finding all link records in the change file that contains links to comprise the desired network year, these will be added to the base of unchanged arcs from the first step

- Filtering out the duplicate links in the change file, by year, to get the version of the link that is closest to your target year

- Using the X, Y coordinates of the node pairs of the links to build routes, appended to the initial route system created, using the appropriate link attributes contained in the change file(s)

- Converting the route system to a stand-alone ArcInfo coverage for use in a variety of applications; further conversion to Shapefile format for use in the TP+ program

It should be mentioned here that this process builds routes for both the base links and their counterparts (if any) in the related reverse link change files. The screenshot on the right below shows the route system-turned-coverage for the current master route system, depicted in the screenshot on the left. As one can see, the new route system illustrates past configuration of the master network, when some of the links in the current network did not yet exist. The new network year coverage’s AAT contains the original link attributes for the specified year.

Creation of transit network link data in GIS

The COG transportation network consists of both highway and transit (bus and rail) links. Under the previous master network editing protocol, bus transit links did not technically exist as spatial entities. Instead, network coders and modelers create ‘real’ bus links on the fly on an as-needed basis in MINUTP modeling software. The bus transit line data were contained in ASCII text files with a MINUTP-specific format so they can be read in that software.

Our task was to develop a way to transfer the ASCII files of bus routes into the master transportation network coverage. A SAS program enabled us to select out the bus route name and node numbers and place them into a separate file that would eventually become the stops file. This stops file is used to create bus routes within the master network. Another procedure extracted out all other relevant information to create an attribute file that would later be joined to the newly created routes.

The ASCII file was imported into ArcInfo and joined with the master network node attribute table (NAT) to produce the stops file. An AML program automated the route building process, using the stops file to select the nodes necessary to generate each bus route. A total of 12 bus routes were generated, 6 each for peak and off-peak travel times. The network staff members now have a simple, automated way to generate bus links whenever COG receives updated mode files from local transit agencies. The bus routes can now be edited within a GIS environment, as is the case with the rest of the master transportation network.

This project also included updating COG’s rail transit data so it would seamlessly link to the master transportation network.. At this time, the bus and highway network share a common spatial base, but the rail networks, with their separate rights-of-way, are contained in two additional coverages, representing Washington Metropolitan Transportation Authority (WMATA) Metro Rail lines and Maryland and Virginia commuter rail lines respectively.

In order to accurately relate these files to the master network, the rail transit coverages had to be modified. A member of the COG GIS staff updated the coverages’ attributes to ensure the A (from) and B (to) node numbers matched those node numbers reserved in the master network for rail link data. Staff also checked and corrected link directionality and link ID field against the corresponding attributes in the master network.

ArcInfo and TP+

As mentioned earlier, the transition from MINUTP to TP+ transportation modeling software has enabled COG GIS to take advantage of TP+’s ability to directly read Esri shapefiles. Thus, the planned method of data transfer between ArcInfo and TP+ will be through the shapefile format. The processes developed to facilitate data creation and editing will ultimately produce a shapefile. That is, when the network staff members complete their network editing exercises, they will have a shapefile version of their specified network in their workspace. The shapefile can then be imported into TP+ and used in Viper, the graphical component of TP+, for transportation modeling, forecasting, and planning operations.

In order to continue the seamless integration of GIS and transportation modeling software, we are also in the preliminary stages of developing methodology on how to post edits made to the networks in TP+ format back into an ArcInfo environment. So far, we have developed the following guidelines regarding editing in TP+, with regards to the GIS master transportation network:

- spatial editing changes will still take place within ArcInfo
- attribute-only changes can take place in either TP+ or ArcInfo
- project-based changes will be done in TP+

Basically, all spatial editing will be performed in ArcInfo. Attribute changes, so long as they do not affect the link identification attributes (ID, A or B node, etc.), can be performed directly in TP+. In these instances, an exported shapefile version of the TP+ network is related back to the master to update attribute data. Most network users will utilize a project-based copy of the master network, and thus can complete most transportation planning applications in TP+/Viper.

Final Thoughts/Future Steps

While we have made significant progress in providing seamless links between our GIS and transportation modeling software, the work continues. Current projects either in development or under consideration include a stand-alone application for generating a file for specific network year, automated procedures for accepting attribute edits directly from TP+ into ArcInfo, a menu-based system for editing the transit links (bus and rail) in a GIS environment, and procedures for exporting transit link data in a format for use in TP+.

COG’s future plans include the development of an enterprise-wade solution for the management of spatial data throughout the organization. A migration plan is in progress that will guide COG from the current file-based data management structure to a more robust relational database management structure utilizing the GeoDatabase with ArcSDE and Oracle. Eventually, the entire organization will upgrade to ArcGIS to take full advantage of this stronger database model. The overall goal is for COG’s GIS staff to continue to provide the rest of the organization with the most timely, accurate, and accessible spatial data possible, while utilizing the best available technology for each particular task.

Acknowledgement

I would like to acknowledge fellow COG GIS staff members Martha Kile and Yew Yuan for their assistance with this project.

For more information, contact

Charlene Howard
Department of Transportation Planning
Metropolitan Washington Council of Governments
777 North Capitol Street, NE, Suite 300
Washington, DC 20002-4239
202.962.3384
202.962.3202 fax
charlene@mwcog.org
check out MWCOG's website