Generalization of Multiple Scale Maps from a Single Master Database

Abstract: This paper describes some of the work being done on map generalization at the American Automobile Association (AAA). AAA produces various map products at different scales for our members, including detailed city maps, vicinity maps, regional maps, and atlases. Our goal is to maintain all of the required geographic data in a single master database and to generate all of our map products from this one database. A closely related goal is make updates only once, directly to the master database, and to have those changes automatically appear in all derived maps the next time they are produced.

Introduction

This paper is an update to a presentation given at the 1998 Esri User Conference, "Producing Multiple Scale Maps from a Single Master Database" , available online at: http://www.Esri.com/library/userconf/proc98/PROCEED/TO350/PAP327/P327.HTM. That paper described AAA's approach to map generalization at the start of our GIS map development effort. This update describes some of the changes in our generalization philosophy, and what our current thinking is for the future of map generalization at AAA.

Objectives for GIS Map Production - Mission Statements

• Create a seamless, nationwide master database to support all map production needs
• Produce multiple map products from a single master database
• Update features only in master database, synchronize changes to all map products
• Cartographic edits only on map products, not the master database
• Yearly updates to map products will be able to re-use last year's work, and also include all changes to the master database that might affect a given map product.
• Support electronic travel-related products, such as the Internet Triptik.

Ideal - Live Generalization of product from master database

To summarize, an ideal generalization environment for us would be as follows:

• The source data would be stored in a seamless nationwide (worldwide?) database such as SDE
• A map could be generalized on the fly, in real time, in its own projection with minimal interactive work required.
• There would be no duplication of data. The map would consist of features from the master database, generalized in real time to enhance their appearance or behavior on the map.
• The user would be able to make adjustments to the output of generalization to add, remove, or otherwise change the appearance of objects as needed.

We find that it is extremely difficult to create maps in different scales and projections directly out of the master database without duplicating data. At least we have not found a way to do it that works for a wide range of scales. Someday processing speeds and new functionality will allow us to achieve this goal of not duplicating data. In the meantime, we will have to develop a synchronization program to propagate changes that are made at the master level to all of the relevant product layers.

Evolution of our Approach to Generalization

 From ArcStorm Visibility files to SDE Product Layers

The performance of selection sets and Mapprojection under ArcStorm was tolerable with UNIX Arcinfo 7.2.1. When we switched to Informix SDE and UNIX Arcinfo 8.0, Mapprojection's performance was noticeably slower, and deemed unacceptable by our cartographers and developers alike. We decided to shift our approach to generalization.

In order to improve the performance of the system with SDE, we did away with the old Visibility files and Mapprojection. Instead, we took the visibility file concept a step further. The visibility files used to be plain INFO files with one column - the unique ID of each feature in a map product. That was all we needed to keep track of which features in the master database should appear on a particular product. We decided to add a shape column, essentially turning the visibility file into a coverage or shapefile. Since we now had an actual coverage we could manipulate, we applied the product projection directly to the extracted data, doing away with the Mapprojection command, and this improved performance tremendously.

What we ended up with, instead of visibility files, was product layers in SDE for each map product. Cartographic displacements and other generalization processing could be done directly on the product layers. We decided to keep all attribution on the master database only, in order to prevent users from accidentally making master-level changes to the product layers.

Right now, product layers are just the shape, a unique id, and a symbol item; not much different than a CAD file. We are giving some thought to leaving the full attribution on them. This would give us greater flexibility and control over the appearance of features in a map product. Another advantage subsequent map revisions would be able to see exactly what was on the old map, including underlying attributes. We are still debating the merits of this change. The concern is that it will be difficult to coordinate edits to the master and product layers. We would have to develop some sort of synchronization program to propagate changes from the master database down to the product layers.

Ramping up to Generalization

Our first prototypes using GIS software (UNIX Arcinfo) were the Alabama/Georgia State maps, and the Atlanta CitiMap. Based on these early prototypes, we decided to focus our development efforts on creating a system for CitiMap production. The principal advantage of doing this was that CitiMaps were much closer to our source data scale, and therefore had fewer generalization issues. We used nearly all of the features in our source database for the city scale maps. Our generalization needs consisted of:

• Centerlines - Creating a centerline network out of divided roads, such as Interstates, which are digitized in both directions of travel.
• Weeding out features of interest (FOIs). Initially this was done manually, but we now have Automated Extraction routines to pre-select layers for each map type.
• Maplex for Text Placement.
• Displacing features as needed to ensure that they do not obscure each other. This is similar to the problem of text placement - accommodating lots of information on a map, preserving spatial relationships, yet moving things around a bit to better accommodate all the information.

Centerline Layer

Although Arcinfo now has a CENTERLINE command which will create centerlines out of separately digitized directions of travel, this command was not available when we created our centerline layer. We contracted this work out, and received a centerline layer that required some interactive cleanup work. Most of the cleanup was at intersections, where centerlines sometimes went astray. Also, we did considerable work to create ramp extensions, so that existing rampwork would hook up to the new centerlines.

I want to emphasize that we did not use Arcinfo's CENTERLINE command, so these comments about cleanup may not be applicable to the CENTERLINE command.
 
 

	Figure 1. Fixing collapsed roads at intersections
	Figure 2. "T"-ing off intersections where doubly digitized roads meet
	Figure 3. Extending Rampwork (1,2) to meet up with centerlines (3, 4)

 

Automated Extraction (Pre-selection)

Feature Matrix

Turning the Feature Matrix into Extraction Rules

One of our senior cartographers was a key member in developing the Feature Matrix, and took on the task of writing the precise Extraction Rules. Because he helped develop the matrix, he had a good understanding of the logic behind why features occur on each map type.

We wrote the extraction rules in pseudo code, in such a way that a programmer could easily write code from it in whichever platform we ended up using (UNIX or NT). As a result of this effort, we created one set of "Extraction Rules" for each map series. Each set of documents listed all the extraction logic for up to 7 possible inset types per map, for each of 11 possible GIS layers.

Turning the Extraction Rules into Program Code

During this coding phase, a programmer took the cartographer's Extraction Rules and translated them into AML code. We added an option to the map production user interface to allow cartographers to run the Automated Extraction routines on their own, as one of the first steps in their map production process. After Automated Extraction, the layers needed for the map are ready for the cartographer to use.

Evaluation of Automated Extraction

Selection from SDE

Unfortunately, in the version of Arcinfo we are using (7.2.1), you must do the attribute selection first (LAYERQUERY), and follow that with a spatial selection (LAYERSEARCH or LAYERFILTER). Starting in 8.0, there is a LAYERSEARCH ORDER option which will allow you to change this default and do spatial selects first, but unfortunately we are still using 7.2.1 because we found that SDE draws much faster with Arcinfo 7.2.1 than with Arcinfo 8.0.

Now that Arcinfo 8.1 is final, we will have to evaluate performance again (comparing 8.1 and 7.2.1). If Arcinfo 8.1 draws SDE as fast as Arcinfo 7.2.1, then we would be able to get back on track with the latest version of Arcinfo and take advantage of the LAYERSEARCH ORDER command to optimize our spatial and attribute selections.

Content Management

Figure 4. Results of Automated Extraction for Tourbook Spotting map for West Palm Beach. The default extraction is then manipulated by the cartographer as needed to produce the final map. Some features need to be added from master, others removed from the product.	Figure 5. Finished Tourbook Spotting map for West Palm Beach. The default extraction has been modified as needed, the map was finished with ArcMap.

Generalization requirements for different map types

As mentioned earlier, the greater the difference between the source scale and the product scale, the more generalization work there is to be done. Our experience with generalization is a direct result of the types of maps we have developed, and the chronological order in which we have produced them. If we started our GIS map production system with smaller scale maps like the Regional maps rather than the large scale maps like the city maps, we might be saying different things today about generalization.

Sheet Map production

We have recently started producing smaller scale Vicinity, State, and Regional maps, and are now encountering many more "opportunities" for generalization. Currently some generalization is being done manually, but we intend to automate as much as we can once we have identified the operations needed. We are still compiling information on the generalization requirements for these map products.

As the scale of the product becomes smaller, it is much more likely that multiple features will occupy the same space, and will need to be reconciled to be clearly visible. Some features would be lost completely without generalization. The only way to convey all the information is to abstract and generalize features, compromising real world locations for legibility.

Our Regional maps present new generalization challenges. The Southeastern States map is easily one third of the country. Most of the visitors to the southeast, particularly those who drive there, are from the northeast, and some from places as far away as Chicago and Canada. In order to accommodate these travelers, the back panel of our Southeastern States Regional Planning map extends as far north as Chicago, Detroit, Windsor in Ontario, and New York.

We are still investigating the generalization issues for producing Regional maps of this scale from a detailed, street-level database. The main challenge will be the enormous volume of data to filter.

Tourbook Map production with ArcMap

We needed a system that could make a large number of small maps easily and quickly. Rather than add more custom functionality to the UNIX Arcinfo system, which is no longer evolving as a product, we decided to see if the new ArcGIS 8.1 desktop tools could be used to create Tourbook maps. The desktop tools have a lot of built-in functionality that appear to surpass the custom tools we developed for UNIX.

Because the Tourbook maps cover a very wide range of scales, this presents an opportunity for us to make a lot of progress on our automated generalization procedures. The Tourbook map production system is in its early stages of development, so many issues related to generalization on the desktop are still being explored.

We are going ahead with cartographic production using ArcMap, but we are going continue to run Automated Extraction for Tourbook maps on UNIX because we believe the processing speed for the extraction is better on UNIX than with ArcMap. At least this was our conclusion during the ArcGIS 8.1 Beta period. Now that Arcinfo 8.1 is final, we will have to reassess, and see if our Automated Extraction can be done efficiently on the desktop.

We are also Beta testing the Adobe Illustrator Export from ArcMap, and the possibility of using Adobe Illustrator for finishing the Tourbook maps. Under this scenario, we would get the map as far as possible with ArcMap, and just use Adobe Illustrator to add graphic effects to the map.

Internet Triptik

The Internet Triptik relies on pre-generalized datasets for specific scale ranges. Depending on your scale, you will draw from one of a number of predefined datasets for specific scale ranges. This approach is quite different from our paper map products, and is driven by the need for fast performance over the Internet.

Generalization Issues for Various Paper Map Types

• Line simplification. We do simplify lines for the Internet Triptik layers, but we don't always simplify lines for paper products. In general, we simplify only where necessary to improve performance of displays, not necessarily for smoother appearance.

Figure 6. Line detail of Chesapeake Bay at vicinity scale, from the Washington DC Beltway and Vicinity map.

• Road Convergence. As they converge on urban areas, major roads tend to occlude each other. Due to the scale of the map and the thickness of the road line symbol at that scale, a single line could obscure a number of nearby roadways that run parallel or tangent. For example, Interstate 95 and the Florida Turnpike converge and run within a few hundred feet of each other for several miles (see Figure 7). At the state and regional scales, these roads would blend together. Until we find an automated way to do it, we will manually displace one or all roads affected on each map product.

Figure 7. Convergence of Interstate 95 and Florida Turnpike. This shows actual positions of roads without generalization.	Figure 8. Convergence - generalized on State map product.	Figure 9. Convergence - generalized on Regional map product.

• Barrier Islands. Some coastal areas of the United States have barrier islands located just offshore. Examples of these include the Florida Keys, the Outer Banks, the Jersey shore, etc. These islands are connected to each other and to the mainland by bridges and causeways. At a state or regional scale, some of the islands are so narrow that they disappear under the thickness of the highway symbol. The smaller islands have to be enlarged and displaced along with the roads that traverse them, so that they can remain visible. The examples below are from the Miami area.

Figure 10. Offshore islands in Miami area without generalization. At this scale, roads cover up the smaller islands.	Figure 11. Generalized roads and islands on the State map product.	Figure 12. Generalized roads and islands on the Regional map product. At this scale islands are very abstract.

• City Tints. Our state and regional map products show major city boundaries in yellow tint. The city boundaries sometimes need to be enlarged for the yellow tint to be visible at smaller scales. See Figure 13.
• City Line weights. Since major roads converge on cities, AAA has historically "knocked down" the symbology of roads to a thinner line weight within the city tint boundaries. This reduces clutter, and allows the roads to be displayed more clearly. See Figure 14.

Figure 13. City Tints enlarged for visibility on regional map product.	Figure 14. City line weights knocked down in urban areas. This example is for the Jacksonville area.

Updating the Database and Map Products

One approach would be to simply generate the map again. Perform all the generalization steps again on the new database to derive a new map. Who cares what the old map looked like - just create it again with current data. If map production were fully automated, this would be an acceptable course of action. However, the map production system is not fully automated. There are a number of manual, interactive steps involved in producing a map. The output of our Automated Extraction needs to be reviewed and adjusted by the cartographer. A considerable amount of research in invested in making sure that the map includes the appropriate features, that they are properly located and labeled. The index needs to be checked and verified. Since a lot of manual effort is put into map production, we would like to preserve this effort if possible, and not have to recreate it year after year.

The other approach would be to have a system of notifying the cartographer of any information which could influence the next revision of a map. The cartographer could then retrieve the previous version of the map, examine the changes recommended by the system, make any adjustments necessary, and be done with the map. The advantage of this approach is that all generalization work is preserved and does not need to be done again.

Update is an issue at the fringe of the generalization puzzle. If I had a system that could generalize a map perfectly every time, I would probably not care so much about map update - I would simply re-generate the map with new data. But since we don't currently have a fully automated map generalization system, I do want to preserve the manual, interactive effort that went into making the map in the first place, so I don't have to re-do that effort every time the map is updated. We don't want to have to treat each revision as a brand new map.

Updating the Master Database

The fact that our database is so dynamic does have an impact on how we update our map products. For example, the map I create this year will have to be updated next year. Over the course of a year, the underlying database will have changed. The problem is then, how to bring the appropriate changes that have occurred to the master database down to the product layers, bringing in the new information while at the same time preserving product-specific changes and generalization.

We are trying to deal with database update and map update as separate issues. In other words, the problem of how to preserve master level changes from one vendor release to another is different than the challenge of how to apply changes in the master database to particular map products. In this view, database update has nothing to do with the maps that were derived from the data. From this perspective, it is easy to see map update as a separate issue. We have a good workflow in place to update the database with new vendor data, while at the same time preserving AAA value-added information. We feel we have the database update problem resolved, and are now ready to focus more attention on the map update process.

Updating Map Products - preserve previous version's work while accessing latest information

The relationship between generalization and map update may seem like a bit of a stretch, but it is a very real need for us. It could be argued that it is not a generalization problem at all, but we would like to see continued research in the GIS community regarding how generalization and map update should work together.

Ideally, when it came time to update a map, the system would make any necessary additions and deletions for us. Under our current model of creating product coverages or layers, it would be useful if an automated generalization process could look for new features and remove or replace old features automatically based on Permanent ID, or by comparing last year's generalization to this year's.

Until then, we have to handle the Map Update process manually. Our current approach to map update is described below, in the hopes that future breakthroughs in the automation of generalization will make this kind of interactive work unnecessary.

Re-extract features

• Find new features in the database that should go on the map.
• Find features on the map that have been deleted from the database, determine if they should be removed from map.

Change Detection - Compare old product layers with new extracts

• Identify changed features, such as a realigned highways. In this case, the old features need to be dropped from the map and the new features added from the master database.
• Preserve cartographic displacements. If a feature is displaced for cartographic reasons, preserve that displacement, even for features that are replaced or realigned.
• Preserve other forms of generalization. If features are simplified, or typified, and they need to be replaced, the same generalization operations will have to be performed again on the replacement features.

Performance of UNIX Workstation Arcinfo and NT Desktop Arcinfo

We are just starting to work with the desktop. We know we can make maps with ArcMap once the data layers are prepared through the UNIX Automated Extraction. We have yet to get into geodatabases and modeling generalization behavior into our objects. It is also unclear to us how much generalization technology is currently available through ArcGIS, or how advances will be phased into future product releases.

As we become more comfortable with Arcinfo 8.1 desktop, and as the performance of this product improves, we may be able to do Automated Extraction, and all generalization, on the desktop.

We look forward to working with the object-oriented technology of the ArcGIS desktop. This is new technology for us, and we are particularly interested in the following capabilities:

• Building behavior into an object so that it would know how to represent itself in different products and at different scales
• Building sophisticated behavior that would consider the relationships between different objects in selecting which objects should appear on a map.
• Performance of these operations in a real-time production environment
• Can the object-oriented approach facilitate the map update process ? Smart objects that automatically update themselves in map products.
• Is ArcMap capable of producing one of our very large sheet maps ? Is there a size limitation for acceptable performance ?

Miguel Garriga

Project Manager

GIS Development & Support

1000 AAA Drive, mailstop 56

Heathrow, FL 32746

mgarriga@national.aaa.com

Geoff Baldwin

Senior Cartographer

GIS Cartography

1000 AAA Drive, mailstop 56

Heathrow, FL 32746

gbaldwin@national.aaa.com