Bingcai Zhang

BAE SYSTEMS Mission Solutions Inc.

A. Stewart Walker and Scott B. Miller

LH Systems, LLC

GIS databases become more valuable when they are kept up to date, contain image layers, contain 3D information, and have high spatial accuracy. The Esri Spatial Database Engine (ArcSDEä ) provides an interface to integrate GIS data with non-GIS data, and puts GIS data directly under the control of a database management system. SOCET SETâ , we are introducing a 3D feature extraction system that directly links to ArcSDEä and takes advantage of ArcSDEä for real-time applications. ArcSDEä permits users to extract 3D features directly from controlled stereo imagery while inserting them into the GIS database. This system allows multiple operators to extract 3D features to the same feature database and/or update the feature database while it is being used for real-time applications, and has no limits on the feature database size. 3D features are tightly integrated with non-GIS data to maximize their value and application. This is a necessary step towards web-based applications using geospatial data and imagery.

The largest cost of managing a GIS database is creating and updating the GIS data. The simple process of converting data from hardcopy format to a digital format accounts for more than 50% of the cost of GIS operations (Stevenson, 1995). Manually digitizing GIS data from hardcopy maps is error prone and contains inherent inaccuracies. Using imagery, rather than hardcopy maps, helps GIS operations in three key ways: cost; accuracy; and currency (up-to-date). Costs are reduced because collection from imagery is efficient, accurate, and current. In addition, many processes that formerly required a human operator, such as terrain generation, are now performed by an unattended computer. Accuracy is increased because image-based data is derived from photographs of the actual ground, rather than paper maps or non up-to-date sources that can be of dubious origin. Currency is improved because imagery can be collected upon demand at relatively low cost, and the data extracted from it can be used to create or revise a GIS database (Zhang and Olander, 2000). The use of stereo imagery permits full 3D population of the GIS database with high accuracies. All of these benefits come from using computer-based photogrammetric systems, such as SOCET SETâ . The improvements over the last decade in low cost off-the-shelf computer based digital photogrammetry, has created efficient methods of photogrammetric data capture at low cost. This low cost and high productivity permits the effective use of photogrammetry as a direct GIS revision and population tool. Figure 1 illustrates the ease of connection between 3D imaging sources and GIS.

When photogrammetry is used, data generation and conversion costs can be reduced significantly. For example, the SOCET SETâ software product can generate one million terrain data points per hour (Zhang and Miller, 1997; Zhang et al. 1998). Conversion costs are minimized, because SOCET SETâ can directly edit GIS data in ArcSDEä databases without the need for any file format conversions. Accuracy is ensured, because photogrammetry employs sophisticated algorithms that compute the precise ground position (latitude, longitude, and elevation) of each pixel or point extracted from in the image.

Imagery can come from many sources. Commercial satellites, based on classified government spy technology, are now available for use by the GIS community. IKONOS, QuickBird, SPOT 5, and OrbView are three satellites that can or will collect imagery with spatial resolution from 2 to 0.5 meters. However, the most common source of imagery is still aerial based. Film based aerial cameras are very common and provide reliable high quality imagery. In addition, digital cameras are coming into widespread use and include such systems as simple frame digital cameras to more sophisticated ones Digital cameras, such as those from TerraSource and the ADS40 from LH Systems. The image sources supported by SOCET SETâ are listed in Table 1.

The most important function of a photogrammetric system is to associate, or compute, ground locations (latitude, longitude, and elevation) for selected pixels in the imagery. This computation is done by an algorithm called the "sensor model", which is unique for each type of image source.

The sensor models use stereoscopic principles, similar to the binocular vision of two-eyed animals as illustrated in Figure 2, to compute precisely the ground locations of objects in the imagery (Olander and Walker, 1998). Generally, the accuracy of data derived from the imagery is around 0.5 pixels or better. For example, if the image resolution is 6 inches per pixel, then the accuracy can be 3 inches. Because stereoscopic algorithms are used in photogrammetry, two images are required for all regions that are being processed. If there is only a single image, it is still possible to generate GIS data, but without obtaining the full benefits of photogrammetry, because the 3-dimensional characteristics of the ground cannot be extracted.

The SOCET SETâ feature extraction module uses controlled imagery to collect 3D GIS data as illustrated in Figure 3. As discussed above, digital imagery must be controlled prior to 3D feature extraction can occur. Controlled imagery is used to create a 3D view within the S SOCET SETâ stereo image server as illustrated in Figure 3. The 3D feature extraction process generates geometric data of features such as building, roads, etc. from the Stereo Image Server. This process also computes feature attributes or obtains feature attributes from user inputs and simultaneously writes geometric data and attributes out through ArcSDEä to a RDBMS.

The Esri Spatial Database Engine (ArcSDEä ) provides an interface to integrate GIS data directly with stereo imagery, and puts GIS data directly under database management system control. With SOCET SETâ , we are introducing a 3D feature extraction system that directly links to ArcSDEä and takes full advantage of ArcSDEä for real-time applications. With this system, users can extract 3D features directly from controlled stereo imagery. This system allows multiple operators to extract 3D features to the same feature database and/or update the feature database while it is being used for real-time applications, and has no limits on the feature database size. 3D features are tightly integrated with non-GIS data to maximize their value and applicability.

One of the most important applications of the GIS database is to support decision making. In many cases, GIS data alone may not be sufficient and non-GIS data are required for decision making. In this digital era, database management systems are widely used to store and manage non-GIS data. When non-GIS data and GIS data are both managed by the same database management system, decision making using non-GIS data and GIS data are becoming much easier. As a result, GIS databases can support more applications.

Real-time applications require concurrency control that is readily available from a database management system such as Oracle Spatial. ArcSDEä takes advantages of the available concurrency control capability from the database management system and SOCET SETâ 3D feature extraction uses this capability to support real-time applications by locking. Locking is available on entire layers, or on a specific geographic area within a layer. Two types of locking methods are available, read or write lock. Read lock prevents any modifications on the locked area from any user (features here cannot be edited or updated), and write lock allows modifications only from a specific user (other users may perform reads on the data). If a read lock is imposed on a set of features, then a write lock, in turn, cannot be imposed on this set of features and vice versa. This locking capability allows multiple users to extract 3D GIS data to the same ArcSDEä database at the same time. For example, user A extracts 3D data with stereo models 1-10, user B extracts 3D data with stereo models 11-20 in the same project. This locking capability also allows other users such as the fire department to access the ArcSDEä database while at the same time as it is being updated.

Data recovery is another readily available capability when using the ArcSDEä interface. Most database management systems include this capability, for example Oracle Spatial. When computers crash or power is lost, data recovery comes to the rescue. By using an online connection to the RDBMS, data is rarely lost. When something unexpected happens, data recovery can recover the 3D data that has been extracted.

Unlike most packages that store feature data into a file system and load the entire file from disk to memory such as ArcInfo coverage files, shapefiles, or SOCET SETâ 3D topological feature database files, the ArcSDEä interface does not load the entire database into memory. Instead, the database management system provides paging. Only required records are retrieved from the database on disk to memory. Therefore, the database size is only limited by the amount hard disk space, which is normally much larger than the computer’s main memory. Thus, the database size is virtually unlimited.

SOCET SETâ and ArcSDEä support a large number of coordinate systems. However, some of these are not identical in both systems. Some SOCET SETâ coordinate systems are not supported by ArcSDEä and vice versa. To resolve this problem, we need to perform coordinate system transformations as illustrated in Figure 4. We firstly convert all coordinates to a commonly supported system (WGS-84 Geographic) before converting to a SOCET SETâ project coordinate or to an ArcSDEä coordinate. This approach is independent of the projection/datum subset or vertical reference. This way, SOCET SETâ will convert any of its supported coordinate systems to a common format that ArcSDEä can easily convert back to any of its supported coordinate systems and vice versa.

This approach has the following advantages:

1. Independent of the coordinate systems, projections, datum, or vertical references supported in either SOCET SETâ or ArcSDEä Projection Engine.
2. Less software needed since Geo/WGS-84/Ellipsoid is a pre-defined coordinate system in PE (as opposed to converting a PE coordinate system to a SOCET SETâ coordinate system and vice versa).
3. Maintainability: if any new coordinate systems or projections are added to either SOCET SETâ or ArcSDEä PE, no code will need to be changed in these routines.

A SOCET SETâ feature database specification is defined in an ASCII file called the extraction specification. The extraction specification defines the kinds (classes) of features that are permissible in the feature database, and defines the types of attributes associated with each class of feature. This file is always a part of a feature database. ArcSDEä does not use the SOCET SETâ extraction specification. Instead, ArcSDEä uses layers that are equivalent to SOCET SETâ classes.

SOCET SET users have the option of selecting a set of ArcSDEä layers to load into the 3D feature extraction. The extraction specification can be built on-the-fly using these selected layers. Alternatively, these existing layers may be matched with class definitions from an existing extraction specification. In addition, the user can create a new set of ArcSDEä layers by selecting one or more classes from an existing extraction specification. An alternative to continually selecting the same groups of layers for each editing session is to have a profile that contains a list of layers to always be edited at the same time. This is analogous to having multiple classes in a single SOCET SETâ feature database. This data file (profile) resides in the project data directory and contains information that allows a set of ArcSDEä layers to function as an extraction specification. This file contains information such as ArcSDEä layer names and bounding rectangles (for feature locking or query limiting). Upon selection of this file in the 3D feature extraction, an existing extraction specification mentioned in this file may be used to match with the layers. Alternatively, an extraction specification with all these layers could be generated on-the-fly.

When creating an ArcSDEä layer, a bounding rectangle is important for specifying the extent of the layer (map extent). If expansive ArcSDEä layers are to be queried, but the area of interest is only a portion of that area, then a bounding rectangle would be useful to limit the query to that specific area. A bounding rectangle is also important for feature locking. We can lock features based on layer, area, or both. This can prevent more than one user from editing specific features. (Locking based on display minimum bounding rectangle or individually picked features is also possible). Instead of having the operator directly specify what the Bounding Rectangle is, the 3D feature extraction calculates this area automatically by generating the MBR that consists of the union of all image support files within the project.

3D geometric data or vector data are categorized into six geometry types in SOCET SETâ as shown in Table 2. The six geometry types are then converted into five ArcSDEä shapes. POINT_GEOMETRY features have (X,Y,Z) coordinates and a set of attributes which describe the feature characteristics such as height. POINT_GEOMETRY features can be used for features that their dimensions are relatively too small to be presented as an area feature. TEXT_GEOMETRY features have (X,Y,Z) coordinates, a string, and a set of attributes. TEXT_GEOMETRY features can be used as labels for instance. LINE_GEOMETRY features have a number of vertices with (X,Y,Z) coordinates and a set of attributes. LINE_GEOMETRY features can be used for roads, rivers, etc. POLYGON_GEOMETRY features have a number of vertices with (X,Y,Z) coordinates and a set of attributes. POLYGON_GEOMRTY features can be used for area features such as lakes, lot boundaries, etc. A MULTILINE_GEOMETRY feature may have one or more LINE_GEOMETRY feature as its elements. MULTILINE_GEOMETRY can have a set of feature attributes as well as a set of element attributes. For example, to represent highway I-15, we may need a set of feature attributes such as Name, and a set of element attributes such as surface type, number of lanes, etc. For highway I-15, the Name is the same for all its elements/sections, but the surface type, number of lanes may be different from element/section to element/section. POLYHEDRON_GEOMETRY features may have a number of POLYGON_GEOMETRY features as their elements. POLYHEDRON_GEOMETRY features are especially useful for 3D volumetric features such as buildings and houses. For 3D visualizations, each volumetric feature may attach an image patch to its polygon facet. This information is stored in ArcSDEä as blob type.

Table 2. SOCET SETâ geometry types versus ArcSDEä Shapes.

SOCET SETâ 3D feature extraction provides a set of fence tools to manipulate and query features, including (1)polygon clipping, which can click features using a polygon; (2)trim/extend, which solves dangling nodes problems; (3)linear feature intersection and stacked nodes generation, which are mainly used for linear features such as streets and highways; (4)database manipulation such as copying, deleting, moving, grouping, updating attributes on a list of features selected either by attribute query or by fence polygon selection.

SOCET SETâ 3D feature extraction provides special tools for texture patch, which is mainly used for 3D visualization. Other special tools include generic features and model placement. This tool, similar to Arc/Info 8.0 division, can extract large numbers of generic feature such as street lamps efficiently. 3D models are used for volumetric feature extraction such as buildings. Dimensional attributes can be computed on-the-fly. There are more special tools such as Volume Create etc. All tools are designed and optimized for efficient 3D GIS feature extraction from imagery.

3D geometric data are extracted by a set of sketch tools that are optimized for efficient 3D feature extraction from stereo imagery. The tools are similar to CAD/CAM/GIS tools, but have significant differences: (1)all vertices have (X,Y,Z) coordinates; (2)computer vision techniques are used to perform semi-automatic feature extraction from a pair of stereo images; (3)tools are optimized to work in a stereoscopic view environment. In a stereoscopic view environment, users may become tired quickly because their eyes have to focus and refocus frequently. The sketch tools must be as efficient as possible in order to reduce the amount time required for users to extract vector data. Based on many years of experiences and customer feedback, we recently re-engineered a new sketch tools which provide a set of efficient tools specifically designed for feature extraction from stereo images. There is also a set of semi-automatic vector data extraction tools based on computer vision techniques.

Attributes are facts about a feature, such as its size, purpose, age, condition, parcel number, material, history, or address. In many GIS applications, attributes are just as important as the geometry (location and shape) of the feature. Attributes suffer from the same accuracy and currency problems as the geometric data. Using imagery can ensure that attributes are accurate and up-to-date. SOCET SETâ automatically computes size attributes (area and length), and other attributes such as condition, material, and purpose can be readily ascertained by simply interpreting the imagery.

Eight dimensional attributes can be computed.

1. Area of a polygon feature computed in 2D space;
2. Length of best fitting rectangle of minimal area around the feature;
3. Length along center line of feature;
4. Width of best fitting rectangle of minimal area around the feature;
5. Width based on feature area;
6. Maximum Z-elevation (above mean sea level) of the feature;
7. Height of the feature above the terrain;
8. Angle of orientation measured clockwise from North in degrees.

Dimensional attributes computed in SOCET SETâ are preserved. ArcSDEä can compute some dimensional attributes such as area and perimeter. However, we use the dimensional attributes computed in SOCET SETâ because SOCET SETâ computes more than just area and perimeter.

The enumerated attribute types are very useful to guide and help users select the correct attributes. This type of attribute is very important for NIMA outsourcing and for SOCET SETâ feature extraction in general. Enumerated attribute types are preserved by ArcSDEä , though ArcSDEä does not directly support the enumerated type. Users are able to treat enumerated types just as they treat them without ArcSDEä . Our approach is to store the SOCET SETâ Specification file (ASCII file) into ArcSDEä and each enumerated attribute as an integer. When opening a ArcSDEä database, we read in the SOCET SETâ Specification file such that we can establish the relationship between the integer value and the enumerated string.

SOCET SETâ supports complex feature types (POLYHEDRON and MULTILINE). A complex feature can have more than one element (part) and each element can have its unique attributes called element attributes. Element attributes are preserved in ArcSDEä . ArcSDEä does not have the concept of element attributes. We treat element attributes as feature attributes by adding an element_ prefix into its name in ArcSDEä . Element attributes behave the same in SOCET SETâ 3D feature extraction with or without the ArcSDEä interface.

Graphic attributes are mainly used for drawing. Available graphic attributes are: (1)foreground color; (2)background color; (3)line width; (4)line style; (5)font size; (6)font type; and (7)font style. SOCET SETâ supports up to 216 different overlay graphic colors. Graphic attributes are stored in ArcSDEä database such that each feature can have its own graphic attributes.

We observe from industry trends that web-based applications as having great potential for the geospatial information industry including digital photogrammetry. Interfacing with ArcSDEä and Oracle Spatial is a necessary step towards web-based applications for digital photogrammetry. Expanding digital photogrammetry to a technique broader than simply modeling two human eyes to establish a 3D stereo view and perform 3D measurements opens up many potential applications. For example, people are using digital photogrammetry to measure the size of a dolphin without taking it captive. Internet technology, together with web-based digital photogrammetry software, can make photogrammetry more accessible to a wide range of users and create many more new applications.

Imagery is an essential part of data for digital photogrammetry and GIS data generation. ArcSDEä and Oracle Spatial support imagery. We believe SOCET SETâ should take full advantages of ArcSDEä and Oracle Spatial and store its imagery together with their support/control data in the RDBMS. The SOCET SETâ terrain data (digital terrain models) is stored in imagery formats for GRID and in feature database formats for TIN. Once imagery is stored in ArcSDEä and Oracle Spatial, all SOCET SETâ data is under RDBMS control. This will enable SOCET SETâ to develop web-based applications for potential new markets.

Olander, N. and A.S. Walker. (1998). Modeling Sensors in Software. Proceedings of ISPRS Commission II Symposium, Cambridge, England, July.

Stevenson, P.J. (1995). The Problem of Data Conversion. Geo Info Systems, February, pp. 28-32.

Zhang, B. and S. Miller. (1997). Adaptive Automatic Terrain Extraction. Proceedings of SPIE, Volume 3072, Integrating Photogrammetric Techniques with Scene Analysis and Machine Vision (edited by D. M. McKeown, J. C. McGlone and O. Jamet). pp. 27-36.

Zhang, B., N.F. Olander, F.C. Paderes, S. Miller, and A.S. Walker. (1998). Automatic TIN Generation from Imagery. Proceedings of ASPRS-RTI annual conference.

Zhang, B. and N.F. Olander. (2000). GIS Data Extraction From Imagery. Proceedings of Esri User Conference.