Jean Loodts

Digital Photogrammetry and ArcInfo:Link to the "Hidden" Third Dimension

Since a few years digital photogrammetric workstations (DPW) allow automatic digital elevation model (DEM) acquisition from medium to small-scale aerial photographs (1:20000 - 1:40000).

Recently, however,there has been a tremendous increase in the use of large-scale imagery (1:5000 - 1:10000). For such scales, automatic DEM acquisition fails. The present DPW software is not open to build new models. Large-scale automatic stereo correlation fails specifically when the terrain contains man-made structures (i.e buildings, bridges, etc.). Large-scale imagery shows severe distortions along linear features.

All these problems are linked to a misconception of DEMs, represented as regular arrays. In order to be able to include breaklines, we need Triangular Irregular Networks (TINs). We also need algorithms to extract linear features and algorithms to indicate the areas where correlation is necessary.

What we need in fact is an image analysis development tool which is strange enough absent in all DPWs. A macro language is lacking also.

The main paradox of the present DPWs lies in the concept of photogrammetry itself. The goal of photogrammetry is to gather data in any GIS. The present situation is that no direct links exist between GIS and DPW.

1. PROPOSITIONS: It is shown that the grid module can be used to display stereo images and stereo orthophotos. An anaglyph solution is obtained using the stack concept.It is shown that the grid module can be used to define new interest points, to extract breaklines and to perform image segmentation.It is shown that a macro language (AML) and the grid language are the tools needed to model problems in digital photogrammetry.It is shown that ArcInfo offers an ideal platform for raster-vector interactions.It is shown that Digital Photogrammetry and GIS have a common language based on spatial relationships.The Arctools environment plays a central role in managing a DPW. The view tools (under Grid or Arcplot) allow vector superposition of different coverages (point, line, etc.).We switch quickly from mono to stereoscopic or 3-D representations. Such tools are essential for a quality check process.Finally, an overview of new functionalities to be built will be listed.
2. STEREO MATCHING ON LARGE-SCALE PHOTOGRAPHS: Our approach in the stereo matching process is illustrated with a set of images.The present automatic DTM extraction software tries to correlate images such as the one in fig. 1a on a predefined set of points situated on a regular array. Image pyramids are used to speed up the process. This kind of algorithm fails on large-scale images because:- surface discontinuities cannot be described using regular grid points;- statistical correlation procedures give erroneous disparity values over homogeneous areas.When the same kind of algorithm is applied on a derived maximum gradient image (fig. 1b), much better results are obtained near discontinuities, though not systematically.The idea is to replace the regular point distribution by a well-suited point distribution. Fig 1c shows such point distribution resulting from a new morphological interest point operator.A tesselation process performed on these points gives a TIN representation of the surface (fig. 1d).Figures 1a to 1d represent the steps applied to the right component of a stereo pair. The same process is applied to the left component.A statistical correlation process is performed on the point distribution (fig. 1c); this creates a disparity map (fig. 2c) which is then converted into a link coverage. A stereo image made of fig. 1b is really helpful to check the disparity map results.Overlay of the TIN structure (fig. 1d) with image gradients allows to perform vector extraction (extraction of breaklines) on the left and right images (fig. 2a and 2b).Using the link coverage we are able to project the left vector component onto the right vector component in order to display the matching quality (fig. 2d).It is now possible to build a DTM on the basis of the disparity map (fig. 3a). Fig. 3c is the 3-D version of the image in fig. 1a; this image clearly shows correlation defects (i.e. roof edges). Part of these defects may be eliminated after a homologous feature process applied on the left and right vector component. Fig. 3b shows the effect of such a breakline improvement applied on a house. This homologous feature process represents the first topological constraint which improves the DTM quality. We are currently looking for new topological correlation operators to solve problems related to occluding features.
3. STEREO MATCHING IMPLEMENTATION UNDER ArcInfoThe stereo matching process works with 2 sets of datasets: the left and right components of a stereo pair in an epipolar projection.For each component we have a set of datasets.The most important dataset is the grid version of the stereo image, followed by a cover dataset (line and point feature) in order to build a new XYZ cover or a TIN.All other datasets linked to each component are constraints or complementary information datasets necessary for the stereomatching process. They consist of covers and grids.The vector-raster data model conversion is an essential algorithmic component to solve the stereo problem.Each stereo component may thus be seen as a GIS structure in itself:

_structure_diagram

The stack dataset, only valid for grids, is a convenient tool to display raster stereo data.In photogrammetry, we usually work with a set of images along one flight strip. Each photo in a strip belongs at least to 2 stereo pairs. This means that each dataset component is linked to at least 2 projective equations.In a stereo GIS, only 2 possibilities can solve the projection problem:a) each dataset has to be duplicated for each projection file;b) each dataset should share multiple projection files.The last solution is certainly the best in stereo GIS.Apart from the left and right components, a third component is introduced in order to fulfill the matching process and link the first two components. This last component contains a dataset part which creates and summarizes the third dimension (see fig. 3) as follows:- the stack dataset helps the stereo display in an anaglyphic way;- the DTM dataset (TIN/Lattice) is created from a stereopair. In the future, this dataset should have updating facilities (such as GRIDEDIT and ARCEDIT) and will be used to design digital monoplotters;- a link dataset is derived from the computed disparity or from a DTM in order to project the left vector component onto the right vector component (or inverse) as a tool to display and to check the quality of the matching process.Each dataset part of each component is integrated in one Arctools View structure.The last part of the third component contains a set of functionalities for stereomatching, such as:- a set of correlators (statistical and topological);- display facilities in order to validate the correlation process (i.e. profile display, scatterogramme, etc.).This functionality part is really an important part of a stereo GIS (even in GIS) just to validate the data acquisition. Compared to the present DPWs (with old-fashioned workflows) working on digital stereo images with raster-vector conversion, with correlation tools and with selective overlay of datasets, we increasingly need tools (i.e. scientific visualisation tools) to control the results of any interactive or batch functionality. This is a way to explore the environment and to interact with data.

4. PRESENT 3-D DATA STATUS UNDER ArcInfo: The existent functionalities to generate 3-D data look as follows:

Existent_functionalities_diagram

Some missing tools (?) could already solve a lot of 3-D data difficulties.Concerning the query on 3D, an equivalent function of [CELLVALUE ] in [TINVALUE ] could be interesting.In short, 2 main problems remain:- How to edit a TIN or EDIT an Arctin coverage with breaklines?- How to use profile INFO files?5. DISCUSSIONHow can we explain that a GIS such as ArcInfo has the tools to solve photogrammetric problems and that the present DPW do not? a) The AML macro language is particularly useful to design new algorithms with links to different modules and an easy display interface which allows interaction with data. b) The ArcInfo GRID module coherently solves raster-vector and vector- raster conversions. This is quite unique. In digital photogrammetry such conversions are essential if we try automated feature extraction or if we want to create computer-assisted procedures. An improvement of the IMAGEGRID function inside a spatial window could be a tool to complete raster-vector interactions. c) The grid language is not cryptic: finally a language that we can understand! There are no reasons to complicate simple concepts (except for masochists).In a time when a lot of attention is spent on new operating systems and object-oriented languages, the grid language already presents object-oriented features which could be extended to other datasets.Such an extension already exists in the Arctools environment and can be updated without any difficulties.The only defect to the grid module is its poor performance, it is not really an image processing system .d) Open GISThe main paradox of the present DPWs lies in the concept of photogrammetry. The goal of photogrammetry is to gather GIS in data. Despite claims, no links exists between GIS and DPWs. Our point of view (as shown in the different illustrations) is that we cannot solve image understanding (and digital photogrammetry belongs to image understanding) without GIS tools and spatial analysis tools.In image understanding we are obliged to detect, to locate spatial constraints, to build knowledge models. We achieve these obligations by building new datasets (see illustration on the stereo matching process). The present DPWs only use statistical correlation processes. That is why they fail on large-scale images.The improvement of stereomatching necessitates a multiplication of joining marks (points,lines, surfaces) such as shown in fig. 1c and fig. 2a. That is why we build different new datasets, derived from the original images in order to create strong spatial relationships.In an iterative method applied on a first statistical process, each dataset improves the matching quality.This is the reason why we strongly believe in topological correlators. As far we understand, the same stepwise procedure seems to occur in the human visual system.Concerning the GIS dataset part (DB data model), it is clear that some improvement is necessary (see chapter on 3-D data) and can be done. A lot of 3-D data structures can be extracted and can be linked to a relational DB. It is not necessary to build a complete 3-D topological structure.e) Possible future of DPWs with ArcInfoThe coupling of digital orthophotos (part of DPWs) and GIS works perfectly. This coupling works on a continuous cover of orthophotos in a monoplotter mode.One possible evolution of DPWs could be a transition towards digital monoplotters (such monoplotters exist) with a direct link to GIS data and GIS functionalities. This is not wishful thinking, it is reality.Multiple windows environments (such as Imageview) can be used to display stereo views, 3-D views and other information layers.Most of the time feature extraction (lines, points) concerns objects lying on a surface (DTM) which can be described by a monotonic function (z = f(x,y)).It means that for each position x,y there is only one z value. Even in case of bridges over a feature (road, river, etc.), we can describe this feature with vectors defined by a set of points before and after the bridge,just as we do for cartographic purposes. These vectors are invisible even on photos.The only exceptions to this monotonic DTM function known of are special buildings (with many Z values).With a monoplotter you can digitize features directly on one image and using the DTM or an online correlation process we directly get the X,Y,Z coordinates of the feature. The correlation process and topological operators can be controlled by different types of stereo data: the original images, transformed images, acting on raster or vector data.Draping twodimensional vectors on a monotonic surface is most of the time sufficient to get X,Y,Z coordinates.Having the orientation parameters of all images (projective equations) we can easily switch from one image to another to extract some occluding features.Feature extraction cannot be performed without a certain knowledge of the nearby spatial environment and spatial relationships. This knowledge must and can be obtained using existing data. This knowledge may be represented as different layers in a usual GIS. That is why GIS tools are needed, and why photogrammetry and especially digital photogrammetry must be completely integrated in a GIS (raster and vector GIS) and image processing (raster GIS means image processing) environment.A digital monoplotter is also the best answer in an environment where stereo images are substituted by multiple images,a possible future photogrammetric direction for large-scale photography.6.

CONCLUSIONS: Digital photogrammetry and GIS share the same problems. There is no need to reinvent the wheel... A well-tuned open system is a necessity to exchange data, to design new algorithms, to customize apllications.In this sense, opening is no more synonymous with nightmares.

7. APPENDICES:


Figure_1

Figure 1: Extract of the right component of an epipolar stereo pairUpper- left image (1a): The aerial photography to be correlated Upper-right image (1b): The maximum gradient of this imageLower-left image (1c): Interest points extracted on this area Lower-right image (1d): Triangular network built on interest points

Figure_2

Figure 2: Upper-left image (2a): Vectors extracted on the left component (breaklines)Upper-right image (2b): Vectors extracted on the right component (breaklines)Lower-left image (2c): Generated disparity map (link map)Lower-right image (2d): Projection of left vectors on right vectors with link map

Figure_3

Figure 3:Upper-left image (3a): Computed DTM without breaklinesUpper-right image (3b): 3-D representation of a house without and with breaklinesLower image (3c): 3-D representation of figure 1 without breaklines

Figure_4

Figure 4:Grid Arctools environment used for stereo matching. A new form menu is introduced in order to detect homologous features.

8. AUTHOR INFORMATIONDr. Jean Loodts can be contacted at Eurosense NV,
Nervi�rslaan 54, B-1780 Wemmel, Belgium.
Telephone: +32 (0)2 460 70 00, Fax: +32 (0)2 460 49 58
E-mail: info@eurosense.be