13° EEUC '98 - Firenze

BEYOND WEB-MAPPING: CATALOGUING, PROCESSING AND DOWNLOADING ARCINFO DATA ON INTERNET

Andrea Musone
Intecs Sistemi SpA, via L. Gereschi 32, 56127 Pisa (Italy), phone +39 50 545111, fax +39 50 545200, Web:
http://www.pisa.intecs.it/, email: andream@pisa.intecs.it

Abstract

In the world of distributors and users there is the need of documenting and accessing geodata. Users have their own GIS applications (many based on Esri products), and they need to know which suitable geodata are available and where, and how to access them. Internet Map Servers don’t solve all their problems, since GIS applications often move beyond simple map making, only a few have been ported on the Internet/Web, and porting is costly. Apart of legal constraints, access to geodata is limited by the poor knowledge of data availability, the geographical distribution of users and distributors and the data stored in multiple, heterogeneous GISs. Therefore users need to locate the data, determine (through the metadata) if the data meet their application needs, and transfer the data. These needs move beyond the download of predefined export files.
It is presented the architecture of an open, ArcInfo-based, Client-Server geodata processing server on the Internet for documenting (through metadata), accessing, filtering, transforming and transferring heterogeneous geodata, via Internet. The Client, either on ArcView or Web-accessible, lets users navigate a Catalogue of metadata, with queries on geodata type, scale, dates, theme, structure and distributor name. Spatial queries are also possible (e.g. intersect). Other metadata are available, such as geo-objects, attributes and values, browse graphic, available formats and projections, plus others if needed (CEN/TC 287 or ISO/TC 211 standards).
The user can then issue a data processing request to the Server, specifying the desired geo-objects, attributes, attribute values, format, projection and clip area. The generic request, via Internet, is sent to the Server where it is parsed and, via IAC, it is translated into AML (open for other GISs languages) and executed by ArcInfo. When terminated, the URL where to retrieve the data (e.g.
ftp://ftp.pisa.intecs.it/pub/rivers.e00) is asynchronously notified to the user through email.


INTRODUCTION

Worldwide large amounts of geodata are available in an unlimited number of different forms, like:

  • raster data (e.g. satellite images, aerial photographs, scanned maps, etc.);
  • vector data ( e.g. cadastral data, thematic maps, cartography, etc.);
  • alphanumeric data (e.g. statistics, measurement data, etc.);
  • CAD data;

The data volume is increasing exponentially in amount, capacity and variety. Data are stored on heterogeneous, world-wide distributed, geographical databases (GISs). To potential data users the information on data availability and accessibility are very poor. It is getting more and more difficult to use data efficiently and bringing them within GIS applications. E.g., currently only 2% of all existing remote sensing satellite data are being used within GIS applications.
Users have their own applications (usually based on commercial GISs) which, with a few exceptions, are still proprietary, vertical and only partially open. These applications need geodata to work upon and therefore the fundamental need, until OpenGIS-compliant software become widespread, is still to "feed" said GIS applications with proper data, having the desired content, format, projection and extension.
The following set of requirements is therefore emerging, which is usually associated with terms like "towards GIS interoperability" or "towards open GISs", etc. In brief, efforts to promote geodata access focus on supporting the capability to:

  • locate the data (either on LAN or WAN);
  • determine (through metadata) if the data meet the user needs;
  • request data processing (filter, project, clip, etc.);
  • transfer the data on own computers;
  • import data in own GIS applications;

The universally accepted answer to these requirements are the "Spatial Data Infrastructures". Several and authoritative initiatives have started already, either at International level (GSDI, Global Spatial Data Infrastructure), in the US (NSDI), in Europe (ESDI) and in Italy (e.g. by AIPA). Some of these are still in progress while others have already become reality.
In all European and world countries there is emerging an almost uniform operational scenario, based on the data provider, data distributor and data user paradigm. All together they represent the "actors" involved in the production, distribution and use of geographical data. Regardless of the specific role, their needs can be further specified as follows:

  • Users need to easily import geodata from a variety of operational GIS systems into their own GIS applications; for example someone using a commercial GIS software to prepare a Town Planning Scheme of a municipality should be able to import gas facility information from one provider, natural resource information from another and so on, in order to easily combine them into his/her GIS application;
  • Users need to easily locate and import geodata from both national and international data infrastructures (data distributors), and other sources into their own GIS;
  • Users and Organizations need to be able to prepare and publish geodata they have compiled; this means making the data available to others on the network;
  • Organizations need tools for cataloguing and organizing their geographic data holdings; these catalogues hold metadata ("data about data") and references to the geographic information itself;
  • Users and Organizations want a shared policy for pricing the data use rights, avoiding cost duplications;

The requirements of users having own GIS applications can't be tackled easily with the Web mapping technology. Though powerful and innovative (Esri solutions first) it is still unconceivable that more than a limited part of the current GIS applications might be ported on the Web, with users simply accessing "application servers" with their Web browser, and having available the very same functionality. This is hopefully advised soon to come (mainly thanks to OpenGIS), but not yet beyond the corner. Therefore the GIS interoperability through data export and transfer is still expected to last for a while.

GIS BROKER DEMONSTRATOR

With the introduction of powerful desktop computers, corporate servers, the Internet and Intranet technology, the Web, low-cost desktop GIS viewers (e.g. ArcView) and the like, the requirements above can become implementation reality.
With the partial contribution of the CEC, via ESPRIT project GEO2DIS and Telematics project GeoMed, it has been built a GIS Broker in Client-Server architecture. It consists of an open Client-Server geodata processing server on the Web for documenting (through metadata), accessing, filtering, clipping, projecting, transforming and transferring via Internet heterogeneous geodata, hosted in heterogeneous GISs (see the simplified diagram in fig.1).


fig.1
Generic architecture of the GIS Broker and the Data Servers

The user invokes the Client, generically identified as Data User Tool (DUT), in order to discover suitable geodata for his/her application. The Client is available either on ArcView or on the Web (via a mere browser) and on any Unix/PC platform. The Client lets users interactively navigate a Catalogue of metadata by formulating alphanumeric queries on these items:

  • geodata type (vector, raster, etc.);
  • equivalent scale;
  • production and last update date;
  • theme;
  • structure (simple, dataset series, etc.);
  • name of distributor and producer;

Spatial queries are also possible on a displayed Background Cartography of the concerned area, i.e. intersect & "fully within" operators between an arbitrarily user drawn Area Of Interest (AOI) and the geodata spatial extensions (alias "footprints"). Note that footprints in the Catalogue are stored in their source shape, and not rounded to their bounding box. In reply to a query the user receives the successful hits as full metadata records. These comprise the items above plus other metadata elements, such as geographical objects (e.g. lines), attributes and their values, browse graphic, available formats and projections (one-of many), and other elements where needed (e.g. for compliance with CEN/TC 287 or ISO/TC 211 standards). The user can then display the records at will and see their spatial extension on the Background Cartography.
The Catalogue is either stored on the local filesystem (or somewhere else on the LAN) and accessible with the ArcView Client, or on the Web and simply accessible with a Web browser. The two Catalogue databases are different but with the same contents and but fully interoperable through an open interchange format (metadata export files). The GUI of the two Clients have been made very similar (see fig. 2).
Usually the Catalogue is on the same LAN where it is hosted the geodata server (generically identified as DDT, Data Distribution Tool), and documenting its data holdings. However the architecture is flexible to separate the two elements and thus to fulfill arbitrary organizational needs: various Catalogues containing metadata of datasets hosted and distributed by other Data Distribution Tools (the metadata record contains the address of the DDT hosting the dataset).
Typically users with a Web browser access the Catalogue of a distributor. Then they can either navigate and query the on-line Catalogue (with the functionality seen above) or, in case that performances and specific spatial queries are required (e.g. buffering), download the whole or part of the Catalogue and use the ArcView-based Client. With the metadata import/export mechanism a given distributor, or user locally, may have metadata of any other data distributor.
Once chosen the desired dataset in the Catalogue, the user can issue a data extraction request defining these further restrictions:

  • geo-objects (default: all);
  • attributes (default: all);
  • attribute values (default: no restrictions);
  • format (default: source);
  • projection (default: source);
  • clip area (default: full extension);

The Catalogue contains, for each dataset, the controlled list of available formats (e.g. E00, SDTS, DXF) and projections (e.g. UTM, Gauss-Boaga). The generic request, via Internet, is sent to the Data Distribution Tool hosting the desired data. The Server parses the request (stated in GeoQL, extension of ODMG '93 OQL), and translates it onto the specific language of the GIS hosting the data (through programs called Transformers). The specific request is then executed by the involved GIS and, when the process terminates, the Server asynchronously returns to the user, through electronic mail, the address of the produced data (e.g. ftp://ftp.pisa.intecs.it/pub/rivers.e00). The user can then retrieve the data and use it in his/her GIS application.
Two Transformers have been developed: one for the O2 DBMS, by O2 Technology (F), and the other for ArcInfo. This last Transformer has been implemented exploiting the IAC (Inter-Application Communication) functionality of ArcInfo. Through IAC, ArcInfo is run in "server mode" and, as such, it does not lock a license. Upon arrival, the Client request is queued and served only when the ArcInfo possible interactive session terminates or is idle, waiting for user input.

TOWARDS THE FUTURE

At the time of the Web implementation the ArcView Internet Map Server (AV IMS) was not available on the desired Unix platform. Therefore the Web Client has been fully implemented in Java, exploiting a Java applet accessing the Catalogue (hosted on Informix Dynamic Server) via a JDBC-compliant API. It is under design the porting of the Web Client on AV IMS, in turn accessing the unified Esri SDE and Informix Spatial DataBlade technologies.


fig.2
Graphical User Interface (GUI) of the Client


Go to the introductionGo to the conference programmeGo to the presentation by authorGo to the presentation by categoryGo to the Poster session
Go to the list of exhibitorGo to the novalities in Esri productsGo to the credits

Back to home page

[Introduction] [Conference programme] [Presentation by authors] [Presentation by category]
[
Poster session] [List of european Esri distributors] [List of exhibitor] [Esri products news]
[
Credits]