Jim Ball and Ian Rodgers
ArcInfo has been used to develop an Environmental Information System for atmospheric data. The system combines the output from computer models of atmospheric stack and cooling tower plume dispersion and deposition with monitored air quality data, and third-party datasets (such as SSSI boundaries), to provide a range of information about power station stack emissions, their dispersion and their potential environmental effects. This multi-user application includes custom grid and contour visualisation, automatic context-sensitive legend generation and complete "loading" and "saving" of all displays. It uses AML, C, AWK and UNIX shell scripts for OpenWindows or Motif environments.
National Power is the UK's largest electricity generator with about 25% of the UK market and an increasing investment in electricity generation worldwide.
Effective management of the environment is vital to the commercial success of National Power. As well as managing the existing assets with the environment in mind, it is necessary to develop the business strategy to take account of the likely direction of future legislation and technology - and the views of everyone with an interest in the business.
The National Power environmental approach is guided by five principles:
These principles require National Power to -
The NP environmental performance is backed by a team of environmental specialists who provide the underlying scientific and technical expertise.
The Environment Unit uses modelled and measured information about the environmental effects of power station operation both for development of environmental strategy and for input to site-specific Environmental Effects Evaluations and impact assessments.
Historically these modelling studies were performed using separate systems and integrated only at a later stage. The continuing drive for increased productivity motivates the search for opportunities to integrate existing systems within a single framework. This integration has several clear benefits:
This paper describes the development of such an integrated Environmental Information System (EIS) using ArcInfo, to be used in its initial form for atmospheric environmental studies. A feature of the project was that it had very tight timescales for delivery; the ways of working introduced to ensure successful completion are discussed below.
Three main overall areas of functionality were identified:
1. management, visualisation and manipulation of a variety of underlying datasets.
2. management, visualisation and manipulation of output from computer models.
3. analysis of environmental impacts by the combination of 1 and 2.
Three main computer models are currently used within National Power for atmospheric dispersion calculations on a local or UK-wide scale: MATADOR, UK-ADMS and VISPACT. The aim of the initial version of the EIS was to take the standard output files from these models and bring them into the GIS as data layers for display and analysis operations. The more complex task of direct (process-to-process) integration of the models within ArcInfo has not been attempted at this stage.
MATADOR is a model of the long-range transport and deposition of sulphur and nitrogen emitted from UK and continental sources. It calculates atmospheric concentrations, and deposition by dry and wet processes, on a uniform grid of user-defined size (typically 10km). It was required:
UK-ADMS is a model of the atmospheric dispersion of gaseous releases to the atmosphere, typically used from the point of emission to a distance of 50 km. Results may refer to a single "instantaneous" plume, or to the accumulated result of emissions over some longer period. Information is output about atmospheric concentrations in plan, at a number of vertical heights above ground.
UK-ADMS produces output fields on a uniform grid either in Cartesian (x,y,z) or Polar (r,.,z) of user-defined size (typically 1 km). It was required:
VISPACT is a model of the atmospheric dispersion and visibility of water vapour and droplets from cooling towers, typically used from the point of emission to a distance of 5 km. Results refer to the accumulated frequency of occurrence of ground-level fogging.
VISPACT produces output fields along 12 radial lines, equally spaced in azimuth. It was required:
It is in this area that the spatial analysis capabilities of the system add particular value. A number of the calculations made would have been quite complex to write from scratch using a programming language. Some examples of the analysis functionality that was required of the system are given below.
Analysis functionality to be associated with the models included:
This section outlines some of the data quality issues that were addressed during the pre-processing of the background datasets originating from third party suppliers used within the EIS. Therefore, this section does not provide a comprehensive listing of all the background datasets used in the EIS, but focuses instead on those datasets where poor quality provided a stumbling block to the integration of the data into the system. For any developer contemplating the integration of third party data into their GIS, the quality issues raised here will hopefully provide some guidance in their acquisition of data. The main point that can be identified is that in order to minimise re-processing, a GIS developer must be able to enumerate the geographic extent, and quality requirements of each dataset to be used within their system, before shopping for data. The converse of this requirement is that data providers in the UK must start providing comprehensive data quality information targeted at discerning GIS users.
The OS provided National Power with the OS GB Base Data dataset. This dataset is an off-the-shelf product. All the datasets extracted from within the OS data required some form of pre-processing before use within the EIS.
The original data was supplied in DXF format geographic tiles, and converted into thematic data layers in ArcInfo format, and the level of extra pre-processing needed on the datasets was dependent on the original data quality and the final demands of the EIS on the data. The datasets listed below, outline some of the problems we had to address.
The country ownership of islands off the UK main land-mass was not specified within the original OS dataset. This dataset was amended to include country ownership details.
The county ownership of islands off the UK main land-mass was not specified within the original OS dataset, and the sea-ward boundaries of many counties were not included. The boundaries were added, and islands were either allocated to counties (e.g. Isle of Wight to Hampshire) or were given their own "county" identifier (e.g. Shetland Isles).
This dataset held the polygonal representation of the major GB urban areas as defined by the OS. Each urban area was meant to have an associated centroid from which the polygon attribute data could be accessed. Within the DXF data polygon boundary definitions and polygon centroids were held on different DXF layers. In order to create a fully attributed ArcInfo dataset, the centroids and the boundaries needed to be amalgamated, and there were several problems resulting from amalgamation:-
This dataset was characterised by the atomisation of the geographic road definitions. For example the M25 motorway was represented by approximately two hundred arcs.
Site of Special Scientific Interest (SSSI) polygon definition data was to be provided from the appropriate English, Welsh, and Scottish authorities.
The problems encountered with these datasets highlight the importance of the flexibility of the development environments used by third party agencies supplying data, and the importance of cross-country co-ordination between agencies supplying similar data to the eventual utility of a GIS dataset.
In terms of completeness, general accuracy, and ease of integration into the EIS, the Welsh data was the best third party dataset added to the EIS. This probably reflects the fact that the Countryside Council for Wales already uses ArcInfo for its own GIS needs. The data provided was topologically correct, in ArcInfo format, and in the correct projection. However as will be outlined below, the set of Welsh SSSIs did include some SSSI polygons that fall within current English borders.
English Nature provided the English SSSI dataset. This was provided in Intergraph design file format (DGN). Every SSSI in England was provided as a separate design file, and the full set of data completely filled two 500 megabyte exabyte tapes. (To give an idea of the size of the data being managed, processing time was three days constant CPU time for a Sun Sparc 10.) Problems with the originally-supplied data meant that full pre-processing of the data had to be done twice.
The data had a number of problems in its integration into the EIS including missing SSSI boundaries, unclosed SSSI boundaries, and missing SSSI centroids.
It was a requirement of the EIS to integrate all the SSSI datasets into one coverage. On comparison between the English and Welsh SSSI boundaries, many were seen to overlap between the two countries. For some SSSIs, it seems that both English and Welsh agencies are responsible for their monitoring and management. Eventually, ownership of SSSIs was apportioned to either country according to the location of the current national boundaries of England and Wales.
Three datasets were obtained. One contained the census statistics for all Enumeration Districts (EDs) in the UK (thirty megabytes in size), one contained the boundary definitions of all postcode sectors within the UK, and one contained ED boundaries for north east England.
In the final output, there were postcode sectors that were missing population density values. There were three main reasons for this:-
The third error case above was rectified in the output postcode population density coverage by using complex cursor processing. In the absence of additional information, the first two error cases could not be fully rectified. Therefore there are some parts of the UK in the output postcode coverage where the population density values are specifically set to NULL to indicate missing data.
The initial design of the EIS required the application of the philosophy and general methods of formal systems analysis and design in a way that would clearly identify the structure and "feel" of the eventual solution, without being impenetrable, or taking too long to complete.
The sensible partitioning of a GIS solution into a set of functional areas, and from that point a set of functions, critically provides a development team with a structure to work around, and provides the developer and the customer with the opportunity to talk to each other using a common frame of reference. Without sensible partitioning, a GIS is liable to be chaotic in development, and unlikely to meet all its original requirements.
It can be a gamble if your intended customers do not like your ideas, but partitioning should be attempted before a proposal for a new system is submitted. This is especially important for short term projects where the ability to "hit the ground running" from the first day will often determine whether the project is going to be on time and to budget.
It was determined that as well as the areas concerned with ADMS, MATADOR, VISPACT and point data, facilities for three additional functional areas would also need to be provided:-
So, even at the proposal stage it was possible to present an overall framework based on identified functional areas that would constitute the broad structure of the final EIS.
Having identified all the functional areas within the EIS, it was possible to begin the mapping of the system requirements into actual processes within each functional area. At the proposal stage, the success or otherwise of this process can be attributed as much to experienced "common sense" as to formal design methods.
For example, it was "common sense" that each of the main model functional areas (MATADOR, ADMS, VISPACT) would need flexible import routines to allow for the creation of grids and TINs from the input ASCII model files, otherwise subsequent analysis would not have been possible. On the other hand, an explicit requirement to graph time-series data held at point locations within the EIS meant a suitable function would need to be created to meet this end. Whether using "common sense" or working from the requirements derived from an ITT, it is essential to ensure that every requirement can be mapped to a proposed module within the final solution.
Therefore, the process of identifying the main functions within each functional area was iterative, stopping only once it was decided that no further progress was possible without actually starting the detailed requirements analysis.
With most GIS implementations, a function identified in this iterative process can be either functions that are common to most if not all the functional areas within a proposed system, or functions that are "peculiar" to one functional area only.
From a design perspective, the greater the number of "common" functions identified, the fewer the number of programs in the final system, and the more integrated the operation of the overall system.
The final act of the initial system analysis and design was to create diagrams that showed the interaction of the "common" and "peculiar" processes within each functional area. These diagrams were a cross between Yourdon data flow diagrams (DFDs), and Interface design diagrams. The aim of the diagrams was to outline the proposed system structure for each functional area by indicating the relationships between the proposed system menu interfaces and the functions that were initiated from the menus.
Today's competitive environment will often dictate that any proposed GIS solution must be cost-effective, quick, and complete. In such an environment, the close collaboration of a developer and their client is essential for things to get resolved quickly. For RAD to work, collaboration must be effective. For collaboration to be effective, the developer and the customer must have a clear mutual understanding of the requirements of a GIS and its proposed solutions.
The first stage of the project was detailed requirements analysis. The aim of this stage was to identify a common "baseline" for the development work that both National Power and NRSC could agree.
The initial DFDs were used by both parties as the starting point for this analysis. Over a period of several days of collaborative discussions involving all relevant personnel the detailed requirements were identified, centred on the functionality indicated within the DFDs.
Any GIS project requires a framework for the continuous organised reporting of progress, and thus the opportunity to resolve any issues that may arise as soon as they become apparent. For the EIS, the approach taken involved the phased delivery of the final system.
Development took place both at NRSC headquarters and National Power headquarters and target dates were set for the completion of each functional area. At the end of each time period, the appropriate completed functional area was delivered, installed, and demonstrated at National Power headquarters. Any issues or problems with that release of the functional area were thrashed out there and then. At the next target date, as well as the next due functional area, any revisions to installed functional areas were also delivered.
This rolling program of functional area release gave an opportunity to amend the functional areas as the project progressed, and gave National Power the opportunity to see if the system was going to be as expected throughout the development period. We suggest that phased delivery is the most suitable method for rapidly implementing a GIS solution. However it is recognised that not all projects will allow for it, either because of a poor relationship between the developer and the customer, or because the initial design work was not possible at the proposal stage.
At the end of the project, the EIS was also tested against a Test Specification document that represented a restatement of the detailed requirements. Given that the system had undergone phased delivery, we believe the number of failures, omissions, and problems were minimised, as by the time site acceptance testing was begun, the National Power team already knew part of what to expect.
The EIS is targeted to provide functionality to a wide variety of users within National Power. This includes specialists in the modelling packages that the EIS caters for, users needing to find out what data holdings National Power have, and management needing to know how a "project" is progressing.
The Top-level functional area organises work that takes place within the EIS on a "project" basis. A project is defined by the model and point datasets available to it over the National Power network. The model output might focus on a particular area within the UK, or might focus on a particular pollutant, or might focus on a set of new data that needs to be added to the EIS data holdings.
Using projects means a mechanism now exists within National Power to logically organise the output from a variety of modelling systems into a coherent group in one location on the network.
This section provides brief descriptions of those areas within the system where the technical content is of most interest.
For the main model functional areas (MATADOR, ADMS, VISPACT), the EIS provides customised import facilities to bring data in from the modelling systems. Each functional area has a particular set of import facilities, as the complexity and contents of an input file differs between each modelling systems.
In principle though, within each functional area a user can employ the import facilities to choose:-
For the UKADMS and MATADOR functional areas, one model file can produce many different output grids and tins depending on what set of data within a model file the user chooses.
Within the UKADMS and MATADOR functional areas, a user can also specify the output grid resolution in metres for the chosen model data. This gives a user the chance to specify the level of detail required for the output grid.
The import facilities use an AML front-end, with an ANSI C process running in the background that interrogates the chosen model file to obtain the user's data preferences, and then restructures the chosen set of data within the file into a suitable format for ArcInfo to handle.
Within the points functional area, the EIS provides file based editing facilities:-
The data files conform to a pre-defined format that the SAS package may produce, as SAS is where most of the point data are currently stored. The functionality uses AWK and NAWK to restructure the chosen input file into a format that AML can read. The process makes sure that the all the attribute names and values in the file are valid for the destination coverage. Having confirmed its validity, the data is automatically added to the destination coverage.
This functionality places no internal restrictions on the size of the files provided, so can be used for "long-haul" automatic batch additions or updates to existing points coverages.
The EIS is multi-user. The first thing a user does on entering the EIS is "open" an existing project or create a new project. The act of entering either a new or an existing project creates a unique temporary workspace under the project directory for that user. In this way, each user has a unique workspace during an EIS session. This overcomes problems that can occur if many users share the same workspace, and allows the EIS to "tidy up" after itself when a user exits the system by deleting the temporary workspace that was being used.
Additionally, the EIS implements a simple coverage locking mechanism to ensure that features on a particular coverage are only edited by one user at a time. Although there are many different methods for implementing locking, the EIS uses a "lock file" approach. If a user requests facilities to edit a dataset and a "lock file" exists for that coverage, then that user will be informed that someone else is already editing the coverage. On completion of an editing function, the "lock file" will be removed to allow other users to perform editing activities.
The coverage level locking provided is sufficient for the EIS, given that the only real dataset editing occurs in the Points functional area, when an authorised user is either adding, deleting, or updating points features and their attributes. These operations are reasonably infrequent given that power stations are not built every day! Depending on whether the appropriate modules are purchased, the locking facilities may well be extended in the future to allow feature level locking rather than coverage level locking.
The EIS allows for the customised visualisation of model or background data grids. There are three options provided with this functionality:-
On specifying a particular visualisation option, the input grid will be drawn using the applied legend schemes, and the legend definition will be added to the overall key display area.
The EIS allows for the custom visualisation of a contour surface derived from a pollution model grid or tin.
A user can create a contour surface by reference to the minimum, maximum, mean, and standard deviation values of an input grid, and the minimum, maximum, and mean values of an input tin. The EIS prompts for a starting contour value and a fixed interval from that value in order to define the limits of the surface to be contoured, and the density of the output contour map. A user can also choose whether to include the contour values as text on the map display as well as in the contour legend.
The contouring functionality currently available within ArcInfo is less rich than that provided by modern scientific visualisation packages. This can lead to some frustration in users familiar with these alternatives.
The general screen display within the EIS is split between a map area and a key area. Every screen display within the EIS holds a dynamic key. During the operation of the system, the key changes dynamically to reflect the current contents on display in the map area. If a dataset is added to the display then the complete key classes within that dataset are also added. If a dataset is removed from display then the key classes for that dataset will also be removed from the display.
This dynamic key is implemented as a logical sub process of the main "draw" routine. Whenever the screen is refreshed to display new sets of data in the "draw" routine, the key generation process picks up the details and appropriate symbology for all the datasets on show, from point, polygon, or line datasets, to grid and TIN datasets.
The EIS allows a user to compare two data layers; for example depositions and critical loads for selected areas in the UK. The logical and graphical selection tools available for polygon features in the General Map Navigation functional area are all available to this function, so a user can custom define the set of areas for which this calculation is applied.
The functionality provides a difference grid for the selected polygons highlighting those areas where pollution exceeds the sustainable load, and provides a statistical analysis of the differences observed. The EIS also provides the ability to graphically query the output grid to identify the difference values in selected cells, and the output grid can be saved for future reference or additional analysis.
Throughout the EIS, there is functionality to save the current display to a named directory, and functionality to load previously saved displays to the screen. This allows an user to treat a session within the EIS as an on-going process. Displays can be saved in the same way as a user would save a document within a word-processor, the only difference being that the EIS has to deal with map "documents".
Saving a display involves recording the current map extent, recording the logical contents of the display, creating a listing of all the datasets currently on display and their internal characteristics, and also involves copying any temporary datasets from a user's unique workspace to the named save directory.
From the information described above a display can be re-created, exactly as it was when saved. The EIS load functionality provides a listing of all the saved displays available within a project, and a user can choose which saved display to load to the screen to replace the current view.
The EIS provides functionality to create customised line graphs of time-series pollution variables stored against point features such as power stations. This functionality is available from the top-level menu within the points functional area. On choosing a point feature against which time-series data is stored, a user is presented with a graphing menu. From the menu a user can choose a set of variables to graph, choose the time period to graph between, and choose the symbology for each line on the graph. The graph key holds the variable names and measurement units, and a user can pick any printer or plotter visible from the local workstation to print the output graph. As the menu stays up until explicitly dismissed, the functionality allows for the quick generation of multiple graphs for a selected point feature.
Throughout the EIS, functionality exists to allow a user to produce hard-copy maps of the current screen display. From the custom print menu, a user is able to specify the output size of a plot (A4 or A3), the orientation of a plot (landscape or portrait), the plot title, whether to add a scale bar, and whether to add a north arrow. On confirmation of the menu, the EIS displays the custom plot file as it will appear in hard-copy, and gives a user the option to print or not. A user can then either specify any printer or plotter visible from the host workstation to send the plot to, or send the plot to a named PostScript file for use outside the system.
This paper has aimed to describe how an environmental information system has been built using ArcInfo, and some of the decisions and difficulties that needed to be resolved along the way. In summary:
The system is now in use and has already proved its value as a means both of gaining added insight into the interpretation of atmospheric dispersion models and how they may be used, and also in communicating the results of environmental effects analyses both internally and externally. It has also acted as a catalyst in the generation of new ideas from users about additional functionality and areas of application that would be of value.
This paper is published by permission of National Power PLC and the National Remote Sensing Centre Ltd.
Jim Ball,
Senior Applications Engineer,
National Remote Sensing Centre Ltd.,
Delta House,
Southwood Crescent,
Southwood,
Farnborough,
Hampshire,
GU14 0NL,
UK
Telephone.: (01252) 541464
Fax: (01252) 375016
email: jball@nrsc.co.uk
Dr Ian Rodgers,
Senior Consultant,
National Power PLC,
Electron Building,
Windmill Hill Business Park,
Whitehill Way,
Swindon,
Wiltshire
SN5 6PB,
UK
Telephone.: (01793) 896235
Fax: (01793) 896391
email: Ian.Rodgers@research.natpower.co.uk