David Tulloch, Ben Niemann, and Earl Epstein

ASSESSING THE OUTCOMES OF MPLIS IMPLEMENTATION: IDENTIFYING AND DESCRIBING BENEFITS


BACKGROUND

Whether directly or indirectly, a large portion of GIS research addresses the increased efficiency and effectiveness of systems within organizations. Much less concern has been placed on broadening the view of these systems to the community context in which they exist. Within the community context, a system's value is not simply measured by the benefits accruing to the operating organization -- efficiency and effectiveness -- but also by those benefits enjoyed by the community -- equity. Consideration of this broader context with its added benefits stream suggests that MPLIS development has not completed its life-cycle until the system reaches a final stage of development that we call democratization, characterized by broader community participation in land management. Inherent in this concept is that a MPLIS that does not achieve democratization (i.e. the system produces outcomes of improved equity) should not be considered fully developed.

As a tool for studying MPLIS development within this altered perspective, we have proposed a model (Tulloch et al. 1996) that describes the life-cycle of community systems over time. It is based on a review of previous research on the nature of GIS/LIS development in organizations, the results of surveys of GIS/LIS implementation activity, and anecdotal and personal experiences of the authors. The model describes the necessary stages in a linear development process in a community as well as indicators of those stages, factors that determine a change from one state to another and benefits. The model is intended as a common basis for understanding of and communication about system development by decision-makers, system developers, and academic theoreticians equally well.

This paper reviews the specific category of indicators called benefits. Indicators are the measurable variables that define the identifiable and measurable stages of MPLIS development. The model relies upon seven categories of indicators, further divided into fifty-three specific factors. For example, the nature and extent of data in a digital format can be used as an indicator of a system that has entered into the recordkeeping stage.

BENEFITS: THE THREE E's

How to describe the benefits obtained from system investments remains a significant issue. Economists recognize that benefits are both tangible and intangible. The identification of tangible benefits and their measurement in monetary terms make it possible to calculate a benefit/cost ratio for investments. However, economists also emphasize that it is equally important to identify and characterize intangible benefits and introduce these in any discussion of potential system investments.

The model places the benefits of MPLIS development into three broad categories: efficiency, effectiveness, and equity. Efficiency results where traditional activities are performed at a reduced cost, generate more products, are accomplished more quickly or in some combination. Effectiveness results when more or better information is generated from traditionally available data because of digitally stored data and the software for sophisticated analysis of that data. The emphasis shifts from benefits associated with traditional tasks to benefits associated with actions that rely upon system products. Equity results from a perceived or real increase in effective participation by citizens and organizations in decisions about land and resources. Although too space consuming to include here, a system's total benefits can be expressed mathematically (Tulloch et al. 1996).

Traditionally, attention to system benefits has focused on efficiency and effectiveness (Smith and Tomlinson 1992; Antenucci et al. 1991; Gillespie 1994). Efficiency is a common measure of benefits because is it easily expressed in monetary terms reflecting the amount of money saved through better management of system operations. Effectiveness is somewhat more difficult to determine. However, estimates of the monetary values of these benefits can be made based upon the savings associated with activities made possible by the additional information generated from the old data. These are also commonly sought measures of benefits because they relate closely to the internal needs of organizations and are easily understood by non-technicians and decision-makers in those organizations.

Recently attention has turned to the benefits associated with the broader use of system products beyond the organization and throughout the community. These related outcomes have been characterized as societal benefits (Clapp et al. 1989), equity (Kishor et al. 1990; Cowen 1994), decision making (Pinto and Onsrud 1995), and democratization (Mead 1994; Lang 1995). As a general trend, this attention is evidenced by the amount of research concerning community issues such as public access and liability (e.g. Epstein and Roitman 1987; Epstein, Hunter, and Agumya 1996) and concerns about the outcomes associated with the growing relationships between GIS and society (e.g. McMaster et al. 1996; MSC/NRC 1997). Here the set of product users is potentially much greater in number and type than those users in the organizations that initiate and develop the system. The perspective encompasses the whole community of public and private organizations and people interested in the value to them in their land-related work that comes from the products of MPLIS Development. The context for these benefits is the full array of public and private policy plans, decisions, and actions where use of land and its resources are allocated.

The benefits from the use of system products by increased numbers of community members are labeled equity benefits. This label is appropriate because land-related decisions made by a more representative segment of the community means that GIS/LIS becomes democratized. This democratization represents an important advance in MPLIS development, holding out the potential for full community awareness, utilization and support for the system based upon a sense of increased and balanced participation in the allocation of land and its resources. Emphasis on these types of benefits is another important and different aspect of the community perspective that serves as a primary assumption of the model.

There is also another important reason for describing these benefits as equity benefits. Its achievement depends upon wide distribution and easy access to system products. This fundamental characteristic of dissemination and access has always been a standard for public data and information and a subject of policy controversy (e.g. Epstein 1991; Brown 1992). The long-term democratic interest in the dissemination of records and information used by governments to execute their legislative and legal mandates is exhibited by the state and federal open records and freedom of information laws which are built upon the democratic principal that access to material used by governments for their public business is essential in a society where people need to know what their governments are doing. A negative view of broad, easy access by many to GIS/LIS data restricts the full development of systems. Access to information is the key to garnering the large unrealized potential for equity benefits.

ANECDOTAL EVIDENCE

Because system benefits are usually accrued only after several stages of the MPLIS development process, it can be difficult to find examples of benefits, especially equity, as an outcome of system development in communities. However, we will endeavor to present a few recent occurrences which demonstrate the potential of this benefit.

When the land information officer in Waukesha County, Wisconsin automated the tax assessment records, a few lost parcels were discovered. These parcels accounted for several thousands of dollars in lost revenue each year. The first benefit generated was that of a more efficient operating office -- faster and cheaper to maintain. The second benefit came from the office's use of the system to better perform their traditional tasks, as illustrated by the discovery of the lost parcels.

For roughly a decade, Winnebago County, Wisconsin has been developing a MPLIS to serve its 140,000 residents in as many ways as possible. In recent years the database development has been completed and the system has been maintained and used for a variety recordkeeping and analysis purposes. One unexpected opportunity arose when investments in system development led to the discovery of inaccuracies in the FEMA produced Flood Insurance Rate Maps (FIRM) of the county. The maps are adopted by localities and used to determine whether homeowners are required to purchase flood insurance and also to determine where new structures can and cannot be built. In response to community concerns about the maps, the county used existing data to construct new floodplain models and maps. The county system showed that, in fact, a number of homes had been inappropriately located in or out of various zones. Because of the high quality of the data and the "transparency" of the analysis, the county's efforts have been recognized as a suitable alternative to the old maps. The broad impact on the community is an equity benefit.

System efficiencies allowed for the timely production and update of necessary maps, address lists, and other system products. The result of the analysis -- a complex hydrologic modeling process not previously feasible -- was a more effective means by which the county could fulfill its mandates. Both of these examples also have the subtle potential for the social benefit of equity because they may have both altered the public's confidence in those government agencies and may have a long-term impact on participation.

The Register of Deeds in Dane County, Wisconsin, has automated the title records and indexes for the county and maintains the digital material as the county's official property records. The entire database is available in digital form at the cost of reproduction (as mandated by state open records laws). Title insurance companies, for whom a complete copy of all relevant land transactions is necessary for competitive operation, are able to cheaply acquire the entire database for their use. While the existing firms are able to use this database to update their existing data at low cost, thereby increasing profits, new firms are able to enter the market avoiding the previously prohibitive start-up costs associated with building a database. The result has been the entry of new title insurance companies and an increase in competition which is estimated to have driven the cost of title insurance for homebuyers down an average of about $300 per transaction. This system, with an estimated cost of $500,000, results in a savings for homeowners of an estimated $6-7 million annually.

The Register of Deeds Office finds that the system makes its basic recordkeeping activity more efficient, requiring less time and space. Effectiveness is harder to judge because the primary duties of the office are recordkeeping and promoting access to data rather than an application that requires sophisticated data analysis. The automated system allows the general public to access land records easier and faster with an increase in direct citizen participation in use of the material and an indirect increase in participation through the added title insurers. Hence the accrual of the equity benefit. The reduction in title insurance illustrates and provides measures for the benefits of this increased participation.

BENEFITS MATRIX

Finally, we offer a matrix (Figure 1) as a simple tool for organizing potential benefits. This matrix includes examples from Wisconsin communities that have experienced a variety of benefits from the MPLIS development process. The matrix is provided as a new tool designed for organizing outcomes that have already occured while also encouraging brainstorming within communities to discover new benefits or anticipated benefits. Readers who use the matrix are encouraged to contact the authors at dtulloch@students.wisc.edu with examples of benefits that their own communities have experienced.

Figure 1 -- The benefits matrix as completed for a variety of benefits experienced by Wisconsin communities.


CITATIONS

Antenucci, J. C., K. Brown, P. L. Croswell, M. J. Kevany, and H. Archer. (1991). Geographic Information Systems : A Guide To The Technology. New York : Van Nostrand Reinhold.

Brown, K. (1992). "A Response to Earl Epstein," URISA Journal, 4: 6-8.

Clapp, J. L., J. D. McLaughlin, J. G. Sullivan, and A. Vonderohe. (1989)."Toward a Method for the Evaluation of Multipurpose Land Information Systems," URISA Journal, 1, 39-45.

Cowen, D. J. (1994). "The Importance of GIS for the Average Person," in GIS in Government: The Federal Perspective, Proceedings of the First Federal Geographic Technology Conference, Washington DC, 7-11.

Epstein, E. F. (1991). In My Opinion. In: URISA Journal, 3 (1): 2-4.

Epstein, E. F., and H. Roitman. (1987). "Liability for Information," URISA '87 Conference Proceedings, Vol. 4, 115-125.

Epstein, E. F., G. Hunter, and A. Agumya. (1996). "Liability and Insurance for the Use of Geographic Information," URISA '96 Conference Proceedings, Vol. 1, 294-301.

Gillespie, S. R. (1994). "Measuring The Benefits of GIS Use: Two Transportation Case Studies," URISA Journal, 6 (Fall): 2, 62-67.

Kishor, P., B. J. Niemann, D. D. Moyer, S. J. Ventura, R. W. Martin, and P. G. Thum. (1990). "Lessons from CONSOIL Evaluating GIS/LIS," Wisconsin Land Information Newsletter 6:1, 1-11.

Lang, L. (1995). "The Democratization of GIS," GIS World, 8:4 (April), 62-67.

Mapping Science Committee, National Research Council. (1997) The Future of Spatial Data and Society: Summary of a Workshop. Forthcoming. National Academy Press: Washington, D.C.

McMaster, R. B., B. J. Niemann, Jr., S. J. Ventura, D. D. Moyer, D. L. Tulloch, E. F. Epstein, and G. Elmes. (1996). " and Society," University Consortium for Geographic Information Science White Paper.

Mead, R. A. (1994). "Field-Level Diffusion Eases GIS Implementation Efforts," GIS World, 7:11 (November), 50-52.

Pinto, J. K., and H. J. Onsrud. (1995). "Sharing Geographic Information Across Organizational Boundaries: A Research Framework," in URISA '95 Conference Proceedings, Vol. I: 688-694.

Smith, D. A., & R. F. Tomlinson. (1992). "Assessing the Costs and Benefits of Geographical Information Systems: Methodological and Implementation Issues," International Journal of Geographic Information Systems. 6: 3, 247-56.

Tulloch, D. L., E. F. Epstein, and B. J. Niemann, Jr. (1996). "A Model of MPLIS Development in Communities: Forces, Factors, Stages, Indicators, and Benefits," GIS/LIS '96 Proceedings 325-348.



David L. Tulloch
Bernard J. Niemann, Jr.
Land Information and Computer Graphics Facility
University of Wisconsin-Madison
Madison, Wisconsin 53706
dtulloch@students.wisc.edu

Earl F. Epstein
School of Natural Resources
Ohio State University
Columbus, Ohio 43210
epstein.4@osu.edu