GIS in Education: An Examination of Pedagogy

Abstract

Geographic information systems (GIS) have been identified as one of the most critical and important software programs for implementing computer-based technology in social studies and science education. Much of the research about effective GIS integration is "garnered from intuition." This study will present the results of three different teaching methods based on the Jasper Woodbury Problem Solving Series and provide a model for successful GIS implementation into any preservice teacher education program.


"No exemplary models for integrating GIS into preservice teacher preparation programs exist."
Bednarz & Audet, The Status of GIS Technology in Teacher Preparation Programs, (1999)
Since the inclusion of geography as a core subject in the Goals 2000: Educate America Act, there has been widespread acceptance among citizens of the United States of the goal of developing students who are internationally competitive as well as productive and responsible citizens in a global economy. In response to this desire for a geographically literate society, the National Geography Standards 1994 were developed. The function of the standards is to help students and teachers develop a clear understanding of what geography is and how to effectively apply that understanding to life (National Geography Standards Project, 1994).

The effective teaching of the National Geography Standards has been the focus of K-12 social studies curricula throughout the nation. It has been identified that in order to effectively teach the standards, teachers require a clear understanding of a geographic information system (Bednarz, 1995; Sui, 1995). A geographic information system (GIS) is software that allows a user to store, retrieve, manipulate and display geographic data about any place in the world (Environmental Systems Research Institute, 1998). Even though such an understanding of GIS is necessary, it has not been adopted in the K-12 American classrooms at a rate that the National Science Foundation, the Environmental Systems Research Institute (Esri), and geography educators had once hoped (Environmental Systems Research Institute, 1998; Fitzpatrick, 2002). Fitzpatrick (2002) noted that Esri's goal in 1992 was for K-12 educators to be the largest single group of users by 1996, a goal still not achieved. The key reason for this slow pace of GIS integration, according to Bednarz and Audet (1999), is that no "exemplary models for integrating GIS into preservice teacher preparation programs exist" (p. 65).

In response to this lack of exemplary models for teaching GIS, the purpose of this study is to develop and research the effectiveness of three GIS instructional models for university-level instructors to use within preservice teacher education courses.

Rationale for the Study

The geography education reform movement of the 1990s had three principal goals. The first was to forge a consensus among geographers, educators, and the general public regarding the goals of geography education. The second was to examine technology use by geography educators. The third was a revision of geography curricula that would combine current issues in geography technology to enhance student achievement (Nellis, 1994).

In response to the first goal, Geography for Life: National Geography Standards 1994 was written outlining eighteen standards that K-12 students should meet to become geographically literate. The National Geography Standards have been described as a "vital contribution" toward helping students "use their minds well, so they may be prepared for responsible citizenship, further learning, and productive employment in our Nation's modern economy . . ." (National Geography Standards Project, 1994). Unfortunately, the second and third goals have not resulted in much technology implementation or curriculum revision in college or K-12 geography classrooms (Bednarz & Audet, 1999).

The curricula in geography education focuses on how individuals observe, conceptualize, analyze, and evaluate information from a spatial perspective (Fredrick & Fuller, 1997). Three types of technology that can provide this spatial dimension have been identified: exploratory systems such as Atlas CD-ROMs; database systems such as geographic information systems; and simulation systems such as National Geographic's Weather Machine (Fitzpatrick, 1990). Of all these, database systems and geographic information systems (GIS) have been recognized as critical for the implementation of computer-based technology in geography education (Bednarz & Audet, 1999; Keiper, 1999; Kerski, 2001). In fact, authors of the National Geography Standards have identified GIS as the only technology that can assist students in meeting all of those standards (Bednarz, 1995; Sui, 1995).

State of Current Teaching of GIS

Bednarz and Audet (1999) have identified three main reasons that current approaches to teaching GIS in K-12 classrooms have not been effective: teachers inadequately trained to use GIS, a lack of teaching models to guide their pedagogy, and preservice teachers education programs that do not teach GIS in "a meaningful way" (p. 64). Teachers who are adequately trained to use GIS in their classroom endure major frustrations and obstacles in their efforts to successfully integrate technology into their instruction (O'Neill, 1995). Fitzpatrick in communication with Audet and Paris (1997) has estimated that fewer than 20% of GIS software packages are being used in effective GIS teaching. Many teachers are excited about using GIS, but their motivation dissipates when they encounter technical and pedagogical problems. Thus, argues Brownwell (1997), the effectiveness of current approaches recommended for GIS should be questioned, and preservice teachers with time and access to university computer labs should become proficient in both the technical and pedagogical approaches to teaching GIS in the classroom. Bednarz (1999) not only calls upon teacher education programs to more effectively prepare preservice teachers to implement GIS-based curricula, but upon the research community to provide more than anecdotal evidence to support the claims that GIS can enhance students' learning.

This study, in response to Brownwell's (1997) and Bednarz's (1999) appeal for the development of an effective GIS teaching model for K-12 classrooms and for university preservice teachers, develops and researches three GIS teaching models for preservice teacher education programs. The GIS pedagogies are based on three instructional models employed by teachers of the Jasper Woodbury Problem Solving Series published by the Cognition and Technology Group of Vanderbilt (CTGV). To help students develop effective problem solving skills, the CTGV encourages teachers to use anchored instruction which "provides a way to recreate some of the advantages of apprenticeship training in formal education settings involving groups of students" (CTGV, 1990, p. 6) by creating learning environments that emphasize scaffolding and generative thinking (CTGV, 1992). These instructional approaches are the same ones that many educators have identified as necessary for successfully teaching GIS to learners at all age levels (Bednarz, 1995; Keiper, 1999). Therefore, the goals of the Jasper Woodbury Problem Solving Series and GIS are for students to "learn to become independent thinkers and learners rather than simply become able to perform basic computations and retrieve simple knowledge" and to develop their ability to "identify and define issues and problems on their own rather than simply respond to problems that others have posed" (CTGV, 1990).

The three instructional models CTGV (1990) identified for teaching the Jasper series are: "Basics First, Immediate Feedback Direct Instruction," "Structured Problem Solving," and "The Guided Generation Model." The first model, "Basics First, Immediate Feedback Direct Instruction," focuses on the order in which content and tasks in a curriculum are presented. This model is grounded in a "extreme reductionist view that all components of a skills must be mastered before the components can be assembled into the skill they comprise" (CTGV, 1990). The second model, "Structured Problem Solving," focuses on the importance of learners making errors and struggling with a task. This model is based on the premise "that errorless learning is ideal; at the other is the assumption that important lessons of learning occur only when students make errors or reach impasses and are then helped to correct their initial misconceptions" (CTGV, 1990). The third model, "The Guided Generation Model" is grounded in the role of the teacher in the learning process. This can span the range of "authoritative provider of knowledge to a resource who at times is consulted by the students and at other times can become the student whom others teach" (CTGV, 1990).

In addition to developing and researching GIS instructional models, because GIS use is grounded in a learner's ability to successfully navigate and learn with a spatial perspective (Audet & Paris, 1997), this study will analyze the effect of learners being field dependent or field independent on GIS learning. Field dependence and field independence is an established spatial ability cognitive style that correlates with the ability to successfully perform in a computer-based instructional environment (Riding & Cheema, 1991). In addition to this relationship, the key difference between field independent and field dependent learners is their visual perceptiveness or their ability to distinguish the parts of an image or visual environment from the whole (Riding & Cheema, 1991). Given that GIS is a visual environment where distinction between details is critical, this variable may provide a greater understanding to develop a GIS model of learning for preservice teachers.

Statement of the Research Questions

This study is designed to answer the following questions based on the development and research of GIS teaching models for university instructors to implement within their preservice teacher education programs:

  1. To what extent will preservice teachers, who are taught with direct instruction, be able to perform on a GIS basic skills exam and problem solve in a GIS environment immediately after instruction and two weeks later?
  2. To what extent will preservice teachers, who are taught with a structured problem solving model of instruction, be able to perform on a GIS basic skills exam and problem solve in a GIS environment immediately after instruction and two weeks later?
  3. To what extent will preservice teachers, who are taught with the guided generation model of instruction, be able to perform on a GIS basic skills exam and problem solve in a GIS environment immediately after instruction and two weeks later?
  4. How does a preservice teacher who is field dependent or field independent influence learning within the three pedagogical approaches - direct instruction, structured problem solving, and guided generation?

Methods

In order to provide a context for the present study, descriptions of the research methods are organized into several different sections. They include descriptions of the subjects, materials, design, research setting, procedure, data scoring, and statistical analyses.

Subjects

The pool of available subjects in this investigation consisted of 160 volunteer university preservice teachers (hereafter teachers) who were enrolled in the post-baccalaureate teaching program at the University of Minnesota. The teachers were recruited from seven sections of the EdHD 5007: Technology for Teaching and Learning course.

Materials

The materials for this study consisted of the geographic information system software entitled ArcVoyager, three web-based modules with strict pedagogical guidelines - "Basics First, Immediate Feedback Direct Instruction," "Structured Problem Solving," and "Guided Generation," the Group Embedded Figures Test (GEFT) to measure the students' field articulation, the project exams - "The Case of the Missing Ship" (Esri, 1997) and "Magic Dan's Extreme Sea & Ski Resort" (Esri 1998), and a response confidence survey/ demographic questionnaire.

ArcVoyager

A geographic information system (GIS) is used to analyze relationships between features distributed over space. Environmental Systems Research Institute (Esri) is the developer of geographic information systems (GIS) softwares ArcView and ArcVoyager that allows users to view, analyze and interpret data about any place in the world. ArcView allows users to add, manipulate, and interpret data layers that provide an environment for geographic exploration as users develop maps, charts, and tables in response to posed questions about the world. ArcVoyager was developed to accompany ArcView and consists of carefully selected data, projects and instructions to aid a learner in understanding a geographic information system.

The design of the ArcVoyager software includes the "ArcVoyager Guide" that aids GIS learners at the learning levels mapped out in Table 1. It was developed to aid the learner in understanding what GIS is, provide the user with ready-to-use data, and allow geographic exploration for students.

Table 1: ArcVoyager GIS Learning Levels and Description

GIS Learning Levels Description
Level 1 Engages just the help file, providing a tightly controlled text and graphic-based introduction to basic concepts and skills of spatial thinking with a computer.
Level 2 Engages an ArcView project, with a world atlas and a constrained interface that lets the user explore in a tightly controlled environment. Users learn easily about choosing, adding, and deleting themes; turning themes on and off; shuffling layers; zooming in andout; and identifying features, all looking at about 25 different layers.
Level 3 Engages a more open and powerful but still customized interface, with more capabilities involving tools that students and teachers use most often. Here, users start with a vast set of data at the ready, and can add an unlimited amount and modify it all in standard ArcView fashion.
Level 4 Engages the Level 3 interface and provides an open set of data without pre-arranged starting layers."
Note. From "ArcVoyager Makes ArcView Easy" by Environmental Systems Research Institute (Esri), 2002, http://www.Esri.com/industries/k-12/voyager.html .

Teachers who participated in this study used ArcVoyager for their GIS software. This study focused on the teachers' abilities to perform GIS at levels one through three as shown in Table 1.

Instructional Approaches Materials

Three instructional methods were used in this study to identify the best approach for GIS learning and to develop a model to guide future teachers who use GIS in their instruction. Each instructional method was based on suggested instructional approaches to teach the Jasper Woodbury Problem Solving Series from the Cognition and Technology Group at Vanderbilt (CTGV, 1992). The CTGV named their instructional methods "Basics First, Immediate Feedback Direct Instruction," "Structured Problem Solving," and "Guided Generation." The "Basics First" method focused on students acquiring the basic subskills and subconcepts prior to being presented a complex problem. The "Structured Problem Solving" method focused on students receiving instruction that minimized errors and reduced confusion and the "Guided Generation" method introduced complex tasks at the same time as the basic subskills and subconcepts. The materials developed for each method of teaching GIS were as follows.

Basics First, Immediate Feedback Direct Instruction Materials

The materials developed for the "Basics First" model were developed for a directed method of instruction where the instructor is the purveyor of knowledge while making sure students acquire the necessary GIS subskills and subconcepts.

The "Basics First" GIS model was a web-based module developed with Macromedia Dreamweaver that included text and graphics to illustrate the main GIS subskills and subconcepts. This module was designed to help learners build their basic cognitive foundation in the use of a geographic information system prior to participating in the project exams. The content presented in the module consisted of ten sections:

  1. Macintosh Basics

  2. What is a Geographic Information System (GIS)?

  3. What is ArcView GIS?

  4. What is ArcVoyager GIS?

  5. GIS: A Thinker's Tool.

  6. Exploring the ArcVoyager Guide.

  7. Basic Viewing Area Layout

  8. Working with Map Themes

  9. Identifying and Drawing Information on the ArcView Map

  10. Viewing a Theme Legend

  11. Moving Around the Map View
Each of the ten sections was decontextualized to emphasize the basic skills. Therefore, basic GIS skills needed to complete the project exams were covered in this module before they practiced the task themselves and before they independently took the project exam.

Structured Problem Solving

The teaching materials developed for the "Structured Problem Solving" model were developed so students could minimize their errors without confusion while working only on activities that would hopefully generate the correct result.

A worksheet with ten complex practice questions (http:///www.geographyeducation.com/gisweb/structuredindex.html) was used that required students to use basic subskills and subconcepts in parallel with learning the ten sections in the "Basics First" web-based module. In order that students could minimize their confusion within this module, Quicktime video tutorials (http://www.geographyeducation.com/gisweb/structuredindex.html ) were developed. The QuickTime videos included graphics, animation, and narration that provided a web-based tutorial for students to successfully complete the ten complex practice questions. By providing a QuickTime video, students could play, stop, and replay the video, reducing confusion and minimizing errors while completing the practice questions.

The Guided Generation Model

The "Guided Generation" materials were developed to emphasize generative learning. According to Wittrock (1974), generative learning occurs when learners themselves generate meaningful relationships involving the ideas that they want to learn about. For example, a Minnesota student, after learning about latitude and climate might make the connection between lower latitudes and warmer temperatures. Thus, the students are generating connections between the concepts they are learning. However, instructors also need to make sure that students do not make the incorrect connection such as students noting that even at higher altitudes, it is going to be warmer in lower latitudes.

In addition to generative learning, scaffolding was described as a teaching strategy used by instructors of the guided generation model (CTGV, 1992) . Vygotsky (1978) described scaffolding as when an instructor identifies where a student is in his or her learning and then uses instructional tools to build upon the student's experiences to take them to a higher level of understanding.

The materials in this module consisted of a video news clip (http://www.geographyeducation.com/gisweb/guidedindex.html) that imitated a news report announcing the "Case of The Missing Ship" (Esri 1998), a message that was received in the "cartography" room from the "decoding" room where a ship had gone down on earth, but no one knows exactly the location of the disaster. This ship was on a "highly secret" spying mission. Within this video, the news anchor provided nine clues that could have been used with a geographic information system (GIS) to find the ship's location. The video provided "anchored" instruction for the GIS learners. Anchored instruction "provides a way to recreate some of the advantages of apprenticeship training in formal educational settings involving groups of students" (CTGV, 1990).

In addition to the video news clip, in order to scaffold the students' learning, a web-based module featuring QuickTime videos was developed to instruct students on four domains needed to solve the problem of finding the ship. These domains consisted of the biosphere (people, plants, animals), atmosphere (air), lithosphere (rocks), and hydrosphere (water).

Field Articulation Measurement

Field dependence and field independence is a spatial ability cognitive style that correlates with the ability to successfully perform in a computer-based instructional environment (Riding & Cheema, 1991). In addition to this relationship, the key difference between field independent and field dependent learners is their visual perceptiveness or their ability to distinguish the parts of an image or visual environment from the whole (Riding & Cheema, 1991). The Group Embedded Figures Test (GEFT) measures the cognitive style of field dependence or field independence. The GEFT is a timed test that requires participants to locate 18 geometric shapes located on a more complex, patterned geometric shape. The individuals who locate most of the simple shapes were considered field independent while field dependent individuals are those who locate less than half of the shapes (Witkin et al., 1971). A median split was used to determine high and low field articulation or field independence and field dependence respectively.

The split-half reliability of the GEFT, published in the test manual and using the Spearman-Brown formula, is 0.82 for men and women. The validity of the GEFT, as determined by comparing the GEFT with the Embedded Figures Test, is an r = -.82 for men and r = -.63 for women (Witkin, Oltman, Raskin, & Karp, 1971). The overall usefulness and validity of the GEFT with undergraduate, graduate, and adult populations has been supported by numerous examinations that have established internal consistency, reliability, and validity (Day, McRaie, & Young, 1990; Panek, Funk, & Nelson, 1980).

Project Exams

The teachers' abilities to learn and use GIS were measured by how well they performed on two project exams and two GIS basic skills exams.

The project exams measured how well the teachers used GIS to identify a given location on a map when given nine clues. The exam questions were set in an authentic situation that the students could relate to in their everyday life. Both project exams have been used extensively in GIS training throughout the nation and have been found to be valid and reliable (Fitzpatrick 2002). Fitzpatrick (2002) noted that the project exams clearly measure how well students learn and use GIS at all age levels while providing a consistent measurement over the last four years.

The first measurement of GIS learning was the project exam, "Case of the Missing Ship" (Esri 1997). This exam is "accomplished by applying good critical thinking" (Esri 2002) while effectively using a geographic information system. The teachers were required to find a location on a map using a geographic information system and data layers that were provided along with the clues from the exam. The exam develops the scenario where a ship on a secret mission has crashed on earth; however, no one knows the exact location. Given nine clues, the teachers were asked to find the exact location on a map using ArcVoyager (see: http://www.geographyeducation.com/gisweb for the exam).

The focus of the second exam was to identify the delayed retention of GIS knowledge. The second measurement of GIS learning was "Magic Dan's Extreme Sea & Ski Resort" (Esri 1998). This exam required using the GIS skills learned for the first exam, "Case of the Missing Ship" (Esri 1997), within a dissimilar situation - a "business sitting". This exam called upon the teachers to assume the role of a site analyst and find the best location to develop a resort. Once again, they were given nine clues needed to find a successful business location and were asked to identify this location on the map (see http://www.geographyeducation.com/gisweb for the exam).

A GIS basic skills exam was also used to assess the teachers' ability to understand basic GIS procedural tasks. The basic skills exam consisted of ten multiple-choice questions (see http://www.geographyeducation.com/gisweb for the exam).

Response Confidence

Kulhavy, Yekovich, and Dyer (1976) found that during an exam, seldom does a test taker know their answer is completely correct or incorrect. In fact, they found that the students create a "hierarchy of confidence" regarding how correct their answers are (p. 522). They measured this hierarchy using a scale with five equidistant points ranging from 1 (low confidence, random guess) to 5 (highest confidence, sure of answer). Therefore, because the teachers could possibly guess and receive a high score when taking the project exams, how confident the teachers were in their answer was measured by teachers ranking themselves on a five point equidistant Likert scale ranging from 1 (low confidence, random guess) to 5 (highest confidence, sure of answer).

Questionnaire

Fitzpatrick (2002) noted that the GIS national training he has been involved with over the past ten years have found three main measurable variables that have impacted GIS learning - time taken to complete the exam, background in earth science and/or geography, and how comfortable a user feels using technology. In order to account for variable factors such as these that may have influenced GIS learning, a questionnaire was developed that consisted of ten questions. The questions asked about the teacher's gender, number of geography courses taken, number of earth science courses taken, how comfortable they felt in the use of technology, if they had ever worked with GIS software before and if so, to what extent, cumulative grade point average, major in content area, and time taken to complete the exam.

Design

The design of this study was a quasi-experimental 4 X 3 factorial design. The dependent variables included cognitive engagement and procedural knowledge with two levels in each representing the exams immediately after instruction and two weeks later. The four independent variables were four different teaching methods - Basics First, Structured Problem Solving, Guided Generation, and the Control Group (Table 2).

Table 2: Experimental Group Assignment

Cognitive Engagement Procedural Knowledge
Teaching Methods GIS Problem Solving Problem Solving
Delayed-Retention
GIS Basic Skills Basic Skills
Delayed-Retention
Basics First . . . .
Structured Problem Solving . . . .
Guided Generation. . ..
ArcVoyager Guide (Control). . . .

Setting

The instruction for all GIS methods took place in a Macintosh computer lab. All teachers had their own computers. All three web-based modules were burnt on a CD-ROM so access time to text, graphics and videos was not a problem. The instructor had a Macintosh computer connected to a projector that was viewed on an 8 by 8 foot theatre screen for instruction. All modules and videos were available to the teachers and were also used for teaching by the instructor on the theatre screen.

Procedures

Teachers who volunteered to participate in the study were divided according to the days they were available to have a maximum of 24 teachers in the computer lab at one time. The instruction began with each student completing the Witkin's Group Embedded Figures Test (GEFT) to determine his or her field articulation. The GEFT procedure strictly followed the GEFT manual (Witkin et al., 1971) which required a maximum of 20 minutes for the instructor to give the entire exam. Upon completing the GEFT, the students launched Microsoft Internet Explorer and went to a predetermined bookmark where they answered the demographic questions online. These questions were recorded automatically in a database.

To account for instructor bias in the delivery of each teaching method, an instructor observer was trained to make systematic observations based on Moore's (2001) "making systematic observations" guidelines. The observer collected descriptive data - "data that have been organized, or quantified by an observer, but do not involve a value judgment" (p. 294), and valued data - "data that involve the judgment of an observer" (p. 294), using frequency and duration measurements. The data from each module instruction were compared to make sure instructor bias was minimal.

Of the 160 teachers that were invited to participate in the study, 40 were instructed in the "Basics First" method, 35 were instructed in the "Structured Problem Solving" method, 35 were instructed in the "Guided Generation" method, and 32 received the control group instruction.

Basics First, Immediate Feedback Direct Instruction

Using the "Basics First" ArcVoyager module, the instructor taught all ten sections of the module in a directed method of teaching where he described a subskill or subconcept while providing a graphic that highlighted the subskill or subconcept. After each subskill or subconcept instruction, the instructor asked the teachers to raise their hand if they had any questions. If there were questions, the instructor answered the question fully before proceeding to the next section of the module. The teachers could see the text and graphic for the instruction on the instructor's theatre screen and could also follow along on their own computer. Once a subskill or subconcept was introduced, the teachers practiced the skill using ArcVoyager.

An example of the "Basics First" method was instructing the teachers how to add data layers within ArcVoyager. First, a description and graphic of a data layer was provided to the teachers in the module and they were instructed what a data layer is and how it is imported into ArcVoyager. Second, the teachers were asked if they had any questions about adding a data layer. If there were questions, the instructor made sure they were completely answered. Lastly, after all of the questions were answered, the teachers practiced the skills themselves.

This method of teaching continued until all subskills and subconcepts were covered for all ten sections of the module. Upon the completion of the module, the teachers were given 90 minutes to complete the exam identifying the location they believed was the answer. The teachers were able to ask questions of the instructor at any time throughout the exam. The instructor did not provide answers to the exam question, but focused on answering procedural knowledge questions. The teachers wrote the latitude and longitude of their answer and recorded their response confidence. Upon completion of the first project exam, teachers completed the GIS basic skills exam.

The teachers' delayed-retention was measured when the students reconvened in the same computer lab to take the second project exam titled "Magic Dan's Extreme Ski Shop" (Esri 1998) two weeks later. The teacher once again identified the location they believed was the correct answer by writing the latitude and longitude and recorded their response confidence. The teachers also completed the second GIS basic skills exam. They were given 90 minutes to complete both exams.

Structured Problem Solving Instruction

The teaching approach in this module consisted of the following three parts: First, the instructor presented each subskill and subconcept of the "Basics First" module. Unlike the "Basics First" approach, the instructor did not ask for confirmation of understanding. This section was presented using the instructor's theatre screen and the teachers could also refer to it on their own computer. Second, the instructor gave the teachers a worksheet with ten complex problem questions to be solved. Answering the ten questions required the teachers to understand many of the concepts and perform many of the skills needed in their project exams. In order to reduce confusion and minimize errors, the teachers had access to a web-based module showing QuickTime videos that modeled task performance to help them complete the ten questions. The teachers were able to begin the first project exam whenever they felt they were ready, either after or during the completion of the ten complex questions.

The teachers completed the project exam titled "The Case of the Missing Ship" (Esri 1997). The teachers wrote the latitude and longitude of their answer and recorded their response confidence. Upon completion of the first project exam, students completed the GIS basic skills exam.

The teachers' delayed-retention was measured when they reconvened in the same computer lab to take the second project exam titled "Magic Dan's Extreme Ski Shop" (Esri 1998) two weeks later. The teachers once again identified the location they believed was the correct answer by writing the latitude and longitude and recorded their response confidence. The teachers also completed the second GIS basic skills exam. They were given 90 minutes to complete both exams.

Guided Generation Instruction

The "guided generation" instruction began with teachers watching a news report announcing the "Case of the Missing Ship" (Esri, 1997). The news anchor described the scenario, identified the nine clues that were available to solve the case, and gave the assignment - to solve where the ship is located using ArcVoyager.

Once the teachers watched the "missing ship" news clip, the teachers were immediately given the exam to work on synchronously with access to the same web-based module as the "Structured Problem Solving" group. However, in addition to the "Structured Problem Solving" instruction, web-based QuickTime videos that instructed students on the four content domains that were needed to solve the problem of finding the ship were developed to scaffold their learning. These domains consisted of the biosphere (people, plants, animals), atmosphere (air), lithosphere (rocks), and hydrosphere (water). These videos scaffolded the teachers' learning as it provided instruction on how to add the data layers of the earth science content knowledge. For example, one of the clues in solving the exam is as follows: "Records indicate that the January temperature is around 25 degrees Celsius, perhaps a little higher, perhaps a little lower. Data indicate that, on the day of the ship's disappearance in June, the site was between 20 and 30 degrees Celsius." Therefore, the teachers needed to know how to import the layer, "Atmosphere - Air", that contained the temperature data on the day the ship went down, June 1, 1995. One of the videos showed how to import, identify, and select the correct layer to find the data on temperature.

The teachers used the clues from the news clip and the web-based modules on the use of ArcVoyager and the four content domains to solve the problem. The teachers wrote the latitude and longitude of their answer and recorded their response confidence. Upon completion of the first project exam, students completed the GIS basic skills exam.

The teachers' delayed-retention was measured when they reconvened in the same computer lab to take the second project exam titled "Magic Dan's Extreme Ski Shop." The teachers once again identified the location they believed was the correct answer by writing the latitude and longitude and recorded their response confidence. The teachers also completed the second GIS basic skills exam. They were given 90 minutes to complete both exams.

Control Group - ArcVoyager Guide

The control group's instruction began with an overview of their task - to individually use ArcVoyager to solve the exam titled "The Case of the Missing Ship" (Esri 1997). They were given two handouts to prepare them for this task. First, they received a handout titled "ArcVoyager Quick Start Tutorial" (see http://www.geographyeducation.com/gisweb ) which provided an overview of the ArcVoyager Guide and ArcVoyager. Second, the teachers received the exam titled "The Case of the Missing Ship" (Esri 1997). They were instructed to use the tutorial with the ArcVoyager guide to solve the missing ship project exam.

Using the clues from the exam, the teachers wrote the latitude and longitude of their answer and recorded their response confidence. Upon completion of the first project exam, teachers completed the GIS basic skills exam.

The teachers' delayed-retention was measured when they reconvened in the same computer lab to take the second project exam titled "Magic Dan's Extreme Ski Shop" (Esri 1998) two weeks later. The teachers once again identified the location they believed was the correct answer by writing the latitude and longitude and recorded their response confidence. The teachers also completed the second GIS basic skills exam. They were given 90 minutes to complete both exams.

Personal Interviews

Personal interviews were conducted to better understand the teachers' experiences using a geographic information system. Five teachers were interviewed for each pedagogical level for a total of twenty interviews. They were asked questions to help the author better understand the benefits and drawbacks of each pedagogical model. The interview questions can be seen at http://www.geographyeducation.com/gisweb).

Data Scoring

A geographic information system is a spatial environment. Therefore, the two project exams, "The Case of the Missing Ship" (Esri 1997) and "Magic Dan's Extreme Sea & Ski Resort" (Esri 1998), were scored based on the distance of the teachers' answers from the actual answer location. The distance was measured in total degrees latitude and longitude from the correct location. The basic skills tests were graded on the scale of the number of correct out of ten.

Statistical Analyses

The first step to the statistical analyses was the descriptive statistics for the demographic variables and the group means of the dependent and independent variables. The second step, a multiple analysis of variance (MANOVA), was used to see the main and interaction effects of the categorical variables (teaching methods - Basics First, Structured Problem Solving, Guided Generation) on multiple dependent variables. An alpha level of .05 was used to determine if there was significance between at least one group mean with the others (Hand & Taylor, 1987).

A test of group differences was the second step in MANOVA. If the overall F test was significant, a test of group differences was used to identify where the differences occurred. The test used was the Wilk's lambda U, the most common where there are more than two groups formed by the independent variable, as in this study (Hand & Taylor, 1987).

Qualitative Analyses

A constant comparative method (Glaser & Strauss, 1967) was used to guide the development of the salient categories and patterns in the data of the twenty interviews. First, compiling the data that pertained to each person developed an individual data set for each participant. Then, the data was read noting emerging patterns across individuals. The patterns were compiled, shared with other GIS experts, and reread searching for confirming and disconfirming evidence for the patterns. A consensus on the salient patterns that emerged from the data was identified.

Initial Results

This study was designed to determine the best pedagogical approach for teaching preservice teachers GIS based on three pedagogical models - direct instruction, structured problem solving, and guided generation and to identify if field articulation has any impact on GIS learning. Specifically, this study was designed to answer the following research questions:

  1. To what extent will preservice teachers, who are taught with direct instruction, be able to perform on a GIS basic skills exam and problem solve in a GIS environment immediately after instruction and two weeks later?

  2. To what extent will preservice teachers, who are taught with a structured problem solving model of instruction, be able to perform on a GIS basic skills exam and problem solve in a GIS environment immediately after instruction and two weeks later?

  3. To what extent will preservice teachers, who are taught with the guided generation model of instruction, be able to perform on a GIS basic skills exam and problem solve in a GIS environment immediately after instruction and two weeks later?

  4. How does a preservice teacher who is field dependent or field independent influence learning within the three pedagogical approaches - direct instruction, structured problem solving, and guided generation?
This study has just completed and as this paper is being written, data analyses are still being performed. The initial results have shown the following findings.

Student Demographic Data

In order to answer the research questions, several different data analyses were conducted. First, the background variables of the students who participated in this study were compared in order to establish the quasi-equivalence of the treatment and comparison groups. Second, an analysis of teachers' GIS cognitive engagement and procedural knowledge was conducted to determine if there was a difference between groups. Finally, personal interviews were conducted and analyzed at all pedagogical levels to better understand the teachers' GIS knowledge and abilities.

The background variables of the students who participated in this study were analyzed in order to establish the quasi-equivalence of the treatment groups. Within this study, random assignment to groups was impossible, therefore, it is important to demonstrate that the students in the groups have similar demographic characteristics. If it can be shown that the students in the groups have similar background characteristics, then it is more likely that any significant gains in GIS performance are due to treatments and not some other background variable.

The results of the demographic questionnaire were analyzed by using a t-test for the continuous variable and a Pearson's Chi-square test for the categorical variables. Because the four groups had different sample sizes, a pooled variance estimate was used. All analysis was performed using the Statistical Package for the Social Sciences (SPSS, 2000).

The demographic results indicated that, for the most part, they were equivalent. There were no significant differences among variables between any of the groups except between the "Basics First" and the "Guided Generation" groups. The results that were significant between these groups were the "total number of countries traveled to" and the "total number of states traveled to." Given this difference, all performance data in this study were analyzed by these two demographic variables to determine if the differences were merely due to the number of places one has traveled to which was found to be insignificant.

Comparison of Teachers' GIS Cognitive Engagement

The primary purpose of this study was to determine what pedagogical model was most effective when educating preservice teachers how to use a geographic information system. In order to determine the most effective model, independent T tests were used to compare the treatment groups to the control group at the .05 significance level. As shown in Table 3, even though all three treatment groups performed better, none of them performed significantly better.

Table 3: Cognitive Engagement Performance - Exam 1
Treatment GroupNMean*Std. Deviationp**
Control 32 47.701 62.9725
Basics First 40 30.609 50.3295 0.205
Control 32 47.701 62.9725
Structured Problem Solving 35 36.4222 53.4472 0.431
Control 32 47.701 62.9725
Guided Generation 35 38.54107 38.5417 0.353
* Lower number is better performance.
** Significant at the .05 level.

However, as shown in Table 4, when delayed-retention was measured two weeks later, the "Basics First" and "Structured Problem Solving" group performed significantly better than the control group.

Table 4: Cognitive Engagement Performance - Exam 2
Treatment Group N Mean* Std. Deviation p**
Control 32 141.394 93.9213
Basics First 40 55.107 50.3292 0.000
Control 32 141.394 93.9213
Structured Problem Solving 35 36.422 53.4472 0.001
Control 32 141.394 93.9213
Guided Generation 35 133.335 102.5423 0.748
* Lower number is better performance.
** Significant at the .05 level.

When cognitive engagement was measured between groups, rather than just to the control group, there were no significant difference sbetween the three treatment groups (Basics First, Structured Problem Solving, Guided Generation).

Comparison of Teachers' GIS Procedural Knowledge

The teachers' procedural knowledge was analyzed by performing a T test based on their GIS basic skills exam. As shown in Table 5, all three treatment groups performed significantly better than the control group at the .05 level.

Table 5: Procedural Knowledge - Exam 1
Treatment Group N Mean* Std. Deviation p**
Control 32 5.031 1.8749
Basics First 40 7.700 1.5392 0.000
Control 32 5.031 1.8749
Structured Problem Solving 35 7.7714 1.5546 0.000
Control 32 5.031 1.8749
Guided Generation 35 6.600 1.4994 0.001
* Higher number is better performance.
** Significant at the .05 level.

Two weeks later after the first exam, the GIS basic skills test was given again. As shown in Table 6, once again, all treatment groups performed better than the control group at the .05 level and the "Basics First" and "Structured Problem Solving" groups performed significantly better.

Table 6: Procedural Knowledge - Exam 2
Treatment Group N Mean* Std. Deviation p**
Control 32 4.563 1.8826
Basics First 40 5.820 1.8098 0.005
Control 32 4.563 1.8826
Structured Problem Solving 35 6.531 1.8468 0.000
Control 32 4.563 1.8826
Guided Generation 35 5.433 1.775 0.066
* Higher number is better performance.
** Significant at the .05 level.

When procedural knowledge was analyzed between groups, rather than just to the control group, there was no significant difference between the three treatment groups (Basics First, Structured Problem Solving, Guided Generation).

Field Articulation and GIS Performance

Field dependence and field independence is a spatial ability cognitive style that may impact the ability to successfully perform in a computer-based instructional environment (Riding & Cheema, 1991). The fourth research question of this study pushed to find if field articulation might influence how a person is able to perform in a GIS environment. It was found that field articulation had no significant influence on either GIS cognitive engagement or procedural knowledge.

Pearson Correlations for all Treatment Groups

Table 7 shows the significant correlations of all variables when compared to GIS cognitive engagement and GIS procedural knowledge.

Table 7: Correlations - Cognitive Engagement & Procedural Knowledge
Cognitive Engagement Performance - Exam 1
Significant Correlations Pearson Sig. (2-tailed)*
# of Science Courses Taken -0.196 0.022
GPA -0.557 0.001
Response Confidence - Exam #1 0.438 0.000
Response Confidence - Exam #2 -0.187 0.029
Procedural Knowledge - Exam #1 -0.212 0.013
Procedural Knowledge - Exam #2 -0.363 0.049
Cognitive Engagement Performance - Exam 2
Significant Correlations Pearson Sig. (2-tailed)*
Rating of GIS Knowledge -0.213 0.009
Response Confidence - Exam #2 -0.452 0.000
Procedural Knowledge - Exam #1 -0.336 0.000
Procedural Knowledge - Exam #2 -0.219 0.010
Procedural Knowledge - Exam 1
Significant Correlations Pearson Sig. (2-tailed)*
Rating of GIS Knowledge 0.233 0.006
Rating of Computer Skills 0.203 0.017
Response Confidence - Exam #1 0.218 0.011
Response Confidence - Exam #2 0.360 0.000
Cognitive Engagement - Exam #1 -0.212 0.013
Cognitive Engagement - Exam #2 -0.336 0.000
Procedural Knowledge - Exam #2 0.705 0.000
Procedural Knowledge - Exam 2
Significant Correlations Pearson Sig. (2-tailed)*
Rating of GIS Knowledge 0.208 0.015
Rating of Computer Skills 0.171 0.046
Response Confidence - Exam #1 0.222 0.009
Response Confidence - Exam #2 0.307 0.000
Cognitive Engagement - Exam #2 -0.279 0.010
Test Time - Cognitive Eng. - Exam #2 0.221 0.010
Procedural Knowledge - Exam #1 0.705 0.000
* Significant at the .05 level.

As Table 7 indicates, the correlations between procedural knowledge and cognitive engagement were significant for both cognitive engagement exams and procedural knowledge exams. The number of science courses significantly correlated with the first cognitive engagement exam as well as GPA. Response confidence for both cognitive engagement exams significantly correlated with all exams except cognitive engagement exam two where only the second response confidence was significant. Self rating of GIS knowledge correlated with both procedural knowledge exams and the second cognitive engagement exam. Self rating of computer skills correlated with procedural knowledge exams one and two.

Discussion

This study was designed to determine what GIS pedagogy is most effective when teaching preservice teachers how to use a geographic information system. GIS has many times been taught with paper-based tutorials - the pedagogy for the control group in this study. The three treatment groups - "Basics First" (Basics), "Structured Problem Solving" (SPS), and "Guided Generation" (Guided), received three strictly developed pedagogies based on the work of teachers teaching the Jasper Woodbury Problem Solving Series from the Cognition and Technology Group at Vanderbilt.

Cognitive Engagement Exams

This study found that teachers completing the first cognitive engagement exam did not perform significantly better depending on what treatment group they were in. However, when the teachers returned two weeks later to take the second cognitive engagement exam, teachers who were in the Basics and SPS treatment groups all performed significantly better. The mean scores were markedly different with the Basics group scoring 86 degrees and the SPS group scoring 105 degrees closer than the control group to the correct answer.

An in-depth analysis of why teachers may have performed better in the Basics and SPS treatment groups was provided in the personal interviews. Teachers in the Basics group identified two main reasons why they believed they performed well. They included 1) access to the instructor to ask questions and 2) opportunity to practice GIS procedural steps until they felt "comfortable." Even though the teachers had access to the web-based module, none of the teachers interviewed said they used it beyond the initial instruction. They noted that it was much "quicker," "easier," and "less stressful" to simply ask the instructor. One teacher said, "I would much rather wait until [he] had time to answer my question than go searching for it on the web-based module." They also felt that the majority of their questions were answered in the beginning because they were able to practice the "basics" until they had no further questions.

Teachers in the SPS treatment group identified two main reasons why they felt they were successful. They included 1) the opportunity to practice GIS within a situation that was relevant to the exam and 2) the ability to watch the movie at any time throughout the exam. Four of the five students that were interviewed stated that the practice questions "definitely helped" as they were able to "see exactly why [they] were doing the GIS steps." All of the students mentioned that they used the QuickTime videos that were provided for them. They said they "bypassed" the graphics and "jumped right to the videos." One student commented, "It was great! I could fast forward, rewind, stop, - I had complete control over finding the answer and matching what I was supposed to do in ArcVoyager with the QuickTime videos."

When the students in the Basics and SPS treatment groups were asked if they felt they understood what GIS is and how it could be used, all ten of them answered "yes." They believed they "knew enough to be successful" within their classroom. They felt they new what the benefits of using GIS are and commented on them in detail. The next step they felt was to start developing lessons that integrated the GIS technology.

The Guided treatment group did not perform significantly better than the control group on both cognitive engagement exams. Within the interviews, the teachers commented that they understood the task as the "video was excellent in setting the stage", but they felt "lost" and "frustrated" when the tried to answer the problem. The students commented that they felt they understood the GIS "functions, but didn't know how to use them." Four of the teachers commented that they would have liked some extra guidance.

Procedural Knowledge Exams

The procedural knowledge exams asked the teachers to identify the correct answer for a GIS function such as "What button do you click if you would like to import a data layer?" All treatment groups performed significantly better than the control group on both procedural knowledge exams. Therefore, the instruction that all treatment groups received improved their knowledge of the GIS interface significantly. The Basics and SPS treatment groups were able to apply that knowledge in an authentic situation, but the Guided treatment group was not. One Guided teacher noted she couldn't "apply [her] knowledge of the GIS to what [she] was supposed to accomplish in the exams."

Implications of This Study

As noted in the beginning of the "results" section, the results that are being reported in this paper are only the initial results. The data that have been presented will continue to be analyzed. However, regarding the data that have been presented thus far, the following consideration should be taken when developing GIS instruction for preservice teachers. They are 1) instructors of GIS have a tremendous influence on GIS learning when they are available to their students when teaching, 2) GIS instruction that is not accompanied by an instructor within the room should have practice questions that set the GIS task within an authentic situation while giving students access to QuickTime videos for GIS skill understanding and review, 3) GIS learners need instruction and direction in their learning, and 4) understanding the procedural knowledge of a GIS influences learning, but not without guidance.

The Basics treatment group performed extremely well in all exams. They noted that access to the instructor was most important in their GIS learning. They would rather wait to ask the instructor than looking on the web-based module for the answer. Therefore, the quality of the GIS instructor was very important when learning GIS. If instructors with different GIS backgrounds and understanding use the Basics pedagogy, one must ask the question, "Will the GIS learners perform at such a high level?" This also raises the second point of online GIS learning.

The SPS treatment group also performed significantly well with only access to the complex practice questions and QuickTime videos. Therefore, if one were going teach GIS online or would not be able to offer assistance at a level that is required in the Basics treatment group, the SPS approach would be most beneficial. Learners in the Guided treatment group noted this assistance in GIS learning as the missing variable.

The Guided treatment group felt that they understood what they were supposed to do after they watched the anchoring video (http://www.geographyeducation.com/gisweb/guidedindex.html ) but didn't "know where to start." This was even when they had scored significantly better on the procedural knowledge exam. They noted that they needed to know how to apply the GIS skills to solving the task at hand. Therefore, a constructivist approach that provides no instructor guidance with GIS learning was not found to be effective.

This study strived to find an effective pedagogy when teaching preservice teachers GIS. Of the four approaches used, two were identified where the teachers performed significantly better on both GIS cognitive engagement and procedural knowledge. These two identified pedagogies were very different and provide us with two methodologies to use depending if we are able to give GIS learners assistance within the classroom throughout a four hour period or if we are encouraging the GIS learners to learn on their own with only the tools that we have provided them. The initial finding in this study will provide GIS educators two successful ways of teaching preservice teachers as well as other GIS educators. It is hoped that these pedagogies will encourage others to pursue avenues of research related to GIS pedagogy in preservice teacher education as well as in the k-12 classroom.

References

Audet, R. H., & Paris, J. (1997). GIS Implementation Model for Schools: Assessing the Critical Concerns. Journal of Geography, 96, 293-300.

Bednarz, S. W. (1995). Reaching New Standards: GIS and K-12 Geography. Retrieved April 10, 2001, from the World Wide Web: http://www.odyseey.maine.edu/gisweb/spatdb/gislis95/gi95006.html

Bednarz, S. W., & Audet, R. H. (1999). The Status of GIS Technology in Teacher Preparation Programs. Journal of Geography, 98(2), 60-67.

Brownwell, K. (1997). Technology in teacher education: Where are we and where do we go from here? Journal of Technology and Teacher Education, 5, 117-138.

CTGV. (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19(6), 2-10.

CTGV. (1992). The Jasper Experiment: An Exploration of Issues in Learning and Instructional Design. Educational Technology, Research and Development, 40(1), 65-80.

Day, D., McRaie, L. S., & Young, J. D. (1990). The Group Embedded Figures Test: a factor analytic study. Perceptual and Motor Skills, 70(3), 835-839.

Environmental Systems Research Institute, I. E. (1998). White Paper: GIS in K-12 Education. Esri Press.

Esri. (1997). Case of the Missing Ship. Retrieved February 5, 2001, from the World Wide Web: http://www.Esri.com/k-12

Fitzpatrick, C. (1990). Computers in geography instruction. Journal of Geography, 89, 148-149.

Fitzpatrick, C. (2002). Personal Communication. In A. Doering (Ed.). St. Paul.

Fredrick, B., & Fuller, K. (1997). What we see and what they see: Slide tests in geograpy. Journal of Geography, 97, 63-71.

Keiper, T. (1999). GIS for Elementary Students: An Inquiry Into a New Approach to Learning Geography. Journal of Geography, 98, 47-59.

Kerski, J. J. (2001). The Implementation and Effectiveness of Geographic Information Systems Technology and Methods in Secondary Education. Paper presented at the Esri Education User Conference, San Diego, CA.

National Geography Standards Project. (1994). Geography for Life: National Geography Standards 1994. Washington DC: National Geographic Society.

Nellis, D. (1994). Technology in geographic education; reflections and future direction. Journal of Geography, 93(1), 36-69.

Panek, P. E., Funk, L. G., & Nelson, P. K. (1980). Reliability and validity of the Group Embedded Figures Test across the life span. Perceptual and Motor Skills, 50, 1171-1174.

Riding, R., & Cheema, I. (1991). Cognitive Styles--An Overview and Integration. Educational Psychology: An International Journal of Experimental Educational Psychology, 11(3-4), 193-215.

SPSS. (2000). Statistical package for the social sciences (Version 10.0). Chicago: Author.

Sui, D. (1995). A pedagogics framework to link GIS to the intellectual core of geography. Journal of Geography, 94, 578-579.

Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press.

Witkin, H. A., Oltman, P. K., Raskin, E., & Karp, S. A. (1971). A Manual for the embedded figures test. Palo Alto: Consulting Psychologists Press.

Wittrock, M. C. (1974). Learning as a generative process. Educational Psychologist, 11, 87-95.


Aaron H. Doering
University of Minnesota

Author Note
Correspondence concerning this article should be addressed to Aaron H. Doering, Department of Curriculum and Instruction, University of Minnesota, 159 Pillsbury Dr. SE, 130B Peik Hall, Minneapolis, Minnesota 55455. Electronic mail may be sent to adoering@umn.edu.