The White Sands Missile Range Environment and Safety Directorate (WS-ES) is developing a hybrid system using Esri tools (including ARC IMS) and Microsoft Site Server to index and access the Directorate's electronic information. The White Sands Environment and Safety Data Management System (ESDMS) includes Site Server components which automatically "crawl" shared directories, classifying shared files according to the Tri-Services Spatial Data Standards (SDS) and the Federal Geographic Data Committee (FGDC) Content Standards for Digital Geospatial Metadata. Current ArcInfo coverages and ArcView shape files are being converted to the Spatial Database Engine (SDE) database format and will be indexed by the Site Server engine. The entire data set will be made available to authorized users via a ColdFusion application interfacing to ArcIMS, which handles the user interface for web display of spatial and non-spatial information. The project has presented several unique challenges and promises to radically change the way WS-ES users view their information storage facilities. The paper will describe the design and implementation of the various subsystems and specifics of the ArcIMS interface development project.
The first step in designing a knowledge management system is to gain a thorough understanding of the members of the target audience - how are they performing their required tasks now, how will this system make the performance of their jobs easier, or more complete? Through on-site, telephone, and e-mail interviews, it was apparent that the audience consisted of a variety of professionals (biologists, botanists, geologists, geographic information specialists, etc.) who are dedicated to their crafts and proud of their accomplishments at White Sands Missile Range (WSMR). They view themselves as stewards of the WSMR resources. Each organization developed its own set of maps and reports as part of their responsibilities for safeguarding natural and cultural resources at WSMR. These various pieces of information, while easily located during their development, were now filed away in a variety of ways, few of which were easy for other division members to locate. It was obvious that a single access point was needed for the various information repositories. This single access point would be the WS-ES Data Management System.
Since each organization within the ES Directorate created and maintained its own set of maps and reports, access to these items by members of other ES organizations was on a case-by-case approval. A typical user would have up to a dozen different userids and passwords to remember. Needless to say, this was awkward at best and discouraged data sharing. Additionally, redundant data was being produced due to the inability or difficulty of inventorying directorate-wide data resources.
There were also concerns about sharing data that was considered sensitive (document and maps that identified archaeological sites or the locations of certain wildlife) or works-in-progress.
Several challenges faced us - the system must be implemented with no impact to the existing business processes, minimal manual input by data owners or users was permitted, application must be web-enabled, no client software would be required, and, of course, do it as cheaply as possible. With these requirements in mind, we knew we had to use software that either already existed at WSMR and/or was compatible with software already in place. By staying in the Microsoft family of tools, we were able to keep costs down and ensure that users could run the application. Esri's ArcIMS was chosen because it would enable the application to display not only the materials created using Esri tools, but also Intergraph materials. Esri's Spatial Database Engine (SDE) would be used in conjunction with a SQL Server database to index the geospatial and related non-geospatial files. Macromedia (Allaire) ColdFusion was already in use at WSMR, so it was a natural choice, especially since ArcIMS includes the ColdFusion Connector. Visual Basic and Visual C++ were used to create the Administrative Utility component.
DMS is comprised of not only multiple software tools, but also multiple processes that combine to make a complete system. Primarily, these processes are data acquisition, data indexing and cataloging, and data access.
The processes begin with the acquisition of the data to be shared. Using Site Server, we built a crawl definition and then crawled multiple servers and their respective folders to obtain a comprehensive inventory of the more than 100,000 data files within the Environment and Safety directorate. It was immediately apparent that multiple copies of documents existed. For example, if twenty people attended a meeting and the minutes of that meeting were sent to those twenty attendees, each of the twenty people filed a copy of the minutes on one or more shared servers, as well as on his/her own workstation. Although the DMS cannot prevent this from happening, it does identify duplicate files via a report.
We recommended that users identify and use one shared folder on each server as the "common" folder that would hold the "official" documents and other files. Files in the Common folder would be designated as read only. The folder would contain form templates, meeting minutes, final reports, published maps, etc. All work in progress would be held in the author's folder on the server. Only the data files in the Common folder would be accessible through the DMS application.
Those files that were identified by the data owners as restricted due to their sensitive nature were flagged in the DMS database. It was recommended that whenever possible, sensitive files would be contained in folders separate from "Public" files. The indexing process of the DMS would flag all files within a designated restricted folder as restricted. When it was not feasible to separate sensitive from public files, sensitive files would be flagged manually through the Administrative Utility. Each data owner was provided a complete listing of his/her files. The data owner could then determine which files would be restricted and which would be public. The DMS administrator would set the parameters with the DMS application accordingly.
All geospatial files identified to be included in the DMS are converted using the ArcSDE tool. The SDE database will be accessed in the next process to gather information needed to associate geospatial files with their related non-geospatial files (for example, a coverage/map and the MS Word document report that discusses the findings or conclusions made as a result of the information displayed in the map).
Once the data files have been crawled and a de-selection process weeded out redundant or superfluous files, the Administrative Utility processes the gather log contents. The folder names are checked against the Restricted Directories table in order to flag all files in a restricted directory as restricted. The process then evaluates each file extension to determine the appropriate proxy tool to use to scan the file to extricate the title, abstract, key words, publication date, place, etc. to be loaded into the database. The SDE database is queried for information pertaining to geospatial data files; this information is coupled with the geospatial file's metadata file information to populate the application database. Photographs are logged into the database with a minimum of information - the full path and file name, the fact it is a photo, its creation date, etc.
The application further processes the data by matching data content against the SDS definitions to correctly catalog the data according to SDS criteria. This ensures that wherever possible, each file can be accessed by its appropriate SDS entity set, class, or type name. Those files that do not contain sufficient information can be cataloged manually via the Administration Utility.
By implementing the DMS, a single userid and password are needed to access any data released for viewing within the ES Directorate. Users do not need to know which organization owns the files they are accessing. A survey of the standard workstation configuration for WS-ES revealed that all users would have the minimum requirements for using the DMS. That is, each user must have Microsoft Office 2000 since many of the shared files were created using Office 2000 products. ArcIMS enables users to display maps. Free browser-based plug-ins are available online, if needed.
Because we wanted to keep the development tools within the same family of tools already being used at WSMR, our choices were pretty easy. We chose a Dell PowerEdge 4400 as the database server. It would host the production database (WSES) and the web application. We chose a Dell PowerEdge 300 as the application process server. It would be used to perform the Site Server crawl process, data indexing and cataloging processes, and host the SDE and processing (WSES_Admin) databases. The NT operating system was installed on each server. Keeping the various processing activities on separate servers greatly enhanced the performance of the application.
The Microsoft BackOffice Suite of tools was used (SQL Server, Site Server, Word, Excel, PowerPoint, etc.), as well as Visual Basic 6.0. Esri ArcIMS (Internet Map Service), ArcSDE (Spatial Database Engine), and ColdFusion Connector were used. Macromedia ColdFusion 4.5 was used to develop the application web interface screens.
The DMS processing database (WSES_Admin) contains tables populated with the SDS Entity Set, Class, and Type name information. This information was captured from the CADD GIS Technology Center website.
The DMS application provides several functions:
The heart of the DMS is the data search capability that enables the user to enter one or more keywords that pertain to the data he/she wishes to access.
The user can also select from a pull-down list of entity set names or further filter the search by entity class and/or type name.
When the user submits the keywords (exact match, match on any word, or match on all words), or the entity set, class, and/or type name, a resulting list of files is displayed.
This display provides the title, data type, creation date and an abstract of the file. It also denotes whether the file is restricted.
If the user is displaying the restricted file information for the first time, a message to "Request Permission" will be in the "Restricted" column. By clicking on the "Request Permission", the application sends an e-mail message to the data owner. The data owner is immediately notified via e-mail of the request and can then, within the DMS application, grant or deny access to the restricted file. An e-mail is automatically sent to the requestor notifying him or her of the action taken by the data owner. If the data owner needs additional information about the requestor, the application will display the requestor's information such as phone number, organization, address, simply by clicking on the requestor's name. If granted access, the user will be able to view the file; the restricted file will have a reminder to safeguard the sensitive information. If denied access, the restricted file will no longer be displayed as a choice in the results list, thus preventing multiple access requests for the same file.
The Title displayed in the Results list is really a link to the file itself; clicking on the title will invoke the appropriate application to display the file in a new browser window. In the case of a map file, ArcIMS is invoked.
Development and implementation of Phase I of the WS-ES DMS succeeded on multiple levels. By conducting a full inventory of the Directorate's servers, several issues were identified which could then be addressed - multiple copies of files, version control of published files and forms, reducing personal files on government servers, and reducing back-up and recovery times by reducing redundant and unofficial files.
An additional benefit was to serve as the impetus to a new naming convention for the server folders containing geospatial data. Incorporation of the Spatial Database Standards into the everyday vocabulary of the Directorate will ease the ability of users to think in terms of the entity set names when discussing geospatial data.
The blending of Esri, Macromedia, and Microsoft products creates a hybrid system that enables its users to access diverse data types through a single portal, oblivious to the owner, location, or data type, while still safeguarding sensitive data.
MEVATEC Corporation served as mentor to Information Technologies Consulting Group, Inc. (ITC) during the development of the WS-ES Data Management System. ITC provided ColdFusion programming support and also assisted in preparation of deliverable documents. Primary contributors were Tom High and Dave Pileggi of the ITC Orlando, Florida office.
Dale Dunham and Gary Melendez of Geographic Information Services, Inc. (GIS, Inc.) of Birmingham, Alabama assisted MEVATEC and ITC in developing the WS-ES DMS Feasibility Study.
Brenda Parton
Senior Systems Analyst
MEVATEC Corporation