By

James Walker and Thomas Conry

Abstract: The implementation of an enterprise-wide GIS in a large metropolitan county presents many technological and logistical hurdles that must be overcome. With an ever-increasing demand for access to the GIS from more than 50 remote offices and over 200 concurrent GIS users, providing high-performance data access to 400GB of raster and vector GIS data becomes an issue.

How to serve the most recent GIS data to over 50 remote offices? How to provide top of the line computing performance that is easily accessible to the GIS user? How to efficiently perform hardware, software, and operating system upgrades on remotely distributed GIS workstations? How to provide a full suite of GIS applications to over 10,000 county employees with minimal hardware, software, and infrastructure upgrades?

One solution, Server Based Computing, was implemented in Fairfax County, Virginia to resolve the above hurdles and provide many additional benefits. "Server Based Computing" is a new implementation of an old computing paradigm. The personal computer's Windows environment can now be hosted by large server farms, which communicate to "thin" web-distributed clients on the end-user's computer with minimal network bandwidth.

The Server Based Computing solution provides many benefits including answers to the questions at the beginning of this article. The most recent GIS data is located on centralized database and file servers with high-speed network connectivity to a farm of GIS application servers. Users access the GIS application server farm through a web browser, which only consumes 10-20 kb/user of bandwidth. Hardware, software and operating system upgrades are performed by seamlessly upgrading individual nodes in the server farm. And finally all 10,000+ employees have access to the GIS with no additional client software installations or PC upgrades.

This paper is aimed at people who have a medium to advanced knowledge of Windows NT system and network administration. The aim of the paper is to provide one possible solution to the ever-increasing demand for GIS technology and data within large organizations. Lessons learned at Fairfax County will be provided to show pitfalls that may be encountered when implementing a Server Based Computing solution for an enterprise-wide GIS.

In Fairfax County, GIS began in 1992 with the purchase of ArcInfo running on 3 RS-6000 UNIX workstations. This was a typical beginning for GIS in most jurisdictions both prior to this time and for several years after 1992. After all GIS was relegated to mini-computers in the 80's and high powered, expensive UNIX workstations in the late 1980's to mid 1990's.

Then came the desktop GIS revolution. All of the sudden the GIS analyst could store, manipulate, query and analyze small amounts of data on inexpensive Intel/Windows PCs. In 1995, Fairfax County developed a new GIS strategy to "Quick Start" the GIS hardware into as many County Agencies as possible. With the same budget that was going to put UNIX based GIS hardware into 5 agencies, Fairfax County was now able to put Windows based GIS hardware into 22 County agencies.

Also, at this same time a new data conversion strategy was developed to "Quick Start" the GIS data. Instead of spending millions of dollars to digitize the typical cadastral and planimetric data, the GIS Quick Start data conversion effort focused on the large scale data used to produce 30 County-wide wall maps. These wall maps had previously been maintained by traditional cartographic methods.

A year or so after the successful implementation of the "Quick Start" GIS program, the ArcInfo software was migrated to the Windows NT desktop. At about this same time the first incarnation of the Internet Map Server was coming on the scene. These two advances completed the move of GIS to the Desktop. Users could access the full set of GIS tools loaded on a high-end Windows NT desktops; view, query and analysis tools on mid-range Windows desktops; and view, and query tools through a thin-client on a low-end PCs running a web browser.

The latest phase of this technological merry-go-round is to serve all three sets of GIS tools (ArcInfo, ArcView, and Map Objects) to mid-range or low-end PC desktops from high-speed application and data servers. The rest of this paper will discuss Fairfax County's experience in this latest phase of computer and GIS technology, Server Based Computing.

There are several methods for distributing GIS software and data to the end-user (Client Server; Windows NT Server, Terminal Server Edition with Citrix MetaFrame, and Esri's Internet Map Server). The following section describes these delivery methods and the reasoning for using or not using each method.

The client server method for providing access to GIS software and data consists of locally loaded GIS software with network connectivity to the data server. This method of accessing the enterprise GIS is frequently used in small centralized GIS organizations. Two prerequisites for this type of implementation are high-powered workstations and high-speed networks (preferably 100Mb switched Ethernet or higher). This is frequently the preferred method for serving the GIS user that performs CPU intensive modeling. Listed below are some of the benefits and limitations of this system deployment strategy.

• The processing demands of user do not affect the processing requirements of other users.
• Individual user control over their client hardware, (if they need more power they can buy a faster PC).
• Application software is loaded and executed locally.
• Snap shots of data can be loaded on local disk storage to increase performance.
• Good platform for small installations.

• Individual users have the ability to modify the operating system and hardware, which frequently requires an IT support professional to correct the user inflicted damage.
• Upgrading large numbers of PCs to support GIS applications can be very expensive.
• Requires high bandwidth (preferably 100Mb switched Ethernet or higher) between the client and the data server.

In order to facilitate the maintenance and distribution of the most current GIS data, Fairfax County implemented a centralized GIS data warehouse. The next step was to determine the best method for proving access to this data without having to upgrade several hundred PCs and the network connectivity to those PCs. The solution we decided upon was server based computing. The server based distribution of the GIS software and data was tested for several months in the summer of 1999, and was then implemented in the fall of 1999. With Fairfax County's GIS users spread over a 400 square mile area in over 50 separate facilities this has quickly become the preferred method of GIS access for most users.

This implementation method requires one or more high-powered servers running Windows NT Server 4.0, Terminal Server Edition with Citrix MetaFrame 1.8 and enough licenses of the GIS software to meet the user demand. For the best performance one or more high-bandwidth connections (100Mb switched Ethernet or higher) from the server to the GIS data warehouse are required. Since the Citrix MetaFrame servers and GIS data warehouse servers should be located in a centralized computer room, the cost of these high-bandwidth connections is relatively inexpensive.

The communication link between the Citrix application servers and the client PCs need only be 16Mb Token ring or 10Mb-switched Ethernet. This relatively slow connection is more than adequate since the ICA protocol (Citrix communication protocol) only requires 10 - 20 Kb/s. The only network traffic between the application servers and the client PCs are screen scrapes, keystrokes, and mouse clicks. All data transfers occur between the application servers and the data warehouse servers.

In addition to running Citrix MetaFrame 1.8 on each application server, a thin client must be loaded on the client PC. There are two types of thin clients available from Citrix, one is a thin client application that is manually installed on each PC, the second is an automatically installed web browser plug-in. Fairfax County primarily uses the web browser plug-in to minimize desktop visits by IT support staff.

The GIS applications are made available to the end-user through the Fairfax County InfoWeb (Intranet). Presently, this is made possible by publishing a hyperlink to the software on the application server (Citrix MetaFrame can create the necessary support file and hyperlink to make this connection). As this paper is being written Fairfax County is testing a new Citrix product, NFUse. This product will create an HTML program neighborhood that will display all available published applications through one hyperlink. Otherwise, a separate hyperlink must be created for each application.

Listed below are the benefits and limitations of the application server distribution method.

• Low power, low-bandwidth clients make perfectly good GIS workstations when using Citrix MetaFrame application servers.
• All 10,000 County employees have access to the GIS as long as they have a PC connected to the network or modem.
• Since the user accesses the GIS though a web browser, user modifications to their personal PC have virtually no impact on the performance of the GIS software.
• High bandwidth connectivity between the application servers and the data warehouse servers provides high performance access to the most current GIS data.
• The GIS software is only loaded on the application servers, not the 6,000 personal PCs spread through the organization. Therefore, software patches, upgrades, and new extensions are made available in a very timely manner.
• Problems created by heterogeneous networks, and several different Operating Systems are eliminated in regards to the GIS software environment.
• Help Desk support using Citrix session shadowing. Help Desk staff are able to remotely view and manipulate the end-user's GIS application.
• Staff visits to the desktops are significantly reduced.

• Microsoft NT Server, Terminal Server Edition limits the clients color palette to 256 colors on the screen. The color palette for export files and print jobs are not affected by the 256-color display limitation.
• The processing demands of a specific user may affect the processing requirements of other users. This problem can be reduced with proper system design and monitoring.
• If only one server is in use, a hardware or software failure could disrupt services for several hours or until the system problem is resolved.

Another form of server based computing being implemented in Fairfax County is Esri's Map Objects Internet Map Server 2.0. This software also provides thin-client access to the GIS data warehouse through customized web-enabled visual basic mapping applications. With an application published through Citrix MetaFrame the user has full access to all of the functionality of the GIS software whether it is Workstation ArcInfo of ArcView. However, with Map Objects Internet Map Server a software developer must first create an application using Visual Basic, for example. Then the user will only have access to the limited functionality that Map Objects provides.

This implementation method requires one or more high-powered servers running Windows NT Server 4.0. For the best performance one or more high-bandwidth connections (100Mb switched Ethernet or higher) to the GIS data warehouse are required. Since the Internet Map Servers and GIS data warehouse servers should be located in a centralized computer room, the cost of these high-bandwidth connections is relatively inexpensive.

As with any other web based application the communication link between the Internet Map Servers and the client PCs can easily be handled by modem connection speeds of 28.8Kb or 56Kb. All data transfers occur between the application servers and the data warehouse servers. The GIS applications are made available to the end-user through the Fairfax County InfoWeb (Intranet) and the Fairfax County Internet web site.

Listed below are the benefits and limitations of the Internet Map Server distribution method.

• Low power, low-bandwidth clients make perfectly good GIS workstations when using Internet Map Server.
• All 10,000 County employees have access to the GIS as long as they have a PC connected to the network or modem.
• Since the user accesses the GIS though a web browser, user modifications to their personal PC have virtually no impact on the performance of the GIS software.
• High bandwidth connectivity between the application servers and the data warehouse servers provides high performance access to the most current GIS data.
• The GIS software and customized Visual Basic applications are only loaded on the Internet Map Servers, not the 6,000 personal PCs spread through the organization. Therefore, software patches, upgrades, and new extensions are made available in a very timely manner.
• Staff visits to the desktops are significantly reduced.

• The processing demands of a specific user may affect the processing requirements of other users. This problem can be reduced with proper system design and monitoring.
• Map Objects Internet Map Server GIS applications must written by existing staff or contractors.
• The functionality of Map Objects Internet Map Server is limited to the subset of GIS functions available to Map Objects.

When providing GIS technology to a large user base, the requirements of the users need to be considered very carefully. Purchasing a high-end gigahertz workstation for a user that creates a handful of page size maps for an annual report is just as wasteful as purchasing a network PC for a user performing sophisticated overlay analysis and 3D modeling. The following three tiered approach to classifying the GIS users has been developed to assist GIS Analysts and System Analysts in properly specifying hardware and network requirements for GIS applications and end-users.

In the first tier are the "professional" GIS users that perform frequent edits to the GIS database and other intensive interactive tasks such as Spatial and 3D modeling using high-end NT workstations, performing the work with their local CPU. In order to perform these tasks on the local workstation's processor(s) the system will need to have enough local disk space to hold the necessary data for the project or the system will need at least 100Mb switched Ethernet connectivity to the GIS Data Warehouse.

In the second tier are the "intermediate" GIS users that perform typical display, query, analysis, and mapping tasks using normal desktop PCs that access the application server farm. By accessing the application server farm, virtually all processing and data transfer tasks are performed on powerful high throughput servers. Since all the processing and data transfers occur between the application server farm, the file server, and the Oracle/SDE database server, the requirement for a high bandwidth network is limited to the computer room where the application, data, and file servers are located. The only network traffic that occurs between the client PC and the application server farm are screen scrapes and mouse or keyboard clicks, which require about 10k of network bandwidth.

In the third tier are the "casual" GIS users that use a very generic data browser or a highly customized and easy to use application. These users access the application server farm or the internet map server farm using a network or normal desktop PC that may have network connection speeds as slow as 56Kbs.

As with many GIS departments that came to existence five to ten years ago, the GIS Branch in Fairfax County was responsible for planning and implementing its own Information Technology plan. Also, since the GIS software frequently requires hardware and operating systems that fall somewhere between the Proven Leading Edge of Technology and the Bleeding Edge of Technology, the GIS Branch is often times required to blaze these new Information Technology frontiers.

With the GIS Department's start in the UNIX world we have looked at extending our investment in RS-6000 servers and disk storage devices. Also, when referencing Esri's White Paper "System Design Strategies" July 1999, the scalability of the RS-6000 far exceeds that of the Intel platform running Windows NT. The GIS Data Warehouse consists of an RS-6000 model S70 Advanced Server with four CPUs and two GB of RAM. This server is running AIX 4.3.2 and Samba 2.0.5 for file sharing.

The link between AIX and the NT environments is made by Samba. It is a UNIX implementation of the Server Message Block protocol (SMB), which enables any client running SMB (any Microsoft Operating System) to natively see the UNIX file systems as Windows NT file systems. The Samba product, which is free via GNU, eliminates the need to purchase, install, configure and maintain NFS clients on the end-users PCs.

The disk storage devices that are served by the S70 Advanced Server are the IBM 7133 Serial Storage Architecture (SSA) disk sub-systems running in RAID-5. This storage sub-system is a very scalable architecture, which on some tests has out performed other storage sub-systems including the fastest SCSI and Fiber channel systems. The 7133 SSA is one of the key building blocks for the IBM Enterprise Storage Server, which is being closely examined as a high speed, highly fault tolerant, heterogeneous storage server for the GIS and possibly for the County as an enterprise.

The S70 Advanced Server also serves as the database server for Esri's Spatial Database Engine 8.0.1 running on Oracle 8.1. This GIS Data Warehouse consists of over 1.8 million files and over 30 million unique data elements.

The Internet Map Server farm is made up of two Intel based servers running Windows NT Server, Terminal Server Edition. Each server has 4GB of ram with 18GB of RAID-5 disk storage. One server is a dual 500Mhz-xeon processor with 2MB L2 cache, and the second server is a quad 500Mhz-xeon processor with 2MB L2 cache. Each server is connected to the network with dual GB Ethernet network interface cards to provide high throughput and failure redundancy. The dual processor runs two instances of each Internet mapping application and the quad processor runs four instances of each Internet mapping application.

The Application Server farm is made up of four Intel based servers running Windows NT Server, Terminal Server Edition and Citrix MetaFrame 1.8. Each server has two 733Mhz CPUs, 1GB of ram with 27GB of RAID-5 disk storage. Each server is connected to the network with dual GB Ethernet network interface cards to provide high throughput and failure redundancy. Each server is capable of serving 5-10 ArcInfo users, or 10-20 ArcView users, or 20-40 MapObjects users.

A key component to the successful deployment of an enterprise GIS includes the use of development servers. Once a program such as GIS moves from a pilot project to a full-fledged enterprise application it is very important to thoroughly test new hardware, software, patches/fixes, and tuning modifications. The end-user will quickly become frustrated with a system that becomes unstable from untested software or system tuning procedures that actually mistune a system.

The GIS Branch has an RS-6000 model 591 with 3/4 GB ram and 72 GB of disk storage that is used to test system and network performance. This server is also used to test Oracle and SDE configurations. The GIS Branch also maintains development and beta servers for Windows NT Server with Internet Information Server, Windows NT Server Terminal Server Edition, with Citrix MetaFrame 1.8, Windows NT Workstation, and Windows 95/98. The Intel-based development and beta servers consist of older model workstations and servers that are still capable of providing adequate processing performance for testing and development.

The GIS Department maintains its own Windows NT Domain with a Primary Domain Controller and a Backup Domain Controller, both of which run on Intel Pentium 133 MHz PCs. As the Department of Information Technology migrates the entire County from a mix of Novell and OS2 servers and a variety of Windows workgroups to a centralized NT domain scheme the GIS NT domain will be gradually phased out.

The GIS Branch relies on a 300Mhz Intel server with 18 GB of RAID-5 disk storage running Windows NT Server, Terminal Server Edition to provide access to a network installation of the GIS software and FlexLM license management services. This server also provides countywide access and spool areas to the following output devices:

• HP 8500DN color laser printer.
• HP 8100DN high speed B&W laser printer.
• Two HP 1055CM 36" color inkjet plotters.
• HP 2500CP 36" color inkjet plotter.
• Two HP 3500CP 54" color inkjet plotters.
• A variety of workgroup and personal class laser and inkjet printers.

A 300Mhz Intel server with 18 GB of RAID-5 disk storage running Windows NT Server, Terminal Server Edition is used to provide access to Microsoft's System Management Server and SQL Server. This server is also ready to be substituted as a replacement for the GIS Print and Software Server described above. This server is primarily used to manage software licenses not managed by the FlexLM software, and to provide a method for automated software deployment, system monitoring and reporting of system health within the GIS Branch.

The GIS Branch and several other high-end power users throughout the County require workstation class systems to do the amount of complex modeling and other development work which is best suited for stand alone workstations. The most recent systems purchased are 550 MHz xeon processor with 256 MB ram, 18 GB 10,000 rpm SCSI disk drive, 100 Mb Ethernet Card, High Speed Open GL Active Graphics Port graphics cards, and 21" high resolution monitors. Several GIS will be connected to a gigabit-switched Ethernet to facilitate 3D modeling and other network and/or graphics intensive tasks.

The network within the GIS suite is 100Mb-switched Ethernet running on category 5 wiring. The GIS suite is connected to a gigabit switched Ethernet backbone. The computer room, which is managed by the Department of Information Technology, is wired with both 100Mb and Gigabit switched Ethernet. The GIS Internet Map Servers, Application Servers and the GIS Data Warehouse are connected to the Gigabit switched Ethernet. The Department of Information Technology, including the Geographic Information Services Branch, is aggressively investigating Storage Area Network (SAN) technologies to improve data throughput between application and data servers.

The rest of the County consists of several hundred remote offices connected to the Wide Area Network or connected to the network through dial-up modems. Approximately fifty of the remote offices are actual users of the GIS. Through the course of this year, seven of the large Human Services Centers will be upgraded to a fractional DS-3, the 13 police stations will be upgraded to full T-1 (1.544 mbps) service, and the 40 fire stations will be connected to network with T-1 or 56K frame-relay. Two other large customers of the GIS are the 23 libraries and nearly 200 public school facilities.

In addition to the remote offices listed above the County consists of two large office campuses. Both of these office campuses are running on a backbone consisting of Gigabit Ethernet over single-mode fiber. The County is in the process of converting from a shared token-ring environment to a switched Ethernet environment providing 10/100Mb Ethernet to the desktop.

The desktop systems within the County consist of Intel based processors ranging from 133Mhz to 600Mhz Pentium PCs, with between 32MB to 128MB of ram. The County uses a four-year PC lifecycle for replacing the desktop PCs. The older systems have 15" screens, while the newer systems have 17" screens. If the end-user will spend a significant amount of time working with the GIS a 21" screen is recommended.

The process of publishing an application on the Citrix application server is a straightforward process. Once the GIS software is installed on the server, the application can be "published" using the Citrix Application Publishing Tool. It provides the ability to customize a variety of settings and access security.

Citrix also provides some simple scripting commands that can be used to streamline the login process. In addition to handling network drive mappings, the script commands can easily cleanup any temporary files left on the server by the user, and provide another method for customizing how a particular application starts-up. This extra control is also helpful in providing information billboards to the end-user. These billboards can display information concerning the new data, new GIS tools, and upcoming system maintenance.

Even though the end-user has access to local resources such as printers, these resources lack the performance expected by many users. In order to facilitate printing from the GIS software the printer drivers should be loaded locally on the Citrix servers. The GIS Branch installed its printers on the Citrix server for use by departments that do not have internal facilities to produce high quality, large format output products.

Once the application is published, all specified users can run the GIS software from their desktop. To do this the end-user needs to login to the Citrix NFUse program neighborhood located as a hyperlink on the Fairfax InfoWeb (Intranet). After logging in, the end-user only sees the applications they have been granted permissions to see. This is very useful when dealing with sensitive information that many departments maintain (Police, Health, Child Services, etc.)

When the end-user clicks on a published application's icon the program starts up and appears to be running on the end-users PC. Actually, the application is running on the Citrix server and only screen scrapes of the programs output is displayed on the end-user's PC. In addition to appearing as though the application is running on the local PC the end-user is also able to access local printers, local disk drives, and local communication ports.

At this time Esri only provides license management software for its ArcInfo software. Microsoft's System Management Server was used to ensure license compliance with the number of ArcView and Map Objects applications owned by the County. Microsoft System Management Server was already being used to create hardware and software inventories, as well as, pushing installs of software to the GIS Branch's PCs.

To assist with the management of the Citrix application server farms two additional products were purchased from Citrix. Load Balancing Services, this software load balances the user load across all servers in the in the application farm. Citrix Resource Management Services monitors 33 system activities from network load to number of process threads. The Resource Management Services software has been very helpful in a number of areas. First, hardware and software performance bottlenecks can be diagnosed by reviewing system performance charts. Accurate counts on the number of users per agency can be examined to justify the purchase of additional hardware and software to meet the end-users needs. The accurate user lists are also helpful in targeting newsletter mailings, and for identifying departments that may need assistance in implementing GIS technologies.

Citrix Resource Management Services screen capture.

The process of publishing an application on the Internet Map Server is also straightforward. Unfortunately, the process for creating an application to be published requires a lot more effort. The creation of a Map Objects Internet Map Server application is not discussed in this paper, however, the publication of the application is covered. Once the application is created a hyperlink to the HTML page containing the IMS application is all that is needed.

To assist with the management of the Internet Map Server farm the Citrix Resource Management Services software package was purchased. As described above, this software monitors 33 system activities from network load to number of process threads.

Through the pilot and implementation of the Citrix and Map Objects Internet Map Server projects several lessons were learned. One of the primary lessons deals with high availability of both servers. As computing control is taken out of the end-users domain it is very important that the computing power not be interrupted by hardware or software problems. Once the end-users ability to work has been interrupted no matter how brief the end-user begins to doubt the feasibility of server based computing. Because of this, it is important that when these two methods of server based computing are implemented, that every effort is made to maintain redundant systems both internally to the servers and through the deployment of server farms. A server farm can be thought of as a Redundant Array of Inexpensive Computers (RAIC).

Instead of purchasing one "super server" to meet the Internet Map Server processing needs and one "super server" to meet the Citrix MetaFrame server processing needs it is very beneficial to purchase several smaller servers in place of each "super server". By purchasing several smaller servers, the high availability recommendations specified above are met. Additionally, by splitting the workload to many smaller servers a variety of bottlenecks can be easily negated (CPU processing, system disk IO, memory IO, network IO).

It is also important that appropriate technical support be available for both the Internet Map Server and for Citrix MetaFrame server. In regards to the Internet Map Server the primary support is needed at the front end, the development of the web-enabled applications requires skilled Visual Basic and Web programmers. Technical support in terms of the server is generally covered by Esri technical support that is covered through the software maintenance agreement.

In regards to the Citrix MetaFrame server the primary support is needed at the operating system level (Microsoft NT server and Citrix MetaFrame server). Both Microsoft and Citrix rely heavily on 3^rd party partners to provide technical phone and on-site support. Technical support from the source companies is often twice the cost of the business partner.

In the Fairfax County installation, the best response times have occurred when users have all of their data stored either on the Citrix server on another server with high speed links to the Citrix server. Similarly, plotting is faster when the printer drivers are loaded on the Citrix server. When data are stored locally, (which can be done with Citrix), there is significant data traffic between the server and the client. Typically those links are slower and the overall impact is slower processing.

Desktop installation of Citrix will have some bumps. We occasionally have found applications that interfere with Citrix. Similarly, poorly configured PCs may not run the software well.

James Walker

GIS System Administrator

Geographic Information Services Branch

County of Fairfax

12000 Government Center Parkway, Suite 117

Fairfax, VA 22035-0010

phone 703-324-4064, fax 703-324-3937

email james.walker@co.fairfax.va.us

Thomas Conry

GIS Manager

Geographic Information Services Branch

County of Fairfax

12000 Government Center Parkway, Suite 117

Fairfax, VA 22035-0010

phone 703-324-3909, fax 703-324-3937

email thomas.conry@co.fairfax.va.us