Hanming Tu

How AutoAML Can Help In Application Development and Document Management

ABSTARCT: Application and document management sometime can be very time-consuming and stressful. The AutoAML provides a nice tool for you to create templates for your AMLs and Menus, to manage your applications, and to document your applications, database or any file systems in HTML format so that you can publish your applications, databases or documents very quickly. The purpose of the paper is to highlight the AutoAML package and its functions as well as to review the technical challenge encountered in using AML functions and UNIX commands in recurring search and inserting HTML tags.

Table of Content

Some HTML Document Samples

Introduction

Have you ever had the pressure to write many AMLs in which lots of routines are the same but you have to write them repeatedly? Have you ever faced the situation that you have to document all your AMLs but you do not know from where to start and how to quickly get the called-by and calls-to relationships among your AMLs and menus in your application? Are you asked to document all the data layers that someone has put into an ArcStorm database, a library, or a place no one knows for sure? Application and document management sometime can become really time-consuming and stressful. The AutoAML, a collection of AMLs that the author has put together, provides a nice tool for you to ease your application development and document management.

The AutoAML is a collection of AMLs written in ArcInfo version 7.0.3 and IBM AIX 3.2.5 and later tested in ArcInfo 7.0.4 and AIX 4.1.4. It can be run in two modes: menu driven or ArcInfo commands. AutoAML can

  1. generates AML and Menu templates which include all the standard routines to perform error handling and preserve environment variables;
  2. documents all the AMLs and construct the calling flows among your AMLs and menus in HTML format;
  3. conduct data and file inventory and document them in the HTML documents;
  4. provides programs to monitor and record your programs' progress and performance.
For detailed information about sub-programs in AutoAML, you can see from the application document example (click here ).

This paper introduces the issues that AutoAML intends to address, then extends to discuss how to solve the problems related to application development and document management in the subsequent sections. Section 2 discusses approaches used in application development and how AutoAML can help in standardizing AML and menu formats. Section 3 explores document management and presents choices for publishing databases and documents. Section 4 introduces tools for monitoring program performance. Section 5 discusses the limitations in AutoAML and improvements needed in the future.

Back to TOC

Application Development

Many approaches have been explored in application development. The trend from structured approach to objected-oriented approach can been seen in many languages such as Visual Basic, Visual C, and Java, etc. Arc Macro Language (AML), the script language in ArcInfo, is a very good example of structured languages. By carefully planning and designing in the beginning of application development, programs written in AML can be structured so that they can be object-oriented. In order to achieve such a goal, the most important step in the planning is to standardize all the formats presented in AMLs.

Code Standardization

gif Image: Main Menu Two kinds of files are used in ArcInfo application development: one is AML file; the other is menu file. That is the two files that AutoAML ought to standardize. The sub-program genprog.aml can be used to generate standardized templates. All the sub-programs in AutoAML can be invoked from the menu or command mode. The main menu can be started in ARC prompt by typing 
autoaml
Then the genprog menu can be called from the driven menu.

gif Image: GenProg Menu The sub-menu of GENPROG can also be called from the command mode by issuing 

autoaml genprog
From the menu you can specify project ID, program name, version number, program type, and where you want to place the program. The target location can be specified from a list of modular types. User name and full name are obtained from passwd file on the system. You can change the user name and specify a file containing full names of users. Then the program will get user full name from the file you specified.

The usages of autoaml.aml and genprog.aml are: 

Usage: AUTOAML {routine|sub-program}
Usage: GENPROG    {type} {project_ID} {user_id}
               {usr_list_file} {Start_VI:n|y} 
Note: {type:cur|aml|arc|ae|ap|grid|tin|real_dir} for AML 
      {type:7|6|5|4|3|2|1|app} for MENU
Please click AML and Menu for examples of AML template and menu template generated by genprog.aml .

Code Reusibility and Centralization

As demonstrated in Arctools, AML programs can be grouped into classes, functional objects, and functional routines. In AutoAML, I have put all the common routines into one program called ctools.aml. For an excerpted version of ctools.aml, please click here . There are two ways to increase the reusibility of codes: one is to put common routines into separate AMLS; the other is to group generic routines into one AML program and structure the program so that it can become a library containing many functional routines. I prefer the second approach. It is more concise and easier to manage.

Version Control and Version Management

The longer a project lasts, the more possible that new requests will be added into the project and the applicatons associated wiht the project will be more likely to have more complicated version management issue. There generally has more level of version control: version number, release number, modification number, and patching number. The general format is v.r.m.p.

There are two ways used in version management: patching and rolling. Patching approach is commonly used in COBOL for developing applications on mainframe. Once a new request or modification is placed, new codes will be generated to fullfil the task and will overwrite the existing codes only during runtime. The advantage of patching is using less disk space (only duplicate codes that are affected). The first drawback is that many paths need to be to be included in menu and aml paths. It is very easy exceeding the limit of 30 paths in aml and menu path. It is slower than rolling since many directories need to be searched. It requires to modify the main program which sets the paths if any new version, release, modification, or patching number is generated.

Rolling approach requires to copy all the existing programs into a new version or release location, then either add new codes or modify existing programs and need significant amount of disk space if the application is quite large. These approach has many advantages. First, it does not need to change path settings in the main program. Invoking a new version can be done by simply point the global variable to the new version or modification number. Secondly, it runs faster since less paths need to be searhced. If the paths are referenced in the main program as the followings: 

 &amlpath  $VPATH/amls
 &menupath $VPATH/menu
 &atool    $VPATH/atool
Setting VPATH = /apps/v2.1 will enable version 2.1 to be executed.

The following table shows the file structure difference between the approaches. 

  Patching                          Rolling
  --------------                    -----------------
  $APATH/amls/v2.0.0.0/...          /apps/v2.1/amls/...
  $APATH/amls/v1.1.2.0/...          /apps/v2.1/menu/...
  $APATH/amls/v1.1.1.2/...          /apps/v2.1/atool/...
  $APATH/amls/v1.1.1.1/...          /apps/v1.3/amls/...
  $APATH/menu/v1.1/...              /apps/v1.3/menu/...
  $APATH/menu/v1.0/...              /apps/v1.3/atool...
Generally patching approach is better in handling the situation where many modification and patch updates are requested for the applications. Rolling approach is better in managing applications which only major release updates are needed. Back to TOC

Document Management

gif Image: Ge.HTM The term document used here is very broad. It includes program files, meta files, INFO files, coverages, directories, and databases. They can be grouped into three categories: applications, databases, and directories. AutoAML can be used to create HTML documents and links them into a framed page . The sub-program - genhtmll.aml can be invoked through autoaml AML or menu, or it can be initiated in ArcInfo command line. The graph on the right shows the main menu of genhtml . From the menu you can select entry type, then select entries from left window to add to the list on the right. Click the OK button, then it will generate HTML pages based on the categories. Menus will be added in the future so that users can customize logo, color, font, and layout.

Application Documents

Developers often face two difficulties in ArcInfo application development: version consolidation and called-by and calls-to relationship. As presented in the previous section, the file structure in patching approach needs to be consolidated in the final release. AutoAML has functions to bring necessary programs in different levels into the highest version and document the program flow in HTML format.

Database Documents

Databases documented by AutoAML include ArcStorm databases and libraries managed by Librarain. If the description file used to create the ArcStorm database is provided to AutoAML, the source coverage location and library definition will be included in the ArcStrom meta file. The description file was used by mkasdb.aml to create the ArcStorm database.

More detailed reports can be generated if the variable of reporting in detail is checked.

Directory Documents

Directories documented by AutoAML include ArcInfo coverages, INFO files, regular files, and sub-directories. Files are organized by directories, then by types (regular file, coverages, or INFO files). See an example here .

There are couple of challenges encountered in creating HTML documents. First one is directory recursion. The second is inserting HTML tags. The third is getting file types. A loop technique is desgned to accomplish recursion while proerly inserting corresponding HTML tags.

Back to TOC

Performance Monitoring

Two unique tools are included in AutoAML for monitoring programs' performance. One is gettimes.aml for run-time monitoring; the other is logaml.aml for recording the performance of programs.

Run-time Monitoring

gif Image: GetTimes The gettimes.aml is a very nice tool to be used for reporting the progress of your programs. It reports the elapsed time, remaining time, and accumulated descrepancy time caused by gettimes.aml itself. It can be invoked from AutoAML on the command line or from AutoAML's menu. The usage of gettimes.aml is 
Usage: GETTINES < TIME | FULL | UFULL | FTAG  | VMSTIME | VMSFULL >
                {StartTime} {EndTime} {CurrentLoopNo} {TotalLoops}
                {verbose:y|n}
You can type in an expendable command in the field "AML to be run," then watch it progressing. The menu displays times, steps and percentage accomplished. If you include gettimes.aml in the loop of your program, you will get the report like the following: 
  5 of 66: Time elapsed 1:05:12 (h:m:s) 
  Time remaining 2 days 8:23:45 (h:m:s)
  Accumulated discrepancy time by gettimes is 1 seconds.

Background Logging

Logaml.aml is a little program can be used to gather information about user, location where the user issued the program, duration, CPU, disk I/O, system and ArcInfo versions, arguments, and the status of the program. I have included two examples here. You will get the idea of these log files ( full log , simple log ).

Back to TOC

Limitations and Improvements

I do not intend to create a big application. I just grouped a few related AMLs that I wrote in recent years into a presentable form. It by no means is complete nor perfect. Many UNIX commands such as find, file, sed, join, cut and paste, echo, just naming a few, are used especially in genhtml.aml . This may greatly limit its compatibility. A slight variation in UNIX commands' flags will cause the program to be bailed out or produce unwanted results. Many AML's file management functions are used in the application. These funcitons are not efficient in terms of I/O. When comeing to recursion, AML does not have an effective way to accomplish the task.

In order to improve the performance of some sub-programs, C language may need to be introduced. In publishing library database, it is desirable to present libraries in tiled structure, so image map needs to be introduced in HTML document. In documenting application, a flow diagram may be more desirable.

Any comment, or suggestion is greatly appreciated.
Back to TOC

Acknowledgements

Thanks to Weihe Guan and her colleages from SWFWD for testing the first version of genhtml.aml.

Reference

ArcInfo On-line Documentation - ArcDoc.
Hanming Tu
Sr. Programmer Analyst / ArcStorm DBA
Dept. of Computing and Information Services
Chester County, PA, USA
10 N. Church Street, West Chester, PA 19380
Phone: 610-344-6475; Fax: 610-344-6794
E-mail: h_tu@mail.co.chester.ps.us