Larry Van Sickle
wptc has extended the powerful Perl language to access SDE 2.1 databases. Combined with oraperl and other Perl extensions, the extension lets programmers quickly write complex SDE applications. For the most recent update on Perl and SDE, see wptc's web page at http://www.wptc.com.
Perl is powerful, easy to use, and free language. It is used for a wide variety of jobs from simple text processing and data conversion to complex data base processing. Perl is heavily used for system administration taks. It is widely used by Web developers for writing CGI scripts. Perl is the first tool many programmers turn to to get their job done.
Perl is available on a wide variety of computers, including virtually all flavors of Unix, MS-DOS, all versions of Windows, and Macintosh.
Perl has powerful pattern matching and file handling facilities. It has associative arrays, which let you index values by strings or even more complicated data structures. An example of a simple Perl script is:
while (<>) { print if (/http:|ftp:|mailto:/); }
This Perl script will read through a file and print all lines that contain URL's that begin with "http:", "ftp:", or "mailto:". This example shows that simple jobs are very simple in Perl. Perl make simple jobs very simple, and makes complex jobs much easier.
For more information about Perl, particularly about obtaining your own copy of Perl, see the Perl Home Page at http://www.perl.com/perl. Another useful web site is the Perl Institute at http://www.perl.org.
To get a feeling for the power of Perl, look at a Perl script that reads an SDE layer, gathers statistics on the layer, and reports the statistics. This Perl script is 88 lines long, and is shown in Appendix A. A C program that does exactly the same job as the Perl script contains 277 lines of code. It is shown in Appendix_B. Perl does the same job as C in one-third the lines of code.
A Perl library is a set of Perl subroutines. A Perl module is a Perl library that follows certain conventions. The SDE extension for Perl is a Perl module.
Perl comes with a large number of generally useful modules. There are even more useful Perl modules that are freely available at the Comprehensive Perl Archive Network (CPAN) web site.
In addition to libraries and modules, Perl has facilities for calling C from Perl. To call C from Perl, you must create an XSUB file. This file contains a combination of C code and specifications of how C routines are called from C.
The current version of the Perl SDE extension defines a very simple interface between Perl and SDE. Most of the SDE subroutines return a LONG status, or return code, value. Many of the SDE subroutine modify some of their arguments. The Perl SDE extension returns a Perl list, or array, containing the status and the elements of the modified data structure.
For example, SE_dataset_info return a LONG status and modifies the structure referenced by its single argument. SE_dataset_info is defined as :
LONG SE_dataset_info ( DATASET * dataset)
And the C structure DATASET is defined as:
typedef struct {
CHAR dsname[9]; /* Dataset Name, i.e. database file name */
CHAR location[255]; /* DATASET LOCATION, PATH or SPACE NAME */
LONG max_layers; /* MAX LAYERS 1 - MAX_LAYERS IN THIS DATASET */
LONG max_feat_pts; /* MAX POINTS PER FEATURE FOR THIS DATASET */
LONG falsex, /* FALSE ORIGIN X VALUE, (SYSTEM UNITS) */
falsey, /* FALSE ORIGIN Y VALUE, (SYSTEM UNITS) */
falsez, /* FALSE ORIGIN Z VALUE, (SYSTEM UNITS) */
sysunits, /* NUMBER OF SYSTEMS UNITS PER UNIT TYPE */
unit_type, /* FEET, METERS */
cr_uid, /* CREATORS USER ID */
cr_gid, /* CREATORS GROUP ID */
cr_date; /* CREATION DATE AND TIME */
USHORT majver, /* MAJOR RUNTIME VERSION USED TO CREATE IT */
minver; /* MINOR RUNTIME VERSION USED TO CREATE IT */
LFLOAT cround,
halfsu;
} DATASET;
In Perl the call to SE_dataset_info looks like this:
($returncode,$dsname,$location,$max_layers,$max_feat_pts,$falsex, $falsey,$falsez,$sysunits,$unit_type,$cr_uid,$cr_gid,$cr_date, $majver,$minver,$cround,$halfsu) = &sdelib::se_dataset_info();
Note that the se_dataset_info() call returns a list, and that the list consists of the status returned from SE_dataset_info and the elements of the DATASET data structure.
Many built-in Perl functions return lists of values. Our Perl interface to SDE, while very simple, is fairly natural to Perl programmers.
An XSUB file has a file extension of .xs. It contains specifications for how to call C subroutines from Perl. The specification language is similar to C. We developed a Perl script that reads the SDE #include file and automatically creates much of the XSUB file for the Perl SDE Extension.
The XSUB routine for SE_dataset_info() is:
void
se_dataset_info()
PPCODE:
{
long returncode;
DATASET dataset;
returncode = SE_dataset_info(&dataset);
XPUSHs(sv_2mortal(newSViv((IV) returncode)));
XPUSHs(sv_2mortal(newSVpv(dataset.dsname, 0)));
XPUSHs(sv_2mortal(newSVpv(dataset.location, 0)));
XPUSHs(sv_2mortal(newSViv((IV) dataset.max_layers)));
XPUSHs(sv_2mortal(newSViv((IV) dataset.max_feat_pts)));
XPUSHs(sv_2mortal(newSViv((IV) dataset.falsex)));
XPUSHs(sv_2mortal(newSViv((IV) dataset.falsey)));
XPUSHs(sv_2mortal(newSViv((IV) dataset.falsez)));
XPUSHs(sv_2mortal(newSViv((IV) dataset.sysunits)));
XPUSHs(sv_2mortal(newSViv((IV) dataset.unit_type)));
.
.
.
}
Perl includes a set of tools for adding extensions. These tools process the XSUB file and create a C program which is then compiled into a library. These tools also process the associated Perl module. When this processing is complete, a Perl use command will make the extension available to the Perl program.
Perl 5 introduced object-oriented syntax and constructs into Perl. An improved version of the Perl SDE Extension would have a more object-oriented interface between Perl and SDE. The SDE data structures would be Perl objects.
An example using an object-oriented Perl to SDE interface to retrieve SDE dataset information might look like this:
$dataset = new Dataset(); $dataset->getinfo(); print "Data set name: $dataset->dsname\n"; print "False origin: $dataset->falsex,$dataset->falsey\n";
In this hypothetical example, Dataset is a Perl class. The first statement creates a new instance of a Dataset and assigns it to the variable $dataset. The second statement calls the getinfo() method of the Dataset.
The current Perl SDE Extension does not have this object-oriented structure, but an object-oriented Perl interface to SDE would be a better way to access SDE. We are waiting for SDE 3.0 before pursuing this approach.
Perl is a powerful tool for a wide variety of tasks, but particularly for writing CGI scripts. An extension to Perl that allows Perl scripts to call the SDE API, when combined with Perl's built-in functions and existing modules to access Oracle and other relational databases from Perl, provides an enormous productivity booster for programmers writing SDE applications.
#
# Report layer information and statistics.
#
use Getopt::Std;
use Sdelib;
getopts("d:il:p:s:u:");
#
# SE_start_engine.
#
$returncode = &sdelib::se_start_engine($opt_s,$opt_d,$opt_u,$opt_p);
print !$returncode ? "ok 2\n" : "not ok 2\n" if ($opt_d);
#
# SE_dataset_info.
#
($returncode,$dsname,$location,$max_layers,$max_feat_pts,$falsex,
$falsey,$falsez,$sysunits,$unit_type) = &sdelib::se_dataset_info();
#
# SE_layer_info.
#
($returncode,$layer,$description,$grid1,$grid2,$grid3,$eflags,$cr_uid,
$cr_gid,$cr_date,$maint,$layer_config) = &sdelib::se_layer_info(4);
printf "Layer information\n");
printf " Layer number %ld\n",$layer;
printf " Layer name %s\n",$description;
printf " Grid %ld\n",$grid1;
printf " Layer Config %s\n",$layer_config;
if ($opt_i)
{
$l_minx = 2147483647;
$l_miny = 2147483647;
($returncode,$layer,$fid,$cr_date,$cr_uid,$cr_gid,$mod_date,$mod_uid,
$mod_gid,$numofpts,$entity,$symbol,$anno_x,$anno_y,$angle,$cent_x,
$cent_y,$minx,$miny,$maxx,$maxy,$minz,$maxz,@pts) =
&sdelib::se_get_feature_by_layer($opt_l,"**NOJOIN**");
while (!$returncode)
{
$l_minx = $minx if ($minx < $l_minx);
$l_miny = $miny if ($miny < $l_miny);
$l_maxx = $maxx if ($maxx < $l_maxx);
$l_maxy = $maxy if ($maxy < $l_maxy);
$count++;
($returncode,$layer,$fid,$cr_date,$cr_uid,$cr_gid,$mod_date,$mod_uid,
$mod_gid,$numofpts,$entity,$symbol,$anno_x,$anno_y,$angle,$cent_x,
$cent_y,$minx,$miny,$maxx,$maxy,$minz,$maxz,@pts) =
&sdelib::se_get_next_feature();
}
$l_x_range = ($l_maxx - $l_minx) / $l_bins + 1;
$l_y_range = ($l_maxy - $l_miny) / $l_bins + 1;
printf "Feature Count %ld\n",$count);
printf "X Min %ld X Max %d X Range %ld\nn",$l_minx,$l_maxx,$l_x_range);
printf "Y Min %ld Y Max %d Y Range %ld\nn",$l_miny,$l_maxy,$l_y_range);
#
# Second pass through features.
#
($returncode,$layer,$fid,$cr_date,$cr_uid,$cr_gid,$mod_date,$mod_uid,
$mod_gid,$numofpts,$entity,$symbol,$anno_x,$anno_y,$angle,$cent_x,
$cent_y,$minx,$miny,$maxx,$maxy,$minz,$maxz,@pts) =
&sdelib::se_get_feature_by_layer($opt_l,"**NOJOIN**");
while (!$returncode)
{
$l_x_histogram[($l_f.win.maxx - $l_f.win.minx) / $l_x_range]++;
$l_y_histogram[($l_f.win.maxy - $l_f.win.miny) / $l_y_range]++;
$count++;
($returncode,$layer,$fid,$cr_date,$cr_uid,$cr_gid,$mod_date,$mod_uid,
$mod_gid,$numofpts,$entity,$symbol,$anno_x,$anno_y,$angle,$cent_x,
$cent_y,$minx,$miny,$maxx,$maxy,$minz,$maxz,@pts) =
&sdelib::se_get_next_feature();
}
#
# Print histogram.
#
foreach $i (0 .. $#l_x_histogram)
{
printf ("%d,%ld\n",$i,$l_x_histogram[$i]) if ($l_x_histogram[$i]);
}
printf ("\n");
foreach $i (0 .. $#l_y_histogram)
{
printf ("%d,%ld\n",$i,$l_y_histogram[$i]) if ($l_y_histogram[$i]);
}
#
SE_stop_engine.
#
&sdelib::se_stop_engine();
exit 0;
/* Report on an SDE layer */
#include <stdio.h>
#include <sdeerno.h>
#include <sdetype.h>
#include <string.h>
#include <stdlib.h>
#include <limits.h>
#include <time.h>
#include <sys/timeb.h>
#define max(a,b) (((a) > (b)) ? (a) : (b))
#define min(a,b) (((a) < (b)) ? (a) : (b))
#ifdef DOS
#include <showcode.h>
#else
#include <showcodes.h>
#endif
/*
#include "coords.c"
#include "feature.c"
*/
#ifdef DOS
#include "showcode.c"
#else
#include "showcodes.c"
#endif
#define CHECK(f) if( (returncode=(f)) != SUCCESS ) \
{showcodes("Error " #f , returncode);}
/*
** Prototypes.
*/
void do_create_and_copy();
/*
**
*/
LONG returncode;
UCHAR l_server_name[255];
UCHAR l_dset_name[255];
UCHAR l_user_name[255];
UCHAR l_password[255];
DATASET dset;
DPOINT wpt;
FEATURE_TYPE l_feat_type;
FEATURE l_f;
FEATURE l_f_search;
CHAR l_where[255];
LONG feat_pts;
CHAR response[65];
LONG l_layer;
LONG l_trials;
LONG l_size;
CHAR* l_env_ptr;
LONG l_minx;
LONG l_miny;
LONG l_maxx;
LONG l_maxy;
LONG l_x_range;
LONG l_y_range;
LONG l_do_histogram;
#define L_BINS 1000
LONG l_x_histogram[L_BINS];
LONG l_y_histogram[L_BINS];
int i;
int j;
LONG l_count;
ENVELOPE l_env_range;
ENVELOPE l_env_search;
LONG l_search_method;
LONG l_error;
time_t l_time;
struct timeb l_time_start;
struct timeb l_time_end;
/*
** *****************************************************
*/
void main( int argc, char** argv )
{
/*
** Initialize variables.
*/
l_layer = 0;
l_trials = 0;
l_size = 0;
l_minx = LONG_MAX;
l_miny = LONG_MAX;
l_maxx = 0;
l_maxy = 0;
l_do_histogram = 0;
l_where[0] = 0;
l_server_name[0] = 0;
l_dset_name[0] = 0;
l_user_name[0] = 0;
l_password[0] = 0;
l_error = 0;
/*
** Process options
*/
for (i = 1; i < argc; i++)
{
if (memcmp (argv[i], "-d", 2) == 0)
if (*(argv[i] + 2) != 0)
strcpy(l_dset_name,argv[i] + 2);
else
{
i++;
strcpy(l_dset_name,argv[i]);
}
else if (memcmp (argv[i], "-l", 2) == 0)
if (*(argv[i] + 2) != 0)
l_layer = atol(argv[i] + 2);
else
{
i++;
l_layer = atol(argv[i]);
}
else if (memcmp (argv[i], "-i", 2) == 0)
l_do_histogram = 1;
else if (memcmp (argv[i], "-n", 2) == 0)
if (*(argv[i] + 2) != 0)
l_layer = atol(argv[i] + 2);
else
{
i++;
l_layer = atol(argv[i]);
}
else if (memcmp (argv[i], "-p", 2) == 0)
if (*(argv[i] + 2) != 0)
strcpy(l_password,argv[i] + 2);
else
{
i++;
strcpy(l_password,argv[i]);
}
else if (memcmp (argv[i], "-s", 2) == 0)
if (*(argv[i] + 2) != 0)
strcpy(l_server_name,argv[i] + 2);
else
{
i++;
strcpy(l_server_name,argv[i]);
}
else if (memcmp (argv[i], "-u", 2) == 0)
if (*(argv[i] + 2) != 0)
strcpy(l_user_name,argv[i] + 2);
else
{
i++;
strcpy(l_user_name,argv[i]);
}
else if (memcmp (argv[i], "-v", 2) == 0)
{
printf("usage: %s [-s<server>]", argv[0]);
printf("[-d<dataset>] [-u<user_name>]");
printf(" [-p<password>] [-l<new_layer>] [-i]");
printf("\n");
exit(FALSE);
}
}
/*
** For Testing display result of argument processing
*/
printf ("Server %s\n", l_server_name);
printf ("Dataset %s\n", l_dset_name);
printf ("User %s\n", l_user_name);
printf ("Password %s\n", l_password);
printf ("Layer %ld\n",l_layer);
/*
** Test arguments for consistency and correctness.
*/
if (l_error)
exit(1);
/*
**
*/
do_create_and_copy();
exit(0);
}
void do_create_and_copy()
{
/*
** Start SDE server, get SDE dataset info, allocate feature
*/
/* ***************************************************** */
CHECK( SE_start_engine(l_server_name, l_dset_name,
l_user_name, l_password) );
if (returncode!=SUCCESS) exit(FALSE);
CHECK( SE_dataset_info( &dset ) );
if (returncode!=SUCCESS)
{
SE_stop_engine();
exit(FALSE);
}
returncode = SE_layer_info( l_layer, &l_feat_type );
if (returncode!=SUCCESS)
{
printf ("Error: Layer %d does not exist.\n");
SE_stop_engine();
exit(FALSE);
}
printf("Layer information\n");
printf(" Layer number %ld\n",l_feat_type.ftype);
printf(" Layer name %s\n",l_feat_type.description);
printf(" Grid %ld\n",l_feat_type.grid_size);
printf(" Layer Config %s\n",l_feat_type.layer_config);
if (l_do_histogram)
{
CHECK( SE_alloc_2D_feature( &l_f, ALL ) );
if (returncode!=SUCCESS)
{
SE_stop_engine();
exit(FALSE);
}
returncode=SE_get_feature_by_layer( l_feat_type.ftype,
&l_f, "" );
l_count = 0;
while (returncode == SUCCESS)
{
l_minx = min(l_f.win.maxx - l_f.win.minx,l_minx);
l_miny = min(l_f.win.maxy - l_f.win.miny,l_miny);
l_maxx = max(l_f.win.maxx - l_f.win.minx,l_maxx);
l_maxy = max(l_f.win.maxy - l_f.win.miny,l_maxy);
returncode = SE_get_next_feature( &l_f );
l_count++;
}
l_x_range = (l_maxx - l_minx) / L_BINS + 1;
l_y_range = (l_maxy - l_miny) / L_BINS + 1;
printf ("Feature Count %ld\n",l_count);
printf ("X Min %ld X Max %d X Range %ld\nn",
l_minx,l_maxx,l_x_range);
printf ("Y Min %ld Y Max %d Y Range %ld\nn",
l_miny,l_maxy,l_y_range);
returncode=SE_get_feature_by_layer( l_feat_type.ftype,
&l_f, "" );
l_count = 0;
while (returncode == SUCCESS)
{
l_x_histogram[(l_f.win.maxx - l_f.win.minx) / l_x_range]++;
l_y_histogram[(l_f.win.maxy - l_f.win.miny) / l_y_range]++;
returncode = SE_get_next_feature( &l_f );
l_count++;
}
for (i = 0; i < L_BINS; i++)
if (l_x_histogram[i])
printf ("%d,%ld\n",i,l_x_histogram[i]);
printf ("\n");
for (i = 0; i < L_BINS; i++)
if (l_y_histogram[i])
printf ("%d,%ld\n",i,l_y_histogram[i]);
}
SE_stop_engine();
printf("Engine stopped.\n");
/* ***************************************************** */
}
Larry Van Sickle
wptc
3305 Hancock Drive
Austin, Texas 78731
Telephone: 512-459-9027
Fax: 512-452-1655
EMail: larry@wptc.com