The Weather Processor (WXP) is an integrated set of meteorological data analysis programs developed at Purdue University, Department of Earth and Atmospheric Sciences. These programs perform data ingest, decoding, analysis and display of various types of meteorological information all from a single menu driven shell called wxp.
WXP Version 1
Work began on WXP in 1984 as a set of student projects to handle the real-time plotting and analysis of data from the FAA 604 circuit. The software was used to reduce analysis time and improve objectivity. Normally these analyses would take hours to generate by hand. The first version of WXP was developed on one IBM PC/XT as a combination of several student projects from an Application of Microcomputers to Meteorology class. These simple programs were written in BASIC using the IBM Advanced BASIC interpreter provided with IBM MS-DOS. These routines were specifically targeted at the midwestern United States and used the Color Graphics Adapter (CGA) for its graphics display. The complete package contained a weather plotting program, a radar summary program, a thermodynamic diagram plotting program along with simple ingest and decoding software.
WXP Version 2
This BASIC language version of WXP (Version 1.0) worked well only for limited hardware and software domains. The BASIC language was considered too slow and its graphics capabilities too limited for expanded computational programs to work efficiently. Therefore, continuing code development in BASIC limited the scope of further expansion. For subsequent versions of WXP, a decision was made to rewrite the code in the C language and use the Graphical Kernel System (GKS) as a graphical interface. The acquisition in 1986 of the IBM RT/PC with a UNIX System V based operating system called AIX (Advanced Interactive eXecutive) aided this development. Other benefits of this language conversion were that C is an ANSI standardized language and provides extensive and standardized means to interact with the operating system, thereby increasing the portability of the software to several different types of computers and operating systems. As a greater variety of graphics displays became available, it became clear that the graphical interface for these programs should become more device independent. The GKS library of graphics functions has been adopted functions has been adopted by ANSI as a standard for two-dimensional graphics. As a result of this standardization, several manufacturers provide GKS library bindings that work on a variety of operating systems and enable graphics on numerous output devices.
The new C/GKS WXP code (version 2.0) for the IBM RT/PC was finalized in late 1987. This version added a great deal of functionality to the original software. It included a more powerful data ingestor along with an ingest parameter setting program that could change ingest parameters interactively with the running ingest program. This eliminated the problem of having to stop the ingest program and rerun it with new parameters, resulting in lost data. Additions to surface data plotting included regional data plotting so that the entire United States could be covered either with one view or through smaller subregions of higher resolution. An upper air decoder and plotter was developed as well as an enhanced thermodynamic plotting program. Satellite display software was also included. Since WXP version 2.0 was designed for portability and C Compilers and GKS bindings were available for for MS-DOS, the decision was made to move the code back to its original MS-DOS environment. The MS-DOS version of WXP offered the same functionality as the RT version except that, since MS-DOS is a single tasking operating system, the ingest program could not be run simultaneously with any other program. To accomplish continuous data ingest required the use of networking technology to essentially share data and distribute the ingest process and subsequent analyses among a number of single tasking MS-DOS machines. One IBM PC/XT performed continuous ingest and wrote the data to a central fileserver. Any other computer on that network then could access the data and produce near-real-time analyses (see Figure 1) with the WXP software.
WXP Version 3
Work on a new version of WXP (Version 3.0) began in November 1987 with the assistance of IBM Corporation through a Joint Study Contract. The principle focus of this effort was to further extend WXP analysis routines and shift the network operation of WXP to a UNIX-AIX based environment. In this mode of operation, an IBM RT/PC became the central ingestor and fileserver, as well as serving as an analysis workstation. Additional machines capable of display and analysis may share the central data reserves of the RT fileserver through the Network File System (NFS) (see Figure 2). Since NFS has become available on several different types of operating systems, this allows a variety of machines to access the ingested data. PC-NFS is a software package that incorporates NFS under DOS and allows networked IBM PC's and PS/2's to access data stored on a properly configured Unix fileserver.
Another important aspect of the work done on version 3.0 was the adaptation necessary for incorporation of WXP with the Scientific Data Management (SDM) system. The Unidata Program Center (UPC) located in Boulder, Colorado wrote a comprehensive data ingestor and manager called the Local Data Manager (LDM). The data ingestor portion of the LDM reads the Domestic Data Plus (DD+), International Data Service (IDS) and the NMC Numerical Product Service (NPS) which has now been replaced with the High Resolution Data Service (HDS). The ingestor then passes the data to the data management portion of the LDM which selects various products based on a standard pattern matching algorithm and saves the products to user specifiable data files. A joint effort between Purdue and Unidata modified WXP to read LDM DD+ data using a Unidata naming convention. This effort resulted in Unidata adopting WXP as the Unix based application interface to the SDM system. Recently, the SDM has been broken down into its component packages. At the same time, three other packages have entered the Unix arena to compliment WXP. The WXP has been distributed nationally through Unidata to more than 40 colleges and universities and runs on Sun workstations under SunOS and Solaris, DEC workstations under VMS and Ultrix, HP workstations under HP/UX, Silicon Graphics workstations under IRIX and IBM workstations under AIX. The Purdue Research Foundation (PRF) distributes and licenses WXP to non-university sites and is offered on the above platforms plus DOS.
The major modifications to WXP to produce version 3.0 in late 1988 included LDM / DD+ support, surface and upper air gridding and contouring programs, MDR radar conversion programs as well as a set of data parsing programs. In order to provide WXP programs with a uniform user interface and to allow extra functionality without adding to shell menus and prompts, a command line interface was added to each program. As a result, all necessary pieces of information requested by the individual program may be passed to the program via a single command line, and run independently of the WXP shell. In addition, this allowed WXP programs to extend functionality by increasing parameters addressable through the command line. Finally, this allowed an individual site to develop localized methods of interaction to suit their operational interests or environment.
While all the graphics source code for WXP calls GKS libraries to support graphics, many environments require the use of an intermediate software layer to perform graphics functions. In some cases, this layer acts as a router of graphics commands produced by the GKS graphics library to a set of individual device drivers supporting displays, printers or plotters (see Figure 3). This way several devices may be accessed by the same executable program by just changing where the router sends its commands, often with the use of environment variables. To improve efficiency, some GKS libraries direct graphics to a single hardware output device. An example of this is the GKS binding XGKS where GKS calls are remapped into X Windows graphics calls. Also under X Windows, the X server can be located on a network so that the application program and the display may exist on separate computers. GKS also provides the capability of saving graphical information in metafiles. A metafile contains a sequential list of graphics commands that are written to a file on disk. At some later time, this metafile may be sent to a metafile translator which translates these graphics commands back into physical graphics which are displayed on a specific output device.
WXP Version 3.5
Further modifications to WXP resulted in the release of version 3.5 in Fall of 1989. This version had the same applications as version 2 and 3; however, major changes were made in the lower level functions used by WXP to perform various calculations and produce graphics, as well as for data entry. These modifications were required to allow WXP programs to use more types of data and to support more operating systems. In addition, a defaults file was incorporated into each WXP program in order to allow more end-user customization of the WXP environment. Menu interfaces were modified to improve user productivity and enable program expansion. Finally, the command line interface was modified to eliminate case-sensitive parameters.
WXP Version 4.0
Access to several new data types were introduced for WXP version 4.0 released in the Summer of 1990. Software development at the Unidata project office implemented a new data file type called netCDF for the network transparent Common Data Format. This is a self-describing data structure where variables have specified dimensions and attributes which describe them. The low level routines of WXP were modified to access this new data file type transparent to the user and also provided the capability of producing netCDF files as output from converters, decoders and gridding routines. New programs were developed to plot and contour Model Output Statistics (MOS). A NPS ingest program was developed and a program developed to contour the model gridpoint data. Support for international data were added to WXP which included more complete city database files, synoptic surface data decoders and global map database and region specification. More command line and defaults file parameters further extended user tailorability of the WXP programs.
WXP Version 4.3
The work done in version 4.0 did not meet the functionality required for the SDM version 3 release planned by the UPC. Further work had to be done to implement a resource and command line interface known as UDRES or UniData RESource. This replaced the original defaults file interface developed for WXP 3.5. In making the changes to the underlying code, a WXP_init interface was written which emulated UDRES but provided backwards compatibility to the previous defaults files interface. The major difference of this resource interface was that the resource or defaults file used the X defaults standard. The command line implementation was also cleaned up and code to handle resources was greatly improved. All resources could now be specified either with the command line or in the resource file. Second, the netCDF structures were modified to allow easier access by netCDF operator programs to be developed by the UPC. Third, a new color interface was developed allowing all colors used by WXP to be modified or changed. This was implemented to provide support for monochrome and high resolution monitors. Colors could now be specified either by color index or by color name based on a color table file which could be modified by the user. This file interface allows up to 75 colors can be specified allowing for many different color combinations. Finally, a new surface conversion program "sacvt" was developed to replace "convert". The new conversion program used a new parsing interface which allowed more different types of data to be decoded including METARs. These modifications resulted in version 4.3 in late summer 1991 and was included in the SDM version 3 release by the UPC.
WXP Version 4.5
Many new programs were being developed along with the additions needed to satisfy the deliverables of SDM 3 but were not included into WXP 4.3 because of the time constraints placed on the SDM release. These programs were finished and debugged and added in the WXP 4.5 release during the winter of 1991-92. This release included an upper air cross-sectional contouring program, an animation, overlay and annotation program, X based satellite display program and programs to help in graphical overlays. The most of the work on WXP 4.5 concentrated on the X windows interface in an effort to realize the full potential of X to solve many of the deficiencies that resulted from using GKS as the main graphics layer. First, the XGKS layer was replaced by a more robust GKS to X interface "ansi2x11" which allowed programs to plot information on windows not opened by that program. If a base window is opened, any WXP program can plot data to it allowing for many overlay possibilities. Second, a program called "wxploop" was developed which will create this base window. Also, writing to several pixmaps instead of windows and animating the pixmaps sets up wxploop to animate plotted data. The major drawback in the past to the X windows implementations was the lack of high-resolution hardcopy capabilities. Once created, the window may be window-dumped and printed, but the results is limited to the resolution of the screen which is much lower than most hardcopy devices. Therefore, the ansi2x11 interface was fitted with a postscript back-end that allowed any WXP program to produce high quality laser printer output.
WXP Version 4.6
With the decision of Unidata to remove WXP from the core SDM package, WXP development could now occur without the necessity of another SDM release. During the spring and early summer of 1992, work continued on version 4.6 which included addition of satellite navigation so that data could be plotted correctly over satellite images and a grid mathematics package that allowed the end user to manipulate grid either produced by WXP or by other programs. This meant the inclusion of programs grdmath and vector. This was also the first version to use WXP programs as filters or pipes allowing data to be passed from program to program without intermediate files. This became a necessity with the grid math interface where several component grids would have to be passed from one program to the next. Using pipes greatly simplifies data transfer.
WXP Version 4.7
WXP version 4.7 became available in the fall of 1992 and included several new components. First, a raw data file component was added to each plotting program that allows each program to act as a filter from large converted/decoded data files which have a large number of parameters. The raw file interface allows each component to be dealt with individually. Also, a math package called rawmath was added for raw data manipulation. The program mapplt was modified to add raw data plotting and a program named grid was added to fit raw data to a grid. This addition increases the amount of data handled by WXP by including the capability to use non meteorological data. The second addition is a attributes package which allows line styles, widths, fill patterns and text types to be changed. Also, dither patterns were added to the satellite display program xsat. This is all in an effort to support monochrome displays as well as color. It turns out that most universities have invested greatly in monochrome systems for computer laboratories and this addition helps establish WXP for those sites. Third, additional support has been added to the code base to handle the generation of many of the standard DIFAX facsimile products. This is as a result of the proposed faze out of DIFAX type products in the near future.
WXP Version 4.8
When 4.7 was completed, work began to make WXP more compatible with a Graphical User Interface or GUI. Many subtle changes had to be made in order to allow the GUI to access and handle the communication of resources and program defaults between a variety of programs. Second, a strong push was made to increase the base set of analyses by setting up shell scripts called "wxpplt" and "wxptxt". These scripts use a combination of WXP programs with wxploop to create overlays and loops. Program capabilities were enhanced to allow for more complicated plot generation from within a script. This included the addition of an overlay capability to the DOS version of WXP. Third, WXP was included into the American Meteorological Society's Project Atmosphere as part of the DataStreme project. This meant a completely new ingestor had to be developed to handle the distribution of non-NWS data plus some modifications to upgrade the overall DOS support.
The High Resolution Data Service (HDS) also came online in the summer of 1993 which required another ingestor modification. The old GRIB based ingestor was replaced by a new ingest program that could handle all NWS circuits. In addition, the new version added satellite remapping to any projection, a new raw file interface, SHEF decoding, and support for new MOS and METAR data formats. This led to the release of WXP version 4.8 in the winter of 1993-94.
WXP Version 5.0
WXP 5.0 represents a major conceptual change from previous versions of WXP. Earlier versions of WXP were written with a top down mentality. In other words, there was a set of analyses that needed to be produced and programs were developed to produce these analyses from the data types available. As a result, analysis specific programs were developed. Since, analyses were tied to specific data types, a different analysis program had to be written for each data type. As a result, as the number of different analyses grew and the number of different data types grew, the number of WXP programs grew.
This worked for a while but as the complexity of the analyses increased, the amount of redundant code in each program became too difficult to manage. Starting in WXP 4.0, a library of common functions was developed but even through version 4.8, less than 30% of the actual functionality of a WXP program was buried in this library.
With WXP 5.0, 95% of the functionality is now in the WXP library. This allows WXP programs to share capabilities previous reserved to specific programs. For example, the old surface data plotting program sfcwx could only plot surface data. If you wanted to overlay a contour plot of sea level pressure, you needed both sfccalc, the surface contouring program and wxploop, the overlay and looping program. In WXP 5.0, the functionality for contouring, overlaying plots and looping are in a central WXP library and sfcwx can perform all 3 functions.
Second, carrying that much of the code in each program meant nearly 50% of the code was redundant from one program to another. This meant adding new functionality was cumbersome considering the number of WXP programs. The library now offers a single point at which to make changes and the changes propogate throughout all WXP programs. This required WXP to be rewritten with a object oriented style so that various component don't step on each other. Even though WXP is written in C, it has alot of C++ concepts in its writing.
Finally, WXP 5.0 inverts the data model. In previous versions, WXP had specific data models for each type of data. This meant surface, upper air, radar, satellite and gridded data were all handled differently. With new data types being added almost monthly, WXP needed a different data model. WXP now treats data as either point data (referred to as raw data), gridded data or image data. The plotting model uses a central coordinate model for all plotting. A robust coordinate transformation routine called maptran can handle everything from XY plots to SkewTs to Mercator projections.
WXP also uses internal resources to specify user configurable parameters. In previous WXP versions, resources were called into the main program and then passed to the library functions. In WXP 5.0, the resources are called, set and reset inside the code segments that need the resource. In other words, the main program no longer knows what resources are being used during its execution.
Here is a list of the conceptual changes:
- The bulk of the code has been moved into the WXP library. Older WXP program performed
the bulk of the tasks in the main program module. This made each program somewhat unique
and the character of each program was unique. Now, different WXP programs use the same
library base unifying the programs and making the interface more consistent. Also,
programs can cross link, gaining capabilities from other programs. For example, sfcwx now
can plot and contour data. This will also work for upairwx.
- The simplification of the data model makes data easier to plot. WXP now uses three data
models: GRID, RAW, IMAGE. There are routines that perform operations on each type. Before,
WXP had redundant code to plot and contour data in EACH WXP program. Updating one
capability meant modifying 10 or more programs. Now, only the single library function
needs to be touched.
- The use of resources has been internalized to each function. Before, ALL resources were
managed by the main program and then passed to each function as a value. Now, each
function queries the resources as it needs it. This way the main program is simplified and
the use of resources can now be extended without worry of side effects. Functions now work
independent of the main program.
- Program parameters have now been externalized. Many parameters to WXP programs have been
moved outside WXP to variable files, map files, symbol files, etc. This way WXP can be
upgraded and extended without modifying the source code. There are two major areas this
- File naming conventions - the way WXP accesses files is now based on a file naming convention file which specifies exactly how file names are to look. Originally, WXP users were tied down to a file naming format but now, the user is open to name the files with any convention they wish.
- Variable files - the way WXP processes variables is different than before. In
earlier WXP versions, programs had a hard coded list of parameters they could plot and the
user had a limited selection of varaibles they could plot and a limited way of viewing
them. Now, the variable file defines how variables are to be plotted, can define new plot
types, overlays and even new variables through a function interface.
- Finally, much of the WXP code has been simplified. The library makes the writing of a program simplier. For example, a model data sounding program was written in 1 hour. The program is only 35 lines long. Also, upgrading functionality is MUCH easier to do. The modular approach means that new functionality can be added and at the same time, reduce overall code size. The new WXP has twice the functionality in the same amount of source code as version 4.8.
WXP 5 also has support for new data types such as:
- NLDN lightning
- RCM radar and summaries
- NIDS (all types)
- NOWRad radar mosaics
- Unisys radar mosaics
- GRIB model data from NCEP and other sources
- Hurricane plotting
- TAF decoding
- NOAAPORT satellite imagery
- Unisys satellite imgery
- Support for Postscript output has been enhanced
- New output to metafiles and HPGL
As a final addition, support was added for the Win32 API which now allows WXP to run on Windows 95, 98 and NT. To aid in this transition, Tcl/Tk based GUI has been developed for use with WXP 5.
For further information about WXP, email firstname.lastname@example.org
Last updated by Dan Vietor on July 21, 1998