    Title: RECIO DESIGN AND DEVELOPMENT NOTES
Copyright: (C) 1994-1996, William Pierpoint
  Version: 2.15
     Date: October 26, 1996


1.0 DATA STRUCTURES

1.1 REC structure for each record stream

    * defined in recio.h.
    * 3 (!dos) or 4 (dos) static RECs for recin, recout, etc (included in
      ROPEN_MAX count).
    * allocate dynamic array of RECs dimensioned to ROPEN_MAX-NREC in ropen().
    * Each REC has two associated buffers:
      1) record string buffer containing current record;
         allocate when first record read;
         reallocate if record becomes larger.
      2) field string buffer containing current field;
         allocate when first field read;
         reallocate if field becomes larger.
    * deallocate dynamic RECs and associated buffers in rclose() and
      rcloseall() if all record streams closed; deallocate associated
      buffers for recin with an exit function registered with atexit().


1.2 REC r_flags assignments

    Bit	    Description
   -----    -------------------------------------------------------------
     0	    If clear, colno start at 0; if set, colno start at 1
     1      if clear, read mode; if set, write/append mode
     2      if clear, current field not quoted by r_txtch;
            if set, current field quoted by r_txtch.
    3-6     Reserved for future use.
     7	    If clear, EOF not reached; if set, EOF reached
    8-11    If clear, no error; else rerror number
   12-15    if clear, no warning; else rwarning number


1.3 Accessing and Setting Information

    How do I
    * access the name of the record stream?           rnames()
    * access the current context number?              rcxtno()
    * access the current record number?               rrecno()
    * access the current field number?                rfldno()
    * access the current column number?               rcolno()
    * access the record string buffer?                rrecs()
    * access the field string buffer?                 rflds()
    * go to a specific field?                         rgotofldno()
    * determine the number of fields in a record?     rnumfld()
    * determine if column numbers start at 0 or 1?    rbegcolno()
    * determine if the current field was quoted?      ristxtfld()
    * determine if there are more records left?       reof()
    * determine if there is an error on the stream?   rerror()
    * determine if there is a warning on the stream?  rwarning()
    * access the error message for the stream?        rerrstr()
    * access the last warning message on the stream?  rwarnstr()
    * force an error on a record stream?              rseterr()
    * clear an error on a record stream?              rclearerr()
    * force a warning on a record stream?             rsetwarn()
    * clear a warning on a record stream?             rclearwarn()
    * scan the record buffer more than once?          rresetrec()
    * increase the size of the record string buffer?  rsetrecsiz()
    * increase the size of the field string buffer?   rsetfldsiz()
    * replace the data in the field string buffer?    rsetfldstr()
    * replace the data in the record string buffer?   rsetrecstr()
    * get the position of a field in a record?        rgetfldpos()
    * return to a previous field position?            rsetfldpos()
    * set the field delimiter character?              rsetfldch()
    * set the text delimiter character?               rsettxtch()
    * set the time format string?                     rsettmfmt()
    * set the context number?                         rsetcxtno()
    * set column numbering to start at 0 or 1?        rsetbegcolno()
    * dynamically copy a string?                      scpys()
    * dynamically concatenate a string?               scats()
    * trim white space from the ends of a string?     strims()



2.0 CODE STRUCTURES

2.1 RECIO Structure Chart

    ͻ                  ͻ
     rstr.c                     recio.c 
    ͼ                  ͼ
                                      
             ͻ     input      output   ͻ
         Ķ rget.c     Ķ rput.c 
       char   ͼ column             char   ͼ column
     delimited        delimited         delimited        delimited
    ͻ      ͻ      ͻ      ͻ
     rgetf.c Ĵ    Ķ rcgetf.c        rputf.c Ĵ    Ķ rcputf.c 
    ͼ      ͼ      ͼ      ͼ
                                                       
    ͻ      ͻ      ͻ      ͻ
     rgets.c Ĵ    Ķ rcgets.c        rputs.c Ĵ    Ķ rcputs.c 
    ͼ      ͼ      ͼ      ͼ
                                                       
    ͻ      ͻ      ͻ      ͻ
     rgett.c Ĵ    Ķ rcgett.c        rputt.c Ĵ    Ķ rcputt.c 
    ͼ      ͼ      ͼ      ͼ
                                                       
    ͻ      ͻ      ͻ      ͻ
     rbget.c     Ķ rcbget.c        rbput.c     Ķ rcbput.c 
    ͼ        ͼ      ͼ        ͼ
                                      
                       ͻ      ͻ        ͻ
                        rwarn.c  Ķ rerr.c  Ķ  rfix.c  
                       ͼ       ͼ        ͼ
                                              
                       ͻ       ͻ
                        rwmsg.c          remsg.c 
                       ͼ       ͼ


2.2 Callback Error Functions

    The callback error function technique used by the RECIO library can
    be applied within your programs to other abstract data types.  For
    abstract data types within the standard C library, you will need to
    write wrapper functions.

                                 ͻ
                                  errnofn 
                                 ͼ
                                      
                        etc
                                                   
                  ͻ    ͻ    ͻ
                   rerrfn       ferrfn       merrfn  
                  ͼ    ͼ    ͼ
		     REC            FILE          MEMORY


2.2.1 Errno Callback Error Function Skeleton [errnofn()]

    switch on error number [errno]
    case out of memory [ENOMEM]
    case out of range [ERANGE]
    case out of record or file pointers [EMFILE]
    case permission denied [EACCES]
    case invalid argument [EINVAL]
    default [anything else]
    endcase


2.2.2 RECIO Callback Error Function Skeleton [rerrfn(rp)]

    if valid record pointer [risvalid(rp)]
        if past end of file [reof(rp)]  (if reof test removed, past EOF will
        else [error number set]           become R_EMISDAT or R_WEMPSTR)
            switch on error number [rerror(rp)]
            case read data errors [R_ERANGE || R_EINVDAT || R_EMISDAT]
            case write data errors [R_ENOPUT]
                switch on context number [rcxtno(rp)]
                case RECIN
                    switch field number [rfldno(rp)]
                    case 1 (first field read)
                    case 2 (second field read)
                    ...
                    endcase
                ...
                default [missing or unknown context number]
                endcase
            case out of memory [R_ENOMEM]
            case reading binary file [R_ENOGET]
            case fatal errors [R_EINVAL || R_EINVMOD]
            default [anything else]
            endcase
        endif
    else [invalid record pointer]
        call error function that handles errno errors [errnofn()]
    endif


2.3 RECIO Callback Warning Function Skeleton [rwarnfn(rp)]

    if valid record pointer [risvalid(rp)]
        switch on warning number [rwarning(rp)]
        case read data warnings [R_WEMPSTR || R_WTMFMT]
        case write data warnings [R_WWIDTH]
        case code runtime warnings [R_WNOREG]
        default [anything else]
        endcase
    endif


2.4 RECIO Field Parsing Algorithm

The RECIO library parses a field from an input record according to 
regular expressions that are specified below using PERL programming 
language notation (see the references in section 4.0).  Four cases must 
be considered that depend on whether the field and text delimiters are 
set to whitespace or to something else.  To illustrate the cases below, 
that something else is the comma character for the field delimiter and 
the double quote character for the text delimiter.

case 1: fldch = ' '; txtch = ' ';
        \s*(\S*)(\s*(\s|$))

case 2: fldch = ','; txtch = ' ';
        ([^,]*)(,|$)

case 3: fldch = ' '; txtch = '"'; 
     A: \s*"(([^"]|"[^"]*"|"\S)*"?)("\s*(\s|$))
     B: \s*"(([^"]|"\S)*"?)("\s*(\s|$))

        notes: (1) use case 3 when first nonwhitespace character is txtch; 
                   otherwise use case 1.
               (2) use case 3A when txtch is the double quote character;
                   use the form of case 3B for any other nonwhitespace 
                   text delimiter. (See case 4, note 2 example.)

case 4: fldch = ','; txtch = '"';
     A: \s*"(([^"]|"[^"]*"|"\s*[^,])*"?)("\s*(,|$))
     B: \s*"(([^"]|"\s*[^,])*"?)("\s*(,|$))

        notes: (1) use case 4 when first nonwhitespace character is txtch; 
                   otherwise use case 2.
               (2) use case 4A when txtch is the double quote character;
                   use the form of case 4B for any other nonwhitespace 
                   text delimiter.  (Example: "use case "4A", okay?", is 
                   one field using case 4A, but would break after the 
                   first comma if using case 4B.  Case 4A is limited to 
                   double quote characters since other delimiters might 
                   not be used in matching pairs.  For instance, the 
                   single quote character is also used as an apostrophe.)

When the parsing algorithm is finished, the field buffer contains the 
equivalent of the PERL variable $1.  The field parsing code can be 
found in the _rfldlen and _rfldstr functions in rget.c.



3.0 DEVELOPMENT NOTES

3.1 fgets (Microsoft C 5.1)

Previous notes of mine indicate that Microsoft's fgets function does not
work correctly when it reads a line of text that consists of only a newline.
However this can be worked around by first setting the string buffer to an
empty string.  If you plan on retaining the newline, you will need to test
this further.  The fgets function is used in the rgetrec function.  


3.2 fopen (Borland C 3.1)

fopen() calls __openfp() calls open().  Borland's "Library Reference"
documents error numbers for open(), but not for fopen().  These error
numbers are ENOENT, EMFILE, EACCES, and EINVACC.  Because ropen() screens
the access code, the EINVACC error should not occur from the recio library.


3.3 strtol & strtoul (Borland C 3.1)

These functions stop consuming input once they overflow, setting ERANGE.
Hence endptr can point into the middle of a sequence of valid characters
having the expected form as given in ANSI X3.159-1989, Sections 4.10.1.5
and 4.10.1.6.  IMHO this characteristic is not in conformance with the
ANSI standard as endptr should only point to the first unrecognized
character or to the terminating null character.  Borland's strtod does
not have this problem.

Note that ANSI X3.159-1989 Section 4.10.1.6 allows strtoul (unsigned
long) to have an optional negative sign.  A negative unsigned long?!
Borland 3.1 strtoul converts a negative long to an unsigned number
without error.  But I prefer to trap any negative numbers input to
unsigned fields.  So str2ul is a wrapper function for strtoul that
first tests for a negative number and if one is found, flags the data
as invalid and returns zero.

The test suite includes -0 as a data value.  The strtol function traps
this as an ERANGE error and returns the overflow limit.  The rfixi and
rfixl functions substitute zero.

3.4 strtod (Borland C 3.1)

Unlike the strtol and strtoul functions, strtod does not clear errno first.
Starting with recio vers. 2.02, errno is cleared before any conversion
where recio code checks the value of errno after the conversion.  It is also
cleared when a record stream is successfully opened.

3.5 mktime (Borland C 3.1)

The Borland C 3.1 mktime function has valid range from 1 Jan 1970 00:00:00 
to 19 Jan 2038 03:14:07.  Actual upper limit can vary by a few hours 
depending on the timezone setting.  For EST, upper limit is 18 Jan 2038 
22:14:07.  When the recio library reads a 2-digit year, it first represents 
it in a struct tm from 1951 to 2050.  (You can control this with the rsetbegyr 
function.)  The mktime function is used to convert to a time_t type.  Hence, 
when compiled with the Borland C 3.1 compiler, the recio library time_t 
routines fail for any dates beyond 18 or 19 Jan 2038, or before 1 Jan 1970.

The minimum and maximum values of time_t are controlled by the recio.h
symbolic constants TIME_T_MIN and TIME_T_MAX.  These are used only by 
the time functions in rfix.c.  Borland C 3.1 represents time_t using 
a long.  Thus TIME_T_MIN is set to 0 and TIME_T_MAX is set to LONG_MAX.
LONG_MAX is defined in your compiler's header file limits.h.  You should 
find your compiler's representation of time_t in your compiler's header 
file time.h.

3.6 ltoa and ultoa (ANSI-C)

Your ANSI-C compiler might lack the ltoa and ultoa functions as these are 
missing from ANSI X3.159-1989.  These functions convert a long and an
unsigned long to a string.  The recio library uses these functions.  If 
they are missing from your compiler, you can write a simple version of
these functions by using the sprintf function.  The sprintf function limits 
the radix (base) to 8, 10, and 16, but this will probably be adequate for 
most applications.  The prototypes for recio.h are:

extern char *ltoa(long l, char *str, int base);
extern char *ultoa(unsigned long ul, char *str, int base);

The logical place for the ltoa and ultoa functions is in rput.c next to 
the dtoa function.  The code is as follows:

/****************************************************************************/
char *                       /* return pointer to string                    */
    ltoa(                    /* convert long number to string               */
        long   l,            /* number to convert                           */
        char  *str,          /* string buffer to use                        */
        int    base)         /* radix of the number                         */
/****************************************************************************/
{
    switch (base) {
    case 8:
        sprintf(str, "%lo", (unsigned long) l);
        break;
    case 10:
        sprintf(str, "%ld", l);
        break;
    case 16:
        sprintf(str, "%lx", (unsigned long) l);
        break;
    default:   /* error */
        *str = '\0';
        rseterr(NULL, EINVAL);
    }
    return str;
}

/****************************************************************************/
char *                       /* return pointer to string                    */
    ultoa(                   /* convert unsigned long number to string      */
        unsigned long  ul,   /* number to convert                           */
        char          *str,  /* string buffer to use                        */
        int            base) /* radix of the number                         */
/****************************************************************************/
{
    switch (base) {
    case 8:
        sprintf(str, "%lo", ul);
        break;
    case 10:
        sprintf(str, "%lu", ul);
        break;
    case 16:
        sprintf(str, "%lx", ul);
        break;
    default:   /* error */
        *str = '\0';
        rseterr(NULL, EINVAL);
    }
    return str;
}

In recio.h, the value of NSBUFSIZ is used to dimension the internal string 
buffer that the recio library uses when it calls the ltoa, ultoa, and dtoa 
functions.  This value must be large enough to hold every possible number 
(integral and floating point) without overflowing the string buffer.  In 
determining the value, keep in mind the following points:

    * Integral numbers contain more digits as the radix gets smaller.

    * Reserve space for the exponent of a floating point number,
      including the 'E'.

    * Reserve space for possible sign(s).  Floating point numbers can
      contain two signs, e.g. -1.11E-01.

    * Reserve space for the decimal point in floating point numbers.

    * Reserve space for the terminating null.

    * Valuable information on the representation of numbers is contained 
      in your compiler's header files float.h and limits.h.

    * A conservative rule of thumb is: make NSBUFSIZ large enough to 
      hold your largest integral value expressed in radix 2.  Thus if 
      your largest integral number is a 32 bit value, make NSBUFSIZ 34.  
      If using 64 bit integral values, make NSBUFSIZ 66.

4.0 REFERENCES

Wall, L. and Schwartz, R.L.  Programming Perl.  O'Reilly & Associates,
Sebastopol, CA, 1991, p. 103 ff.

Schreiner, A.T. and Friedman, Jr., H.G.  Introduction to Compiler 
Construction with UNIX.  Prentice-Hall, Englewood Cliffs, NJ, 1985, p. 25.
