Floating point on ARM machines and Octave errors


Just a few random thoughts...

The rambling

Back in the old days, (before the advent of OE), I did some releases of Octave and R for the Zaurus machines. Things are slightly different now, Octave builds quite happily using bitbake and OE, R still doesn't, but I lack time to fix it (though hopefully now that I have broadband at home this will change).

Anyway, I use a Nokia 770 nowadays, and so I built and installed Octave on it. Lo and behold, the same old error message that I'd seen on my Zaurus versions of Octave were produced (see here for a link to the Octave ml questions & replies). The error message in question is shown when Octave starts up:

    warning: lo_ieee_init: unrecognized floating point format!

I left the original discussion saying that I'd come back with more results when I had a working EABI toolchain (as the EABI toolchain for OpenZaurus was going to introduce VFP softfloat, which means that floating point numbers have the same endianness as the processor itself). The issue was that Octave looks at the floating point format to decide on the endianness of the processor, this means that for FPA softfloat machines running in littleendian mode, stream writes probably won't work correctly, as Octave thinks the machine is bigendian.

Along similar lines, R adds some extra floating point types (NA for example) by manipulating unused bits in the current NaN defined by the floating point (IEEE 754) standard, which means that it also needs to know the endianness of the floating point numbers (which it assumes is the same as the processor).

I wonder why Octave uses this dynamic endianness testing method, rather than testing and setting it using the configure file (as R does)?

 Anyway, I got busy and the EABI toolchain took longer to develop than people had though, so I'm now thinking about this again, but instead of a Zaurus, I'm using a Nokia 770, which has an EABI toolchain and VFP softfloat (I should note that Angstrom, the next step in the evolution of OpenZaurus, now produces working EABI toolchains and images for the Zaurus, so I will have to get back to using it again).

The problem

Now, the Nokia 770 is littleendian in both its processor setup and floating point storage, but it still fails the test that Octave uses to determine the endianness.

The following code is called to perform the test:

-------------------------------------------------------------------
mach-info.cc
-------------------------------------------------------------------
void
oct_mach_info::init_float_format (void) const
{
#if defined (CRAY)

  // XXX FIXME XXX -- this should be determined automatically.

  native_float_fmt = oct_mach_info::flt_fmt_cray;

#else

  float_params fp[5];

  INIT_FLT_PAR (fp[0], oct_mach_info::flt_fmt_ieee_big_endian,
           1048576,  0,
        2146435071, -1,
        1017118720,  0,
        1018167296,  0);

  INIT_FLT_PAR (fp[1], oct_mach_info::flt_fmt_ieee_little_endian,
         0,    1048576,
        -1, 2146435071,
         0, 1017118720,
         0, 1018167296);

  INIT_FLT_PAR (fp[2], oct_mach_info::flt_fmt_vax_d,
           128,  0,
        -32769, -1,
          9344,  0,
          9344,  0);

  INIT_FLT_PAR (fp[3], oct_mach_info::flt_fmt_vax_g,
            16,  0,
        -32769, -1,
         15552,  0,
         15552,  0);

  INIT_FLT_PAR (fp[4], oct_mach_info::flt_fmt_unknown,
        0, 0,
        0, 0,
        0, 0,
        0, 0);

  equiv mach_fp_par[4];

  mach_fp_par[0].d = F77_FUNC (d1mach, D1MACH) (1);
  mach_fp_par[1].d = F77_FUNC (d1mach, D1MACH) (2);
  mach_fp_par[2].d = F77_FUNC (d1mach, D1MACH) (3);
  mach_fp_par[3].d = F77_FUNC (d1mach, D1MACH) (4);

  int i = 0;
  do
    {
      if (equiv_compare (fp[i].fp_par, mach_fp_par, 4))
    {
      native_float_fmt = fp[i].fp_fmt;
      break;
    }
    }
  while (fp[++i].fp_fmt != oct_mach_info::flt_fmt_unknown);

#endif
}
-------------------------------------------------------------------

This calls a function in libcruft/misc/d1mach.f, which in turn calls a function in libcruft/misc/machar.c, and returns the values xmin, xmaxm epsneg and eps for comparison.

What does the endianness test test?

The upshot of all of this is that for a littleendian machine, the machar() function returns the values of xmin, xmax, epsneg and eps. These values are then compared against the hardcoded values from the oct_mach_info::init_float_format() function:

  INIT_FLT_PAR (fp[1], oct_mach_info::flt_fmt_ieee_little_endian,
         0,    1048576,
        -1, 2146435071,
         0, 1017118720,
         0, 1018167296);

or in hexadecimal:

        0,    100000,         (xmin)
        FFFFFFFF, 7FEFFFFF,   (xmax)
        0,  3CA00000,         (epsneg)
        0,  3CB00000;         (eps)


Now this the the output from a variety of platforms:

x86 (old version of octave)
===========================
eps      2.2204460492503131e-16          0   3CB00000
epsneg   1.1102230246251565e-16          0   3CA00000
xmin    2.2250738585072014e-308          0     100000
xmax    1.7976931348623157e+308   FFFFFFFF   7FEFFFFF

PXA255 using FPA softfloat (old version of Octave)
==================================================
eps      2.2204460492503131e-16   3CB00000          0
epsneg   1.1102230246251565e-16   3CA00000          0
xmin    4.9406564584124654e-324          0          1
xmax                        inf   7FF00000          0

x86_64
======
eps      2.2204460492503131e-16          0   574F0C00
epsneg   1.1102230246251565e-16          0   574F0C00
xmin    2.2250738585072014e-308          0   574F0C00
xmax    1.7976931348623157e+308   FFFFFFFF   574F0C00

Nokia 770 (OMAP 1710)
=====================
eps      2.2204460492503131e-16          0   3CB00000
epsneg   1.1102230246251565e-16          0   3CA00000
xmin    4.9406564584124654e-324          1          0
xmax                        inf          0   7FF00000

Nokia 800 (OMAP 2420 [has VFP hardware])
========================================
eps      2.2204460492503131e-16          0   3CB00000
epsneg   1.1102230246251565e-16          0   3CA00000
xmin    2.2250738585072014e-308          0     100000
xmax    2.2471164185778946e+307   FFFFFFFF   7FBFFFFF

These data were produced using the output of a modified machar, which you can grab from here. Compile like so:
gcc -DDP -DTEST machar.c

or for the Nokia 800 use a command like this:

gcc -DDP -DTEST -mfpu=vfp -mfloat-abi=softfp machar.c

Alternatively you can use the output from Octave itself:

octave:1> format bit
octave:2> eps
eps = 0011110010110000000000000000000000000000000000000000000000000000 
octave:3> realmax
realmax = 0111111111101111111111111111111111111111111111111111111111111111 
octave:4> realmin
realmin = 0000000000010000000000000000000000000000000000000000000000000000 
octave:5> exit

You could go directly to hex output (skip using modified machar) doing something like this (just wrote it off the top of my head so it may not actually work):

octave:1> format bit
octave:2> a = eps
octave:3> a_1 = a(1:32)
octave:4> a_2 = a(33:64)

octave:5> dec2hex(bin2dec(a_1))
octave:6> dec2hex(bin2dec(a_2))

Curiously, the above output is from my x86_64 system, and although the Octave output is correct, the output from the modified machar is not. I must look into this and see what's wrong.

So, things to note:
  1. x86 produces the correct (according to the test) result.
  2. x86_64 doesn't produce the expected results, but it passes the test - must look into this...
  3. Both ARM machines without VFP hardware (i.e. the PXA255 and Nokia 770) produce the same output, but with the words in different positions due to the endianness differences
  4. The Nokia 770 doesn't produce the answer that the test is looking for, nor does the Nokia 800 - why?

Conclusions

The difference in the outputs is caused by a difference in the values of minexp and maxexp between the x86 and Nokia 770 (note that for the Nokia 800, only the value of xmax is different).

The question is, are these differences allowed? If they are allowed, then the Octave test needs altering, if they're not allowed, then the floating point implementation needs fixing.

Testing floating point (ieee754) compliance

So, now to test compliance with the floating point standard. One test I found is called paranoia.
I compiled it like so: gcc paranoia.c -lm -o paranoia.softvfp.out

Both my x86_64 linux box and my Nokia 770 passed the tests with no failures (I'll put up the results when I remember the USB link cable for my Nokia 770).

I've not tested the compliance of the Nokia 800, if anyone wants to try it then use the following command line and email me the results:
gcc -mfpu=vfp -mfloat-abi=softfp paranoia.c -lm -o paranoia.vfp.out