This bug occurs with operations that convert floating point numbers into integers. Floating point numbers are stored in an 80-bit format, while integers are stored in 16-bits (for short integer) and 32-bits (for long integers). Sometimes, converting an 80-bit floating point number into an integer on the Pentium II and the Pentium Pro produces the Dan-0411 bug. This happens when the converted floating point number won't fit into the smaller integer format. According to Intel's Flag Erratum Technical Description site, the erratum occurs only when the FIST[P] instruction is either a 16- or 32-bit operation; either the 'to nearest', 'to zero' or 'up' rounding modes are being used; the sign bit of the floating-point operand is negative; and the floating-point operand being converted is significantly more negative than can be described by the integer size being targeted.

When the operand is large enough that it would not fit into the target data size the processor is supposed to do the following:

when invalid operation exception is masked:	when invalid operation exception is unmasked:
Return the MAXNEG value to memory. Set the IE (Invalid Operation) bit in the Floating Point status word to flag the overflow. Do not set the PE (precision error) bit the Floating Point status word.	Do not return a result to memory. Keep the original operand intact on the stack. Set the IE (Invalid Operation) bit in the Floating Point status word to flag the overflow. Do not set the PE (precision error) bit the Floating Point status word. Vector to the user numeric exception handler.

However, in the case that the operand is large enough that it would not fit into the target data size, the processor responds with returning the MAXNEG value to memory; not setting the IE (Invalid Operation) bit in the Floating Point status word to flag the use of an invalid operand; setting the PE (precision error) bit the Floating Point status word; not invoking an exception handler; and in the case of a FISTP instruction the Operand will have been popped from the floating point stack (Flag Erratum Technical Description).

The chance of this bug occuring is 1 in 8.6 billion, when storing 16-bit integers, and 1 in 562,950 billion, when storing 32-bit integers. The bug was not found in the Pentium and the Pentium MMX processors. When talking about the bug Collins gives an interesting analogy about a launch failure of the Ariane 5 rocket, which happened because a floating point to integer conversion overflowed and the overflow was not handled right. However, the Pentium II and Pentium Pro FPU bug was found to be minor. Most PC applications, such as word processors, data bases, most spreadsheets, and most games, do not use floating point math and can not be affected by the bug. Further, most of the software vendors, such as Microsoft, Oracle, IBM, Netscape, and others, have indicated that the bug does not affect their products. Intel has also suggested the following workarounds to avoid the occurence of the bug:

• Range checking performed prior to execution of the FIST[P] instruction will prevent the overflow condition from occurring, and may already be implemented as a standard coding style.
• Software can use the presence of MAXNEG in the result integer to indicate that an out of range conversion may have occurred.
• Based on the knowledge that numbers capable of triggering this erratum have one of two properties (1) they have a fractional component OR (2) they are larger than the maximum signed 64 bit integer size. A combination of three instructions can be used in place of either a FISTP16 or a FISTP32 instruction that will allow the above errata to be worked around.

• Main
• Pre-Pentium bugs
• Pentium FDIV bug
• Pentium MMX bug
• Bibliography

---

koncadu@earlham.edu