Common problems
From OpenGENIE
Jump to navigationJump to searchWhy does A/B give the wrong answer?
If A and B are both integers, then integer division is performed; if either is a floating point number then the integer value is first converted into a floating point number before division is performed. Thus:
1/2 == 0 # 1 and 2 are of Integer type
but
1/2.0 == 0.5 # 2.0 is a floating point number, so integer 1 is converted to 1.0 before division is done
If you are unsure what data type two variables may be, then you can force real division using either of these statements
RES = AS_REAL(A) / B RES = 1.0 * A / B # equivalent to (1.0*A)/B as "*" and "/" are equal priority so the expression is evaluated left to right