Assembly, overflow detection

As it has been noted, signed integer overflow occurs when the signs of the summands are the same and when the sign of their sum is different.

Examples (with 8-bit numbers for brevity):

0x7F (+127) + 0x00 (+0) = 0x7F (+127)  
0x7F (+127) + 0x01 (+1) = 0x80 (-128) overflow  
0x80 (-128) + 0x7F (+127) = 0xFF (-1)  
0x80 (-128) + 0xFF (-1) = 0x7F (+127) overflow

IOW,

overflow = (sign1 is equal to sign2) AND (sign of sum is NOT equal to sign1)

You can express equality using XOR:

0 XOR 0 = 0 - equal
0 XOR 1 = 1 - not equal
1 XOR 0 = 1 - not equal
1 XOR 1 = 0 - equal

And then:

overflow = (NOT(sign1 XOR sign2)) AND (sign of sum XOR sign1)

And you can code this part something like this:

addu $10,$8,$9 # bit 31 of r10 = sign of sum
xor $10,$10,$8 # bit 31 of r10 = sign of sum XOR sign1
xor $2,$8,$9 # bit 31 of r2 = sign1 XOR sign2
nor $2,$2,$2 # bit 31 of r2 = NOT(sign1 XOR sign2)
and $10,$10,$2 # bit 31 of r10 = (NOT(sign1 XOR sign2)) AND (sign of sum XOR sign1)

Now, if you want to spread bit 31 of r10 to all other bits of the register (this way you'll get all zeroes or all ones in r10), you can use the arithmetic shift right instruction:

sra $10,$10,31

For positive integers it should hold that (a+b)>a. If it isn't, one obvious reason is an overflow. For that there's slt instruction. The logic is naturally inverted for negative numbers.

In c the working code is:

 overflow==((a+b)<a)^(b<0);

and in MIPS assembler:

 add $8, $8, $9
 slt $11, $9, $0    // is 'b' < 0 ?
 slt $10, $8, $9    // is 'a+b' < b ?
 xor $10, $10, $11  // combine the expressions

I have no idea, why one wants to test separately conditions of negative vs. positive OF, but those would be:

 negative_overflow == overflow && (b<0) 
    and $11, $11, $10
 positive_overflow == overflow && (b>=0)
    slt $9, $0, $9     // 0<b
    and $10, $10, $9   // wasting 'b'

You will need to examine the sign bit of your result. In two's complement addition if you add 2 positive numbers and end up with a negative number you have overflowed. Likewise, if you add 2 negative numbers and end up with a positive number you've overflowed. Adding a positive and a negative number cannot overflow.

Signed overflow occurs only when we add two numbers with the same sign and get a different sign in the result (why?). Therefore:

    subu $t0,$t1,$t2 # subtract and don’t trap
    xor $t3,$t1,$t2 # check to see if the signs of the inputs are different
    bgez $t3,skip # if they aren’t different, no overflow
    xor $t3,$t0,$t1 # check to see if the signs of the difference and first input are different
    blz $t4,overflow # if they’re different, there has been overflow
skip:
    # setup a register you wish with all zeroes
overflow:
    # setup the register with all bits one

If you want to dispense with the bgez, you have to get creative; you could use the results in $t3 to figure out the result and use sign extension to setup $10 as necessary.