What causes this program to segmentation fault?

Question

I am attempting to calculate the size of pushed data onto the stack. When attempting this I receive a segmentation fault. My objective is to point ebp into esp before pushing any data onto the stack. After I push data onto the stack I am measuring the size of said data in bytes ebp - esp and storing in ebx and printing to stdout using printf.

For example:

; compile with:
; nasm -f elf test.asm && gcc -m32 -o test test.o
section .text
global  main
extern printf

main:
    ; set the frame pointer to the beginning of the stack before-
    ; data is pushed.
    push  ebp
    mov   ebp, esp

    push  ebx
    push  0x00       ; <- null terminating byte/string truncation
    push  0x64636261 ; <- data

    mov   ebx, ebp
    sub   ebx, esp   ; start of stack - end of stack  = sizeof(data) stored in ebx

    push  ebx
    push  format_str
    call  printf
    add   esp, 8

    pop   ebp
    pop   ebx
    ret

section .data
format_str db "%s", 2, 0

When compiling this code I receive the output:

Segmentation fault (core dumped)

Expected output:

It's considered bad form to modify questions in such a way that answers are invalidated. By all means make a note at the end that a suggestion doesn't work, but wholesale changes are not a good idea. — paxdiablo
– paxdiablo, Commented Nov 10, 2020 at 0:05
Your string data is never removed from the stack. You are missing a mov esp, ebp. — Jester
– Jester, Commented Nov 10, 2020 at 0:06
My apologies, was a mistake on my part. Still it doesn't fix the segmentation fault unfortunately. Any ideas? — asd_2323
– asd_2323, Commented Nov 10, 2020 at 0:07

paxdiablo · Accepted Answer · 2020-11-10 01:13:08Z

3

For a start, items should be popped in the reverse order to which they were pushed, if you want them back in their original registers. What you have here (with irrelevant lines removed):

push ebp
push ebx
pop ebp
pop ebx

will not restore ebp to its previous value, and this is very likely to cause problems moving up through the stack frames.

Additionally, you may be better off following the normal practice of restoring esp from ebp rather than blindly adding eight. That would be something like this at the end:

; add esp, 8 ; not the normal way.
mov   esp, ebp
pop   ebp

Doing it this way removes the need for you to manually calculate how many bytes you need to take off the stack, a calculation that you actually got wrong since you didn't take into account everything you pushed.

And, finally, before you do that, you have to make sure the esp is in the right place so that the pop ebp will work. That means popping everything you pushed (other than ebp itself). Since you pushed (after ebp) ebx, 0x00000000, 0c64636261, ebx, and format_str, you should make sure all of those are off the stack before attempting to pop ebp.

Taking all that into account gives you something like:

main:
    push  ebp
    mov   ebp, esp

    push  ebx           ; (+1)
    push  0x00          ; (+2)
    push  0x64636261    ; (+3)

    mov   ebx, ebp
    sub   ebx, esp

    push  ebx           ; (+4)
    push  format_str    ; (+5)
    call  printf
    add   esp, 16       ; (-5, -4, -3, -2)
    pop   ebx           ; (-1)

    mov   esp, ebp
    pop   ebp
    ret

Each of those (+N) comments represent a 32-bit value that has been pushed on the stack, and the (-N) comments indicate which instructions reverse those pushes. The add esp, 16 reverses four of them (at four bytes apiece), and is done that way since we don't care what happens to those items. That leaves the final pop to recover the original ebx (which we do care about).

That final reload of esp is, I think, unnecessary in this case since it's been restored to the correct value by previous steps. Whether you leave it in for prolog/epilog consistency is up to you.

edited Nov 10, 2020 at 1:13

answered Nov 10, 2020 at 0:01

paxdiablo

888k243 gold badges1.6k silver badges2k bronze badges

Sign up to request clarification or add additional context in comments.

13 Comments

asd_2323 Over a year ago

The code still seg faults after making this modification.

Jester Over a year ago

pop ebx will not work because there is a bunch of stuff in the way.

Jester Over a year ago

Looks good. An alternative would be to not use ebx but some caller-saved register such as edx and then the whole mess could be reduced.

paxdiablo Over a year ago

asd_2323: each of your push instructions, yes. However, were you to do something like push ax (in x86 or x64), only two bytes would be pushed. It depends on what you push, something that's limited by the architecture as well (for example, I don't think you can push four bytes on x64, though I haven't looked into it closely).

Michael Petch Over a year ago

The only nitpick is that the call to printf doesn't conform to the more modern Linux i386 System V ABI which would require 16 byte alignment at the point of the printf function call. I think it is misaligned by 4 bytes.

|

Collectives™ on Stack Overflow

What causes this program to segmentation fault?

1 Answer 1

13 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

13 Comments

Your Answer

Sign up or log in

Post as a guest

Related