7

In a demand paged system like linux where pages maybe~4k from what I read, it ensures protection by checking if the stack or heap size exceeds the number of pages given to each. WHen I create two variables

char *s = (char *) malloc(100);   
char sa[100];

In a for loop I can write s[i] = 'c'; almost 4000 times before it comes up with memory error whereas with sa[i] = 'c'; EDIT: I get a segmentation fault or stack smashing error for anything greater than array size.

I can understand in the first case there is a page fault and it sees that no more pages have been allocated to heap hence a memory violation. But what happens in the second case does gcc keep a check at runtime for all the preallocated variables?.
EDIT: I am posting the entire code below

int main(int argc,char* argv[]){
char *s = (char *) malloc(20);
char sa[400] = {0};
int i ,count;
printf(" enter the number of chars to write: ");
scanf("%d",&count);
for (i=0;i<count;i++){
printf("%d\n",i);
sa[i] = 'a';
//s[i] = 'a';
}
free(s);

}
7
  • Language-wise both cases are totally undefined. Added a gcc tag for the implementation specific question. Commented Aug 14, 2011 at 9:45
  • FYI it is by no means guaranteed that you can only write the length of the array with the stack allocated variable. I got up to 2880 on my x86_64 machine (where it hits a new page as &sa[i] == 0x7ffffffff000). Commented Aug 14, 2011 at 9:46
  • I assume we shall ignore the question of why you are writing to an array outside of the allocated range, right? Commented Aug 14, 2011 at 9:46
  • so I read about gcc and the canaries (or guard bytes... does gcc check for the guard byte overwrite each time the stack grows?. Commented Aug 14, 2011 at 9:49
  • @christian-semrau yes... Commented Aug 14, 2011 at 9:51

1 Answer 1

2

On a lot of 32-bit operating systems the stack grows downwards. You only use positive indexes into the array so it depends how deeply nested your function call is. As you index out of the array, you'll first overwrite the canary. So the stack smashing error is first. Next, you'll start overwriting the function arguments and return address. Without the canary, that will cause the function return to jump into nevernever land, usually producing a segfault. Not always, it might accidentally land on valid code, the logic behind stack buffer overflow attacks.

As you keep going, you'll eventually write past the top of the stack into unallocated pages. Segfault then. On a small test program with few nested calls that happens quickly, couple of kilobytes give or take.

Also try it with negative offsets. That can keep going for a while, not otherwise causing any mishap since you're writing into unallocated stack space. The segfault comes when you write past the allocated stack size, typically a megabyte. On Windows you'd trigger the stack guard page, generating an exception for which this site is named.

Sign up to request clarification or add additional context in comments.

6 Comments

doesnt the OS treat stack,heap,code as different segments?. I was thinking initially the stack gets a page , heap gets a page so as they grow more pages are allocated. so how can stack overflow overwrite the code section?.
It doesn't. Not sure why you drew that conclusion.
I read in an OS text that the c compiler generates program segments as code,stack,heap globals etc, but linux puts the code into the [user code] segment , all other data(stack,heap etc) in [user data] segment and then uses paging. So I get that linux does not segment stack and heap. But if the initial allocation for the [user data] segment is lesser than required (e.g. recursion) then how does it expand the stack?.
Linux is constantly changing, not sure what it does these days. Windows uses reserved memory pages that get turned into real ones by page faults. Linux didn't support that for a while, I wouldn't be surprised if it does now. None of which is relevant, you perhaps missed the "stack grows downward" opening sentence. You are writing into pages that are already allocated.
Because the writing that causes the segfault happens before the function return that detects the canary being stomped. Key insight is exactly when the canary is checked. Not until the function returns.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.