3

Dereferencing a null pointer results in undefined behavior. In practice it usually means that my program will crash. But why doesn't the OS crash? Because if my program dereferences a null pointer, and my program is run by the OS, then, according to the rules of logical transitivity, this means the OS tried to dereference a null pointer. Why doesn't the OS enter a state of "undefined behavior"?

Improve this question
7
  • 2
    From the point of view of the C++ standard your OS could crash Commented Feb 22, 2012 at 12:21
  • 2
    "In practice it means that my program will crash." -- wrong. In practice it means that anything can happen. Commented Feb 22, 2012 at 12:21
  • 2
    Because the people who wrote the OS knew what they were doing? Commented Feb 22, 2012 at 12:21
  • @Xeo reworded a little. Btw, from the perspective of C++ anything can happen, but from the perspective of the OS there are probably well-defined rules on how to handle this situation. (At least that's what I would expect.) Commented Feb 22, 2012 at 12:24
  • 3
    It's usual and helpful for most operating systems to make stronger guarantees about the ability of one unprivileged process to interfere with another. Commented Feb 22, 2012 at 12:26

9 Answers 9

Reset to default
11

The C++ standard doesn't define the behaviour, either to guarantee a crash, or to do anything else. That doesn't prevent the OS from defining the behaviour - it's not a C++ program, so it doesn't have to abide by the "rules"[1] of C++ programs. Even so, the OS won't dereference the pointer itself.

On most modern platforms, accessing the target of the dereferenced pointer will cause the memory-management hardware to raise an exception (often called a "segmentation fault" or "protection fault"). This is caught by the kernel, which can determine which process did it, and either kill the process, or send it a signal.

So, on such a platform, the default behaviour of a process that dereferences a null pointer will be to crash; there is no reason whatsoever for the OS itself to crash.

[1] By which I mean the informal "rules" that a program should be well-formed and avoid undefined behaviour - not to be confused with the formal "rules" for C++ implementations specified by the language standard.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Note that the act of dereferencing the pointer itself will not raise a segfault, since you just get a reference back from that in C++. However, accessing anything through a null pointer (or reference) will do.
@Xeo: Indeed, I was using the word "dereference" a bit loosely.
"it's not a C++ program, so it doesn't have to abide by the rules of C++ programs" uhm, pardon me, but that statement makes no sense. C++ does not require "undefined behavior"; it merely asserts that it does not define the behavior. Someone else very well might. The language that the OS is written in is irrelevant (and the fact that it's likely C if not C++ completely runs counter to your point anyway). You should just remove that sentence entirely, and the post will make sense without it.
@Mehrdad: I was referring to the informal "rules" of C++ programs (that they should be well-formed and avoid undefined behaviour), not the formal "rules" of C++ implementations (that they should implement the behaviour defined by the language). It is relevant that the OS doesn't have to be written in well-defined C++, and hence can deal with (for example) invalid pointers by defining behaviour that the C++ language doesn't. Sorry if I didn't include a full enough definition of each word I used.
3

Memory access is protected in every major OS. You cannot simply write a program that manipulates memory that was not allocated for it (assuming a pointer is not initialized for example, it could be ANY address). So, every time a program tries to access some address space that does not belong to it, the OS will send a signal to terminate the program (resulting in the ultimate famous "Segmentation fault", familiar to any C/C++ programmer).

Improve this answer

1 Comment

+1 at this one at the time I was writting mine. StackOverflow should auto-update answers.
2

Because the OS has to do something, and crashing would make for a rather bad user experience.

The OS isn't being written to run on the abstract machine of the C standard. It's being written for real hardware that behaves in real ways to different situations that the standard calls "undefined," so it can (and really must) take those real behaviors into account. If it didn't, the operating system would perform differently on different hardware, which kind of defeats the purpose of having an OS, doesn't it?

And before you say "undefined behavior is undefined, let the user of bad code wreck what they want," imagine the security problems of a single accidental buffer overrun being able to segfault an entire server.

Improve this answer

Comments

2

First of all, UB means "anything can happen". In practice however modern OSes offer memory protection - when a program tries to dereference a null pointer that attempt triggers an interrupt inside the CPU which is caught and handled by the OS and the OS then stops the program and then continues running as if nothing bad happened.

Improve this answer

4 Comments

Can I handle this interrupt from within my C++ program?
@StackedCrooked, as far as I know, the only signal that can't be caught is SIGKILL/SIGSTOP (edited as I read signal.h =)). You can always read: man signal.h
@StackedCrooked: This will depend on implementation. In Visual C++ such cases can be caught but can't be sensibly handled.
@StackedCrooked, Unix translates the interrupt in a signal (SIGSEGV), now handling it meaningfully can be complex.
1

There are no rules of logical transitivity when it comes to UB. Your assumption is wrong.

UB does mean that anything can happen, so on a poorly written OS, your program might actually crash the OS. Don't rule it out.

Also, your program doesn't crash because you dereference a NULL pointer. It crashes because the OS tells it to crash.

Improve this answer

4 Comments

You second statement seems to contradict your first.
@StackedCrooked I don't see how.
You say that UB-ness of a program can affect the OS.
@StackedCrooked yes, but not because it's anyway transitive to the OS. It's because UB in a C++ program can lead to anything happening. Well, now that I wrote it down, I see your point. But it's not what I meant.
1

The OS sets up a fault handler which is called if a memory access violates rules imposed by the OS - such as an access to the null address. If your program is about to dereference a null pointer this fault handler is called and the program will be ended before it accesses the disallowed memory region. So your program does actually never dereference a null pointer, it is catched while trying.

The mechanism for detecting forbidden memory accesses is often done with hardware support like page tables or memory segmentation.

If the OS kernel itself dereferences a null pointer, it usually halted while trying to do so. You will get a blue screen, kernel oops or similar. If it keeps going, that may actually result in "undefined behaviour".

Note that the term "undefined behaviour" is only exactly defined in C or similar languages, the processor doesn't really care - usually what happens if you try to access a memory region for which you don't have sufficient rights is very well defined in the context of the architecture.

Improve this answer

4 Comments

Well, the hardware detects this, and sends a signal to the OS (i.e., calls a handler registered by the OS). The OS then reacts to it by killing the program.
@ErnestFriedman-Hill The hardware does everything, including executing the OS, the program, and handling access to memory address 0. What exactly is your point?
You say "the OS detects that you program is about to dereference a null pointer and ends it...". There are several other answers here that correctly point out that dedicated memory-management hardware detects the access and notifies the OS, which then takes action. It's not as if the OS looks at every instruction your program executes.
Ok, I see that was imprecise. What exactly happens is that the OS jumps through a fault handler - as you surely know. So talking about "notification" isn't that exact either :-) The "dedicated memory management hardware" (you are talking of the page tables?) has been set up by the OS before and are often considered to be structures which are part of the OS, in spite of being directly read by the hardware.
1

Because most of the programs run in user mode, and the OS runs in kernel mode. The Kernel mode is near to the physical hardware (they say close to the metal). Kernel mode programs (OS, some services, drivers etc) runs in ring 0 of CPU. User mode programs runs on higher ring. User mode programs running on ring N of CPU, cannot access programs or memory running on anything less than N. If they try to, they wont be allowed!

All programs get their logical address, and OS assigns it. OS does the logical to physical addressing when program tries to read or to write some memory. If program tries to access the address, which it doesn't have permission, the OS will throw the exception. This exception may be handled by program itself (a local exception-handler, in same thread). If not, any attached global exception handler. Debugger may also come into picture, if local EH doesn't handle it. It depends on OS, how/when to route exception to debugger and/or to the global exception handler. It also depends on type of exception (like null-pointer access), if OS allows local/global/debugger to handle it or not. If no one handles it, the OS would terminate the process (and possibly creating crash dump, segmentation fault core dump).

If the process it not being debugged (Windows specific), and some debugger is installed, OS may allow user to debug it.

If the kernel mode program does something nasty, it would take down the OS. I'm not Linux guy, so don't know behavior of Linux. But, in case of Windows, BSOD would brighten your monitor with blue color!

Improve this answer

Comments

0

Because if my program dereferences a null pointer, and my program is run by the OS, then, according to the rules of logical transitivity, this means the OS tried to dereference a null pointer. Why doesn't the OS enter a state of "undefined behavior"?

This is wrong. There's something called memory protection and thats WHY your program is terminated. Is the OS that is protecting itself (in the terms of memory use).

Improve this answer

Comments

0

Sorry, what rules of 'logical transitivity'? One of the things an operating system is designed to do is to protect programs from the misbehaviour of other programs. In particular, the O/S should not crash just because your program tries to do something silly.

On operating systems without memory protection, accessing via a null (or any invalid) pointer could indeed cause the O/S to crash (if the O/S happened to use location 0 for something interesting).

But that has nothing to do with logical transitivity. That has to do with your program accessing memory that belongs to another program. Either programs could crash in those circumstances.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.