Out of memory is the only detectable error ... other errors such as freeing memory that has already been freed can lead to crashes.
One strategy for out of memory checking in C is to use wrappers for malloc and realloc (you could possibly call them xmalloc and xrealloc) that check for out of memory and if so take an error action ... printing a message and exiting, or possibly trying to free memory pools and then retrying the allocation. This puts all the testing in one place, produces consistent failure messages, and guarantees that all allocation attempts are checked for failure. Possible downsides are discussed in the comments below.
Historically, this strategy was rare in C code (consistent with a general low quality throughout the code written in this ancient language), but nowadays some mature library frameworks incorporate this sort of thing (although the implementations leave something to desire; again, see comments below). Another approach, which is highly advisable, is to abandon C and move to a more modern language ... possibly C++, in which any failure of new results in a bad_alloc exception.
As for your question ... if malloc fails, it returns NULL; there is no pointer to free. (free(NULL) is a no-op). If realloc fails, then the original allocation remains unchanged. You can find these things out by reading the manual pages or specifications such as http://pubs.opengroup.org/onlinepubs/7908799/xsh/realloc.html
malloccall can succeed, but the memory isn't actually available and your process gets killed. The operating system oversubscribes memory by default. It isn't usually a big issue.echo "2" > /proc/sys/vm/overcommit_memory)