No. You're only allowed to access something as an integer if it is an integer.
But here's how you can manipulate the binary representation of an object by simply turning the logic around:
uint32_t n;
unsigned char * p = (unsigned char *)&n;
assert(sizeof n == 4); // assumes CHAR_BIT == 8
p[0] = 0x94; p[1] = 0x15; p[2] = 0xFD; p[3] = 0x23;
The moral: You can treat every object as a sequence of bytes, but you can't treat an arbitrary sequence of bytes as any particular object.
Moreover, the binary representation of a type is very much platform dependent, so there's no telling what actual integer value you get out from this. If you just want to synthesize an integral value from its base-256 digits, use normal maths:
uint32_t n = 0x94 + (0x15 * 0x100) + (0xFD * 0x10000) + (0x23 * 0x1000000);
This is completely platform-independent and expresses what you want purely in terms of values, not representations. Leave it to your compiler to produce a machine representation of the code.
char* n = number;