Can I set up a serial terminal (RS-232) without a pty?

If you start getty from the command line, you might need to run it as root and wrap it into a loop:

while true; do getty -8 9600 ttyS0; done

One invocation of getty gives you just one session, or a set of usually 3 login attempts, or until the serial port gets a hang-up signal. If picocom sends a hang-up initially to reset the serial line, it would cause the first getty to exit.

If a getty process is not terminated early by a serial port hang-up signal, then it will set up the TTY device, send the contents of /etc/issue to the terminal, and then present a login prompt. If a valid username is entered, getty will exec() (note: without fork()ing first) /bin/login, which will usually invoke the PAM libraries to handle the password prompt and the rest of the user session set-up. As part of that procedure, the identity of the process will be switched to the user that is logging in, and then the process that started as getty and then turned into login will again call exec() to become the user's shell. If that process dies for any reason, the user's session will end, and a new getty process will be started for a new login.

Running a getty manually in a loop might be a good way to "visualize" what needs to be happening, but it is not the proper way for long-term use.

All the getty processes are really intended to be started by init and restarted whenever the process that started its life as getty (and gets transformed into the user's shell on a successful login) dies... so on a modern Linux distribution that uses systemd, the command to make a getty run on a serial port connected to a terminal is usually:

systemctl start [email protected]

To make it persist over reboots, you'll also need a

systemctl enable [email protected]

If you are using classic SysVinit instead, there should be commented-out example lines in /etc/inittab file: just uncomment the appropriate line, then run telinit q to make SysVinit re-read its configuration, and init should immediately start a getty process for the terminal and keep restarting it as necessary.