Adding items to a list with if condition in Common Lisp

As was explained in the comments, for a function call to be made it must be enclosed in parentheses:

> (defun out-of-limits (l)
     (mapcar #'limits  l) )
  ;  ^                  ^       here

Now calling (out-of-limits (list 1 2 3)) is the same as calling

(list (limits 1) (limits 2) (limits 3))

This means we don't need to try to push the results into any variable like out by hand. They will be collected into a return list anyway.

Thus we just need to write

(defun limits (x)
  (if (> x 2)
      (list x)
      (if (< x 5)
          (list x)
          ())))

but this is the same as

(defun limits (x)
  (if (or (> x 2) (< x 5))
      (list x)
      ()))

Let's now try it

> (out-of-limits (list 1 4 10))
((1) (4) (10))

Indeed the or test is wrong. Do you now see why?

That's right. Any number will satisfy it. Any number is either greater than 2 or smaller than 5. The two regions are overlapping.

But you meant to keep numbers that are either smaller than the lesser limit, or greater than the larger limit. We need to change the test accordingly, to

(defun limits (x)
  (if (or (< x 2) (> x 5))
      (list x)
      ()))

Now we have

> (out-of-limits (list 1 4 10))
((1) () (10))

What's left is to append all these inner lists together into one. This is achieved by using #'mapcan instead of #'mapcar:

(defun out-of-limits (l)
     (mapcan #'limits  l))

Now all the internal parentheses will disappear. Testing, we get

> (out-of-limits (list 1 4 10))
(1 10)

as required.