Yes, you need to explicitly call lo_unlink(). I assume you just DELETEd the row that held a reference to it, and that will not remove the actual large object.
If you only ever reference it from the same place, you can always create a trigger to do it automatically for you.