ALTER TABLE ... DROP COLUMN is always very fast, because it is a metadata operation. If it takes long for you, that is because there are concurrent long running transactions using the table: ALTER TABLE statements that modify the table structure require an ACCESS EXCLUSIVE lock that conflicts with everything else.
You'll have to wait for a time when nobody uses the table to run your statement.
The bigger problem is that ALTER TABLE ... DROP COLUMN will not reduce the size of the table — it will only cause PostgreSQL to "ignore" the column data in the existing rows.
To actually get rid of the column, you'd have to preform a no-effect data modification like
UPDATE tab SET id = id;
which will rewrite the whole table. However, that will increase the table size considerably because of PostgreSQL's multi-version architecture.
So you'd end up doing it in batches like
WITH some_ids AS (
SELECT ctid FROM tab
WHERE id BETWEEN 1 AND 100000
)
UPDATE tab
SET id = tab.id
FROM some_ids
WHERE some_ids.ctid = tab.ctid;
VACUUM tab;
and run VACUUM on the table after each batch.
An alternative, simpler way to shrink the table after dropping the column is to pg_dump, DROP and restore it.
