4

I am using PostgreSQL as my database for a project at work. We use triggers in quite a few places to either maintain computed columns, or tables that essentially act as a materialized view.

All this worked just fine when simply utilizing row level triggers to keep all this in sync. However when we wrote scripts to periodically import our customers data into the database, we ran into issues with either performance or problems with number of locks in a single transaction.

To alleviate this I wanted to create a statement-level trigger with access to the modified rows (inserted, updated or deleted). However as this is not possible I instead created a BEFORE statement-level trigger that would create a temporary table. Then an AFTER row-level trigger that would insert the changed data into the temporary table. At last an AFTER statement-level trigger that would read the changes and perform necessary updates, and then drop the temporary table.

All this works just fine, assuming that within the triggers, no one would re-trigger the same flow again (as the temporary table would then already exist).

However I then learned that when using foreign key constraints with ON DELETE SET NULL, it is simply implemented with a system trigger that sets the column to NULL. This of course is not a problem at all, except for the fact that when you have several foreign key constraints like this on a single table, all referencing the same table (let's just call this files). When deleting a row from the files table, all these system level triggers to handle the ON DELETE SET NULL clause all fire at the same time, that is in parallel. Which presents a serious issue for me.

How would I go about implementing something like this? Here is a short SQL script to illustrate the problem:

CREATE TABLE files (
    id serial PRIMARY KEY,
    "name" TEXT NOT NULL
);

CREATE TABLE profiles (
    id serial PRIMARY KEY,
    NAME TEXT NOT NULL,
    cv_file_id INT REFERENCES files(id) ON DELETE SET NULL,
    photo_file_id INT REFERENCES files(id) ON DELETE SET NULL
);

CREATE TABLE profile_audit (
    profile_id INT NOT NULL,
    modified_at timestamptz NOT NULL
);

CREATE FUNCTION pre_stmt_create_temp_table()
RETURNS TRIGGER
AS $$
BEGIN
    CREATE TEMPORARY TABLE tmp_modified_profiles (
        id INT NOT NULL
    ) ON COMMIT DROP;
    RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';

CREATE FUNCTION insert_modified_profile_to_temp_table()
RETURNS TRIGGER
AS $$
BEGIN
    INSERT INTO tmp_modified_profiles(id) VALUES (NEW.id);
    RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';

CREATE FUNCTION post_stmt_insert_rows_and_drop_temp_table()
RETURNS TRIGGER
AS $$
BEGIN
    INSERT INTO profile_audit (id, modified_at)
    SELECT t.id, CURRENT_TIMESTAMP FROM tmp_modified_profiles t;

    DROP TABLE tmp_modified_profiles;

    RETURN NULL;
END;
$$ LANGUAGE 'plpgsql';

CREATE TRIGGER tr_create_working_table BEFORE UPDATE ON profiles FOR EACH STATEMENT EXECUTE PROCEDURE pre_stmt_create_temp_table();
CREATE TRIGGER tr_insert_row_to_working_table AFTER UPDATE ON profiles FOR EACH ROW EXECUTE PROCEDURE insert_modified_profile_to_temp_table();
CREATE TRIGGER tr_insert_modified_rows_and_drop_working_table AFTER UPDATE ON profiles FOR EACH STATEMENT EXECUTE PROCEDURE post_stmt_insert_rows_and_drop_temp_table();

INSERT INTO files ("name") VALUES ('photo.jpg'), ('my_cv.pdf');

INSERT INTO profiles ("name") VALUES ('John Doe');

DELETE FROM files WHERE "name" = 'photo.jpg';
7
  • 1
    Support for statement level triggers with virtual OLD and NEW tables may appear in PostgreSQL 9.5, where's been a fair bit of discussion on how to do it. There's not really a great way to improve on what you have without something like that, the concurrency issues otherwise are rather problematic. Commented Jan 8, 2015 at 10:23
  • That sounds great. I have been watching the roadmap for quite a while. I have noticed that it has been on the list there, but I wasn't getting my hopes up for this feature to make it for some time, as I couldn't find a list stating when it was planned to be done. Now I just need to figure out what to do before then... Commented Jan 8, 2015 at 11:07
  • You say that When deleting a row from the files table, all these system level triggers to handle the ON DELETE SET NULL clause all fire at the same time, that is in parallel. Which presents a serious issue for me. That's not true. They fire serially, one after the other. What problem does this cause for you exactly? Performance? Other? Commented Jan 8, 2015 at 12:24
  • The problem is that the BEFORE statement-level trigger can't create the temporary table when it is fired for the second foreign key column. Using the posted script you should see the issue. I tried to verify that they fire in parallel by increasing a counter in the BEFORE statement trigger and then printing out the counter in the AFTER statement trigger. All values printed were the same. Commented Jan 8, 2015 at 12:57
  • 1
    Ah, ok. They don't fire in parallel, but they can't see each others' effects, and that seems to be what's giving you trouble. Unfortunately I find the combination of temp tables and triggers to be fragile at best, and unworkable most of the time. Commented Jan 8, 2015 at 13:21

2 Answers 2

1

It would be a serious hack, but meanwhile, until PostgreSQL 9.5 is out, I would try to use CONSTRAINT triggers deferred to the end of the transaction. I am not really sure this will work, but might be worth trying.

Sign up to request clarification or add additional context in comments.

1 Comment

...but even if it worked, the downside would be that your materialized data would be inconsistent until committing the transaction. That might be acceptable in certain scenarios, but is generally fragile.
0

You could use a status column to track inserts and updates for your statement-level triggers.

In a BEFORE INSERT OR UPDATE row-level trigger:

    SET NEW.status = TG_OP;

Now you can use statement-level AFTER triggers:

    BEGIN
        DO FUNNY THINGS
         WHERE status = 'INSERT';

        -- reset the status
        UPDATE mytable
           SET status =  NULL
         WHERE status = 'INSERT';
    END;

However, if you want to deal with deletes as well, you'll need something like this in your row-level trigger:

    INSERT INTO status_table (table_name, op, id) VALUES (TG_TABLE_NAME, TG_OP, OLD.id);

Then, in your statement-level AFTER trigger, you can go like:

    BEGIN
        DO FUNNY THINGS
         WHERE id IN (SELECT id FROM status_table 
                       WHERE table_name = TG_TABLE_NAME AND op = TG_OP); -- just an example

        -- reset the status
        DELETE FROM status_table
         WHERE table_name = TG_TABLE_NAME AND op = TG_OP;
    END;

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.