Wrong approach. Table was mutating (as the trigger updates the same table which is being modified by the main transaction), so you tried to save yourself by declaring it an autonomous transaction, but then insert caused trigger to fire and update the table which fired the trigger again and so on, until you run out of resources.
Beside that, another HUGE mistake you did: don't ever do anything in SYS or SYSTEM schema.
ORA-04088: error during execution of trigger 'SYSTEM.MODIFIY_BYY'
Leave them alone, they own the database and - if you do something you shouldn't have, you'll destroy the database. Do your training in another schemas (such as SCOTT or HR if available, or create one of your own).
Back to your problem: you don't have to directly update the table, just use pseudorecord (:new or :old, whichever is appropriate to what you do).
For a sample table:
SQL> create table family
2 (f_num number,
3 f_name varchar2(20),
4 f_age number,
5 upby varchar2(20)
6 );
Table created.
trigger would look like this:
SQL> create or replace trigger modify_byy
2 before insert or update or delete on family
3 for each row
4 begin
5 if inserting then
6 :new.upby := 'insert'; --> note this
7 elsif updating then
8 :new.upby := 'update'; --> and this
9 elsif deleting then
10 dbms_output.put_line('deleted');
11 end if;
12 end;
13 /
Trigger created.
Testing:
SQL> set serveroutput on;
SQL> insert into family(f_num, f_name, f_age) values (1, 'Little', 10);
1 row created.
SQL> select * from family;
F_NUM F_NAME F_AGE UPBY
---------- -------------------- ---------- --------------------
1 Little 10 insert
SQL> update family set f_age = 20 where f_num = 1;
1 row updated.
SQL> select * from family;
F_NUM F_NAME F_AGE UPBY
---------- -------------------- ---------- --------------------
1 Little 20 update
SQL> delete from family where f_num = 1;
deleted
1 row deleted.
SQL>
However, trigger name (as well as upby column name) suggest that trigger should actually record who did the last change. If so, it would be simplified to
SQL> create or replace trigger modify_byy
2 before insert or update on family
3 for each row
4 begin
5 :new.upby := user;
6 end;
7 /
Trigger created.
SQL> insert into family(f_num, f_name, f_age) values (2, 'Foot', 30);
1 row created.
SQL> select * from family;
F_NUM F_NAME F_AGE UPBY
---------- -------------------- ---------- --------------------
2 Foot 30 SCOTT
SQL>
Handling deletes in this context doesn't make much sense; you'd have to have a separate "log" table to log deletes. And, when you do that, switch inserts and updates to it (the log table, I mean) as well. And add a timestamp column.