-1

Configuration

  1. Database Engine = mybusiness.database.windows.net
  2. Records Database = Records --> dbo.accesslogs
  3. Legal Database = Legal --> dbo.personalinfo
  4. Access Database = Access --> dbo.accessinfo

Problem Description

  1. When people join the company their personal data such as employeeID (PK), name, gender, taxCode, address etc is entered into the dbo.personalinfo table of the Legal database. This is a very controlled database.
  2. People given access to our offices by key-swipe-tokens which are enrolled into the system and their employeeID (FK), tokenID (PK) and other details such where there token works and doesn't work are stored in the dbo.accessinfo table of the Access database.
  3. When anyone swipes their building access token the tokenID (FK), accessPoint, dateAndTime, accessID (PK) is recorded in the dbo.accesslogs table of the Records database. The accessID is unique for every scan (whether successful or not) of a RFID token.
  4. After people leave we need to purge their data from the dbo.accesslogs table. We aim to do this by creating an elastic query from the Legal database, as follows.
    • T-Sql query in the Legal database creates External table of the dbo.accessinfo table from the Access database.
    • T-Sql query in the Legal database creates External table of the dbo.accesslogs table from the Records database.
  5. T-Sql query in the Legal database creates Temporary table called #userrecords which contains every accessID for every tokenID corresponding to that user's specific employeeID.

Question

How do I construct a T-Sql query from the Legal database that uses the accessID from the temporary table to delete the rows of the dbo.accesslogs table in the Records database?

2
  • Are there any foreign keys, triggers or constraints on these tables that might affect delete operation? also Could you share the table schema as well? Commented May 22 at 17:57
  • As I understand it, External tables are Read-Only. For clarity the T-Sql query must run in the Legal database, we are not allowed to make an External table of dbo.personalinfo in any other database. I have edited the original question to show which are the FKs. Which table's schema do you want to see, please? Commented May 23 at 9:17

1 Answer 1

-1

Since you cannot issue a DELETE directly from the Legal database against the accesslogs table in the Records database (because external tables are read-only), the recommended solution is:

  1. Prepare the list of accessID to delete in Legal database using external tables.
  2. Export this list to a staging table in the Records database.
  3. Use a stored procedure in the Records database to perform the deletion.
  4. Trigger the procedure using an orchestrator such as Azure Data Factory, Synapse Pipeline, or a scheduled job.

So, let's start by creating External tables in Legal database:

CREATE EXTERNAL TABLE AccessDB_accessinfo
(
    employeeID INT,
    tokenID INT,
    ...
)
WITH (DATA_SOURCE = [AccessDBDataSource], SCHEMA_NAME = 'dbo', OBJECT_NAME = 'accessinfo');

CREATE EXTERNAL TABLE RecordsDB_accesslogs
(
    accessID INT,
    tokenID INT,
    accessPoint NVARCHAR(100),
    dateAndTime DATETIME,
    ...
)
WITH (DATA_SOURCE = [RecordsDBDataSource], SCHEMA_NAME = 'dbo', OBJECT_NAME = 'accesslogs');

After this, prepare the list of accessIDs to Delete in the Legal Database (Replace this with your specific logic). From the Legal database, use a T-SQL query to join the data and filter by employeeID:

SELECT al.accessID
INTO #userrecords
FROM RecordsDB_accesslogs al
JOIN AccessDB_accessinfo ai ON al.tokenID = ai.tokenID
JOIN dbo.personalinfo pi ON ai.employeeID = pi.employeeID
WHERE pi.employeeID = @targetEmployeeID; 

Now, Export accessIDs from Legal to a staging table in Records:

Since external tables are read-only, the Legal database cannot write directly to the Records database. To bridge this, let's Create a staging table in Records database:

CREATE TABLE dbo.to_be_deleted_accesslogs (
    accessID INT PRIMARY KEY
);

Then create a Synapse pipeline/ Azure Data Factory/SQL job that:

  • Connects to the Legal database.
  • Inserts the results into Records.dbo.to_be_deleted_accesslogs.

Lastly, Call a stored procedure in Records DB to delete from accesslogs:

CREATE PROCEDURE dbo.DeleteOldAccessLogs
AS
BEGIN
    DELETE FROM dbo.accesslogs
    WHERE accessID IN (SELECT accessID FROM dbo.to_be_deleted_accesslogs);
END;

Now Trigger the procedure via:

  • A scheduled PowerShell script
  • Azure Data Factory or Synapse pipeline
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.