0

I'm working on the function Import, i have an excel file which contains some data, that later will be edited by the user, I managed to do the import excel by SmartXLS in C# and update all data to SQL Server Database, however, what I did is to fetch all data in excel file and update all rows into the SQL Table, which affects to the performance and I also updated unedited rows.

I would like to ask that is there any way that I can get only modified cells, rows in Excel and update to the correponding data in SQL Table?

var workbook = new WorkBook();
workbook.read(filePath);
var dataTable = workbook.ExportDataTable();
7
  • 1
    Can you upload to a staging table then update changes within the DB? - sorry not 100% sure where the performance bottle neck is. excel reading rows? inserting to the DB or checking which lines have changed (if so where is this done - is it between datatables in the client application?) Commented May 28, 2015 at 13:45
  • those questions don't need answered.. they are just there to explain my suggestion ;) Commented May 28, 2015 at 13:47
  • Updated rows since when? Commented May 28, 2015 at 13:48
  • Thanks, Not much about the performace problem as now I just have some excel rows, but later we will have hundred or more rows, what I believe that the performance will be reduced, and also, I personally think that it is not so effective if I also get unmodified rows in excel and do the update sql of these data. @gordatron . Sorry I could not get the idea of the staging table ? Commented May 28, 2015 at 13:50
  • 1
    @gordatron that's what i would recommend too. Pull the data from excel to a staging table (maybe you can bulk insert the data - this would be the fastest solution) and then do a Merge-Statement to your Data on SQL-Server Commented May 28, 2015 at 13:50

1 Answer 1

1

Just a Scenarion, maybe it helps you to understand what gordatron and i were talking about:

Following Situation: There is a Table "Products" wich is central storage place for product informations and a table "UpdatedProducts" which structure looks exactly like "Products" table but data maybe different. Think of following scenarion: you export product table to excel in the morning. the whole day you delete, add, update products in your excel table. At the end of the day you want to re-import your excel data to "Products" table. What you need:

  • delete all records from "UpdatedProducts"
  • insert data from excel to "UpdatedProducts" (bulk insert if possible)
  • update the "Products" table

Then a Merge-Statement could look like this:

MERGE Products AS TARGET
USING UpdatedProducts AS SOURCE 
    ON TARGET.ProductID = SOURCE.ProductID
WHEN MATCHED AND TARGET.ProductName <> SOURCE.ProductName OR TARGET.Rate <> SOURCE.Rate 
    THEN UPDATE SET TARGET.ProductName = SOURCE.ProductName, 
                    TARGET.Rate = SOURCE.Rate 
WHEN NOT MATCHED BY TARGET 
    THEN INSERT (ProductID, ProductName, Rate) 
        VALUES (SOURCE.ProductID, SOURCE.ProductName, SOURCE.Rate)
WHEN NOT MATCHED BY SOURCE 
    THEN DELETE

What this Statement does: WHEN MATCHED: Data exist in both tables, we update data in "Products" if ProductName or Rate is different

WHEN NOT MATCHED BY TARGET: Data exist in staging table but not in your original table, we add them to "Products"

WHEN NOT MATCHED BY SOURCE: Data exists in your original table but not in staging table, thy will be deleted from "Products"

Thanks a lot to http://www.mssqltips.com/sqlservertip/1704/using-merge-in-sql-server-to-insert-update-and-delete-at-the-same-time/ for this perfect example!

Sign up to request clarification or add additional context in comments.

6 Comments

Thank you, @CeOnSql it is a clear answer, I could grab what you guys suggested, now what i'm doing is I will pass a list of data collected in excel file, to SQL under TVP parameters, but I still wonder how can I get data in TVP (productname, rate, etc) to insert into the UpdatedProducts table
Or I should use Bulk Insert but im not familiar with this approach
I think the slowest operation in this case is to read the data from excel and store it in a DataTable in C#. If you are able to transform your excel to a csv file then bulk insert would be easy. Here is a description how to save a xls(x) File to csv: stackoverflow.com/questions/2536181/… and here you can find how to bulk insert this temporary csv: stackoverflow.com/questions/19302226/…
And to be sure that Performance of MERGE Statement is OK you should have a (unique) clustered index on ProductID and nonclustered index on ProductName and Rate on BOTH TABLES!
Thanks a lot @CeOnSql, I have just one more thing, is the <> symbol works on the comparison between 2 strings ? and it ignores case or I have to manage the Case Sensitive ?
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.