0

I am experiencing an odd problem that I cannot find any solution for the last week.

Wednesday 16/4 ~17.00 suddenly my APIs deployed in an Azure Function app have become much slower.

The architecture is the following: Airflow runs .py file that writes on Azure SQL Server database. Then an Azure Function app deployed API selects data from that Azure SQL database.

  • We did not change any of the APIs code
  • We did not change .py files
  • We did not change the table formats in Azure SQL Server

Also, from Wednesday evening on, a daily scheduled job in Airflow has experienced these errors:

'life_cycle_state': 'TERMINATED', 'result_state': 'FAILED', 'state_message': 'Workload failed, see run output for details'} and with the errors [{'task_key': 'upload_to_tmp_tables', 'run_id': 672367321749800, 'error': 'org.apache.spark.SparkException: Job aborted due to stage failure: Task 3 in stage 77.0 failed 4 times, most recent failure: Lost task 3.3 in stage 77.0 (TID 132) (10.148.6.142 executor 1): com.microsoft.sqlserver.jdbc.SQLServerException: The elastic pool has reached its storage limit. The storage usage for the elastic pool cannot exceed (256000) MBs.'}]

[2025-04-21, 23:24:40 EEST] {taskinstance.py:3093} ERROR - Received SIGTERM. Terminating subprocesses

{'life_cycle_state': 'TERMINATED', 'result_state': 'FAILED', 'state_message': 'Workload failed, see run output for details'} and with the errors [{'task_key': 'upload_to_tmp_tables', 'run_id': 401438450064268, 'error': "OperationalError: (20047, b'DB-Lib error message 20047, severity 9:\nDBPROCESS is dead or not enabled\n')"}]

Right now, my Azure database has 187 GB space used /600gb. Elastic Pool has 188 GB space used / 250GB.

Data space used in Azure DB is almost the same the last 3 months (+- 100 mb). Data space in the other databases is always the same.

The biggest table I try to write to in the Azure SQL database is 370 MB with a clustered columnstore index and non-unique non-clustered indexes.

Improve this question
3
  • 2
    I might try trusting the error msg "The elastic pool has reached its storage limit.", expanding the elastic pool, and retrying the process. The "upload_to_tmp_tables" suggests temporary tables might be involved, with some unspecified effect on storage. Commented Apr 23 at 12:58
  • The elastic pool problem is fixed by expanding the pool size. Writing in SQL server is much slower than before. Also the main problem is the latency on APIs not the elastic pool problem. I mentioned it in case they are connected. Commented Apr 24 at 5:49
  • Well, what does your API do? If data is growing, at some point things will be slower. You should try acquiring some queries and query plans Commented Apr 24 at 12:42
Related questions
0
Azure SQL Elastic Pool Decrease Storage Limit Below Usage for SQL Database (PaaS)
2
Azure SQL - How to add AD users to SQL Pool Databases
2
Terraform Azure - Move an existing Azure SQL Database into an Elastic Pool
Load 6 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.