The Azure documentation says
"Setting app settings in App Service are like setting them in in appsettings.json, but the values in App Service override the ones in appsettings.json. You can keep development settings (for example, local MySQL password) in appsettings.json and production secrets (for example, Azure MySQL database password) safely in App Service. The same code uses your development settings when you debug locally, and it uses your production secrets when deployed to Azure."
which is exactly what I want.
But when I return the config string it is using, it shows the value from my local appsettings file.