How to merge multiple arrays from ASP.Net Core configuration files?

Question

I would like dynamically load and register services in my application. To do that I need to be able to load configuration files from different projects in solution and merge values from them into single json array. Unfortunately by default in ASP.Net Core configuration overrides values.

I register files with following code (part of Program.cs file):

public static IWebHostBuilder CreateWebHostBuilder(string[] args) =>
        WebHost.CreateDefaultBuilder(args)
            .ConfigureAppConfiguration((webHostBuilderContext, configurationbuilder) =>
            {
                var env = webHostBuilderContext.HostingEnvironment;
                configurationbuilder.SetBasePath(env.ContentRootPath);
                configurationbuilder.AddJsonFile("appsettings.json", false, true);

                var path = Path.Combine(env.ContentRootPath, "App_Config\\Include");

                foreach(var file in Directory.EnumerateFiles(path, "*.json",SearchOption.AllDirectories))
                {
                    configurationbuilder.AddJsonFile(file, false, true);
                }
                configurationbuilder.AddEnvironmentVariables();
            })
            .UseStartup<Startup>();

The code searches for all files with *.json extension inside App_Config\Include directory and adds all of them to the configuration builder.

Structure of the files look following way:

{
  "ServicesConfiguration": {
    "Services": [
      {
        "AssemblyName": "ParsingEngine.ServicesConfigurator, ParsingEngine"
      }
    ]
  }
}

As you can see I have got main section ServicesConfiguration then Services array with objects which have one attribute AssemblyName.

To read those values I use ServicesConfiguration class with list:

public class ServicesConfiguration
    {
        public List<ServiceAssembly> Services { get; set; }
    }

And that list uses ServiceAssembly class:

public class ServiceAssembly
    {
        public string AssemblyName { get; set; }
    }

To load that configuration I use IOptions at constructor level (DI):

Microsoft.Extensions.Options.IOptions<ServicesConfiguration> servicesConfiguration,

And configuration seems to be loaded - but values from files are not merged but overridden by last found file.

Any ideas how to fix that?

Well.. one way I think it could work (although not the prettiest, for sure) is: You could merge the jsons yourself, (get the file, parse it and put somewhere) do the merging, ending up with the List<ServiceAssembly> Services you want, then serialize that again into a json file and call AddJson passing it. — jpgrassi
– jpgrassi, Commented Dec 31, 2018 at 9:44
Thinking better, you could even forget about adding json and using the "traditional" way. You can just read the JSON files yourself, parse them into the class and just register that class as a singleton. This way you can do whatever you want with the JSON files and plus you avoid having to use the IOptions pattern which most people (me included) dislike. — jpgrassi
– jpgrassi, Commented Dec 31, 2018 at 10:02

jpgrassi · Accepted Answer · 2018-12-31 10:45:46Z

So you have an idea on what I meant in my comments here's a potential answer

Since you have to load different "config" files from different projects and apply some merging logic to them, I would just avoid using the "default" configuration system to load the JSON files into the app. Instead, I would just do it myself. So:

Read and deserialize the JSON into a type and keep it on a list
Go through the list containing all configs and apply your merging logic
Register the single ServicesConfiguration as a Singleton
Remove the code you had on your Program.cs to load the custom JSON files

Here's how you could do it:

ServicesRootConfiguration (new class, to be able to deserialize the json)

public class ServicesRootConfiguration
{
    public ServicesConfiguration ServicesConfiguration { get; set; }
}

Startup.cs

public class Startup
{
    private readonly IHostingEnvironment _hostingEnvironment;

    public Startup(IConfiguration configuration, IHostingEnvironment env)
    {
        Configuration = configuration;
        _hostingEnvironment = env;
    }

    public IConfiguration Configuration { get; }

    public void ConfigureServices(IServiceCollection services)
    {
        // other configuration omitted for brevity

        // build your custom configuration from json files
        var myCustomConfig = BuildCustomConfiguration(_hostingEnvironment);

        // Register the configuration as a Singleton
        services.AddSingleton(myCustomConfig);
    }

    private static ServicesConfiguration BuildCustomConfiguration(IHostingEnvironment env)
    {
        var allConfigs = new List<ServicesRootConfiguration>();

        var path = Path.Combine(env.ContentRootPath, "App_Config");

        foreach (var file in Directory.EnumerateFiles(path, "*.json", SearchOption.AllDirectories))
        {
            var config = JsonConvert.DeserializeObject<ServicesRootConfiguration>(File.ReadAllText(file));
            allConfigs.Add(config);
        }

        // do your logic to "merge" the each config into a single ServicesConfiguration
        // here I simply select the AssemblyName from all files.
        var mergedConfig = new ServicesConfiguration
        {
            Services = allConfigs.SelectMany(c => c.ServicesConfiguration.Services).ToList()
        };

        return mergedConfig;
    }
}

Then in your Controller just normally get the instance by DI.

public class HomeController : Controller
{
    private readonly ServicesConfiguration _config;

    public HomeController(ServicesConfiguration config)
    {
        _config = config ?? throw new ArgumentNullException(nameof(config));
    }
}

With this approach, you ended up with the same behavior as you would get from normally registering the IOptions. But, you avoid having a dependency on it and having to use the uggly .Value (urgh). Even better, you could register it as an Interface so it makes your life easier during testing/mocking.

Canabale · Accepted Answer · 2022-09-29 08:48:30Z

1

I was actually having a similar issue in dotnet 6, when trying to merge arrays from multiple appsettings, when I stumbled across this thread.
The solution was for me actually way simpler then thought.
Microsoft.Extensions.Configuration merges arrays through the index:
{ foo: [1, 2, 3] } + { foo: [4, 5] } = { foo: 4, 5, 3 }

But we wanted to be able to declare which entries override others and which ones should be added to the list. And we do this by declaring a GUID as dictionary key instead of an array.

{
  "foo": {
     "870622cb-0372-49f3-a46e-07a1bd0db769": 1,
     "cbb3af55-94ea-41a5-bbb5-cb936ac47249": 2,
     "9410fcdc-28b3-4bff-bfed-4d7286b33294": 3
  }
}
+
{
  "foo": {
    "cbb3af55-94ea-41a5-bbb5-cb936ac47249": 4,
    "1c43fa78-b8db-41f8-809d-759a4bc35ee2": 5,
  }
}
=
{
  "foo": {
    "870622cb-0372-49f3-a46e-07a1bd0db769": 1,
    "cbb3af55-94ea-41a5-bbb5-cb936ac47249": 4, /*2 changed to 4 because key equals*/
    "9410fcdc-28b3-4bff-bfed-4d7286b33294": 3
    "1c43fa78-b8db-41f8-809d-759a4bc35ee2": 5, /*while 5 added to the list*/
  }
}

This may seem inconventient at first, because one would think, that ((IConfiguration)config).GetSection("foo").Get<int[]>();

throws some kind of invalid type exception, since a Guid is not what we know as array index. But it can actually (implicitly!) return the merged "foo" section above as int[].

answered Sep 29, 2022 at 8:48

Canabale

4445 silver badges14 bronze badges

5 Comments

Mikhail Brinchuk Over a year ago

This approach works quite nicely.

Good Night Nerd Pride Over a year ago

Does this work with removing elements from a list too?

Good Night Nerd Pride Over a year ago

And the key doesn't need to be a GUID, right?

Canabale Over a year ago

@GoodNightNerdPride nope. It can be whatever you want it to be. :)

Canabale Over a year ago

About the list thing... Good question. I would probably try a set null approach and filter with linq...

Yevgeniy Chernyshkov · Accepted Answer · 2024-09-26 16:43:18Z

0

One of the way available in .NET 6 is an extension method Configure , which takes an action as a parameter. This create required object, and it can also be used with the IOptions interface.

builder.Services.Configure<ServicesConfiguration>((config) => 
{
  config.Services = /// merging files into a ServicesConfiguration object   

}

answered Sep 26, 2024 at 16:43

Yevgeniy Chernyshkov

1

Collectives™ on Stack Overflow

How to merge multiple arrays from ASP.Net Core configuration files?

3 Answers 3

Comments

5 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

5 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related