Skip to main content
Tweeted twitter.com/StackCodeReview/status/1478878985572528130
updated to version-2
Source Link
t3chb0t
  • 44.7k
  • 9
  • 85
  • 191

Here's a small experiment about handling boilerplate tasks via service pipelines. It's inspired bysimilar to the htmlHTML pipeline but forsimpler and multipurpose.


I've updated the question and this particular use case greatly simplifiedis the 2nd version of this project. I'llThe first one turned out to have some substantial weaknesses like inability to invoke more than one controller service to handle requests. The code is written in C# 10 and .net-6.


Example

Let's start with an examplea demo-app demonstrating how I'd use it:

public static class ServicePipelineDemoServicePipelineDemo2
{
    public static async Task Test()
    {
        // Compose the container.
        var builder = new ContainerBuilder();
        
        // Register middlewarecache servicesdependency,
 as a      builder.RegisterInstance(new MemoryCache(new MemoryCacheOptions())).As<IMemoryCache>().SingleInstance();

        // Register service-pipeline-builder and associate it with ReadFile.Text by name.
        builder.Register(_c => new ServicePipeline<string>Service.PipelineBuilder
        {
            // This pipeline should resolve environment variables that might be used by the .Name property.
            new EnvironmentVariableService<string>EnvironmentVariableService(PropertyService.For<IReadFile>.Select(x => x.Name)),
            // OverrideAll cache-lifetimetext files should be cached for all30min txt-and other files for 15min.
            new CacheLifetimeService<string>CacheLifetimeService(TimeSpan.FromMinutes(15))
            {
                Rules =
                {
                    // Add a condition for read-file request.
                    Condition.For<IReadFile>.When(x => x.KeyName.EndsWith(".txt") ?, ".txt").Then(TimeSpan.FromMinutes(30)) 
 : TimeSpan              }
            },
            new CacheService(c.ZeroResolve<IMemoryCache>(), PropertyService.For<IReadFile>.Select(x => x.Name)),
            // This pipeline should look in embedded resources first before it reads files. 
            // Since this node doesn't have to succeed, the next one is called.
            new CacheService<string>(EmbeddedResourceService<ServicePipelineDemo2> { MustSucceed = false },
            // This overrides the file-service for testing.
            new MemoryCacheConstantService.Text("This is not a real file!"),
            // Finally this node tries to read a file.
            new MemoryCacheOptionsFileService.Read()))
        }).InstancePerDependency().Named<ServicePipeline<string>>Named<Service.PipelineBuilder>(nameof(ReadFile.Text));

        await using var container = builder.Build();
        await using var scope = container.BeginLifetimeScope();

        // Set some environment variable.
        Environment.SetEnvironmentVariable("HOME", @"c:\temp");

        // EverythingCreate isthe setuprequest soand let'sinvoke go!it.
        var result = 
            await new ReadFile.Text(@"%HOME%\notes.txt")
                .CacheLifetime(TimeSpan.FromMinutes(10))
                .Bind(scope)
                .InvokeAsync(scope);

        Console.WriteLine(result); // --> c:\temp\notes"This is not a real file!" or FileNotFoundException (as there is no "notes.txttxt".
    }
}

I'll try to anticipate and answer some of the questions that might arise:

FAQ

  • Why do I use new ReadFile.Text instead of resolving the serivce via dependency injection? -- The constructor allows me to better enforce required parameters. This way I can also useSince this is only a consistent IService<T> interface for all servicesrequest and connect themdoesn't require any dependencies here, I'd say it's fine. Additionally it also specifies the expected return type.
  • How do I associate middlewarepipelines with the actual servicerequests? -- I use Named services. By default the name of the request type is used (like Text here). Such a pipeline would apply to all requests of this type but it's possible to add a Tag to the Items dictionary and use this for the lookup. When there is a middleware then the target service is attached as the last node.
  • Why do I register the pipeline as a Func<>? -- Every pipelineIt should be possible to create independent herepipelines for each request but at the actual usageend it depends on the actual use case so everything is possiblethere can be a single pipeline for a lifetime scope.
  • In which order are the middleware evaluated? -- They are evaluated in the same order as registered.
  • Why Autofac? -- I like Autofac. Microsoft's DI is too limited.
  • C# & .net versions? -- C# 10 and .net-6.
  • Is there a GitHub link? -- There is - everything as a single fileThere is.
  • What would be some other usage scenarios?
    • I canTo completely replace the middleware with something else for testing and throw exceptions or return other results skippingwithout modyfing the actual service doing file readingrequesting code.
    • I canTo add another middleware to create file copies or backups without modyfing the code where I would call WriteFile is requested.
    • I canTo add anothervalidation middleware and validate files.
    • I can create an email-service andTo add json de/serialization middleware attaching files or signatures.
    • I canTo create an email-service and middleware attaching files or signatures.
    • To create a config-service that today might read settings from appsettings.json and tommorow from a database by only chanigng the middleware. Another middleware could at the same time validate settings.
    • I canTo add a telemetry middleware that would log the usage of each servicerequest.
    • To add multiple services of the same type and try to read a file from different locations and use the first successful result.
    • To add a retry middleware.
    • ...

In this project every taskLike the HTTP pipeline there is represend by an interface andalso a base class that implements itrequest but no context and response objects. The result is returned directly. The request specifies what you'd like to do e.g.: ReadFile or CopyFile. It defines a linktravels through the pipeline where each node can do anything with it.

Nodes or middleware are called ISerivce. They point to the next one and have one additional property NextMustSucceed service. By default this is false for normal nodes and true for controller nodes like ReadFile when it's the last one in the chain and a method to invoke itresult must be returned or an exception should be thrown.

The Items dictionary allows to pass additional data.

public interface IItemsIRequest
{
    //IDictionary<string, Allowsobject> toItems store{ additionalget; metadata.}
}

public interface IRequest<T> : IRequest { }

public abstract class Request<T> : IRequest<T>
{
    public IDictionary<string, object> Items { get; } = new Dictionary<string, object>(SoftString.Comparer);
}

public sealed class Unit
{
    private Unit() { }

    public static readonly Unit Default = new();
}

public interface IService<T>IService : IItems, IEnumerable<IService<T>>IEnumerable<IService>
{
    bool MustSucceed { get; }

    // Points to the next service in a pipelinemiddleware.
    IService<T>IService? Next { get; set; }

    Task<T>Task<object> InvokeAsync(IRequest request);
}

public abstract class Service<T>Service : IService<T>IService
{
    public IService<T>?bool NextMustSucceed { get; set; }

    public IDictionary<string, object>IService? ItemsNext { get; } = new Dictionary<string,set; object>(SoftString.Comparer);}

    public abstract Task<T>Task<object> InvokeAsync(IRequest request);

    public IEnumerator<IService<T>>IEnumerator<IService> GetEnumerator() => this.Enumerate().GetEnumerator();

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    //protected Null-Serviceasync Task<object> InvokeNext(IRequest request)
    {
        return
            Next is { } next
                ? await next.InvokeAsync(request)
    public            : Unit.Default;
    }

    private class Empty : Service<T>Service
    {
        public override async Task<T>Task<object> InvokeAsync(IRequest request) => await InvokeNext(request);
    }

    public class PipelineBuilder : IEnumerable<IService>
    {
        private IService First { get; } = new Empty();

        // Adds the specified service at the end of the pipeline.
        public void Add(IService last) => First.Enumerate().Last().Next? = last;

        public IEnumerator<IService> GetEnumerator() => First.InvokeAsyncEnumerate().GetEnumerator();

        IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

        public IService Build() => First;
    }
}

The Service also provides the PipelineBuilder that knows how to connect serivces that are added to it.

For the sake of this experiment I'veI created a single servicetwo ReadFile that would normally read a file. Here it returns a string but it could also be a Stream etc.requests:

public abstract class ReadFile<T> : Service<T>, IVariable<string>, IAssociable<string>
{
    protected ReadFile(string name) => Name = name;

    public string Name { get; set; }

    string IVariable<string>.Value
    {
        get => Name;
        set => Name = value;
    }

    string IAssociable<string>.Key => Name;
}

public static class ReadFile
{
    // Reads files as string.
    public class Text : ReadFile<string>
    {
        public Text(string name) : base(name) { }

        public override Task<string> InvokeAsync()
        {
            return Task.FromResult($"Hallo from '{Name}'.");
        }
    }
}

It implements a couple of interfaces that are supported by middleware services:

  • Text gets a string
  • Stream gets a FileStream.
public interface IVariable<T>IReadFile : IRequest
{
    public Tstring ValueName { get; set; } 

    public FileShare Share { get; }
}

public interfaceabstract IAssociable<outclass T>ReadFile<T> : Request<T>, IReadFile
{
    protected ReadFile(string name) => Name = name;

    public string Name { get; set; }

    public TFileShare KeyShare { get; set; } = FileShare.Read;
}

public abstract class ReadFile
{
    public class Text : ReadFile<string>
    {
        public Text(string name) : base(name) { }

        public Encoding Encoding { get; set; } = Encoding.UTF8;
    }

    public class Stream : ReadFile<FileStream>
    {
        public Stream(string name) : base(name) { }
    }
}

Middleware

Middleware

As far as reading files is concered there are twoit's common to perform such tasks that many apps implementas:

  • resolving environment variables in paths andor
  • caching results.

Both of them are handled by two middleware serivces that look for particular interfaces they support. TheThey work dynamically with properties specified via the constructor. (The cache-service is a dummy but the environment-variable service updates the file-name.)

// Provides caching for service results.
public class CacheService<T>EnvironmentVariableService : Service<T>Service
{
    public CacheServiceEnvironmentVariableService(IMemoryCacheIPropertyAccessor<string> cacheproperty) => CacheProperty = cache;property;

    private IMemoryCacheIPropertyAccessor<string> CacheProperty { get; }

    public override async Task<T>Task<object> InvokeAsync(IRequest request)
    {
        if Property.SetValue(thisrequest, Environment.LastExpandEnvironmentVariables()Property.CacheLifetimeGetValue(request) is var cacheLifetime && cacheLifetime > TimeSpan.Zero)
        {
            Console.WriteLine(cacheLifetime);
        }

        return await Next?.InvokeAsyncInvokeNext(request);
    }
}
 
// Allows to associate cache-lifetime with a service.
public class CacheLifetimeService<T>CacheService : Service<T>Service
{
    public CacheLifetimeServiceCacheService(Func<IAssociable<string>IMemoryCache cache, TimeSpan>IPropertyAccessor<string> lifetimeFunckey) => LifetimeFunc(Cache, Key) = lifetimeFunc;(cache, key);

    private Func<IAssociable<string>,IMemoryCache TimeSpan>Cache LifetimeFunc{ get; }

    private IPropertyAccessor<string> Key { get; }

    public override async Task<T>Task<object> InvokeAsync(IRequest request)
    {
        if (thisrequest.LastCacheLifetime() is var lastcacheLifetime && cacheLifetime > TimeSpan.Zero)
        {
            if Console.WriteLine(last$"Cache-lifetime: is{cacheLifetime}");
 IAssociable<string> associable && LifetimeFunc(associable) is var lifetime)
  }

        return await {InvokeNext(request);
    }
}

// Allows to associate cache-lifetime to with a service.
public class CacheLifetimeService : last.CacheLifetime(lifetime);Service
{
    public CacheLifetimeService(TimeSpan fallback)
    {
  }
      if (fallback }

== TimeSpan.Zero) throw new ArgumentException("Fallback value needs to returnbe awaitgreater Next?than zero.InvokeAsync(");
    }
}

public class EnvironmentVariableService<T> : Service<T>
{
Fallback = fallback;
  public override async}

 Task<T> InvokeAsync()
  private TimeSpan Fallback { get; }

    public List<ConditionBag<TimeSpan>> Rules { ifget; (this.Last} = new();

 is IVariable<string> identifiable)
 public override async Task<object> InvokeAsync(IRequest request)
    {
        var lifetime = Rules.Where(c => identifiablec.ValueEvaluate(request)).Select(c ==> Environmentc.ExpandEnvironmentVariablesGetValue(identifiable)).ValueFirstOrDefault(); 

        }request.CacheLifetime(lifetime > TimeSpan.Zero ? lifetime : Fallback);

        return await Next?.InvokeAsyncInvokeNext(request)!;
    }
}

Pipeline

Controllers

The pipeline is build bySerives that handle the SerivcePipelineReadFile. It knows how to connect serivces that request are added to the and of the pipeline.:

  • FileService and
  • EmbeddedResourceService

I call them controllers although they are like every other middleware and can be placed anywhere in the pipeline.

//public Buildsabstract serviceclass pipeline.FileService : Service
{
    protected FileService()
    {
        MustSucceed = true;
    }

    public class ServicePipeline<T>Read : IEnumerable<IService<T>>FileService
    {
    private IService<T> First  public override async Task<object> InvokeAsync(IRequest request)
        { 
 get; }          if (request is IReadFile file)
            {
                if (File.Exists(file.Name))
                {
                    if (request is ReadFile.Text t)
                    {
                        await using var stream = new Service<T>FileStream(file.EmptyName, FileMode.Open, FileAccess.Read, file.Share);
                        return await stream.ReadTextAsync(t.Encoding);
                    }

    // Adds the specified service at the end of the pipeline      if (request is ReadFile.Stream s)
    public void Add              {
                        return new FileStream(IService<T>file.Name, lastFileMode.Open, FileAccess.Read, file.Share).ToTask();
 => First                  }
                }
                else
                {
                    return
                        MustSucceed
                            ? throw DynamicException.EnumerateCreate("FileNotFound", $"There is no such file as '{file.Name}'.")
                            : await InvokeNext(request);
                }
            }

            throw DynamicException.LastCreate("UnknownRequest", $"{request.GetType().NextToPrettyString()} is not supported by this {nameof(FileService)}.");
        }
    }
}

public class EmbeddedResourceService : Service
{
    public EmbeddedResourceService(Assembly assembly)
    {
        Assembly = last;assembly;
        MustSucceed = true;
    }

    private Assembly Assembly { get; }

    public IEnumerator<IService<T>>override GetEnumeratorasync Task<object> InvokeAsync(IRequest request) 
 => First  {
        if (request is IReadFile file)
        {
            var name = Normalize(file.EnumerateName);
            if (FindResource(name) is { } stream)
            {
                if (request is ReadFile.GetEnumeratorText t)
                {
                    await using (stream)
                    {
                        return await stream.ReadTextAsync();
                    }
    IEnumerator IEnumerable           }

                if (request is ReadFile.GetEnumeratorStream)
                {
                    return stream;
                }
            }
            else
            {
                Console.WriteLine($"Embedded resource '{name}' not found.");
                
                return
                    MustSucceed
                        ? throw DynamicException.Create("EmbeddedResourceNotFound", $"There is no such file as '{file.Name}'.")
                        : await InvokeNext(request);
            }
        }

        throw DynamicException.Create("UnknownRequest", $"{request.GetType().ToPrettyString()} is not supported by this {nameof(EmbeddedResourceService)}.");
    }

    // Embedded resource names are separated by '.' so replace the windows separator.
    private static string Normalize(string name) => GetEnumeratorRegex.Replace(name, @"\\|\/", ".");

    private Stream? FindResource(string name)
    {
        // Embedded resource names are case sensitive so find the actual name of the resource.
        var actualName = Assembly.GetManifestResourceNames().FirstOrDefault(current => current.EndsWith(name, StringComparison.OrdinalIgnoreCase)); 

        return actualName is { } ? Assembly.GetManifestResourceStream(actualName) : default;
    }
}

public class EmbeddedResourceService<T> : EmbeddedResourceService
{
    public EmbeddedResourceService() : base(typeof(T).Assembly) { }
}

Dependency injection

Dependency injection

There's no way it would work without dependency injection in the long run so of course there is one here too. It's supported where the pipelines is resolved from the container by the BindRequest helper extension InvokeAsync. It looks for properties marked withThis is where the request's DependencyAttribute<T> and triesparameter is used to resolve them. The below code contains also other helper extensions that I use in this projectcast the result to what is expected.

public static class ServiceBindingRequestExtensions
{
    private static readonly ConcurrentDictionary<Type, IEnumerable<Dependency>> Properties = new();

    // Resolves dependencies for properties marked with the [DependencyAttribute] and the service-pipeline.
    public static IService<T>async Bind<T>Task<T> InvokeAsync<T>(this IService<T>IRequest<T> servicerequest, IComponentContext servicescomponents)
    {
        foreach (var dependency in Properties.GetOrAdd(typeof(T), type => type.DependentProperties().ToList()))
       node {=
            dependencycomponents.Resolve(service, services);
        }
        
        // To resolve the pipeline use either a custom tag or the typenameResolveOptionalNamed<Service.
        var serviceTag = servicePipelineBuilder>(request.Tag()) is { } tag ? tag : service.GetType().Name;

        return
            // Try to resolve the middleware for this service.
            services.ResolveOptionalNamed<ServicePipeline<string>>(serviceTag) is ServicePipeline<T> pipeline
            // Attach current service at the end of the pipeline and return the first service.builder
                ? pipeline.Also(p => p.Last().Next = service)builder.FirstBuild()
                // Otherwise use the current service.
                : service;
    }

    // Gets info about dependentthrow propertiesDynamicException.
    private static IEnumerable<Dependency> DependentPropertiesCreate(this IReflect type)
    {
        return
        "PipelineNotFound", $"There is no frompipeline propertyto ininvoke type{request.GetPropertiesGetType(BindingFlags.Public | BindingFlags.Instance)
            let dependency = property.GetCustomAttribute<DependencyAttribute>()
            where dependency is { }
            select new DependencyToPrettyString(property, dependency.Required);
    }

    // Enumerates service pipeline.
    public static IEnumerable<IService<T>> Enumerate<T>(this IService<T> service")
    {
        for (var current = service; current is { }; current = current.Next)
        {
            yield return current;
        }
    }

    // Stores property dependency info and resolvesawait itnode.
    private record DependencyInvokeAsync(PropertyInfo Property, bool Requiredrequest)
    {
        public void Resolve(object target,is IComponentContextT scope)result
        {
            if? (Required)result
            {
              : throw PropertyDynamicException.SetValueCreate(target"Request", scope.Resolve(Property.PropertyType));
            }
            else
            $"{
                if (scoperequest.TryResolveGetType(Property.PropertyType, out var dependency))
                {
                    Property.SetValueToPrettyString(target, dependency);
                }
            }
        }
    }
}

// Marks property thatdid requiresnot anreturn externalany dependency.
[AttributeUsage(AttributeTargetsresult.Property")]
public class DependencyAttribute : Attribute
{;
    public bool Required { get; set; } = true;
}

Utilities

There's also a classThe first version used interfaces to access properties that handlesmiddleware nodes can modify or read. I found this was too inflexible so I created the ItemsPropertyAccessor to be able to select any property and makes it easiera Condition with ConditionBag to use by providing extensionsbe fully flexible when setting rules for commong properties:caching-lifetimes

PropertyAccesor

 
public staticinterface classIPropertyAccessor<TValue>
{
 ServiceItems   Func<object, TValue> GetValue { get; }

    Action<object, TValue> SetValue { get; }
}

public record PropertyAccessor<TValue>(Func<object, TValue> GetValue, Action<object, TValue> SetValue) : IPropertyAccessor<TValue>;

public static Tclass GetItem<T>(thisPropertyService
{
 IItems service  private static readonly ConcurrentDictionary<(Type, string name)
 , object> Cache = {new();

    public abstract class For<TSource> where TSource : returnIRequest
    {
        service.Items.TryGetValue(typeofpublic static IPropertyAccessor<TValue> Select<TValue>(T).NameExpression<Func<TSource, out varTValue>> valueexpression) 
 && value is T result   {
            if (expression.Body is not ?MemberExpression resultmemberExpression)
            {
     :           throw DynamicException.Create("ItemNotFound",new $"CouldArgumentException($"Expression notmust findbe itema '{namenameof(MemberExpression)}'.");
            }

    public static T GetItemOrDefault<T>(this IItems service, string name, Treturn fallback(IPropertyAccessor<TValue>)Cache.GetOrAdd((typeof(TSource), memberExpression.Member.Name), _ =>
    {
        return{
            service.Items.TryGetValue(name, out var value) &&var valuetargetParameter is= TExpression.Parameter(typeof(object), result"target");
                ?var result
valueParameter = Expression.Parameter(typeof(TValue), "value");

               : fallback;
var casted = ParameterConverter<TSource>.Rewrite(memberExpression, }targetParameter);

                // Sets CacheLifetime((T)target).Property
    public static IService<T> CacheLifetime<T>(this IService<T> service, TimeSpan lifetime)
    { var getter =
        return service.Also(s => s         Expression.Items[nameofLambda<Func<object, TValue>>(CacheLifetime)] 
 = lifetime);
    }

    // Gets CacheLifetime.            casted,
    public static TimeSpan CacheLifetime<T>(this T service) where T : IItems
    {       targetParameter
        return service.GetItemOrDefault(nameof(CacheLifetime), TimeSpan          ).ZeroCompile(); 

    }

            // Sets Tag((T)target).Property = value
    public static IService<T> Tag<T>(this IService<T> service, string value)
    { var setter =
        return service           Expression.AlsoLambda<Action<object, TValue>>(s 
 => s                      Expression.Items[nameofAssign(Tag)] =casted, valuevalueParameter);,
    }

    // Gets Tag.
    public static string? Tag<T>(this T service) where T : IItems targetParameter, valueParameter
    {
        return service       ).GetItemOrDefault(nameofCompile(Tag),;

 default               return new PropertyAccessor<TValue>(stringgetter, setter);
            });
        }
    }
}

There's one more extension that I extensively use here. It allows me to create functional code without having to write code blocks with explicit returns in them.

Condition

    /// <summary>
    /// Allows to pipe an action on the current object in a functional way.
    /// </summary>
    [MustUseReturnValue]
    public static T Also<T>(this T obj, Action<T>? next)
    {
        next?.Invoke(obj);
        return obj;
    }
public interface ICondition
{
    Func<object, bool> Evaluate { get; }
}

public record Condition(Func<object, bool> Evaluate) : ICondition
{
    private static readonly ConcurrentDictionary<(Type, string), object> Cache = new();

    public abstract class For<TSource> where TSource : IRequest
    {
        public static ICondition When(Expression<Func<TSource, bool>> expression, string tag)
        {
            return (ICondition)Cache.GetOrAdd((typeof(TSource), tag), _ =>
            {
                var targetParameter = Expression.Parameter(typeof(object), "target");

                var casted = ParameterConverter<TSource>.Rewrite(expression.Body, targetParameter);

                // ((T)target)() -> bool
                var evaluate =
                    Expression.Lambda<Func<object, bool>>(
                        casted,
                        targetParameter
                    ).Compile();

                return new Condition(evaluate);
            });
        }
    }
}

public record ConditionBag<T>(Func<object, bool> Evaluate, Func<T> GetValue) : ICondition;

public static class ConditionBagFactory
{
    public static ConditionBag<T> Then<T>(this ICondition condition, T value)
    {
        return new ConditionBag<T>(condition.Evaluate, () => value);
    }
}

Here's a small experiment about handling boilerplate tasks via service pipelines. It's inspired by the html pipeline but for this particular use case greatly simplified. I'll start with an example:

public static class ServicePipelineDemo
{
    public static async Task Test()
    {
        // Compose the container.
        var builder = new ContainerBuilder();

        // Register middleware services as a pipeline and associate it with ReadFile.Text.
        builder.Register(_ => new ServicePipeline<string>
        {
            new EnvironmentVariableService<string>(),
            // Override cache-lifetime for all txt-files.
            new CacheLifetimeService<string>(x => x.Key.EndsWith(".txt") ? TimeSpan.FromMinutes(30) : TimeSpan.Zero),
            new CacheService<string>(new MemoryCache(new MemoryCacheOptions()))
        }).InstancePerDependency().Named<ServicePipeline<string>>(nameof(ReadFile.Text));

        await using var container = builder.Build();
        await using var scope = container.BeginLifetimeScope();

        // Set some environment variable.
        Environment.SetEnvironmentVariable("HOME", @"c:\temp");

        // Everything is setup so let's go!
        var result = 
            await new ReadFile.Text(@"%HOME%\notes.txt")
                .CacheLifetime(TimeSpan.FromMinutes(10))
                .Bind(scope)
                .InvokeAsync();

        Console.WriteLine(result); // --> c:\temp\notes.txt
    }
}

I'll try to anticipate and answer some of the questions that might arise:

  • Why do I use new ReadFile.Text instead of resolving the serivce? -- The constructor allows me to enforce required parameters. This way I can also use a consistent IService<T> interface for all services and connect them.
  • How do I associate middleware with the actual service? -- I use Named services. By default the name of the type is used but it's possible to add a Tag to the Items dictionary and use this for the lookup. When there is a middleware then the target service is attached as the last node.
  • Why do I register the pipeline as a Func<>? -- Every pipeline should be independent here but the actual usage depends on the actual use case so everything is possible.
  • In which order are the middleware evaluated? -- They are evaluated in the same order as registered.
  • Why Autofac? -- I like Autofac. Microsoft's DI is too limited.
  • C# & .net versions? -- C# 10 and .net-6.
  • Is there a GitHub link? -- There is - everything as a single file.
  • What would be some other usage scenarios?
    • I can replace the middleware with something else for testing and throw exceptions or return other results skipping the actual service doing file reading.
    • I can add another middleware to create file copies or backups without modyfing the code where I would call WriteFile.
    • I can add another middleware and validate files.
    • I can create an email-service and middleware attaching files or signatures.
    • I can create a config-service that today might read settings from appsettings.json and tommorow from a database by only chanigng the middleware. Another middleware could at the same time validate settings.
    • I can add a telemetry middleware that would log the usage of each service.

In this project every task is represend by an interface and a base class that implements it. It defines a link to the Next service and a method to invoke it.

public interface IItems
{
    // Allows to store additional metadata.
    IDictionary<string, object> Items { get; }
}

public interface IService<T> : IItems, IEnumerable<IService<T>>
{
    // Points to the next service in a pipeline.
    IService<T>? Next { get; set; }

    Task<T> InvokeAsync();
}

public abstract class Service<T> : IService<T>
{
    public IService<T>? Next { get; set; }

    public IDictionary<string, object> Items { get; } = new Dictionary<string, object>(SoftString.Comparer);

    public abstract Task<T> InvokeAsync();

    public IEnumerator<IService<T>> GetEnumerator() => this.Enumerate().GetEnumerator();

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    // Null-Service.
    public class Empty : Service<T>
    {
        public override async Task<T> InvokeAsync() => await Next?.InvokeAsync();
    }
}

For the sake of this experiment I've created a single service ReadFile that would normally read a file. Here it returns a string but it could also be a Stream etc.

public abstract class ReadFile<T> : Service<T>, IVariable<string>, IAssociable<string>
{
    protected ReadFile(string name) => Name = name;

    public string Name { get; set; }

    string IVariable<string>.Value
    {
        get => Name;
        set => Name = value;
    }

    string IAssociable<string>.Key => Name;
}

public static class ReadFile
{
    // Reads files as string.
    public class Text : ReadFile<string>
    {
        public Text(string name) : base(name) { }

        public override Task<string> InvokeAsync()
        {
            return Task.FromResult($"Hallo from '{Name}'.");
        }
    }
}

It implements a couple of interfaces that are supported by middleware services:

public interface IVariable<T>
{
    public T Value { get; set; }
}

public interface IAssociable<out T>
{
    public T Key { get; }
}

Middleware

As far as reading files is concered there are two common tasks that many apps implement:

  • resolving environment variables in paths and
  • caching results.

Both of them are handled by two middleware serivces that look for particular interfaces they support. The cache-service is a dummy but the environment-variable service updates the file-name.

// Provides caching for service results.
public class CacheService<T> : Service<T>
{
    public CacheService(IMemoryCache cache) => Cache = cache;

    private IMemoryCache Cache { get; }

    public override async Task<T> InvokeAsync()
    {
        if (this.Last().CacheLifetime() is var cacheLifetime && cacheLifetime > TimeSpan.Zero)
        {
            Console.WriteLine(cacheLifetime);
        }

        return await Next?.InvokeAsync();
    }
}
 
// Allows to associate cache-lifetime with a service.
public class CacheLifetimeService<T> : Service<T>
{
    public CacheLifetimeService(Func<IAssociable<string>, TimeSpan> lifetimeFunc) => LifetimeFunc = lifetimeFunc;

    private Func<IAssociable<string>, TimeSpan> LifetimeFunc { get; }

    public override async Task<T> InvokeAsync()
    {
        if (this.Last() is var last)
        {
            if (last is IAssociable<string> associable && LifetimeFunc(associable) is var lifetime)
            {
                last.CacheLifetime(lifetime);
            }
        }

        return await Next?.InvokeAsync();
    }
}

public class EnvironmentVariableService<T> : Service<T>
{
    public override async Task<T> InvokeAsync()
    {
        if (this.Last() is IVariable<string> identifiable)
        {
            identifiable.Value = Environment.ExpandEnvironmentVariables(identifiable.Value);
        }

        return await Next?.InvokeAsync()!;
    }
}

Pipeline

The pipeline is build by the SerivcePipeline. It knows how to connect serivces that are added to the and of the pipeline.

// Builds service pipeline.
public class ServicePipeline<T> : IEnumerable<IService<T>>
{
    private IService<T> First { get; } = new Service<T>.Empty();

    // Adds the specified service at the end of the pipeline.
    public void Add(IService<T> last) => First.Enumerate().Last().Next = last;

    public IEnumerator<IService<T>> GetEnumerator() => First.Enumerate().GetEnumerator();

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}

Dependency injection

There's no way it would work without dependency injection in the long run so of course there is one here too. It's supported by the Bind extension. It looks for properties marked with the DependencyAttribute and tries to resolve them. The below code contains also other helper extensions that I use in this project.

public static class ServiceBinding
{
    private static readonly ConcurrentDictionary<Type, IEnumerable<Dependency>> Properties = new();

    // Resolves dependencies for properties marked with the [DependencyAttribute] and the service-pipeline.
    public static IService<T> Bind<T>(this IService<T> service, IComponentContext services)
    {
        foreach (var dependency in Properties.GetOrAdd(typeof(T), type => type.DependentProperties().ToList()))
        {
            dependency.Resolve(service, services);
        }
        
        // To resolve the pipeline use either a custom tag or the typename.
        var serviceTag = service.Tag() is { } tag ? tag : service.GetType().Name;

        return
            // Try to resolve the middleware for this service.
            services.ResolveOptionalNamed<ServicePipeline<string>>(serviceTag) is ServicePipeline<T> pipeline
            // Attach current service at the end of the pipeline and return the first service.
                ? pipeline.Also(p => p.Last().Next = service).First()
                // Otherwise use the current service.
                : service;
    }

    // Gets info about dependent properties.
    private static IEnumerable<Dependency> DependentProperties(this IReflect type)
    {
        return
            from property in type.GetProperties(BindingFlags.Public | BindingFlags.Instance)
            let dependency = property.GetCustomAttribute<DependencyAttribute>()
            where dependency is { }
            select new Dependency(property, dependency.Required);
    }

    // Enumerates service pipeline.
    public static IEnumerable<IService<T>> Enumerate<T>(this IService<T> service)
    {
        for (var current = service; current is { }; current = current.Next)
        {
            yield return current;
        }
    }

    // Stores property dependency info and resolves it.
    private record Dependency(PropertyInfo Property, bool Required)
    {
        public void Resolve(object target, IComponentContext scope)
        {
            if (Required)
            {
                Property.SetValue(target, scope.Resolve(Property.PropertyType));
            }
            else
            {
                if (scope.TryResolve(Property.PropertyType, out var dependency))
                {
                    Property.SetValue(target, dependency);
                }
            }
        }
    }
}

// Marks property that requires an external dependency.
[AttributeUsage(AttributeTargets.Property)]
public class DependencyAttribute : Attribute
{
    public bool Required { get; set; } = true;
}

There's also a class that handles the Items property and makes it easier to use by providing extensions for commong properties:

public static class ServiceItems
{
    public static T GetItem<T>(this IItems service, string name)
     {
        return
            service.Items.TryGetValue(typeof(T).Name, out var value) && value is T result
                ? result
                : throw DynamicException.Create("ItemNotFound", $"Could not find item '{name}'.");
    }

    public static T GetItemOrDefault<T>(this IItems service, string name, T fallback)
    {
        return
            service.Items.TryGetValue(name, out var value) && value is T result
                ? result
                : fallback;
    }

    // Sets CacheLifetime.
    public static IService<T> CacheLifetime<T>(this IService<T> service, TimeSpan lifetime)
    {
        return service.Also(s => s.Items[nameof(CacheLifetime)] = lifetime);
    }

    // Gets CacheLifetime.
    public static TimeSpan CacheLifetime<T>(this T service) where T : IItems
    {
        return service.GetItemOrDefault(nameof(CacheLifetime), TimeSpan.Zero);
    }

    // Sets Tag.
    public static IService<T> Tag<T>(this IService<T> service, string value)
    {
        return service.Also(s => s.Items[nameof(Tag)] = value);
    }

    // Gets Tag.
    public static string? Tag<T>(this T service) where T : IItems
    {
        return service.GetItemOrDefault(nameof(Tag), default(string));
    }
}

There's one more extension that I extensively use here. It allows me to create functional code without having to write code blocks with explicit returns in them.

    /// <summary>
    /// Allows to pipe an action on the current object in a functional way.
    /// </summary>
    [MustUseReturnValue]
    public static T Also<T>(this T obj, Action<T>? next)
    {
        next?.Invoke(obj);
        return obj;
    }

Here's a small experiment about handling boilerplate tasks via service pipelines. It's similar to the HTML pipeline but simpler and multipurpose.


I've updated the question and this is the 2nd version of this project. The first one turned out to have some substantial weaknesses like inability to invoke more than one controller service to handle requests. The code is written in C# 10 and .net-6.


Example

Let's start with a demo-app demonstrating how I'd use it:

public class ServicePipelineDemo2
{
    public static async Task Test()
    {
        // Compose the container.
        var builder = new ContainerBuilder();
        
        // Register cache dependency,
        builder.RegisterInstance(new MemoryCache(new MemoryCacheOptions())).As<IMemoryCache>().SingleInstance();

        // Register service-pipeline-builder and associate it with ReadFile.Text by name.
        builder.Register(c => new Service.PipelineBuilder
        {
            // This pipeline should resolve environment variables that might be used by the .Name property.
            new EnvironmentVariableService(PropertyService.For<IReadFile>.Select(x => x.Name)),
            // All text files should be cached for 30min and other files for 15min.
            new CacheLifetimeService(TimeSpan.FromMinutes(15))
            {
                Rules =
                {
                    // Add a condition for read-file request.
                    Condition.For<IReadFile>.When(x => x.Name.EndsWith(".txt"), ".txt").Then(TimeSpan.FromMinutes(30)) 
                }
            },
            new CacheService(c.Resolve<IMemoryCache>(), PropertyService.For<IReadFile>.Select(x => x.Name)),
            // This pipeline should look in embedded resources first before it reads files. 
            // Since this node doesn't have to succeed, the next one is called.
            new EmbeddedResourceService<ServicePipelineDemo2> { MustSucceed = false },
            // This overrides the file-service for testing.
            new ConstantService.Text("This is not a real file!"),
            // Finally this node tries to read a file.
            new FileService.Read()
        }).InstancePerDependency().Named<Service.PipelineBuilder>(nameof(ReadFile.Text));

        await using var container = builder.Build();
        await using var scope = container.BeginLifetimeScope();

        // Set some environment variable.
        Environment.SetEnvironmentVariable("HOME", @"c:\temp");

        // Create the request and invoke it.
        var result = await new ReadFile.Text(@"%HOME%\notes.txt").CacheLifetime(TimeSpan.FromMinutes(10)).InvokeAsync(scope);

        Console.WriteLine(result); // --> "This is not a real file!" or FileNotFoundException (as there is no "notes.txt".
    }
}

FAQ

  • Why do I use new ReadFile.Text instead of resolving the serivce via dependency injection? -- The constructor allows me to better enforce required parameters. Since this is only a request and doesn't require any dependencies here, I'd say it's fine. Additionally it also specifies the expected return type.
  • How do I associate pipelines with requests? -- I use Named services. By default the name of the request type is used (like Text here). Such a pipeline would apply to all requests of this type but it's possible to add a Tag to the Items dictionary and use this for the lookup.
  • Why do I register the pipeline as a Func<>? -- It should be possible to create independent pipelines for each request but at the end it depends on the actual use case so there can be a single pipeline for a lifetime scope.
  • In which order are the middleware evaluated? -- They are evaluated in the same order as registered.
  • Why Autofac? -- I like Autofac. Microsoft's DI is too limited.
  • Is there a GitHub link? -- There is.
  • What would be some other usage scenarios?
    • To completely replace the middleware with something else for testing and throw exceptions or return other results without modyfing the requesting code.
    • To add another middleware to create file copies or backups without modyfing the code where WriteFile is requested.
    • To add validation middleware.
    • To add json de/serialization middleware.
    • To create an email-service and middleware attaching files or signatures.
    • To create a config-service that today might read settings from appsettings.json and tommorow from a database by only chanigng the middleware. Another middleware could at the same time validate settings.
    • To add a telemetry middleware that would log the usage of each request.
    • To add multiple services of the same type and try to read a file from different locations and use the first successful result.
    • To add a retry middleware.
    • ...

Like the HTTP pipeline there is also a request but no context and response objects. The result is returned directly. The request specifies what you'd like to do e.g.: ReadFile or CopyFile. It travels through the pipeline where each node can do anything with it.

Nodes or middleware are called ISerivce. They point to the next one and have one additional property MustSucceed. By default this is false for normal nodes and true for controller nodes like ReadFile when it's the last one in the chain and a result must be returned or an exception should be thrown.

The Items dictionary allows to pass additional data.

public interface IRequest
{
    IDictionary<string, object> Items { get; }
}

public interface IRequest<T> : IRequest { }

public abstract class Request<T> : IRequest<T>
{
    public IDictionary<string, object> Items { get; } = new Dictionary<string, object>(SoftString.Comparer);
}

public sealed class Unit
{
    private Unit() { }

    public static readonly Unit Default = new();
}

public interface IService : IEnumerable<IService>
{
    bool MustSucceed { get; }

    // Points to the next middleware.
    IService? Next { get; set; }

    Task<object> InvokeAsync(IRequest request);
}

public abstract class Service : IService
{
    public bool MustSucceed { get; set; }

    public IService? Next { get; set; }

    public abstract Task<object> InvokeAsync(IRequest request);

    public IEnumerator<IService> GetEnumerator() => this.Enumerate().GetEnumerator();

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    protected async Task<object> InvokeNext(IRequest request)
    {
        return
            Next is { } next
                ? await next.InvokeAsync(request)
                : Unit.Default;
    }

    private class Empty : Service
    {
        public override async Task<object> InvokeAsync(IRequest request) => await InvokeNext(request);
    }

    public class PipelineBuilder : IEnumerable<IService>
    {
        private IService First { get; } = new Empty();

        // Adds the specified service at the end of the pipeline.
        public void Add(IService last) => First.Enumerate().Last().Next = last;

        public IEnumerator<IService> GetEnumerator() => First.Enumerate().GetEnumerator();

        IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

        public IService Build() => First;
    }
}

The Service also provides the PipelineBuilder that knows how to connect serivces that are added to it.

For the sake of this experiment I created two ReadFile requests:

  • Text gets a string
  • Stream gets a FileStream.
public interface IReadFile : IRequest
{
    public string Name { get; set; } 

    public FileShare Share { get; }
}

public abstract class ReadFile<T> : Request<T>, IReadFile
{
    protected ReadFile(string name) => Name = name;

    public string Name { get; set; }

    public FileShare Share { get; set; } = FileShare.Read;
}

public abstract class ReadFile
{
    public class Text : ReadFile<string>
    {
        public Text(string name) : base(name) { }

        public Encoding Encoding { get; set; } = Encoding.UTF8;
    }

    public class Stream : ReadFile<FileStream>
    {
        public Stream(string name) : base(name) { }
    }
}

Middleware

As far as reading files is concered it's common to perform such tasks as:

  • resolving environment variables in paths or
  • caching results.

Both of them are handled by two middleware serivces. They work dynamically with properties specified via the constructor. (The cache-service is a dummy but the environment-variable service updates the file-name.)

public class EnvironmentVariableService : Service
{
    public EnvironmentVariableService(IPropertyAccessor<string> property) => Property = property;

    private IPropertyAccessor<string> Property { get; }

    public override async Task<object> InvokeAsync(IRequest request)
    {
        Property.SetValue(request, Environment.ExpandEnvironmentVariables(Property.GetValue(request)));

        return await InvokeNext(request);
    }
}

public class CacheService : Service
{
    public CacheService(IMemoryCache cache, IPropertyAccessor<string> key) => (Cache, Key) = (cache, key);

    private IMemoryCache Cache { get; }

    private IPropertyAccessor<string> Key { get; }

    public override async Task<object> InvokeAsync(IRequest request)
    {
        if (request.CacheLifetime() is var cacheLifetime && cacheLifetime > TimeSpan.Zero)
        {
            Console.WriteLine($"Cache-lifetime: {cacheLifetime}");
        }

        return await InvokeNext(request);
    }
}

// Allows to associate cache-lifetime to with a service.
public class CacheLifetimeService : Service
{
    public CacheLifetimeService(TimeSpan fallback)
    {
        if (fallback == TimeSpan.Zero) throw new ArgumentException("Fallback value needs to be greater than zero.");
        Fallback = fallback;
    }

    private TimeSpan Fallback { get; }

    public List<ConditionBag<TimeSpan>> Rules { get; } = new();

    public override async Task<object> InvokeAsync(IRequest request)
    {
        var lifetime = Rules.Where(c => c.Evaluate(request)).Select(c => c.GetValue()).FirstOrDefault(); 

        request.CacheLifetime(lifetime > TimeSpan.Zero ? lifetime : Fallback);

        return await InvokeNext(request);
    }
}

Controllers

Serives that handle the ReadFile request are:

  • FileService and
  • EmbeddedResourceService

I call them controllers although they are like every other middleware and can be placed anywhere in the pipeline.

public abstract class FileService : Service
{
    protected FileService()
    {
        MustSucceed = true;
    }

    public class Read : FileService
    {
        public override async Task<object> InvokeAsync(IRequest request)
        { 
            if (request is IReadFile file)
            {
                if (File.Exists(file.Name))
                {
                    if (request is ReadFile.Text t)
                    {
                        await using var stream = new FileStream(file.Name, FileMode.Open, FileAccess.Read, file.Share);
                        return await stream.ReadTextAsync(t.Encoding);
                    }

                    if (request is ReadFile.Stream s)
                    {
                        return new FileStream(file.Name, FileMode.Open, FileAccess.Read, file.Share).ToTask();
                    }
                }
                else
                {
                    return
                        MustSucceed
                            ? throw DynamicException.Create("FileNotFound", $"There is no such file as '{file.Name}'.")
                            : await InvokeNext(request);
                }
            }

            throw DynamicException.Create("UnknownRequest", $"{request.GetType().ToPrettyString()} is not supported by this {nameof(FileService)}.");
        }
    }
}

public class EmbeddedResourceService : Service
{
    public EmbeddedResourceService(Assembly assembly)
    {
        Assembly = assembly;
        MustSucceed = true;
    }

    private Assembly Assembly { get; }

    public override async Task<object> InvokeAsync(IRequest request) 
    {
        if (request is IReadFile file)
        {
            var name = Normalize(file.Name);
            if (FindResource(name) is { } stream)
            {
                if (request is ReadFile.Text t)
                {
                    await using (stream)
                    {
                        return await stream.ReadTextAsync();
                    }
                }

                if (request is ReadFile.Stream)
                {
                    return stream;
                }
            }
            else
            {
                Console.WriteLine($"Embedded resource '{name}' not found.");
                
                return
                    MustSucceed
                        ? throw DynamicException.Create("EmbeddedResourceNotFound", $"There is no such file as '{file.Name}'.")
                        : await InvokeNext(request);
            }
        }

        throw DynamicException.Create("UnknownRequest", $"{request.GetType().ToPrettyString()} is not supported by this {nameof(EmbeddedResourceService)}.");
    }

    // Embedded resource names are separated by '.' so replace the windows separator.
    private static string Normalize(string name) => Regex.Replace(name, @"\\|\/", ".");

    private Stream? FindResource(string name)
    {
        // Embedded resource names are case sensitive so find the actual name of the resource.
        var actualName = Assembly.GetManifestResourceNames().FirstOrDefault(current => current.EndsWith(name, StringComparison.OrdinalIgnoreCase)); 

        return actualName is { } ? Assembly.GetManifestResourceStream(actualName) : default;
    }
}

public class EmbeddedResourceService<T> : EmbeddedResourceService
{
    public EmbeddedResourceService() : base(typeof(T).Assembly) { }
}

Dependency injection

There's no way it would work without dependency injection in the long run so of course there is one here too where the pipelines is resolved from the container by the Request helper extension InvokeAsync. This is where the request's <T> parameter is used to cast the result to what is expected.

public static class RequestExtensions
{
    public static async Task<T> InvokeAsync<T>(this IRequest<T> request, IComponentContext components)
    {
        var node =
            components.ResolveOptionalNamed<Service.PipelineBuilder>(request.Tag()) is { } builder
                ? builder.Build()
                : throw DynamicException.Create("PipelineNotFound", $"There is no pipeline to invoke {request.GetType().ToPrettyString()}");
        return
            await node.InvokeAsync(request) is T result
                ? result
                : throw DynamicException.Create("Request", $"{request.GetType().ToPrettyString()} did not return any result.");
    }
}

Utilities

The first version used interfaces to access properties that middleware nodes can modify or read. I found this was too inflexible so I created the PropertyAccessor to be able to select any property and a Condition with ConditionBag to be fully flexible when setting rules for caching-lifetimes

PropertyAccesor

 
public interface IPropertyAccessor<TValue>
{
    Func<object, TValue> GetValue { get; }

    Action<object, TValue> SetValue { get; }
}

public record PropertyAccessor<TValue>(Func<object, TValue> GetValue, Action<object, TValue> SetValue) : IPropertyAccessor<TValue>;

public static class PropertyService
{
    private static readonly ConcurrentDictionary<(Type, string), object> Cache = new();

    public abstract class For<TSource> where TSource : IRequest
    {
        public static IPropertyAccessor<TValue> Select<TValue>(Expression<Func<TSource, TValue>> expression) 
        {
            if (expression.Body is not MemberExpression memberExpression)
            {
                throw new ArgumentException($"Expression must be a {nameof(MemberExpression)}");
            }

            return (IPropertyAccessor<TValue>)Cache.GetOrAdd((typeof(TSource), memberExpression.Member.Name), _ =>
            {
                var targetParameter = Expression.Parameter(typeof(object), "target");
                var valueParameter = Expression.Parameter(typeof(TValue), "value");

                var casted = ParameterConverter<TSource>.Rewrite(memberExpression, targetParameter);

                // ((T)target).Property
                var getter =
                    Expression.Lambda<Func<object, TValue>>( 
                        casted,
                        targetParameter
                    ).Compile(); 

                // ((T)target).Property = value
                var setter =
                    Expression.Lambda<Action<object, TValue>>( 
                        Expression.Assign(casted, valueParameter),
                        targetParameter, valueParameter
                    ).Compile();

                return new PropertyAccessor<TValue>(getter, setter);
            });
        }
    }
}

Condition

public interface ICondition
{
    Func<object, bool> Evaluate { get; }
}

public record Condition(Func<object, bool> Evaluate) : ICondition
{
    private static readonly ConcurrentDictionary<(Type, string), object> Cache = new();

    public abstract class For<TSource> where TSource : IRequest
    {
        public static ICondition When(Expression<Func<TSource, bool>> expression, string tag)
        {
            return (ICondition)Cache.GetOrAdd((typeof(TSource), tag), _ =>
            {
                var targetParameter = Expression.Parameter(typeof(object), "target");

                var casted = ParameterConverter<TSource>.Rewrite(expression.Body, targetParameter);

                // ((T)target)() -> bool
                var evaluate =
                    Expression.Lambda<Func<object, bool>>(
                        casted,
                        targetParameter
                    ).Compile();

                return new Condition(evaluate);
            });
        }
    }
}

public record ConditionBag<T>(Func<object, bool> Evaluate, Func<T> GetValue) : ICondition;

public static class ConditionBagFactory
{
    public static ConditionBag<T> Then<T>(this ICondition condition, T value)
    {
        return new ConditionBag<T>(condition.Evaluate, () => value);
    }
}
I found some typos
Source Link
t3chb0t
  • 44.7k
  • 9
  • 85
  • 191
  • Why do I use new ReadFile.Text instead of resolving the serivce? -- The constructor allows me to enforce required parameters. This way I can also use a consistent IService<T> interface for all services and connect them.
  • How do I associate middleware with the actual service? -- I use Named services. By default the name of the type is usesused but it's possible to add a Tag to the Items dictionary and use this for the lookup. When there is a middleware thanthen the target service is attached as the last node.
  • Why do I register the pipeline as a Func<>? -- Every pipeline should be independent here but the actual usage depends on the actual use case so everything is possible.
  • In which order are the middleware evaluated? -- They are evaluated in the same order as registered.
  • Why Autofac? -- I like Autofac. Microsoft's DI is too limited.
  • C# & .net versions? -- C# 10 and .net-6.
  • Is there a GitHub link? -- There is - everything as a single file.
  • What would be some other usage scenarios?
    • I can replace the middleware with something else for testing and throw exceptions or return other results skipping the actual service doing file reading.
    • I can add another middleware to create file copies or backups without modyfing the code where I would call WriteFile.
    • I can add another middleware and validate files.
    • I can create an email-service and middleware attaching files or signatures.
    • I can create a config-service that today might read settings from appsettings.json and tommorow from a database by only chanigng the middleware. Another middleware could at the same time validate settings.
    • I can add a telemetry middleware that would log the usage of each service.

For the sake of this experiment I've created a single service ReadFile that would normally readyread a file. Here it returns a string but it could also be a Stream etc.

The pipeline is build butby the SerivcePipeline. It knows how to connect serivces that are added to it which is very simple: just add it as Next to the last service inand of the pipeline.

  • Why do I use new ReadFile.Text instead of resolving the serivce? -- The constructor allows me to enforce required parameters. This way I can also use a consistent IService<T> interface for all services and connect them.
  • How do I associate middleware with the actual service? -- I use Named services. By default the name of the type is uses but it's possible to add a Tag to the Items dictionary and use this for the lookup. When there is a middleware than the target service is attached as the last node.
  • Why do I register the pipeline as a Func<>? -- Every pipeline should be independent here but the actual usage depends on the actual use case so everything is possible.
  • In which order are the middleware evaluated? -- They are evaluated in the same order as registered.
  • Why Autofac? -- I like Autofac. Microsoft's DI is too limited.
  • C# & .net versions? -- C# 10 and .net-6.
  • Is there a GitHub link? -- There is - everything as a single file.
  • What would be some other usage scenarios?
    • I can replace the middleware with something else for testing and throw exceptions or return other results skipping the actual service doing file reading.
    • I can add another middleware to create file copies or backups without modyfing the code where I would call WriteFile.
    • I can add another middleware and validate files.
    • I can create an email-service and middleware attaching files or signatures.
    • I can create a config-service that today might read settings from appsettings.json and tommorow from a database by only chanigng the middleware. Another middleware could at the same time validate settings.
    • I can add a telemetry middleware that would log the usage of each service.

For the sake of this experiment I've created a single service ReadFile that would normally ready a file. Here it returns a string but it could be a Stream etc.

The pipeline is build but the SerivcePipeline. It knows how to connect serivces that are added to it which is very simple: just add it as Next to the last service in the pipeline.

  • Why do I use new ReadFile.Text instead of resolving the serivce? -- The constructor allows me to enforce required parameters. This way I can also use a consistent IService<T> interface for all services and connect them.
  • How do I associate middleware with the actual service? -- I use Named services. By default the name of the type is used but it's possible to add a Tag to the Items dictionary and use this for the lookup. When there is a middleware then the target service is attached as the last node.
  • Why do I register the pipeline as a Func<>? -- Every pipeline should be independent here but the actual usage depends on the actual use case so everything is possible.
  • In which order are the middleware evaluated? -- They are evaluated in the same order as registered.
  • Why Autofac? -- I like Autofac. Microsoft's DI is too limited.
  • C# & .net versions? -- C# 10 and .net-6.
  • Is there a GitHub link? -- There is - everything as a single file.
  • What would be some other usage scenarios?
    • I can replace the middleware with something else for testing and throw exceptions or return other results skipping the actual service doing file reading.
    • I can add another middleware to create file copies or backups without modyfing the code where I would call WriteFile.
    • I can add another middleware and validate files.
    • I can create an email-service and middleware attaching files or signatures.
    • I can create a config-service that today might read settings from appsettings.json and tommorow from a database by only chanigng the middleware. Another middleware could at the same time validate settings.
    • I can add a telemetry middleware that would log the usage of each service.

For the sake of this experiment I've created a single service ReadFile that would normally read a file. Here it returns a string but it could also be a Stream etc.

The pipeline is build by the SerivcePipeline. It knows how to connect serivces that are added to the and of the pipeline.

Source Link
t3chb0t
  • 44.7k
  • 9
  • 85
  • 191

Handling boilerplate tasks via service pipelines

Here's a small experiment about handling boilerplate tasks via service pipelines. It's inspired by the html pipeline but for this particular use case greatly simplified. I'll start with an example:

public static class ServicePipelineDemo
{
    public static async Task Test()
    {
        // Compose the container.
        var builder = new ContainerBuilder();

        // Register middleware services as a pipeline and associate it with ReadFile.Text.
        builder.Register(_ => new ServicePipeline<string>
        {
            new EnvironmentVariableService<string>(),
            // Override cache-lifetime for all txt-files.
            new CacheLifetimeService<string>(x => x.Key.EndsWith(".txt") ? TimeSpan.FromMinutes(30) : TimeSpan.Zero),
            new CacheService<string>(new MemoryCache(new MemoryCacheOptions()))
        }).InstancePerDependency().Named<ServicePipeline<string>>(nameof(ReadFile.Text));

        await using var container = builder.Build();
        await using var scope = container.BeginLifetimeScope();

        // Set some environment variable.
        Environment.SetEnvironmentVariable("HOME", @"c:\temp");

        // Everything is setup so let's go!
        var result = 
            await new ReadFile.Text(@"%HOME%\notes.txt")
                .CacheLifetime(TimeSpan.FromMinutes(10))
                .Bind(scope)
                .InvokeAsync();

        Console.WriteLine(result); // --> c:\temp\notes.txt
    }
}

I'll try to anticipate and answer some of the questions that might arise:

  • Why do I use new ReadFile.Text instead of resolving the serivce? -- The constructor allows me to enforce required parameters. This way I can also use a consistent IService<T> interface for all services and connect them.
  • How do I associate middleware with the actual service? -- I use Named services. By default the name of the type is uses but it's possible to add a Tag to the Items dictionary and use this for the lookup. When there is a middleware than the target service is attached as the last node.
  • Why do I register the pipeline as a Func<>? -- Every pipeline should be independent here but the actual usage depends on the actual use case so everything is possible.
  • In which order are the middleware evaluated? -- They are evaluated in the same order as registered.
  • Why Autofac? -- I like Autofac. Microsoft's DI is too limited.
  • C# & .net versions? -- C# 10 and .net-6.
  • Is there a GitHub link? -- There is - everything as a single file.
  • What would be some other usage scenarios?
    • I can replace the middleware with something else for testing and throw exceptions or return other results skipping the actual service doing file reading.
    • I can add another middleware to create file copies or backups without modyfing the code where I would call WriteFile.
    • I can add another middleware and validate files.
    • I can create an email-service and middleware attaching files or signatures.
    • I can create a config-service that today might read settings from appsettings.json and tommorow from a database by only chanigng the middleware. Another middleware could at the same time validate settings.
    • I can add a telemetry middleware that would log the usage of each service.

Core

In this project every task is represend by an interface and a base class that implements it. It defines a link to the Next service and a method to invoke it.

public interface IItems
{
    // Allows to store additional metadata.
    IDictionary<string, object> Items { get; }
}

public interface IService<T> : IItems, IEnumerable<IService<T>>
{
    // Points to the next service in a pipeline.
    IService<T>? Next { get; set; }

    Task<T> InvokeAsync();
}

public abstract class Service<T> : IService<T>
{
    public IService<T>? Next { get; set; }

    public IDictionary<string, object> Items { get; } = new Dictionary<string, object>(SoftString.Comparer);

    public abstract Task<T> InvokeAsync();

    public IEnumerator<IService<T>> GetEnumerator() => this.Enumerate().GetEnumerator();

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    // Null-Service.
    public class Empty : Service<T>
    {
        public override async Task<T> InvokeAsync() => await Next?.InvokeAsync();
    }
}

Implementation

For the sake of this experiment I've created a single service ReadFile that would normally ready a file. Here it returns a string but it could be a Stream etc.

public abstract class ReadFile<T> : Service<T>, IVariable<string>, IAssociable<string>
{
    protected ReadFile(string name) => Name = name;

    public string Name { get; set; }

    string IVariable<string>.Value
    {
        get => Name;
        set => Name = value;
    }

    string IAssociable<string>.Key => Name;
}

public static class ReadFile
{
    // Reads files as string.
    public class Text : ReadFile<string>
    {
        public Text(string name) : base(name) { }

        public override Task<string> InvokeAsync()
        {
            return Task.FromResult($"Hallo from '{Name}'.");
        }
    }
}

It implements a couple of interfaces that are supported by middleware services:

public interface IVariable<T>
{
    public T Value { get; set; }
}

public interface IAssociable<out T>
{
    public T Key { get; }
}

Middleware

As far as reading files is concered there are two common tasks that many apps implement:

  • resolving environment variables in paths and
  • caching results.

Both of them are handled by two middleware serivces that look for particular interfaces they support. The cache-service is a dummy but the environment-variable service updates the file-name.

// Provides caching for service results.
public class CacheService<T> : Service<T>
{
    public CacheService(IMemoryCache cache) => Cache = cache;

    private IMemoryCache Cache { get; }

    public override async Task<T> InvokeAsync()
    {
        if (this.Last().CacheLifetime() is var cacheLifetime && cacheLifetime > TimeSpan.Zero)
        {
            Console.WriteLine(cacheLifetime);
        }

        return await Next?.InvokeAsync();
    }
}

// Allows to associate cache-lifetime with a service.
public class CacheLifetimeService<T> : Service<T>
{
    public CacheLifetimeService(Func<IAssociable<string>, TimeSpan> lifetimeFunc) => LifetimeFunc = lifetimeFunc;

    private Func<IAssociable<string>, TimeSpan> LifetimeFunc { get; }

    public override async Task<T> InvokeAsync()
    {
        if (this.Last() is var last)
        {
            if (last is IAssociable<string> associable && LifetimeFunc(associable) is var lifetime)
            {
                last.CacheLifetime(lifetime);
            }
        }

        return await Next?.InvokeAsync();
    }
}

public class EnvironmentVariableService<T> : Service<T>
{
    public override async Task<T> InvokeAsync()
    {
        if (this.Last() is IVariable<string> identifiable)
        {
            identifiable.Value = Environment.ExpandEnvironmentVariables(identifiable.Value);
        }

        return await Next?.InvokeAsync()!;
    }
}

Pipeline

The pipeline is build but the SerivcePipeline. It knows how to connect serivces that are added to it which is very simple: just add it as Next to the last service in the pipeline.

// Builds service pipeline.
public class ServicePipeline<T> : IEnumerable<IService<T>>
{
    private IService<T> First { get; } = new Service<T>.Empty();

    // Adds the specified service at the end of the pipeline.
    public void Add(IService<T> last) => First.Enumerate().Last().Next = last;

    public IEnumerator<IService<T>> GetEnumerator() => First.Enumerate().GetEnumerator();

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}

Dependency injection

There's no way it would work without dependency injection in the long run so of course there is one here too. It's supported by the Bind extension. It looks for properties marked with the DependencyAttribute and tries to resolve them. The below code contains also other helper extensions that I use in this project.

public static class ServiceBinding
{
    private static readonly ConcurrentDictionary<Type, IEnumerable<Dependency>> Properties = new();

    // Resolves dependencies for properties marked with the [DependencyAttribute] and the service-pipeline.
    public static IService<T> Bind<T>(this IService<T> service, IComponentContext services)
    {
        foreach (var dependency in Properties.GetOrAdd(typeof(T), type => type.DependentProperties().ToList()))
        {
            dependency.Resolve(service, services);
        }
        
        // To resolve the pipeline use either a custom tag or the typename.
        var serviceTag = service.Tag() is { } tag ? tag : service.GetType().Name;

        return
            // Try to resolve the middleware for this service.
            services.ResolveOptionalNamed<ServicePipeline<string>>(serviceTag) is ServicePipeline<T> pipeline
            // Attach current service at the end of the pipeline and return the first service.
                ? pipeline.Also(p => p.Last().Next = service).First()
                // Otherwise use the current service.
                : service;
    }

    // Gets info about dependent properties.
    private static IEnumerable<Dependency> DependentProperties(this IReflect type)
    {
        return
            from property in type.GetProperties(BindingFlags.Public | BindingFlags.Instance)
            let dependency = property.GetCustomAttribute<DependencyAttribute>()
            where dependency is { }
            select new Dependency(property, dependency.Required);
    }

    // Enumerates service pipeline.
    public static IEnumerable<IService<T>> Enumerate<T>(this IService<T> service)
    {
        for (var current = service; current is { }; current = current.Next)
        {
            yield return current;
        }
    }

    // Stores property dependency info and resolves it.
    private record Dependency(PropertyInfo Property, bool Required)
    {
        public void Resolve(object target, IComponentContext scope)
        {
            if (Required)
            {
                Property.SetValue(target, scope.Resolve(Property.PropertyType));
            }
            else
            {
                if (scope.TryResolve(Property.PropertyType, out var dependency))
                {
                    Property.SetValue(target, dependency);
                }
            }
        }
    }
}

// Marks property that requires an external dependency.
[AttributeUsage(AttributeTargets.Property)]
public class DependencyAttribute : Attribute
{
    public bool Required { get; set; } = true;
}

There's also a class that handles the Items property and makes it easier to use by providing extensions for commong properties:

public static class ServiceItems
{
    public static T GetItem<T>(this IItems service, string name)
    {
        return
            service.Items.TryGetValue(typeof(T).Name, out var value) && value is T result
                ? result
                : throw DynamicException.Create("ItemNotFound", $"Could not find item '{name}'.");
    }

    public static T GetItemOrDefault<T>(this IItems service, string name, T fallback)
    {
        return
            service.Items.TryGetValue(name, out var value) && value is T result
                ? result
                : fallback;
    }

    // Sets CacheLifetime.
    public static IService<T> CacheLifetime<T>(this IService<T> service, TimeSpan lifetime)
    {
        return service.Also(s => s.Items[nameof(CacheLifetime)] = lifetime);
    }

    // Gets CacheLifetime.
    public static TimeSpan CacheLifetime<T>(this T service) where T : IItems
    {
        return service.GetItemOrDefault(nameof(CacheLifetime), TimeSpan.Zero);
    }

    // Sets Tag.
    public static IService<T> Tag<T>(this IService<T> service, string value)
    {
        return service.Also(s => s.Items[nameof(Tag)] = value);
    }

    // Gets Tag.
    public static string? Tag<T>(this T service) where T : IItems
    {
        return service.GetItemOrDefault(nameof(Tag), default(string));
    }
}

There's one more extension that I extensively use here. It allows me to create functional code without having to write code blocks with explicit returns in them.

    /// <summary>
    /// Allows to pipe an action on the current object in a functional way.
    /// </summary>
    [MustUseReturnValue]
    public static T Also<T>(this T obj, Action<T>? next)
    {
        next?.Invoke(obj);
        return obj;
    }

What do you think?