How to correctly write async XUnit test?

Question

I am using async xUnit tests and I am noticing inconsistent passing behavior:

public async Task FetchData()
{
    //Arrange
    var result = await arrangedService.FetchDataAsync().ConfigureAwait(false);
    //Assert
}

I have gone through the call stack executed by this test and have verified that all of my library code is calling .ConfigureAwait(false) after each task. However, in spite of this, this test and others will intermittently fail when performing a Run All, but pass the asserts and manual inspection when I walk through on the debugger. So clearly I am not doing something correctly. I have tried removing the call to ConfigureAwait(false) in the test itself in case there is a special xUnit synchronization context, but it did not change anything. What is the best way to test asynchronous code in a consistent way?

EDIT Okay here is my attempt to create a super-simplified example of the code that is running to provide an example of what is happening:

using Graph = Microsoft.Azure.ActiveDirectory.GraphClient;

public async Task FetchData()
{
    var adUsers = baseUsers //IEnumerable<Graph.User>
        .Cast<Graph.IUser>()
        .ToList();
    var nextPageUsers = Enumerable
        .Range(GoodIdMin, GoodIdMax)
        .Select(number => new Graph.User
        {
            Mail = (-number).ToString()
        })
        .Cast<Graph.IUser>()
        .ToList();

    var mockUserPages = new Mock<IPagedCollection<Graph.IUser>>();
    mockUserPages
        .Setup(pages => pages.MorePagesAvailable)
        .Returns(true);
    mockUserPages
        .Setup(pages => pages.CurrentPage)
        .Returns(new ReadOnlyCollection<Graph.IUser>(adUsers));
    mockUserPages
        .Setup(pages => pages.GetNextPageAsync())
        .ReturnsAsync(mockUserPages.Object)
        .Callback(() =>
        {
            mockUserPages
                .Setup(pages => pages.CurrentPage)
                .Returns(new ReadOnlyCollection<Graph.IUser>(nextPageUsers));
            mockUserPages
                .Setup(pages => pages.MorePagesAvailable)
                .Returns(false);
        });

    var mockUsers = new Mock<Graph.IUserCollection>();
    mockUsers
        .Setup(src => src.ExecuteAsync())
        .ReturnsAsync(mockUserPages.Object);

    var mockGraphClient = new Mock<Graph.IActiveDirectoryClient>();
    mockGraphClient
        .Setup(src => src.Users)
        .Returns(mockUsers.Object);

    var mockDbUsers = CreateBasicMockDbSet(baseUsers.Take(10)
        .Select(user => new User
        {
            Mail = user.Mail
        })
        .AsQueryable());
    var mockContext = new Mock<MyDbContext>();
    mockContext
        .Setup(context => context.Set<User>())
        .Returns(mockDbUsers.Object);

    var mockGraphProvider = new Mock<IGraphProvider>(); 
    mockGraphProvider
        .Setup(src => src.GetClient()) //Creates an IActiveDirectoryClient
        .Returns(mockGraphClient.Object);

    var getter = new UserGetter(mockContext.Object, mockGraphProvider.Object);

    var result = await getter.GetData().ConfigureAwait(false);

    Assert.True(result.Success); //Not the actual assert
}

And here is the code being executed on the var result = ... line:

public UserGetterResult GetData()
{
    var adUsers = await GetAdUsers().ConfigureAwait(false);
    var dbUsers = Context.Set<User>().ToList(); //This is the injected context from before
    return new UserGetterResult //Just a POCO
    {
        AdUsers = adUsers
            .Except(/*Expression that indicates whether
             or not this user is in the database*/)
            .ProjectTo<User>()
            .ToList(),
        DbUsers = dbUsers.ProjectTo<User>().ToList() //Automapper 6.1.1
    };
}

private async Task<List<User>> GetAdUsers()
{
    var userPages = await client //Injected IActiveDirectoryClient from before
        .Users
        .ExecuteAsync()
        .ConfigureAwait(false);
    var users = userPages.CurrentPage.ToList();
    while(userPages.MorePagesAvailable)
    {
        userPages = await userPages.GetNextPageAsync().ConfigureAwait(false);
        users.AddRange(userPages.CurrentPage);
    }
    return users;
}

The purpose of the code is to get a list of users who are in AD but not the database and a list of users who are in the database.

EDIT EDIT Since I forgot to include this in the original update, the errors are all occurring on calls to `IUserCollection.ExecuteAsync().

If your test fails intermittently, then it's very likely a race condition somewhere. If you ever find a way to debug them consistently, you'll be considered as a hero by developers worldwide — Kevin Gosse
– Kevin Gosse, Commented Jan 29, 2018 at 15:52
@Nkosi Okay I have done my best to recreate this in its entirety. — jokul
– jokul, Commented Jan 29, 2018 at 16:46
@KevinGosse I doubt it is a race condition because all mocked services are contained within the scope of the test and even so they are using thread-safe read objects like List<T>. — jokul
– jokul, Commented Jan 29, 2018 at 16:51
Quick question, I assume the code under test works in production? — Nkosi
– Nkosi, Commented Jan 29, 2018 at 20:43
Sorry to reply late, the code has been working fine when manually tested through a typical use-case. — jokul
– jokul, Commented Jan 29, 2018 at 22:25

Nkosi · Accepted Answer · 2018-01-29 21:04:49Z

IUserCollection.ExecuteAsync() appears to be configured correctly based on what was shown in the original post.

Now focusing on the following method...

private async Task<List<User>> GetAdUsers() {
    var userPages = await client //Injected IActiveDirectoryClient from before
        .Users
        .ExecuteAsync()
        .ConfigureAwait(false);
    var users = userPages.CurrentPage.ToList();
    while(userPages.MorePagesAvailable) {
        userPages = await userPages.GetNextPageAsync().ConfigureAwait(false);
        users.AddRange(userPages.CurrentPage);
    }
    return users;
}

I am concerned with how user pages was setup in the mock. Given the flow of the GetAdUsers method it would be better to use SetupSequence to mock the repeated calls CurrentPage and MorePagesAvailable.

var mockUserPages = new Mock<IPagedCollection<Graph.IUser>>();
mockUserPages
    .SetupSequence(_ => _.MorePagesAvailable)
    .Returns(true) // First time called to enter while loop
    .Returns(false); // Second time called to exit while loop
mockUserPages
    .SetupSequence(_ => _.CurrentPage)
    .Returns(new ReadOnlyCollection<Graph.IUser>(adUsers)) // First time called to get List
    .Returns(new ReadOnlyCollection<Graph.IUser>(nextPageUsers)); // Second time called to get next page
mockUserPages
    .Setup(pages => pages.GetNextPageAsync())
    .ReturnsAsync(mockUserPages.Object); // No need for callback

Reference Moq Quickstart

Aman B · Accepted Answer · 2018-01-29 18:19:29Z

0

I suspect the problem could be the delay between executing the callback and next request to mockUserPages.CurrentPage

Try separating the User page collection:

var mockAdUserPages = new Mock<IPagedCollection<Graph.IUser>>();
    mockAdUserPages 
        .Setup(pages => pages.MorePagesAvailable)
        .Returns(true);
    mockAdUserPages 
        .Setup(pages => pages.CurrentPage)
        .Returns(new ReadOnlyCollection<Graph.IUser>(adUsers));

//Setup second page
var mockNextUserPages = new Mock<IPagedCollection<Graph.IUser>>();
mockNextUserPages 
        .Setup(pages => pages.MorePagesAvailable)
        .Returns(false);
    mockNextUserPages 
        .Setup(pages => pages.CurrentPage)
        .Returns(new ReadOnlyCollection<Graph.IUser>(nextPageUsers));

//Return next page
    mockAdUserPages 
        .Setup(pages => pages.GetNextPageAsync())
        .ReturnsAsync(mockNextUserPages.Object);

answered Jan 29, 2018 at 18:19

Aman B

2,4081 gold badge20 silver badges28 bronze badges

2 Comments

Nkosi Over a year ago

No, last setup overrides any previous setup.

jokul Over a year ago

Even when doing this, this test and some others without the callback() intermittently fail.

Collectives™ on Stack Overflow

How to correctly write async XUnit test?

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related