How to get all files from a directory in Azure BLOB using ListBlobsSegmentedAsync

Question

While trying to access all files of the Azure blob folder, getting sample code for container.ListBlobs(); however it looks like an old one.

Old Code : container.ListBlobs();

New Code trying : container.ListBlobsSegmentedAsync(continuationToken);

I am trying to use the below code :

container.ListBlobsSegmentedAsync(continuationToken);

Folders are like :

Container/F1/file.json
Container/F1/F2/file.json
Container/F2/file.json

Looking for the updated version to get all files from an Azure folder. Any sample code would help, thanks!

If the my answer is helpful, you can mark it as answer to help other community members find the helpful information quickly, thanks. — Lee Liu
– Lee Liu, Commented Aug 15, 2018 at 9:42

Daniel Liuzzi · Accepted Answer · 2022-07-14 20:30:21Z

9

Update: Getting all files name from a directory with Azure.Storage.Blobs v12 - Package

var storageConnectionString = "DefaultEndpointsProtocol=...........=core.windows.net";
var blobServiceClient = new BlobServiceClient(storageConnectionString);

//get container
var container = blobServiceClient.GetBlobContainerClient("container_name");

List<string> blobNames = new List<string>();

//Enumerating the blobs may make multiple requests to the service while fetching all the values
//Blobs are ordered lexicographically by name
//if you want metadata set BlobTraits - BlobTraits.Metadata
var blobHierarchyItems = container.GetBlobsByHierarchyAsync(BlobTraits.None, BlobStates.None, "/");

await foreach (var blobHierarchyItem in blobHierarchyItems)
{
    //check if the blob is a virtual directory.
    if (blobHierarchyItem.IsPrefix)
    {
        // You can also access files under nested folders in this way,
        // of course you will need to create a function accordingly (you can do a recursive function)
        // var prefix = blobHierarchyItem.Name;
        // blobHierarchyItem.Name = "folderA\"
        // var blobHierarchyItems= container.GetBlobsByHierarchyAsync(BlobTraits.None, BlobStates.None, "/", prefix);     
    }
    else
    {
        blobNames.Add(blobHierarchyItem.Blob.Name);
    }
}

There are more option and example you can find it here.

This is the link to the nuget package.

edited Jul 14, 2022 at 20:30

Daniel Liuzzi

17.3k8 gold badges55 silver badges57 bronze badges

answered Jun 4, 2021 at 7:38

helix-nebula

3064 silver badges5 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Zoltan Grose Apr 23 at 15:42

Small typo in the commented out section above. var prefix = blobHierarchyItem.Name; should be var prefix = blobHierarchyItem.Prefix; The linked example correctly uses .Prefix as well.

ASHISH R · Accepted Answer · 2020-02-06 14:35:05Z

8

C# code:

   //connection string
    string storageAccount_connectionString = "**NOTE: CONNECTION STRING**";

    // Retrieve storage account from connection string.
    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(storageAccount_connectionString);

    // Create the blob client.
    CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

    // Retrieve reference to a previously created container.
    CloudBlobContainer container = blobClient.GetContainerReference("**NOTE:NAME OF CONTAINER**");
    //The specified container does not exist

    try
    {
        //root directory
        CloudBlobDirectory dira = container.GetDirectoryReference(string.Empty);
        //true for all sub directories else false 
        var rootDirFolders = dira.ListBlobsSegmentedAsync(true, BlobListingDetails.Metadata, null, null, null, null).Result;

        foreach (var blob in rootDirFolders.Results)
        {
             Console.WriteLine("Blob", blob);
        }

    }
    catch (Exception e)
    {
        //  Block of code to handle errors
        Console.WriteLine("Error", e);

    }

answered Feb 6, 2020 at 14:35

ASHISH R

4,1891 gold badge24 silver badges16 bronze badges

1 Comment

John Over a year ago

This almost works but it only gets the first 5000 records. How do I get them all?

Gaurav Kaushik · Accepted Answer · 2018-08-17 10:25:09Z

Here is the code for the Answer :

private async Task<List<IListBlobItem>> ListBlobsAsync(CloudBlobContainer container)
{
    BlobContinuationToken continuationToken = null;
    List<IListBlobItem> results = new List<IListBlobItem>();
    do
    {
       bool useFlatBlobListing = true;
       BlobListingDetails blobListingDetails = BlobListingDetails.None;
       int maxBlobsPerRequest = 500;
       var response = await container.ListBlobsSegmentedAsync(BOAppSettings.ConfigServiceEnvironment, useFlatBlobListing, blobListingDetails, maxBlobsPerRequest, continuationToken, null, null);
            continuationToken = response.ContinuationToken;
            results.AddRange(response.Results);
        }
     while (continuationToken != null);
     return results;
}

And then you can return values like:

IEnumerable<IListBlobItem> listBlobs = await this.ListBlobsAsync(container);
foreach(CloudBlockBlob cloudBlockBlob in listBlobs)
  {
     BOBlobFilesViewModel boBlobFilesViewModel = new BOBlobFilesViewModel
     {
          CacheKey = cloudBlockBlob.Name,
          Name = cloudBlockBlob.Name
      };
      listBOBlobFilesViewModel.Add(boBlobFilesViewModel);
   }
//return listBOBlobFilesViewModel;

Lee Liu · Accepted Answer · 2018-08-15 09:36:59Z

4

The method CloudBlobClient.ListBlobsSegmentedAsync is used to return a result segment containing a collection of blob items in the container.

To list all blobs, we can use ListBlobs method,

Here is a demo for your reference:

    public static List<V> ListAllBlobs<T, V>(Expression<Func<T, V>> expression, string containerName,string prefix)
    {
        CloudStorageAccount storageAccount = CloudStorageAccount.Parse("YourConnectionString;");

        CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();

        CloudBlobContainer container = cloudBlobClient.GetContainerReference(containerName);
        container.CreateIfNotExists();

        var list = container.ListBlobs(prefix: prefix,useFlatBlobListing: true);

        List<V> data = list.OfType<T>().Select(expression.Compile()).ToList();
        return data;
    }

Usage and screenshots:

List all blobs' names under one folder:

List all blobs' URLs under one folder:

answered Aug 15, 2018 at 9:36

Lee Liu

2,0891 gold badge14 silver badges13 bronze badges

2 Comments

Gaurav Kaushik Over a year ago

Thanks, but I resolved it using ListBlobsSegmentedAsync and my custom logic, however, ListBlobs is no more in use as far as I have tried.. :)

martonx Over a year ago

ListBlobs is depreciated and removed from Azure SDK.

Collectives™ on Stack Overflow

How to get all files from a directory in Azure BLOB using ListBlobsSegmentedAsync

4 Answers 4

1 Comment

1 Comment

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

1 Comment

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related