0

I am using azure sdk for cpp, and I am using option ListBlobsByHierarchy.

When I use ListBlobsByHierarchy, I get result almost as I have expected, all the blobs are there, all the virtual directories are there. The problems is with folders that are not virtual directories, not blobs.

If I create a blob folder1/folder2/blob.txt, in azure I will see folder1->folder2->blob.txt.

But if I try to list root directory and to get a folder1 as a result, it fails.

I have tried to use ListBlobs, but it reads whole container and implementing filter for root, or any other directory is complicated and very slow.

Is there an options or way to fetch that folder1 with ListBlobsByHierarchy?

1 Answer 1

0

Is there an options or way to fetch that folder1 with ListBlobsByHierarchy?

In my Azure Storage account, I had a folder structure be like:

  sample/
    data.xlsx
  sample/venkat1/
    demo.txt
    file.csv
    sample.pdf
    sample.wav
    samplecert.pfx
    temp2.jpg
    test.jpg  

Portal:

enter image description here

To list blobs hierarchically, you can use the ListBlobsByHierarchy method. However, if you want to list the root directory and get a folder1(sample) as a result, you can try using the ListBlobs method instead.

Code:

#include <azure/storage/blobs.hpp>
#include <iostream>
#include <set>
#include <string>
#include <vector>

int main() {
    std::string connectionString = "DefaultEndpointsProtocol=https;AccountName=venkat326123;AccountKey=T3czZpu1gZ0zzzzzzzzzw2zzz08xcs5p4Wu2+AStD9nyWw==;EndpointSuffix=core.windows.net";
    std::string containerName = "test";

    auto containerClient = Azure::Storage::Blobs::BlobContainerClient::CreateFromConnectionString(connectionString, containerName);
    std::set<std::string> seenDirs;

    Azure::Storage::Blobs::ListBlobsOptions options;
    auto response = containerClient.ListBlobs(options);
    std::cout << "Listing top-level directories:" << std::endl;

    do {
        // List blobs and extract directories from blob names
        for (const auto& blob : response.Blobs) {
            std::string blobName = blob.Name;

            // Split the blob name by '/' to get top-level directories
            size_t pos = blobName.find('/');
            if (pos != std::string::npos) {
                std::string topLevelDir = blobName.substr(0, pos + 1); // Include '/' to indicate directory
                if (seenDirs.find(topLevelDir) == seenDirs.end()) {
                    std::cout << "  Directory: " << topLevelDir << std::endl;
                    seenDirs.insert(topLevelDir);
                }
            }
        }
        if (response.HasPage()) {
            response.MoveToNextPage();
        }
        else {
            break;
        }
    } while (true);
    if (seenDirs.empty()) {
        std::cout << "No top-level directories found." << std::endl;
    }
    return 0;
}

Output:

Listing top-level directories:
  Directory: sample/
Sign up to request clarification or add additional context in comments.

5 Comments

First of all big big big sorry for this super late reply... I'm running some tests and came up with interesting edge case. I have ADLS gen2 with HNS ON, access it via private endpoint of type BLOB, which means I cannot use Datalake interface, but blob. Now if I use ListBlobsByHierarchy with clean options and "/" as delimiter. I cannot see any folder created by datalake interface in root directory.
auto listBlobOptions = Azure::Storage::Blobs::ListBlobsOptions(); auto blobPages = containerClient.ListBlobsByHierarchy("/", listBlobOptions); std::cout << "Blob Reading" << std::endl; for (const auto& blobItem : blobPages.Blobs) { std::cout << "******************" << std::endl; std::cout << blobItem.Name << std::endl; } If I remove delimiter and pass just "" I am reading everything. Which is kind of a problem because now I cannot read the only root directory.
Any ideas? Should I run with "" delimiter and then filter out the rest? I am afraid I could pick up 100K no needed blobs such way....
please rise it with new question with detail information and paste the question link here.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.