Python remove and merge duplicate dictionaries from list while building sub list

Question

I'm trying to take a list of hosts with network information and produce a list of unique subnets with a list of domains associated with the subnet. Example list of hosts, shortened to three but this could be several hundred/thousand of items:

hosts = [{
    'name': 'foo',
    'subnet_address': '192.168.1.0',
    'subnet_mask': '255.255.254.0',
    'domain': 'foo.example.com'
}, {
    'name': 'bar',
    'subnet_address': '192.168.2.0',
    'subnet_mask': '255.255.254.0',
    'domain': 'bar.example.com'
}, {
    'name': 'baz',
    'subnet_address': '192.168.2.0',
    'subnet_mask': '255.255.254.0',
    'domain': 'foo.example.com'
}]

Here's the kind of output I'm trying to achieve, the subnet_address is the unique key and I want to build a list of domain associations for them:

[{
    'subnet_address': '192.168.1.0',
    'subnet_mask': '255.255.254.0',
    'domains': [
        'foo.example.com'
    ]
}, {
    'subnet_address': '192.168.2.0',
    'subnet_mask': '255.255.254.0',
    'domains': [
        'bar.example.com',
        'foo.example.com'
    ]
}]

I've found questions about removing duplicate dictionaries from a list and questions about merging (updating) dictionaries but haven't found anything similar to this yet, merging duplicates but at the same time building a list (set) of domains found elsewhere in the host list.

What exactly is the unique key? subnet_address or subnet_mask. You have to decide on one. — Erich
– Erich, Commented Apr 21, 2020 at 21:28
I updated the question so that subnet_address is the "primary key", although I'm interested to know if subnet_address and subnet_mask could be used together as the primary key..I suppose one could concatenate them together perhaps using the "192.168.1.0/23" format — gratz
– gratz, Commented Apr 21, 2020 at 21:33

Erich · Accepted Answer · 2020-04-21 21:36:33Z

2

You can iterate over the list of hosts.

subnets = []

for host in hosts:
    addr = host['subnet_address']
    added = False
    # try to find a subnet with given address
    for subnet in subnets:
        if subnet['subnet_address'] == addr:
            # if found, add domain to list
            subnet['domains'].append(host['domain'])
            # and remember that we found a matching subnet
            added = True
            break
    if not added:
        # if we didn't find any subnet, add a new one
        subnets.append({'subnet_address': addr,
                       'subnet_mask': host['subnet_mask'],
                       'domains': [ host['domain'] ] })

answered Apr 21, 2020 at 21:36

Erich

1,84817 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

gratz Over a year ago

Thanks, that works! I added a check before appending the domain to the list to ensure the domains list was unique

Collectives™ on Stack Overflow

Python remove and merge duplicate dictionaries from list while building sub list

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related