How do you remove duplicate, nested DOM elements in PHP?

Question

Assuming you have a DOM tree with nested tags, I would like to clean the DOM object up by removing duplicates. However, this should only apply if the tag only has a single child tag of the same type. For example,

Fix <div><div>1</div></div> and not <div><div>1</div><div>2</div></div>.

I'm trying to figure out how I could do this using PHP's DOM extension. Below is the starting code and I'm looking for help figuring out the logic needed.

<?php

libxml_use_internal_errors(TRUE);

$html = '<div><div><div><p>Some text here</p></div></div></div>';

$dom = new DOMDocument;
$dom->preserveWhiteSpace = false;
$dom->formatOutput = true;
$dom->loadHTML($html);

function dom_remove_duplicate_nodes($node)
{
    var_dump($node);

    if($node->hasChildNodes())
    {
        for($i = 0; $i < $node->childNodes->length; $i++)
        {
            $child = $node->childNodes->item($i);

            dom_remove_duplicate_nodes($child);
        }
    }
    else
    {
        // Process here?
    }
}

dom_remove_duplicate_nodes($dom);

I collected some helper functions that might make it easier to work the DOM nodes like JavaScript.

function DOM_delete_node($node)
{
    DOM_delete_children($node);
    return $node->parentNode->removeChild($node);
}

function DOM_delete_children($node)
{
    while (isset($node->firstChild))
    {
        DOM_delete_children($node->firstChild);
        $node->removeChild($node->firstChild);
    }
}

function DOM_dump_child_nodes($node)
{
    $output = '';
    $owner_document = $node->ownerDocument;

    foreach ($node->childNodes as $el)
    {
        $output .= $owner_document->saveHTML($el);
    }
    return $output;
}

function DOM_dump_node($node)
{
    if($node->ownerDocument)
    {
        return $node->ownerDocument->saveHTML($node);
    }
}

I don't have time for a more detailed answer, but I think this would be faster and easier to do with regular expressions! — janoliver
– janoliver, Commented Nov 1, 2011 at 20:12
@janoliver, I can't imagine a regular expression complex enough to take node levels into account. — Xeoncross
– Xeoncross, Commented Nov 1, 2011 at 20:18
For the love of all that is holy, DO NOT use regular expressions to parse XML unless there is absolutely no other choice. If you have malformed markup, try using the tidy extension to clean it up, then use the DOM, SimpleXML, or XMLReader extensions to parse it. — Matthew Turland
– Matthew Turland, Commented Nov 2, 2011 at 23:44

hakre · Accepted Answer · 2011-11-06 15:07:19Z

You can do this quite easily with DOMDocument and DOMXPath. XPath especially is really useful in your case because you easily divide the logic to select which elements to remove and the way you remove the elements.

First of all, normalize the input. I was not entirely clear about what you mean with empty whitespace, I thought it could be either empty textnodes (which might have been removed as preserveWhiteSpace is FALSE but I'm not sure) or if their normalized whitespace is empty. I opted for the first (if even necessary), in case it's the other variant I left a comment what to use instead:

$xp = new DOMXPath($dom);

//remove empty textnodes - if necessary at all
// (in case remove WS: [normalize-space()=""])
foreach($xp->query('//text()[""]') as $i => $tn)
{
    $tn->parentNode->removeChild($tn);
}

After this textnode normalization you should not run into the problem you talked about in one comment here.

The next part is to find all elements that have the same name as their parent element and which are the only child. This can be expressed in xpath again. If such elements are found, all their children are moved to the parent element and then the element will be removed as well:

// all child elements with same name as parent element and being
// the only child element.
$r = $xp->query('body//*/child::*[name(.)=name(..) and count(../child::*)=1]');
foreach($r as $i => $dupe)
{
    while($dupe->childNodes->length)
    {
        $child = $dupe->firstChild;
        $dupe->removeChild($child);
        $dupe->parentNode->appendChild($child);
    }   
    $dupe->parentNode->removeChild($dupe);
}

Full demo.

As you can see in the demo, this is independent to textnodes and commments. If you don't want that, e.g. actual texts, the expression to count children needs to stretch over all node types. But I don't know if that is your exact need. If it is, this makes the count of children across all node types:

body//*/child::*[name(.)=name(..) and count(../child::node())=1]

If you did not normalize empty textnodes upfront (remove empty ones), then this too strict. Choose the set of tools you need, I think normalizing plus this strict rule might be the best choice.

Very nice, solves the problem and optionally removes comments and whitespace! I had no idea that little xpath class was so powerful!

Matthew Turland · Accepted Answer · 2011-11-02 02:05:04Z

1

Seems like you have almost everything you need here. Where you have // Process here? do something like this:

if ($node->parentNode->nodeName == $node->nodeName 
  && $node->parentNode->childNodes->length == 1) {
  $node->parentNode->removeChild($node);
}

Also, you're currently using recursion in dom_remove_duplicate_notes() which can be computationally expensive. It is possible to iterate over every node in the document without recursion using an approach like this: https://github.com/elazar/domquery/blob/master/trunk/DOMQuery.php#L73

answered Nov 2, 2011 at 2:05

Matthew Turland

7633 silver badges11 bronze badges

3 Comments

Xeoncross Over a year ago

This is a good start, however the problem is that very seldom do you have an element with only 1 child. Most of the time there may be only one child - but childNodes->length reports 3 because of the empty whitespace before and after the node. So, #text div #text.

Matthew Turland Over a year ago

@Xeoncross True. I was going strictly by the earlier example, which this didn't apply to. To do what you're describing would merely require looping through childNodes and checking the ->nodeType property of each node to see if it matches the constant XML_ELEMENT_NODE. See php.net/manual/en/dom.constants.php for more information on that. Just use a counter variable to count the number of element nodes and, if there's only 1, proceed as above.

Xeoncross Over a year ago

I built on your recommendation and have posted my progress so far. It's an almost working version.

Xeoncross · Accepted Answer · 2011-11-03 15:32:12Z

The following is an almost-working snippet. While it does remove duplicate, nested nodes - it changes the source order because of the ->appendChild().

<?php
header('Content-Type: text/plain');

libxml_use_internal_errors(TRUE);

$html = "<div>\n<div>\n<div>\n<p>Some text here</p>\n</div>\n</div>\n</div>";

$dom = new DOMDocument;
$dom->preserveWhiteSpace = false;
$dom->formatOutput = true;
$dom->loadHTML($html);

function dom_remove_duplicate_nodes($node)
{
    //var_dump($node);

    if($node->hasChildNodes())
    {
        $newNode = NULL;
        for($i = 0; $i < $node->childNodes->length; $i++)
        {
            $child = $node->childNodes->item($i);

            dom_remove_duplicate_nodes($child);

            if($newNode === FALSE) continue;

            // If there is a parent to check against
            if($child->nodeName == $node->nodeName)
            {
                // Did we already find the same child?
                if($newNode OR $newNode === FALSE)
                {
                    $newNode = FALSE;
                }
                else
                {
                    $newNode = $child;
                }
            }
            elseif($child->nodeName == '#text')
            {
                // Something other than whitespace?
                if(trim($child->nodeValue))
                {
                    $newNode = FALSE;
                }
            }
            else
            {
                $newNode = FALSE;
            }
        }

        if($newNode)
        {
            // Does not transfer $newNode children!!!!
            //$node->parentNode->replaceChild($newNode, $node);

            // Works, but appends in reverse!!
            $node->parentNode->appendChild($newNode);
            $node->parentNode->removeChild($node);
        }
    }
}

print $dom->saveHTML(). "\n\n\n";
dom_remove_duplicate_nodes($dom);
print $dom->saveHTML();

Collectives™ on Stack Overflow

How do you remove duplicate, nested DOM elements in PHP?

3 Answers 3

1 Comment

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related