ElementTree parse xml file - problem with parsing

Question

I have a problem parsing data from xml file. I'm using xml.etree.ElementTree to extract data from files and then save them into .csv. I have all the necessery modules installed on server. I am aware that there is bs4 module with BeutifulSoup, yet I would like to know if is possible to parse this data/xml file using ElementTree. Sorry if the answear is easy or obvious, yet I'm still very much a beginner and with this problem I could not name the problem in a way to find an answear.

While running python script written below I have no errors and no outcome. I don't really know what should I change. I can not find solution. I tried using different child.tag or attributes but with no result.

The xml file that I have problem with.:

<?xml version="1.0" encoding="utf-8"?>
<offer file_format="IOF" version="2.6" extensions="yes" xmlns="http://www.iai-shop.com/developers/iof.phtml">
    <product id="9" vat="23.0" code_on_card="BHA">
      <producer id="1308137276" name="BEAL"/>
      ...
      <price gross="175" net="142.28"/>
      <sizes>
        <size code_producer="3700288265272" code="9-uniw" weight="0">
          <stock id="0" quantity="-1"/>
          <stock id="1" quantity="4"/>
        </size>
      </sizes>
    </product>
    <product>
              ...
    </product>
              ...

and script that I tried to use (here to extract code_on_card, price net, quantity).

(I am aware that there are two childs: stock and quantity, and I'm completely fine with the second one overwrting the first one)

import requests
import os,sys
import csv
import xml.etree.ElementTree as ET

reload(sys)
sys.setdefaultencoding('utf-8')

xml_path = '/file.xml'

xml = ET.parse(xml_path)

with open('/home/file.csv', 'wb') as f:
    c = csv.writer(f, delimiter=';')
    for product in xml.iter('product'):
    product_id = product.attrib["code_on_card"]
        for child in product:
            if child.tag == 'price':
                if child.attrib["net"] != None:
                    hurt_net = child.attrib["net"]
        for size in product.iter('size'):
            for stock in size.iter('stock'):
                if 'quantity' in stock.attrib.keys():
                    quantity = stock.attrib["quantity"]

        line = product_id, hurt_net, quantity
        c.writerow(line)

Files that seem to me to be built on similar scheme work just fine (offer -> product ->child/attrib ), like this one:

<?xml version="1.0" encoding="UTF-8"?>
<offer file_format="IOF" version="2.5">
    <product id="2">
        <price gross="0.00" net="0.00" vat="23.0"/>
        <srp gross="0.00" net="0" vat="23.0"/>
        <sizes>
            <size id="0"  code="2-0"  weight="0" >
            </size>
        </sizes>
    </product>
        ...
    </product>
        ...

EDIT: Outcome should be .csv file containing multpile rows (each for each product in xml file) of code_on_card, price net, quantity. It should look like:

BC097B.50GD.O;70.81;37
BC097B.50.A;76.75;24
BC086C.50.B;76.75;29
BGRT.L;3;96.75;28
....

EDIT2 code as it is, after drec4s answear:

import requests
import os,sys
import csv
import xml.etree.cElementTree as ET

reload(sys)
sys.setdefaultencoding('utf-8')

xml_path = '/home/platne/serwer16373/dane/z_hurtowni/pobrane/beal2.xml'

root = ET.parse(xml_path)

ns = {'offer': 'http://www.iai-shop.com/developers/iof.phtml'}

products = root.getchildren()

with open('/home/platne/serwer16373/dane/z_hurtowni/stany_magazynowe/karol/bealKa.csv', 'wb') as f:
    c = csv.writer(f, delimiter=';')
    hurtownia = 'beal'
    for product in root.iter('product'):
        qtt = [1]
        code = product.get('code_on_card')
        hurt_net = product.find('price').get('net')
        for stock in product.find('sizes').find('size').getchildren():
            qtt.append(stock.get('quantity'))
        quantity = max(qtt)


        line = 'beal-'+str(code), hurt_net, quantity
        c.writerow(line)

somehow I'm getting AttributeError: 'ElementTree' object has no attribute 'getchildren' I've got Ele

It's because of the default namespace (see here on how to parse XML with namespaces). It would also be helpful if you added an example of what your output should look like. — Daniel Haley
– Daniel Haley, Commented Nov 7, 2018 at 23:22
Another thing that would be helpful is code that will run as-is (see stackoverflow.com/help/mcve). — Daniel Haley
– Daniel Haley, Commented Nov 7, 2018 at 23:25
default namespace - I think thats it - need time to bite into it. — Rafae Rafad
– Rafae Rafad, Commented Nov 8, 2018 at 0:23

drec4s · Accepted Answer · 2018-11-08 00:12:04Z

2

This is how I would go and parse an xml file with namespaces. As per official documentation, the easiest way is to define a dictionary specifying the namespace.

from xml.etree import cElementTree as ET

root = ET.fromstring("""
<offer file_format="IOF" version="2.6" extensions="yes" xmlns="http://www.iai-shop.com/developers/iof.phtml">
    <product id="9" vat="23.0" code_on_card="BHA">
      <producer id="1308137276" name="BEAL"/>
      <price gross="175" net="142.28"/>
      <sizes>
        <size code_producer="3700288265272" code="9-uniw" weight="0">
          <stock id="0" quantity="-1"/>
          <stock id="1" quantity="4"/>
        </size>
      </sizes>
    </product>
</offer>
""")

ns = {'offer': 'http://www.iai-shop.com/developers/iof.phtml'}

products = root.getchildren()

for p in products:
    qtt = [] #to store all stock quantities
    product_id = p.get('code_on_card')
    hurt_net = p.find('offer:price', ns).get('net')
    for stock in p.find('offer:sizes', ns).find('offer:size', ns).getchildren():
        qtt.append(int(stock.get('quantity')))

    quantity = max(qtt) #or sum

line = (product_id, hurt_net, quantity)
print(line)

Outputs:

('BHA', '142.28', 4)

Also, I did not understand what was the stock quantity that you needed to extract, since you were only getting the last children(stock) value (change the sum function to max or to whatever you need).

answered Nov 8, 2018 at 0:12

drec4s

8,1178 gold badges35 silver badges56 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Rafae Rafad Over a year ago

I think that this is the answear, yet I can't seem to work it out. Somehow i'm getting AttributeError: 'ElementTree' object has no attribute 'getchildren'

Collectives™ on Stack Overflow

ElementTree parse xml file - problem with parsing

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related