xmldataset¶

Extract Python datasets from XML with a simple profile that mirrors the XML.

A profile lists the elements and attributes you want, indented the same way as the XML, and names the dataset each value belongs to. xmldataset returns every dataset as a list of records (plain dicts), ready for JSON, CSV, pandas or a database.

Text, YAML or JSON: your choice

Profiles can be written in three equivalent formats: an indented text syntax, YAML, or JSON. Every example in this documentation shows all three. Pick a tab and your choice follows you around the site. See Choosing a format.

A first example¶

Given this XML:

<?xml version="1.0"?>
<colleagues>
    <colleague>
        <title>The Boss</title>
        <phone>+1 202-663-9108</phone>
        <email>boss@the_company.com</email>
    </colleague>
    <colleague>
        <title>Admin Assistant</title>
        <phone>+1 347-999-5454</phone>
        <email>admin@the_company.com</email>
    </colleague>
    <colleague>
        <title>Minion</title>
        <phone>+1 792-123-4109</phone>
    </colleague>
</colleagues>

this profile collects every colleague into a dataset called colleagues:

Text

colleagues
    colleague
        title = dataset:colleagues
        phone = dataset:colleagues
        email = dataset:colleagues

YAML

colleagues:
  colleague:
    title: colleagues
    phone: colleagues
    email: colleagues

JSON

{
  "colleagues": {
    "colleague": {
      "title": "colleagues",
      "phone": "colleagues",
      "email": "colleagues"
    }
  }
}

import xmldataset

result = xmldataset.parse_using_profile(xml, profile)

{
  "colleagues": [
    {
      "title": "The Boss",
      "phone": "+1 202-663-9108",
      "email": "boss@the_company.com"
    },
    {
      "title": "Admin Assistant",
      "phone": "+1 347-999-5454",
      "email": "admin@the_company.com"
    },
    {
      "title": "Minion",
      "phone": "+1 792-123-4109"
    }
  ]
}

The Minion has no email address, so their record has no email field.

Features¶

Three profile formats that compile to the same thing: indented text, YAML or JSON, with a JSON Schema for editor completion.
Elements and attributes, with renaming, prefixes, defaults, lists of repeated values and per-value process functions.
Correct records by construction. A record never mixes values from two elements, and values from an enclosing element (a shop’s number, say) are copied into every record inside it. See How records are built.
Streams documents of any size in flat memory, from a string, bytes, a path or a file; iter_records hands you each record as soon as it is complete.
Namespaces, matched by local name or by prefix.
A command-line tool to run profiles (JSON, JSON Lines or CSV output), convert them between formats, and validate them.
Typed, with PyYAML as the only dependency.