Converting FGDC CSDGM to ISO 19115 Automatically Jump to heading#

Converting FGDC CSDGM to ISO 19115 automatically requires deterministic schema translation rather than heuristic parsing. Legacy FGDC XML structures lack the strict cardinality, temporal precision, and controlled vocabulary constraints mandated by ISO 19115-2003 and ISO 19115-1:2014. Direct mapping pipelines routinely fail when encountering unstructured <idinfo> blocks, ambiguous coordinate reference system declarations, or missing lineage sequencing. This guide is a deep dive within FGDC Metadata Mapping, the part of Geospatial Schema Architecture & Standards Mapping concerned with translating federal metadata profiles into modern interchange formats. It walks through a production-ready Python ETL configuration that enforces strict type coercion, resolves schema drift, and guarantees compliance through automated validation and fallback routing.

Prerequisites Jump to heading#

Confirm the conversion environment before running the pipeline against a real metadata corpus:

• Python 3.10+ — the implementation uses structural pattern matching idioms and decimal rounding modes.
• lxml >= 5.1 — provides etree.XMLSchema validation and deterministic find() traversal.
• PyYAML >= 6.0 — loads the declarative mapping matrix.
• pyproj >= 3.6 with bundled PROJ data — resolves free-text horizontal datums to authoritative EPSG codes.
• libxml2 / xmllint CLI on the PATH — performs the final XSD validation pass.
• ISO 19115 XSD bundle — download the official gmd/gco/gml schemas and pin the version in your repository so the validation target never drifts.
• EPSG registry access — either the local pyproj database or the PROJ_NETWORK=ON environment variable for online datum resolution.

Step 1 — Author the declarative mapping matrix Jump to heading#

The conversion engine must operate on a declarative translation matrix to prevent ad-hoc XPath mutations during execution. Define a YAML-driven mapping layer that explicitly binds FGDC elements to ISO 19115 equivalents while enforcing mandatory field population. The mandatory/optional split mirrors the convention used when handling missing mandatory fields in municipal GIS exports — a target path marked mandatory must always emit, falling back to a nil reason rather than being dropped.

# fgdc_to_iso.yaml — declarative mapping matrix (PyYAML >= 6.0)
mapping_matrix:
  # mandatory: must emit (nilReason fallback if source absent)
  idinfo/citation/citeinfo/title:
    iso_path: identificationInfo/MD_DataIdentification/citation/CI_Citation/title
    mandatory: true
  idinfo/citation/citeinfo/pubdate:
    iso_path: identificationInfo/MD_DataIdentification/citation/CI_Citation/date/CI_Date/date
    mandatory: true
  spref/horizsys/geodetic/horizdn:
    iso_path: referenceSystemInfo/MD_ReferenceSystem/referenceSystemIdentifier/RS_Identifier/code
    mandatory: true
  # optional: emit only when source present
  idinfo/citation/citeinfo/geoform:
    iso_path: identificationInfo/MD_DataIdentification/spatialRepresentationType/MD_SpatialRepresentationTypeCode
    mandatory: false
  dataqual/lineage/procstep/procdate:
    iso_path: dataQualityInfo/DQ_DataQuality/lineage/LI_Lineage/processStep/LI_ProcessStep/dateTime/CI_Date/date
    mandatory: false

Implement this matrix using lxml.etree with a deterministic traversal order. Avoid recursive wildcard searches (//) as they introduce non-deterministic node selection and break batch processing consistency — the same determinism principle enforced across batch schema processing pipelines. The Python loader below enforces strict path resolution and handles missing source nodes gracefully:

# Python 3.10+  (lxml >= 5.1, PyYAML >= 6.0)
import yaml
from lxml import etree
from typing import Dict, Optional, Tuple

def load_mapping_config(config_path: str) -> Dict[str, dict]:
    with open(config_path, "r", encoding="utf-8") as f:
        return yaml.safe_load(f)["mapping_matrix"]

def translate_node(
    source_tree: etree._ElementTree,
    fgdc_path: str,
    iso_path: str,
) -> Optional[Tuple[str, str]]:
    """Extract an FGDC value and return its (ISO target path, value)."""
    node = source_tree.find(fgdc_path)
    if node is None or not node.text or not node.text.strip():
        return None
    return iso_path, node.text.strip()

Step 2 — Coerce precision and temporal values Jump to heading#

Precision loss during coordinate and temporal conversion is the primary failure vector in automated pipelines. FGDC stores dates as YYYYMMDD or YYYY strings, while ISO 19115 requires strict YYYY-MM-DD ISO 8601 formatting with explicit time zone offsets. For spatial bounding boxes, FGDC uses <westbc>, <eastbc>, <northbc>, and <southbc>; ISO 19115 requires EX_GeographicBoundingBox with explicit longitude/latitude tags. Hold coordinates to a fixed tolerance, the same discipline applied when setting tolerance thresholds for automated coordinate snapping.

Apply the following thresholds and rules:

• Temporal precision: If temporal precision drops below day-level, inject 00:00:00Z and append a <gco:CharacterString> provenance note indicating automated normalization.
• Coordinate precision: Enforce a 6-decimal precision threshold during float coercion to prevent floating-point drift that corrupts downstream spatial indexing.
• Rounding strategy: Values exceeding the threshold must be rounded using decimal.ROUND_HALF_EVEN to maintain IEEE 754 compliance.
• Null handling: Missing bounding box coordinates trigger a fallback to the dataset’s declared CRS extent or raise a ValueError if no authoritative extent exists.

# Python 3.10+
import re
from decimal import Decimal, ROUND_HALF_EVEN
from datetime import datetime

DATE_PATTERN = re.compile(r"^(?P<year>\d{4})(?P<month>\d{2})?(?P<day>\d{2})?$")

def coerce_date(raw_date: str) -> str:
    match = DATE_PATTERN.match(raw_date.strip())
    if not match:
        raise ValueError(f"Invalid FGDC date format: {raw_date}")

    year = match.group("year")
    month = match.group("month") or "01"
    day = match.group("day") or "01"

    iso_date = f"{year}-{month}-{day}"
    try:
        datetime.strptime(iso_date, "%Y-%m-%d")
    except ValueError:
        raise ValueError(f"Invalid calendar date: {iso_date}")

    return f"{iso_date}T00:00:00Z"

def coerce_coordinate(val: str) -> str:
    d = Decimal(val.strip())
    return str(d.quantize(Decimal("0.000001"), rounding=ROUND_HALF_EVEN))

Step 3 — Route schema drift to fallback handlers Jump to heading#

Schema drift occurs when FGDC profiles omit mandatory elements or use deprecated vocabulary codes. Production pipelines must implement a quarantine-and-retry architecture rather than hard-failing. When FGDC profiles diverge from the baseline, route non-compliant records to a staging directory with structured error manifests instead of aborting the batch.

Handle the following edge cases explicitly:

• Missing fields: If a mandatory ISO 19115 element lacks a direct FGDC equivalent, inject a <gco:nilReason> attribute with the value missing or unknown per ISO 19115-1:2014 Section 6.2.
• CRS mismatches: FGDC often declares horizontal datums via free-text strings. Resolve these using an authoritative EPSG registry lookup. If resolution fails, default to EPSG:4326 and log a CRS_AMBIGUOUS warning. Where the source carries a partially-specified datum, the Datum Transformation Fallback Chains strategy supplies the ordered resolution order.
• CI pipeline failures: Integrate schema validation into your CI/CD workflow. Fail fast on XML well-formedness errors, but allow soft-failures on optional metadata blocks to prevent deployment bottlenecks.

# Python 3.10+  (pyproj >= 3.6)
import logging
from pyproj import CRS
from pyproj.exceptions import CRSError

logger = logging.getLogger("fgdc_iso")

def resolve_datum(free_text: str) -> str:
    """Resolve a free-text FGDC horizontal datum to an EPSG code string."""
    try:
        crs = CRS.from_user_input(free_text.strip())
        epsg = crs.to_epsg()
        if epsg is not None:
            return f"EPSG:{epsg}"
    except CRSError:
        pass
    logger.warning('{"event": "CRS_AMBIGUOUS", "source": "%s", "fallback": "EPSG:4326"}', free_text)
    return "EPSG:4326"

Step 4 — Validate against the ISO 19115 XSD Jump to heading#

Post-conversion validation must occur before data publication. Use the official ISO 19115 XML Schema Definition (XSD) plus Schematron rules to enforce business-logic constraints that the XSD alone cannot express. Refer to the official ISO 19115-1:2014 specification for authoritative schema definitions.

# Python 3.10+
import subprocess
from pathlib import Path

def validate_iso19115(xml_path: Path, xsd_path: Path) -> bool:
    """Validate ISO 19115 XML against the official XSD using xmllint."""
    result = subprocess.run(
        ["xmllint", "--noout", "--schema", str(xsd_path), str(xml_path)],
        capture_output=True,
        text=True,
    )
    if result.returncode != 0:
        print(f"Schema validation failed:\n{result.stderr}")
        return False
    return True

Step 5 — Gate the conversion in CI and log lineage Jump to heading#

A converted document is only trustworthy once a machine has rejected the alternative. Wire validate_iso19115() into the pipeline so non-compliant output never reaches publication, and write a lineage record for every emitted file so auditors can reconstruct how each ISO field was derived from its FGDC source.

Configure your CI runner to:

• Run validate_iso19115() on every pull request touching the metadata directory.
• Block merges if validation returns False or if the error log contains CRITICAL severity tags.
• Archive failed XML payloads to the quarantine directory for manual review by GIS data stewards.
• Append one lineage record per emitted document — source path, mapping matrix version, datum resolution outcome, and a UTC timestamp.

# Python 3.10+
import json
from datetime import datetime, timezone
from pathlib import Path

def write_lineage(out_path: Path, source: str, matrix_version: str, crs_result: str) -> None:
    record = {
        "emitted": str(out_path),
        "source": source,
        "matrix_version": matrix_version,
        "crs_resolution": crs_result,
        "timestamp": datetime.now(timezone.utc).isoformat(),
    }
    with open("lineage.log", "a", encoding="utf-8") as log:
        log.write(json.dumps(record) + "\n")

Verification Jump to heading#

Confirm a converted record both validates against the schema and carries an auditable lineage line before you trust the batch:

def test_conversion_emits_valid_iso_and_lineage(tmp_path):
    out = tmp_path / "out.xml"
    # ... run the conversion that writes `out` ...
    assert validate_iso19115(out, Path("schemas/iso19115/gmd.xsd")) is True
    lineage = Path("lineage.log").read_text(encoding="utf-8").splitlines()
    assert json.loads(lineage[-1])["emitted"] == str(out)

A successful run prints nothing from xmllint (the --noout flag suppresses output on a clean validation) and appends exactly one JSON line to lineage.log such as {"emitted": "out.xml", "source": "in_fgdc.xml", "matrix_version": "1.4.0", "crs_resolution": "EPSG:4269", "timestamp": "2026-06-25T00:00:00+00:00"}. If xmllint emits any element ... Schemas validity error, the record must be quarantined, not published.

Troubleshooting Jump to heading#

↑ Back to FGDC Metadata Mapping, part of Geospatial Schema Architecture & Standards Mapping.

Related Jump to heading#

• How to Map INSPIRE Annex III to Local PostgreSQL Schemas — apply the same declarative mapping matrix discipline to a European directive target instead of ISO 19115
• Best Practices for Spatial Data Dictionary Versioning — pin and version the mapping matrix so conversions stay reproducible across releases
• Handling Missing Mandatory Fields in Municipal GIS Exports — the nilReason fallback pattern that mandatory ISO 19115 elements rely on
• Setting Tolerance Thresholds for Automated Coordinate Snapping — the precision-threshold reasoning behind the 6-decimal coordinate coercion rule