Geospatial Schema Standardization & Mapping Automation

Projection Normalization Workflows

Automated geospatial pipelines fail quietly when coordinate reference systems drift across ingestion sources: a county parcel layer arrives in a State Plane foot system, an aerial tile in Web Mercator, and a survey extract in a legacy ESRI datum, and a naive spatial join silently produces metre-scale offsets. A projection normalization workflow is the discrete pipeline stage that enforces deterministic CRS alignment — resolving every source to a canonical target EPSG code, gating against projection validity domains, and reprojecting with auditable, tolerance-checked transforms — before any spatial join, tiling, or index generation runs. This stage operates at the core of CRS Normalization & Sync, the discipline responsible for guaranteeing that every geometry in the system shares a coherent spatial reference before downstream schema operations begin.

This page is scoped to the resolution-and-reprojection contract: how to declare the source-to-target mapping, sanitize heterogeneous CRS definitions, gate against the target projection’s valid extent, and reproject deterministically. Choosing which datum transformation path runs when source and target datums differ — and how to fall back when a grid is missing — belongs to Datum Transformation Fallback Chains. Applying numeric deviation gates and coordinate snapping once a transform is selected is the job of Unit Conversion & Tolerance Thresholds. Merging the normalized outputs of several authority sources into one coherent dataset is covered by Multi-CRS Dataset Harmonization. Here we focus on replacing ad-hoc GDAL one-liners with a schema-driven, reproducible stage.

Projection Normalization Workflow with Validity Gate and Failure Routing A left-to-right pipeline of four decision stages. First, resolve canonicalizes a heterogeneous source CRS to an EPSG code; an unparseable input branches down to a quarantine sink. Second, the area-of-use gate compares the geometry envelope against the target projection's valid extent; an out-of-domain envelope is quarantined. Third, reproject runs the explicit PROJ transform; a PROJ error is rejected. Fourth, the tolerance gate compares reported transform accuracy to the configured budget; values over budget are rejected and passing records are written as normalized output with audit lineage. Resolve → canonical EPSG Area-of-Use Gate envelope ⊆ extent? Reproject explicit PROJ pipeline Tolerance Gate accuracy ≤ budget? pass inside ok Normalized Output + PASS audit lineage Quarantine Manifest unparseable · outside_area_of_use Reject Log proj_error · accuracy_exceeds_tolerance CRSError outside ProjError over budget

Declarative CRS Mapping Manifest

The stage is driven by a version-controlled manifest rather than imperative scripts, so that every CRS decision is reviewable in a pull request and reproducible across runs. The manifest declares each expected source identifier, the canonical target EPSG code, and whether resolution must be strict (exact authority match) or may fall back to heuristics. Mandatory fields are non-negotiable to prevent silent fallbacks or undefined behaviour during CI/CD execution; optional fields tune datum routing and precision.

Field Requirement Type Description
source_identifier Mandatory string Raw CRS input (WKT v1/v2, PROJ string, ESRI code, or EPSG alias)
target_epsg Mandatory integer Canonical EPSG code for the output projection
strict_validation Mandatory boolean Enforces exact authority matching; rejects heuristic guesses
authority Optional string Explicit authority override (EPSG, ESRI, IGNF). Defaults to EPSG
tolerance_meters Optional float Maximum permitted residual deviation after reprojection. Defaults to 0.0 (exact)
fallback_datums Optional array Ordered datum paths handed to the fallback-chain stage when datums differ
yaml
# crs_mapping.yaml — projection normalization manifest (pyyaml >=6.0)
normalization_rules:
  - source_identifier: "ESRI:102003"        # USA Contiguous Albers (ESRI alias)
    target_epsg: 5070                        # NAD83 / Conus Albers (canonical)
    strict_validation: true
    authority: EPSG
    tolerance_meters: 0.001
  - source_identifier: "PROJ:+proj=aea +lat_1=50 +lat_2=58.5 +lat_0=45"
    target_epsg: 3005                        # NAD83 / BC Albers
    strict_validation: true
    fallback_datums: ["NAD83", "WGS84"]      # consumed by the fallback-chain stage

A small pydantic or voluptuous schema validates this manifest before the pipeline starts. Treat an unparseable manifest as a hard build failure, never a runtime warning — a malformed target_epsg must not be allowed to reach a production reprojection.

Preprocessing Requirements: Sanitizing Heterogeneous CRS Input

Before the manifest’s rules can be applied, source CRS metadata has to be reduced to a single canonical form. Raw inputs arrive as PROJ strings, WKT v1 and v2, deprecated ESRI identifiers, or bare srs_name attributes, and many carry deprecated parameters (+towgs84 blocks, axis-order ambiguity, or unit drift). The resolver parses each input with pyproj.CRS.from_user_input(), strips deprecated parameters, and confirms the result against the EPSG registry. Inputs that resolve ambiguously — or not at all — are routed to a quarantine manifest rather than guessed at. When an ingestion batch mixes formats and missing declarations, the deeper step-by-step EPSG code normalization for mixed datasets procedure walks through inventory extraction, metadata sanitization, and per-record resolution logging in detail.

python
# crs_resolve.py — canonicalize a heterogeneous CRS string (pyproj >=3.6)
from pyproj import CRS
from pyproj.exceptions import CRSError

def resolve_to_epsg(source_identifier: str, strict: bool = True) -> int:
    """Return a canonical EPSG code or raise. No heuristic guessing under strict."""
    try:
        crs = CRS.from_user_input(source_identifier)
    except CRSError as exc:
        raise CRSError(f"unparseable CRS: {source_identifier!r}") from exc

    # min_confidence=100 demands an exact authority match under strict resolution
    matches = crs.list_authority(auth_name="EPSG")
    if strict and not matches:
        raise CRSError(f"no exact EPSG authority match for {source_identifier!r}")

    epsg = crs.to_epsg(min_confidence=100 if strict else 70)
    if epsg is None:
        raise CRSError(f"could not assign EPSG code to {source_identifier!r}")
    return epsg

Execution Engine & Precision Guards

Once a source is canonicalized, the engine gates against the target projection’s valid domain, then reprojects with an explicit transform. Blind transformation introduces topology corruption when source coordinates exceed the valid area of use of the target projection — Web Mercator beyond ~85° latitude, or a State Plane zone applied outside its county band, produces mathematically unstable shifts. The validity gate compares the geometry envelope against the EPSG-defined area_of_use and quarantines out-of-domain records instead of forcing the transform.

Area-of-Use Gate: Envelope Containment Against the Target EPSG Valid Extent A rectangle labelled "EPSG area_of_use" represents the target projection's valid west/south/east/north bounds. A first geometry envelope sits wholly inside this rectangle and is marked PASS, flowing on to reprojection. A second geometry envelope straddles the top-right boundary, with part of it beyond the north and east edges; it is marked QUARANTINE with reason outside_area_of_use, and does not reach the transform. The containment test shown is: minx greater-or-equal west, miny greater-or-equal south, maxx less-or-equal east, maxy less-or-equal north. EPSG area_of_use (west · south · east · north) N,W S,E envelope A ⊆ extent envelope B crosses N & E PASS → reproject QUARANTINE outside_area_of_use test: minx ≥ west · miny ≥ south · maxx ≤ east · maxy ≤ north
python
# normalize.py — validity gate + audited reprojection (pyproj >=3.6, shapely >=2.0)
import logging
from pyproj import CRS, Transformer
from pyproj.exceptions import ProjError
from shapely.ops import transform as shp_transform

logger = logging.getLogger("crs.normalize")

def gate_and_reproject(geometry, src_crs: CRS, target_epsg: int,
                       tolerance_m: float = 0.0) -> tuple:
    target_crs = CRS.from_epsg(target_epsg)
    aou = target_crs.area_of_use
    if aou is None:
        raise ValueError(f"target EPSG:{target_epsg} has no defined area of use")

    minx, miny, maxx, maxy = geometry.bounds  # geographic-degree envelope assumed
    if minx > aou.east or maxx < aou.west or miny > aou.north or maxy < aou.south:
        logger.warning("envelope outside EPSG:%s area of use -> quarantine", target_epsg)
        return None, {"outcome": "QUARANTINE", "reason": "outside_area_of_use"}

    try:
        # always_xy pins lon/lat axis order across geographic and projected CRS
        tf = Transformer.from_crs(src_crs, target_crs, always_xy=True)
        out = shp_transform(tf.transform, geometry)
    except ProjError as exc:
        logger.error("PROJ transform failed src=%s -> %s: %s",
                     src_crs.to_authority(), target_epsg, exc)
        return None, {"outcome": "REJECT", "reason": f"proj_error:{exc}"}

    audit = {
        "outcome": "PASS",
        "src_crs": src_crs.to_authority(),          # e.g. ('EPSG', '102003')
        "target_epsg": target_epsg,
        "transformer_name": tf.description,
        "accuracy_m": tf.accuracy if tf.accuracy is not None else -1.0,
        "tolerance_m": tolerance_m,
    }
    # Tolerance enforcement: reported transform accuracy must be within budget
    if tolerance_m > 0.0 and audit["accuracy_m"] > tolerance_m:
        audit["outcome"] = "REJECT"
        audit["reason"] = "accuracy_exceeds_tolerance"
        out = None
    return out, audit

Every transform is built explicitly with Transformer.from_crs(...) rather than relying on GeoPandas’ implicit .to_crs(), so the chosen pipeline and its reported accuracy are captured for the audit trail. When the source and target datums differ, hand the fallback_datums list to Datum Transformation Fallback Chains to pick a deterministic grid path; enforce the numeric tolerance_meters budget through Unit Conversion & Tolerance Thresholds. Consult the PROJ documentation for transformation-grid management and accuracy semantics.

Failure Modes & Fallback Routing

No record exits this stage in an undefined state. Each failure type maps to one deterministic recovery action, and every routed record writes a reason code to the audit trail.

Failure mode Likely cause Deterministic recovery
CRSError on resolve Malformed WKT, deprecated ESRI alias, or no EPSG match under strict mode Route to quarantine manifest; do not guess a code
Envelope outside area of use Source coordinates beyond target projection’s valid latitude/longitude band Quarantine record; flag for manual zone reassignment
ProjError on transform Missing PROJ grid (NTv2/NADCON) or unsupported pipeline Reject; hand to fallback-chain stage for an alternate datum path
Accuracy exceeds tolerance Selected transform’s reported accuracy above tolerance_meters Reject record; emit accuracy_exceeds_tolerance to reject log
Axis-order ambiguity Geographic CRS reprojected without always_xy Re-run with always_xy=True; assert lon/lat ordering in tests

Transient causes — a grid file that has not finished downloading, a temporarily unreachable PROJ CDN — are not the resolver’s concern. Wrap the engine with the Error Handling & Retry Logic patterns (exponential backoff, circuit breakers) so a transient grid fetch is retried rather than recorded as a permanent rejection.

Compliance Reporting Output

The stage writes one audit record per input geometry to a newline-delimited JSON (NDJSON) lineage log, so a reviewer can later prove which transform ran for any feature. Each PASS record carries the resolved source authority, target EPSG, the named transformer pipeline, its reported accuracy, and the tolerance budget applied. QUARANTINE and REJECT records carry the same identifiers plus a machine-readable reason field, and are also appended to a separate rejection log that downstream reconciliation jobs consume.

python
# audit_sink.py — append-only lineage and rejection logs (Python 3.10+)
import json
from pathlib import Path

def write_audit(record_id: str, audit: dict,
                lineage: Path, rejects: Path) -> None:
    row = {"record_id": record_id, **audit}
    line = json.dumps(row, sort_keys=True)
    lineage.open("a").write(line + "\n")
    if audit["outcome"] in {"QUARANTINE", "REJECT"}:
        rejects.open("a").write(line + "\n")

This lineage feeds the broader compliance dashboard: the transformer_name and accuracy_m fields satisfy ISO 19115 positional-accuracy lineage requirements, and the rejection log gives auditors a defensible record of every geometry that did not pass and exactly why.

CI Integration

Embed the manifest and resolver checks directly into version control so CRS drift cannot reach production. Gate two things on every pull request: that the crs_mapping.yaml manifest still validates against its schema, and that the resolver assigns the expected canonical EPSG codes for a fixed set of golden inputs.

yaml
# .github/workflows/crs-normalization.yml
name: CRS Normalization Validation
on:
  pull_request:
    paths:
      - "schemas/crs_mapping.yaml"
      - "src/normalization/**"
      - "tests/test_normalization.py"
jobs:
  validate-crs:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - name: Install dependencies
        run: pip install "pyproj>=3.6" "shapely>=2.0" "pyyaml>=6.0" "pytest>=7" proj-data
      - name: Validate CRS mapping manifest
        run: python -m src.normalization.validate --config schemas/crs_mapping.yaml --strict
      - name: Run resolver + validity-gate tests
        run: pytest tests/test_normalization.py -v --tb=short

Pair the GitHub Actions gate with a pytest fixture that asserts each manifest rule round-trips to its expected target_epsg, plus a deliberately out-of-domain geometry that must route to QUARANTINE. A pre-commit hook running the manifest schema validator catches malformed edits before they ever reach CI.

Frequently Asked Questions

Why build transforms with Transformer.from_crs() instead of GeoPandas .to_crs()? .to_crs() selects a transformation pipeline implicitly and throws away its identity and reported accuracy. Building the transform explicitly with Transformer.from_crs(..., always_xy=True) captures the pipeline name (tf.description) and tf.accuracy for the audit trail, and lets you enforce a per-record tolerance_meters budget that an implicit reproject cannot express.

Why does the area-of-use gate run before reprojection rather than after? Reprojecting coordinates that fall outside the target projection’s valid domain produces mathematically unstable shifts — Web Mercator beyond ~85° latitude, or a State Plane zone applied outside its county band. Gating the geometry envelope against the EPSG-defined area_of_use first quarantines out-of-domain records, instead of writing corrupted geometry that downstream joins would silently treat as valid.

What is the difference between a QUARANTINE and a REJECT outcome? QUARANTINE marks records that may still be recoverable by a human or a different rule — an unparseable CRS, or an envelope outside the target’s area of use — and holds them for manual zone reassignment. REJECT marks records that failed the transform itself: a missing PROJ grid raising ProjError, or a transform whose reported accuracy exceeds the configured tolerance. Rejected records are handed to the Datum Transformation Fallback Chains stage for an alternate datum path or written to the reject log.

Why pass always_xy=True to every transformer? Many geographic EPSG definitions declare a latitude-first axis order, while most pipeline code assumes lon/lat. always_xy=True pins longitude-then-latitude ordering across both geographic and projected CRS, eliminating the silent coordinate swap that otherwise produces transposed geometry — the failure mode listed as “axis-order ambiguity” in the table above.

Deeper Implementations