pyproj vs GDAL ogr2ogr for CRS Normalization Jump to heading#

pyproj and GDAL’s ogr2ogr sit on the same PROJ engine, yet they fail and succeed in opposite ways under load. pyproj gives you in-process, per-feature control — you choose the exact coordinate operation, gate it against tolerance, and branch on the result in Python. ogr2ogr gives you a single, fast, whole-file conversion that streams gigabytes without materializing them in memory but hides the transformation decision behind a CLI flag. Choosing wrong means either a slow Python loop over a terabyte of tiles or an opaque batch job you cannot audit. This comparison quantifies the trade-off and recommends which to reach for. It sits inside Projection Normalization Workflows, the parent stage of CRS Normalization & Sync that owns how source projections become one canonical target.

The steps follow configure (Step 1, shared backend), execute (Steps 2-3, both tools), validate (Step 4, determinism and throughput), and log (Step 5, decision lineage). The deliverable is a defensible tool choice backed by numbers, not a preference.

Prerequisites checklist Jump to heading#

• pyproj >= 3.6 and GDAL >= 3.8 installed against the same PROJ >= 9.3 build.
• geopandas >= 0.14 for the in-process path’s I/O.
• Identical PROJ grid data visible to both tools (PROJ_DATA set process-wide).
• A representative sample layer (both a small feature-rich file and a large tile set).
• time / a wall-clock harness for throughput measurement.
• A writable path for the decision-lineage record.

Step 1: Pin the shared PROJ backend Jump to heading#

A comparison is only fair if both tools resolve the same PROJ version and the same grids; otherwise you are comparing PROJ releases, not the tools. Assert the versions match before running anything.

# Confirm both front-ends resolve the same PROJ backend — GDAL >= 3.8, pyproj >= 3.6
python -c "import pyproj; print('pyproj PROJ', pyproj.proj_version_str)"
ogrinfo --version                     # e.g. GDAL 3.8.4
projinfo --version                    # PROJ 9.3.x  -> must match pyproj's report
echo "PROJ_DATA=$PROJ_DATA"           # both tools must see the same grid dir

Backend rules:

• Confirm pyproj.proj_version_str equals the projinfo --version PROJ number; a mismatch invalidates every downstream comparison.
• Export one PROJ_DATA so both tools use identical NTv2/NADCON grids, exactly as Handling Missing NTv2 Grid Files in Production requires.
• Pin both in the environment lockfile; a silent GDAL bump can shift a whole batch’s coordinates.

Step 2: Run the pyproj in-process path Jump to heading#

The pyproj path reprojects inside Python with explicit transformer selection and full per-feature control — you can gate each geometry, branch on accuracy, and reject to a quarantine queue in the same loop.

# pyproj >= 3.6, geopandas >= 0.14 — Python 3.10+
import geopandas as gpd
from pyproj import CRS
from pyproj.transformer import TransformerGroup


def normalize_in_process(src_path: str, dst_epsg: int, tolerance_m: float) -> gpd.GeoDataFrame:
    gdf = gpd.read_file(src_path)
    group = TransformerGroup(CRS.from_user_input(gdf.crs), CRS.from_epsg(dst_epsg), always_xy=True)
    if not group.transformers:
        raise RuntimeError("no transformation path; check grid availability")
    best = group.transformers[0]
    accuracy = getattr(best, "accuracy", -1.0)
    if accuracy > tolerance_m:            # a decision ogr2ogr cannot express inline
        raise RuntimeError(f"best accuracy {accuracy} m exceeds tolerance {tolerance_m} m")
    return gdf.to_crs(epsg=dst_epsg)

In-process advantages:

• You choose the coordinate operation and reject out-of-tolerance paths before writing — impossible to express as a single CLI flag.
• Failures route to a structured quarantine queue in the same process, matching the step-by-step EPSG code normalization walkthrough.
• The full transform decision is a Python object you can assert on in tests.

Step 3: Run the ogr2ogr batch path Jump to heading#

The ogr2ogr path converts the whole layer in one streamed pass. It is dramatically faster for large files because it never holds the dataset in Python memory, and a pinned -ct coordinate operation makes the transform explicit.

# GDAL >= 3.8 — pinned coordinate operation for a deterministic, auditable batch
ogr2ogr \
  -t_srs "EPSG:4326" \
  -ct "+proj=pipeline +step +proj=hgridshift +grids=us_noaa_nadcon5_nad27_nad83_1986_conus.tif" \
  -f GPKG output.gpkg input.shp \
  --config OGR_ENABLE_PARTIAL_REPROJECTION YES

Batch advantages:

• Streams arbitrarily large layers without Python-side memory pressure — the decisive win for tile sets.
• A pinned -ct pipeline makes the exact operation explicit and reproducible, closing the “opaque flag” gap.
• Trivially parallelizable across files in a shell loop or the batch patterns in Batch Schema Processing Pipelines.

Step 4: Compare determinism and throughput Jump to heading#

Because both share PROJ, a pinned operation produces bit-identical coordinates — so the choice is throughput and control, not accuracy. Diff the outputs and measure wall-clock time on the same data.

# geopandas >= 0.14 — Python 3.10+
import numpy as np
import geopandas as gpd
from shapely import get_coordinates


def coordinates_match(a_path: str, b_path: str, atol: float = 1e-9) -> bool:
    """Confirm pyproj and ogr2ogr outputs agree when the same PROJ operation is pinned."""
    a = get_coordinates(gpd.read_file(a_path).geometry.to_numpy())
    b = get_coordinates(gpd.read_file(b_path).geometry.to_numpy())
    return a.shape == b.shape and np.allclose(a, b, atol=atol)

The decision matrix and indicative benchmark below summarize a run over one 2 GB shapefile (~5M features) on an 8-core ubuntu-latest-class runner. Treat the numbers as order-of-magnitude, not absolute — they shift with feature complexity, driver, disk, and PROJ version.

Recommendation: Reach for ogr2ogr when the workload is a high-volume, uniform-CRS conversion where throughput and flat memory dominate — large tile sets, nightly bulk reprojection, ETL where every feature takes the same path. Reach for pyproj when the transform decision is per-feature — mixed source CRS, tolerance gating, quarantine routing, or anything that must be unit-tested and audited in code. Many mature pipelines use both: pyproj to decide and pin the coordinate operation, then ogr2ogr with that pinned -ct to execute it at scale.

Step 5: Record the decision and lineage Jump to heading#

Whichever tool runs, record it so the output is reproducible and the choice is defensible.

# Python 3.10+
import json
from pathlib import Path


def log_tool_decision(path: Path, tool: str, proj_version: str, operation: str, throughput_fpm: float) -> None:
    record = {
        "tool": tool,                       # "pyproj" | "ogr2ogr"
        "proj_version": proj_version,
        "operation": operation,             # the pinned coordinate operation
        "throughput_features_per_min": round(throughput_fpm),
    }
    with path.open("a") as fh:
        fh.write(json.dumps(record) + "\n")

Pinning the PROJ version and operation name is the minimum an auditor needs; route these rows into the lineage store consumed by Geospatial Compliance Reporting & Audit Trails.

Verification Jump to heading#

Prove the two tools agree when the operation is pinned, so the comparison rests on control and speed alone:

# 1. Same PROJ backend under both front-ends.
import pyproj, subprocess
proj_py = pyproj.proj_version_str
proj_cli = subprocess.run(["projinfo", "--version"], capture_output=True, text=True).stdout
assert proj_py.split(".")[0] in proj_cli, "PROJ major version mismatch — comparison invalid"

# 2. Outputs are bit-identical with a pinned operation.
assert coordinates_match("out_pyproj.gpkg", "out_ogr2ogr.gpkg", atol=1e-9)

# 3. Both landed on the target CRS.
import geopandas as gpd
assert gpd.read_file("out_ogr2ogr.gpkg").crs.to_epsg() == 4326

On the CLI, ogrinfo -so out_ogr2ogr.gpkg layer | grep "Geometry Column" should report the expected SRS, and re-running with the pinned -ct on any machine sharing the PROJ version reproduces the coordinates exactly.

Troubleshooting Jump to heading#

Related Jump to heading#

• Projection Normalization Workflows — the parent stage that sets target-CRS policy for either tool
• Step-by-Step EPSG Code Normalization for Mixed Datasets — the per-feature control the pyproj path enables
• Handling Missing NTv2 Grid Files in Production — the shared grid backend both tools depend on
• Batch Schema Processing Pipelines — running the ogr2ogr batch path across large file sets
• Geospatial Compliance Reporting & Audit Trails — recording the tool, PROJ version, and operation as lineage