Geospatial Schema Standardization & Mapping Automation

Emit schema.org Dataset JSON-LD for Open Data Portals

Google Dataset Search and most modern open-data catalogues discover a dataset by reading a schema.org/Dataset block embedded in its landing page. Get that block right and the dataset appears in dataset-specific search within days; get it wrong — a swapped bounding-box axis, a prose license, a missing distribution — and the dataset is silently skipped, with no error to tell you why. This procedure builds a valid, harvestable Dataset document step by step. It is the concrete build behind Dataset Metadata & JSON-LD Publishing, which owns the metadata contract and the DCAT-AP half; this page owns the schema.org half and the embedding.

The steps map to the configure → execute → validate → log rhythm the rest of the site uses: assemble the core fields, attach coverage, declare distributions, serialize and embed, then verify. Each step is a focused block you can drop into an existing publishing pipeline rather than a monolithic script.

Prerequisites checklist

Confirm the environment and inputs before building anything. The most common cause of a dataset that never appears in search is a bounding box in the wrong CRS, so verify coverage extraction against a known point before trusting a batch.

Step 1: Assemble the required core fields

Google’s Dataset guidance treats name, description, and at least one of identifier/sameAs as the minimum viable record, with license, creator, and distribution strongly recommended for geospatial data. Build these first as a plain dictionary, keeping the @context as the string "https://schema.org" so the block is a self-contained schema.org document.

python
# build_dataset.py  — Python 3.10+, standard-library json only
from __future__ import annotations
import json
from typing import Any


def core_dataset(
    identifier: str,
    name: str,
    description: str,
    license_uri: str,
    creator_name: str,
    creator_uri: str,
) -> dict[str, Any]:
    """The minimum harvestable schema.org Dataset skeleton."""
    if not license_uri.startswith(("http://", "https://")):
        raise ValueError(f"license must be a URI, got prose: {license_uri!r}")
    return {
        "@context": "https://schema.org",
        "@type": "Dataset",
        "@id": identifier,
        "identifier": identifier,
        "name": name,
        "description": description.strip(),
        "license": license_uri,
        "creator": {
            "@type": "Organization",
            "@id": creator_uri,
            "name": creator_name,
        },
    }

The guard on license_uri matters: a prose license is the most common reason a structurally valid document still fails to enrich a search result. Reject it at build time rather than discovering the omission after harvest.

Step 2: Add GeoShape spatial and temporal coverage

Spatial coverage is what makes a geospatial dataset discoverable by location. Attach it as a Place with a GeoShape, and remember the schema.org box ordinate order is south west north east — latitude first. Extract the box in EPSG:4326; a box left in the dataset’s projected CRS points somewhere in the ocean off West Africa to a harvester that assumes lon/lat.

python
# build_dataset.py (continued)  — geopandas >=0.14, pyproj >=3.6
import geopandas as gpd
from pyproj import CRS, Transformer


def add_coverage(
    node: dict[str, Any],
    dataset_path: str,
    time_start: str | None = None,
    time_end: str | None = None,
) -> dict[str, Any]:
    """Attach spatialCoverage (GeoShape, EPSG:4326) and temporalCoverage."""
    gdf = gpd.read_file(dataset_path)
    if gdf.crs is None:
        raise ValueError("dataset has no CRS; cannot assert spatial coverage")

    minx, miny, maxx, maxy = gdf.total_bounds
    src = CRS.from_user_input(gdf.crs)
    if src.to_epsg() != 4326:
        tf = Transformer.from_crs(src, CRS.from_epsg(4326), always_xy=True)
        minx, miny = tf.transform(minx, miny)
        maxx, maxy = tf.transform(maxx, maxy)

    south, west = round(miny, 6), round(minx, 6)
    north, east = round(maxy, 6), round(maxx, 6)
    node["spatialCoverage"] = {
        "@type": "Place",
        "geo": {
            "@type": "GeoShape",
            "box": f"{south} {west} {north} {east}",  # S W N E, lat first
        },
    }
    if time_start and time_end:
        # ISO 8601 interval; open-ended intervals use "start/.." if needed.
        node["temporalCoverage"] = f"{time_start}/{time_end}"
    return node

The always_xy=True argument enforces (longitude, latitude) ordering out of the transformer so the assignment of west/south from minx/miny is correct. This is the same axis-order discipline that governs every reprojection upstream; getting it wrong here transposes the box even though the code “runs.” The tolerance rules behind the six-decimal rounding are standardized in Unit Conversion & Tolerance Thresholds.

Step 3: Declare DataDownload distributions and catalog

A Dataset with no distribution is metadata about a file nobody can fetch; portals de-prioritize or skip it. List every downloadable representation as a DataDownload with a contentUrl and encodingFormat, and link the dataset to the catalog it belongs to with includedInDataCatalog so harvesters can walk from record to catalog.

python
# build_dataset.py (continued)  — Python 3.10+
def add_distributions(
    node: dict[str, Any],
    distributions: list[dict[str, str]],
    catalog_name: str,
    catalog_uri: str,
) -> dict[str, Any]:
    """Attach DataDownload distributions and the parent catalog."""
    if not distributions:
        raise ValueError("a Dataset must expose at least one distribution")
    node["distribution"] = [
        {
            "@type": "DataDownload",
            "contentUrl": d["access_url"],
            "encodingFormat": d["media_type"],
            **({"name": d["name"]} if d.get("name") else {}),
        }
        for d in distributions
    ]
    node["includedInDataCatalog"] = {
        "@type": "DataCatalog",
        "@id": catalog_uri,
        "name": catalog_name,
    }
    return node

Use precise encodingFormat media types — application/geopackage+sqlite3 for GeoPackage, application/vnd.apache.parquet for GeoParquet, application/geo+json for GeoJSON. A generic application/octet-stream tells a harvester nothing about how to preview or index the file.

Step 4: Serialize and embed in the landing page

Serialize the assembled node deterministically, then embed it in the dataset landing page inside a <script type="application/ld+json"> tag. Server-side embedding is required: Google Dataset Search does not execute page JavaScript reliably, so a document injected by client-side script may never be seen.

python
# emit.py  — Python 3.10+, standard-library json only
import json
from pathlib import Path


def serialize(node: dict, out_path: Path) -> str:
    """Deterministic serialization; identical input yields identical bytes."""
    text = json.dumps(node, indent=2, sort_keys=True, ensure_ascii=False)
    out_path.write_text(text + "\n", encoding="utf-8")
    return text


def embed_in_page(template_html: str, jsonld_text: str) -> str:
    """Inject the JSON-LD into the landing-page <head>, server-side."""
    block = (
        '<script type="application/ld+json">\n'
        f"{jsonld_text}\n"
        "</script>"
    )
    if "</head>" not in template_html:
        raise ValueError("landing-page template has no </head> to inject before")
    return template_html.replace("</head>", f"{block}\n</head>", 1)

Publish the same serialized document a second time as a standalone .jsonld file at a stable URL. CKAN’s harvesters and DCAT-AP catalogues fetch that file directly, and your CI validates it, while the embedded copy earns Dataset Search discovery. One document, two publication targets, no divergence.

Step 5: Log the publication event

Record the emission to the audit trail so the exact bytes harvested on any date are provable later. This closes the loop with the compliance role of the parent practice area.

python
# log_publication.py  — Python 3.10+
import datetime
import hashlib
import json
from pathlib import Path


def log_emission(audit_path: Path, identifier: str, jsonld_text: str) -> None:
    record = {
        "identifier": identifier,
        "doc_sha256": hashlib.sha256(jsonld_text.encode("utf-8")).hexdigest(),
        "published_utc": datetime.datetime.now(datetime.UTC).isoformat(),
        "outcome": "PUBLISHED",
    }
    with audit_path.open("a") as fh:
        fh.write(json.dumps(record) + "\n")

The doc_sha256 is what lets an auditor match the harvested metadata to the pipeline output byte-for-byte. Retain these records alongside the dataset’s lineage manifest for the full retention window.

Verification

Do not trust a zero exit code. Expand the JSON-LD to confirm every field resolves, and assert the coverage box is well-formed and in EPSG:4326 bounds before submitting the page for harvest.

python
# verify.py  — pyld >=2.0
import json
from pyld import jsonld

node = json.loads(open("dist/parcels-2024.jsonld").read())

# 1. Expansion resolves the @context and every term; a typo raises here.
expanded = jsonld.expand(node)
assert expanded, "expansion produced an empty graph"

# 2. Required fields are present.
for field in ("name", "description", "license", "distribution"):
    assert field in node, f"missing required field: {field}"

# 3. The GeoShape box is S W N E and inside EPSG:4326 bounds.
south, west, north, east = map(float, node["spatialCoverage"]["geo"]["box"].split())
assert south <= north and west <= east, "box ordinates transposed"
assert -90 <= south <= 90 and -180 <= west <= 180, "box not in EPSG:4326"

# 4. Every distribution has a fetchable URL and a specific media type.
for d in node["distribution"]:
    assert d["contentUrl"].startswith("http"), "distribution URL not absolute"
    assert d["encodingFormat"] != "application/octet-stream", "media type too generic"
print("schema.org Dataset JSON-LD is valid and harvestable")

On the published page, a structured-data test against the schema.org Dataset specification and the Dataset Search coverage report will report the record as eligible once it is crawled. A successful run prints the confirmation line and produces a standalone document whose SHA-256 matches the audit record from Step 5.

Troubleshooting

Symptom Likely cause Fix
Dataset never appears in Dataset Search JSON-LD injected client-side; crawler never executed it Embed server-side before </head>; re-run Step 4 and re-request indexing.
Location facet points to the wrong place GeoShape.box built as west south or left in projected CRS Rebuild in EPSG:4326 with always_xy=True; assert S W N E order in verification.
License enrichment missing in results license is prose, not a URI Replace with a resolvable license URI; the Step 1 guard now rejects prose.
“Missing field distribution” warning Dataset node has no distribution array Add at least one DataDownload with contentUrl and encodingFormat (Step 3).
Temporal filter never matches temporalCoverage not an ISO 8601 interval Emit start/end in YYYY-MM-DD form; confirm start <= end.
Expansion raises jsonld.ContextUrlError Typo in @context or an unreachable custom namespace Use the plain string "https://schema.org"; avoid hand-editing the context.