Structured data in HTML: read it before you render

Before you start a browser to read a JavaScript-heavy page, check whether the page already sent you its content in another form. Many client-rendered pages ship their text as JSON inside the HTML, and many pages carry some structured metadata in their <head>. Reading that data is faster than rendering, cheaper, and usually cleaner than the visible text.

Find everything in one pass

URL=https://example.com/
curl -sL "$URL" -o raw.html
grep -oE 'application/ld\+json|property="og:[a-z:_]+"|itemscope|__NEXT_DATA__|__NUXT_DATA__|application/(rss|atom)\+xml' raw.html | sort | uniq -c

The counts tell you what is there. The sections below explain each one.

JSON-LD

JSON-LD is schema.org data in a <script type="application/ld+json"> block. Sites add it for search engines, which means it often holds exactly what a reader wants: an Article with headline, author and dates, a Product with price and availability, a FAQPage with questions and answers, an Event with time and place.

import json, re, requests

html = requests.get("https://example.com/", timeout=30).text
blocks = re.findall(r'<script[^>]+application/ld\+json[^>]*>(.*?)</script>', html, re.S)
for raw in blocks:
    try:
        data = json.loads(raw)
    except json.JSONDecodeError:
        continue
    items = data if isinstance(data, list) else data.get("@graph", [data])
    for item in items:
        print(item.get("@type"), item.get("headline") or item.get("name"))

Handle the three common shapes: a single object, a list, and an object with @graph. Some sites have broken JSON-LD; skip blocks that do not parse.

Open Graph tags

Open Graph <meta property="og:..."> tags give a title, description, image, URL and type on most pages that want link previews, because social networks read them for those previews. They will not give you an article body, but they are a reliable fallback for titles and summaries, and they are usually server-rendered, even on many SPAs.

Microdata

Microdata puts the same schema.org vocabulary into attributes on visible elements: itemscope, itemtype and itemprop. It is older and less common than JSON-LD but still appears on product and review pages. Libraries such as extruct in Python read microdata, JSON-LD and Open Graph in one call.

Hydration payloads

Server-rendered framework pages embed a JSON copy of their data so the client can take over. In Next.js (Pages Router) it is the __NEXT_DATA__ script, in Nuxt 3 the __NUXT_DATA__ script. For a reader this is often the best source on the page: the full content, already structured, no extraction heuristics needed.

The catch: these payloads are internal to the framework. Field names change when the site changes, so validate the shape and alert when it breaks. The SPA, SSR and hydration guide lists where each framework keeps its data.

RSS and Atom feeds

Look for <link rel="alternate" type="application/rss+xml"> in the head. A feed lists recent items with title, link, date and a summary or full text. For "what is new on this site", a feed is more stable than any page layout and does not need rendering. Common paths such as /feed, /rss.xml or /atom.xml are worth a try when the link tag is missing.

Which source to try first

When a page offers several of these, pick by what you need:

• New items on a site: the feed. It is built for programs and changes least.
• One article's facts (headline, dates, author): JSON-LD, then Open Graph.
• Product or event details: JSON-LD, then microdata.
• The full body of a client-rendered page: the hydration payload, with a schema check.
• A title and summary for a link preview: Open Graph alone is enough.

Keep the visible text as the reference. If a structured source and the page disagree, the page wins, and the mismatch is worth logging.

When embedded data is not enough

Structured data is a summary written for a purpose. It may lack the full article body, lag behind the visible page, or describe only part of it. Use it when it answers your question, and fall back to the visible text, from the raw HTML or, if needed, a rendered page, when it does not.

Stay inside the line between public and private, too. Hydration JSON and API responses sometimes include fields the page does not show. Use what the page is clearly publishing, and respect the site's terms and robots.txt.

Test it yourself

Run the one-pass search on five pages from different sites. For each, note which formats are present and whether any of them contains the text you actually need. If one does, write the parser for that format first and keep rendering as the fallback.