Systems · 9 min read
Website enrichment needs structure — not more scraping
A company website can answer useful GTM questions. But raw HTML, untraceable model summaries, and unbounded crawling turn a promising signal source into a data-quality problem.
Use public-web data as evidence. Define the field you need, extract it into a schema, keep its source and date, then decide whether it is strong enough to act on.
Why website data belongs in GTM
A historical HN launch described an API that turns public sites into structured outputs — such as company context, pricing, products, integrations, and screenshots. That is the useful abstraction for GTM engineering: not “scrape the web,” but retrieve specific public evidence for a specific decision.
A homepage, pricing page, careers page, or integration directory can help qualify accounts, enrich a CRM, route inbound, and prepare a relevant research brief. It cannot substitute for consent, first-party context, or human judgment.
Start with the decision
| Decision | Public-web field | Safe action |
|---|---|---|
| Should this account enter a motion? | Category, geography, product type, or target segment. | Apply a documented eligibility rule. |
| Which message angle is relevant? | Named integration, product capability, or use case. | Draft a source-linked hypothesis for review. |
| Who should own the inbound lead? | Pricing model, product line, or region. | Route with a fallback when confidence is low. |
| What should a rep know before a call? | Recent launch, customer proof, or stated priority. | Create a short brief with source URLs and dates. |
Use a schema, not an open-ended prompt
Ask for a bounded record with clear definitions. “Research this company” gives a model permission to improvise. A schema creates a reviewable contract.
| Field | Definition | Required evidence |
|---|---|---|
| product_category | The category stated or clearly supported by the site. | Source URL and quoted page fragment internally. |
| pricing_motion | Self-serve, sales-led, usage-based, or unknown. | Pricing or product page; otherwise return unknown. |
| named_integrations | Integrations explicitly listed by the company. | Integration or partner page. |
| research_confidence | Confidence in the extracted value, not in the account’s value. | Extraction rationale and source recency. |
| last_verified_at | When the page was retrieved. | Timestamp from the workflow. |
Build a bounded enrichment flow
- Check the account identity.
Use a canonical domain and prevent redirects, generic domains, and ambiguous matches from entering the workflow.
- Fetch only the pages you need.
Start with an allowlist: homepage, pricing, product, integrations, careers, or a known campaign page.
- Extract into the schema.
Return unknown when the evidence is missing. Do not translate absence into a negative claim.
- Validate before activation.
Review low-confidence results, material claims, and high-value accounts before they reach a rep or outbound tool.
- Store provenance.
Keep the URL, date, extraction version, and any reviewer correction alongside the field.
Respect the web and your own data model
Rate limits, caching, robots guidance, opt-outs, and terms of use are product requirements — not implementation details. So are overwrite rules. Public-web enrichment should not silently replace a recent first-party fact or a verified customer-provided answer.
The best implementation does less than a general-purpose crawler. It collects evidence that directly improves a defined GTM decision, then stops.