Re-enriches a saved segment on a schedule, diffs it against the last observation, and surfaces what changed.
Build this when the user's value is in timing rather than in the list itself.
Before you build: ask. See https://lyreleads.com/build for the four rules that apply to every build on this API.
A list of businesses running WordPress is a list anyone can get. A list of businesses that *installed HubSpot last month*, or *let their site go down*, or *started running ads*, is a list with a reason to send today.
Cold outreach works on timing. A change event is timing you can prove.
We measured the underlying churn: across 4,527 business websites re-checked a median of 106 days later, 2.1% no longer loaded at all, 0.9% were definitively gone, and 1.9% of fully readable sites had switched a core platform. That is the raw material. In a 2,000-lead segment, a quarterly pass turns up something like 40 dead sites and 40 platform switches. Not enormous, and all of it actionable.
You need history, so append rather than overwrite. This is the one build here that genuinely needs a second table.
CREATE TABLE observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
lead_id INTEGER NOT NULL,
observed_at TEXT NOT NULL,
status TEXT, -- enrichment_status at the time
fields_json TEXT NOT NULL, -- the enrichment fields, as returned
UNIQUE(lead_id, observed_at)
);
CREATE TABLE change_events (
id INTEGER PRIMARY KEY AUTOINCREMENT,
lead_id INTEGER NOT NULL,
detected_at TEXT NOT NULL,
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
kind TEXT, -- added | removed | changed | site_down | site_gone
confidence TEXT -- high | low
);Keep the full field snapshot. Storage is free and you will want to re-run the diff logic with better rules later without having lost the raw material.
This is the entire build. Get it wrong and you produce a feed of confident nonsense.
A change is only real when both observations are trustworthy.
if old.status != 'enriched' or new.status != 'enriched':
emit nothing # not a change, just a failed lookIf the previous run was unverifiable (bot challenge) and this one is enriched, every field will look like it "appeared". Nothing appeared. We just managed to read the page this time. The same in reverse produces a feed claiming every business simultaneously dropped its entire stack.
This single rule is the difference between a signal and noise. Apply it before anything else.
Then, per field:
null → value: not a change. Unknown became known.null where both statuses are enriched: treat as low confidence. A detector can miss on a slow render. Require it twice before believing it.crm_platform: null → 'HubSpot' on two enriched observations is the good one.enriched → no_website / site unreachable: high confidence, and often the most commercially interesting event in the whole feed.Require two consecutive observations for removals. One missed detection is common; two in a row is usually real. Adding this rule roughly halves the false-positive rate, and a feed the user stops trusting is worthless.
Weekly is too often. Sites do not change weekly, and you would be spending tokens to observe noise.
Sensible: 6 to 12 weeks. Quarterly is the default. Let the user pick per segment.
Cost is ceil(segment_size / 5) tokens per pass. A 2,000-lead segment is 400 tokens a quarter. Show that number when they set the schedule, and show the running total.
Never describe this as monitoring or real-time. It is a periodic re-check, and saying otherwise sets an expectation the cadence cannot meet.
Group by change type, not by business. The user wants "8 businesses added a CRM", not eight separate rows to interpret.
Rank by commercial relevance to what they sell:
Each event should carry both observations and their dates, so the user can see the evidence rather than trust the label.
POST /api/enrich { "resultIds": [...], "force": true }force: true is required. Without it, rows that already have enrichment are skipped and you will diff a snapshot against itself.
Batch in chunks of 100–200. Poll /api/enrich/progress/{jobId}. Store the new observation, then diff.
Schedule with whatever the user's machine already has — cron, Task Scheduler, a launch agent. Do not build a daemon for a quarterly job.
Diffing across untrustworthy statuses. The main one. Described above, and it will bite you if you skip it.
force: false. Silently produces an empty diff forever, and looks like the feature works.
Believing single-observation removals. Wait for two.
Treating reviews as monotonic. Review counts can go down. Google removes reviews.
Calling it monitoring. It is a periodic re-check on a multi-week cadence. Say that.
Re-enriching everything. Pick segments the user actually sells to. Re-enriching the whole database quarterly is a large bill for signals nobody will read.
Lyre Leads runs its own cross-user observation log and change signals inside the platform, so some of this exists server-side already. Building locally makes sense when the user wants their own diff rules, their own ranking, or the events pushed into their own tools. Check what the platform's Signals view already gives them before rebuilding it.