Sole developer · 2026

Leads Manager

A scraper and pipeline manager for an agency's outbound engine. The scraping was the easy half. The hard half was deciding when two listings from two directories are the same business, and which of them to believe.

Next.jsPlaywrightExpressMongoDB

View live demo

Role: Sole developer
Scope: Scraper service + pipeline portal
Shape: Two services, one MongoDB
Status: In daily use by the sales team

The problem

What was broken

The agency's outbound motion began with someone browsing business directories, copying phone numbers into a spreadsheet, and calling down the list. Follow-ups lived in whoever's memory made the call.

Duplicated sourcing

Sourcing was manual and uncoordinated — two people could work the same listings for a week without either knowing.

Calls with no memory

Once a call happened there was nowhere to record what came of it, so nothing had an owner or a next step. A prospect who said "call me in March" was one forgotten note away from never being called again.

One business, six listings

The same plumber appears on Yellow Pages, Google Maps, Yelp and half a dozen other directories, each with a slightly different version of the truth. Store them all and the team calls the same business six times; collapse them naively and the good data goes out with the duplicates.

Directories that fight back

Business directories do not want to be scraped in volume. A plain automated browser advertises itself in obvious ways and gets blocked quickly.

The product

What I built

Leads Manager is two services sharing a MongoDB database. The scraper is Express driving Playwright through business directories, extracting listings for category-and-city combinations and consolidating them through the identity and priority rules into master business records. The portal is Next.js 15 with route handlers — about 16,000 lines across 92 files — where the sales team actually works: the filterable business directory, call logging, colour triage, callbacks, tasks, users and groups, and notifications, behind JWT auth over HTTP-only cookies.

The portal grew well past the original brief, and that drift was correct. Sourcing leads was never the goal; the goal was a sales process that does not lose things, and the features that accumulated — callbacks with owners, tasks with statuses, managed category lists — are what that actually takes.

Capabilities

Key features

Automated lead sourcing

A Playwright scraper works through directory search results by category and city, designed to run unattended over hours, with every run stored as an inspectable record with a status.

One master record per business

Listings from every source consolidate into a single business record, so the sales team sees one plumber, not six copies of him.

The working directory

A filterable table of businesses — by category, city, call status, call date or free text — with colour tags for visual triage and click-to-dial phone numbers, because copying numbers into a phone forty times a day guarantees a mistyped one.

Call logging & appointments

Notes, answered/no-answer status and appointment state recorded against each business, so the pipeline has memory.

Callbacks as first-class records

"Call me in March" becomes a record with an owner and a date instead of a note in someone's head — the single most direct fix for lost follow-ups.

A small work tracker

Tasks with statuses, tags and comments, users and groups so work has an owner, notifications, and managed city and category lists so filters stay meaningful.

Under the hood

How it's engineered

Identity split from testimony

A Business is the master record, one per real business, keyed on phone number; a BusinessResult is what one source said about it. Every scrape writes a source record and never overwrites another source's version of the facts — the master keeps a full account of who told it what.

An explicit source hierarchy

Sources carry a priority — Google Maps outranks Yelp, which outranks the small directories, which outrank social pages — and a lower-priority source can never overwrite a higher one. The product's opinion about data quality lives in one readable map, so when addresses look wrong the conversation is about one table.

Phone number as the identity key

Deliberately imperfect: a changed number becomes two records, a shared reception line can collapse two businesses. But it's far more stable than names, more reliably formatted than addresses, and in this domain a duplicate call is a much cheaper mistake than a missed lead.

A browser that looks like a browser

Defeating directory blocking meant removing the automation tells — webdriver flags, missing runtime objects, empty plugin arrays — paced requests, and sessions retired before they wear out their welcome. Documented in the repo, because this knowledge rots.

Where it landed

Outcomes

The spreadsheet is gone. Sourcing runs unattended instead of consuming someone's week, the same listing no longer gets worked twice by two people, and every call leaves a record with an owner and a next step. When a prospect says "call me in March," March arrives with a callback record instead of relying on someone's memory.

Two improvements are written down for next. The source priority table encodes my prior about which directory is better maintained, but every source record already carries a data quality marker — a few months of records could derive priority from which source's data actually survives contact with a phone call. And the master record wants an explicit merge history rather than a last-updated-by field: when a record looks wrong today I can see which source owns it, but not the sequence that got it there, and that is the debugging information I have wanted most.

Build something like this