How to Build an Automated B2B Lead Generation Pipeline: Scraping Google Maps, Company Websites & GitHub Contributors with Node.js & Python
Building a reliable B2B lead generation pipeline in 2026 requires more than scraping a single directory. Local business directories and Google Maps listings give you business names, categories, street addresses, ratings, and phone numbers — but they almost never expose verified decision-maker email addresses or LinkedIn / X (Twitter) profiles directly on the listing card. Similarly, when selling developer tools (DevTools), SaaS infrastructure, or technical recruiting services, your highest-converting prospects aren't sitting on generic business directories — they are actively shipping code on GitHub.
In this deep-dive engineering guide, we will build a complete, multi-source B2B Lead Generation & Contact Enrichment Pipeline using Node.js (@crawlee/cheerio) and Python (requests, pandas). We will cover:
1. Stage 1 — Local Business & Google Maps Lead Extraction: Parsing application/ld+json (LocalBusiness, Organization, ProfessionalService), microdata, phone numbers, and aggregate ratings while automatically following official company domains (/contact, /about, /locations) to uncover hidden business emails.
2. Stage 2 — Technical & DevTools Lead Extraction from GitHub: Extracting open-source contributors, repository owners, public emails, personal blogs, X/Twitter handles, and company affiliations without hitting GitHub's strict 60-requests-per-hour unauthenticated REST rate limit.
3. Stage 3 — Lead Deduplication, Email Verification & CRM Export: Filtering out false-positive emails (sentry.io, wixpress.com, asset filenames), normalizing phone numbers to E.164-compatible formats, and exporting clean CSV/Excel datasets.
If you want to skip hosting your own crawlers and call these pipelines immediately via REST API, you can run our production Actors directly on the Apify Store:
- 🌍 Google Maps & Local Business Lead, Email, Phone & Social Extractor
- 💻 GitHub Developer, Contributor & Stargazer B2B Lead Scraper
- 🏢 B2B Website Email, Phone & Social Media Contact Scraper
---
Architecture Overview: Why Multi-Hop Enrichment Wins
Most off-the-shelf scrapers fail on B2B lead generation for one of two reasons:
1. Single-page blindness: They only scrape the landing URL you pass in. If a company puts their hello@company.com or sales@company.com address on /contact-us or /impressum, a single-page scraper returns emails: [].
2. Regex noise: Naive regular expressions match CSS bundles (bundle@2x.png), error-tracking DSNs (12345@o1.ingest.sentry.io), or placeholder strings (user@example.com), polluting your cold outreach sequence and hurting domain deliverability.
Here is the multi-hop enrichment architecture we use in production:
[Input URLs: Google Maps / Directories / Company Domains / GitHub Repos]
├──> Layer 1: JSON-LD Parser (LocalBusiness, Organization, AggregateRating)
├──> Layer 2: Same-Domain Subpage Crawler (/contact, /about, /locations)
└──> Layer 3: Sanitization & Lead Scoring (Filter telemetry, export CSV/JSON)
---
Stage 1: Extracting Local Business Leads, Emails & Phones in Node.js
When crawling local business websites discovered from Google Maps or regional directories, roughly 68% of SMB websites embed rich application/ld+json blocks generated by WordPress (Yoast / RankMath), Squarespace, Shopify, or Wix. These JSON-LD blocks often contain the exact street address, postal code, direct telephone number, and aggregate customer rating in clean machine-readable JSON.
Below is the production Node.js handler using @crawlee/cheerio that combines JSON-LD schema extraction with automatic /contact subpage crawling:
import { CheerioCrawler, createCheerioRouter } from '@crawlee/cheerio';
const EMAIL_REGEX = /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g;
const PHONE_REGEX = /(?:\+?\d{1,3}[\s.-]?)?\(?\d{3}\)?[\s.-]?\d{3}[\s.-]?\d{4}/g;
const BAD_EMAIL_EXT = /\.(png|jpg|jpeg|gif|svg|webp|ico|css|js|woff|woff2|ttf|map)$/i;
const BLOCKED_DOMAINS = ['example.com', 'sentry.io', 'wixpress.com', 'schema.org', 'domain.com'];
export function parseJsonLdBusinesses($) {
const businesses = [];
$('script[type="application/ld+json"]').each((_, el) => {
try {
const raw = $(el).contents().text().trim();
if (!raw) return;
const parsed = JSON.parse(raw);
const items = Array.isArray(parsed) ? parsed : (parsed['@graph'] || [parsed]);
for (const item of items) {
if (item && typeof item === 'object') {
businesses.push(item);
}
}
} catch {
// Ignore malformed JSON-LD blocks on broken templates
}
});
return businesses;
}
export const router = createCheerioRouter();
router.addDefaultHandler(async ({ enqueueLinks, request, $, body, log }) => {
const loadedUrl = request.loadedUrl || request.url;
const domain = new URL(loadedUrl).hostname.replace(/^www\./, '');
const html = typeof body === 'string' ? body : body.toString('utf-8');
// Automatically follow high-value contact & location pages on the same domain
if (!request.userData.isSubPage) {
await enqueueLinks({
strategy: 'same-hostname',
globs: ['**/contact*', '**/about*', '**/locations*', '**/impressum*'],
userData: { isSubPage: true },
});
}
const emailSet = new Set();
const phoneSet = new Set();
// 1. Extract explicit mailto: and tel: links
$('a[href^="mailto:"]').each((_, el) => {
const email = ($(el).attr('href') || '')
.replace(/^mailto:/i, '')
.split('?')[0]
.trim()
.toLowerCase();
if (email.includes('@') && !BAD_EMAIL_EXT.test(email)) {
emailSet.add(email);
}
});
$('a[href^="tel:"]').each((_, el) => {
const phone = ($(el).attr('href') || '').replace(/^tel:/i, '').trim();
if (phone.length >= 7) phoneSet.add(phone);
});
// 2. Scan raw HTML for inline emails with strict false-positive filtering
for (const match of html.matchAll(EMAIL_REGEX)) {
const candidate = match[0].toLowerCase();
const isBlocked = BLOCKED_DOMAINS.some((d) => candidate.includes(d));
if (!BAD_EMAIL_EXT.test(candidate) && !isBlocked) {
emailSet.add(candidate);
}
}
// 3. Scan visible body text for phone numbers (10 to 15 digits)
for (const match of ($('body').text() || '').matchAll(PHONE_REGEX)) {
const rawPhone = match[0].trim();
const digitCount = rawPhone.replace(/\D/g, '').length;
if (digitCount >= 10 && digitCount <= 15) {
phoneSet.add(rawPhone);
}
}
// 4. Enrich from Schema.org LocalBusiness / Organization JSON-LD
const ldItems = parseJsonLdBusinesses($);
let businessName = $('meta[property="og:site_name"]').attr('content') ||
$('h1').first().text().trim() ||
$('title').first().text().trim() ||
domain;
let address = $('[itemprop="address"], address').first().text().replace(/\s+/g, ' ').trim() || null;
let rating = null;
let reviewCount = null;
let category = null;
for (const ld of ldItems) {
if (ld.name && businessName === domain) businessName = ld.name;
if (ld.telephone) phoneSet.add(String(ld.telephone));
if (ld.email) emailSet.add(String(ld.email).replace(/^mailto:/i, '').toLowerCase());
if (ld['@type'] && !category) {
category = Array.isArray(ld['@type']) ? ld['@type'].join(', ') : String(ld['@type']);
}
if (ld.address && typeof ld.address === 'object') {
const addrParts = [
ld.address.streetAddress,
ld.address.addressLocality,
ld.address.addressRegion,
ld.address.postalCode,
ld.address.addressCountry,
].filter(Boolean);
if (addrParts.length > 0) address = addrParts.join(', ');
}
if (ld.aggregateRating && typeof ld.aggregateRating === 'object') {
rating = ld.aggregateRating.ratingValue ?? rating;
reviewCount = ld.aggregateRating.reviewCount ?? ld.aggregateRating.ratingCount ?? reviewCount;
}
}
// 5. Extract Social Media Profiles
const findSocial = (regex) => {
let matched = null;
$('a[href]').each((_, el) => {
const href = $(el).attr('href') || '';
if (!matched && regex.test(href)) matched = href;
});
return matched;
};
const record = {
businessName: String(businessName).replace(/\s+/g, ' ').trim(),
category: category || 'LocalBusiness',
domain,
sourceUrl: loadedUrl,
address,
rating,
reviewCount,
emails: Array.from(emailSet),
phones: Array.from(phoneSet).slice(0, 10),
linkedin: findSocial(/linkedin\.com\/(company|in)\//i),
facebook: findSocial(/facebook\.com\/(?!sharer)/i),
instagram: findSocial(/instagram\.com\//i),
twitter: findSocial(/(twitter|x)\.com\/(?!intent)/i),
tiktok: findSocial(/tiktok\.com\/@/i),
scrapedAt: new Date().toISOString(),
};
log.info(`[Lead Extracted] ${record.businessName} -> ${record.emails.length} emails, ${record.phones.length} phones`);
});
Calling the Deployed Google Maps & Local Business Lead Extractor via Python
Instead of maintaining Node.js infrastructure and proxy pools yourself, you can invoke the cloud Actor tungnguyen84/google-maps-lead-email-phone-extractor directly from Python:
import requests
import pandas as pd
APIFY_TOKEN = "YOUR_APIFY_API_TOKEN"
ACTOR_URL = (
"https://api.apify.com/v2/acts/"
"tungnguyen84~google-maps-lead-email-phone-extractor/"
f"run-sync-get-dataset-items?token={APIFY_TOKEN}"
)
payload = {
"startUrls": [
{"url": "https://apify.com/contact"},
{"url": "https://crawlee.dev"}
],
"enrichCompanyWebsites": True,
"maxBusinesses": 50
}
response = requests.post(ACTOR_URL, json=payload, timeout=180)
response.raise_for_status()
leads = response.json()
df = pd.DataFrame(leads)
print(df[["businessName", "domain", "emails", "phones", "linkedin"]])
df.to_csv("enriched_b2b_leads.csv", index=False)
---
Stage 2: Scraping GitHub Developers & Contributors for DevTools B2B Outreach
If you are recruiting senior engineers or selling developer-focused SaaS (CI/CD, observability, AI coding assistants, cloud databases, security scanners), scraping generic business directories won't help. You need active open-source contributors and maintainers from relevant GitHub repositories.
However, GitHub's official REST API limits unauthenticated requests to just 60 requests per hour, which breaks almost immediately when enriching a repository with 200 contributors.
By parsing GitHub's semantic HTML microformats (span.p-name, div.p-note, span.p-org, span.p-label, li[itemprop="url"]) with @crawlee/cheerio, we can extract contributor profiles, company affiliations, personal websites, X/Twitter handles, and public emails at high speed without burning API quota:
// Extracting developer lead data from a GitHub profile DOM
const username = new URL(loadedUrl).pathname.split('/').filter(Boolean)[0];
const fullName = $('span.p-name, [itemprop="name"]').first().text().trim() || username;
const bio = $('div.p-note, [data-bio-text]').first().text().trim() || '';
const company = $('span.p-org, [itemprop="worksFor"]').first().text().trim() || null;
const location = $('span.p-label, [itemprop="homeLocation"]').first().text().trim() || null;
const website = $('li[itemprop="url"] a, a.Link--primary[rel*="me"]').first().attr('href') || null;
const followers = $('a[href$="?tab=followers"] span.text-bold').first().text().trim() || null;
Running the GitHub Developer & Contributor Scraper via cURL or Python
Our live Actor GitHub Developer, Contributor & Stargazer B2B Lead Scraper accepts both GitHub repository URLs (automatically discovering and enqueuing contributor profiles) and direct developer profile URLs:
curl -X POST "https://api.apify.com/v2/acts/tungnguyen84~github-developer-contributor-lead-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"targetUrls": [
{ "url": "https://github.com/tungnguyen84" },
{ "url": "https://github.com/apify/crawlee" }
],
"maxProfiles": 25
}'
---
Stage 3: Deduplicating & Scoring Multi-Page Leads in Python
When your crawler visits both https://company.com and https://company.com/contact, it outputs one record per page. Before importing into HubSpot, Pipedrive, or Apollo, you should merge records by domain so each company occupies a single row with all discovered emails, phones, and social links combined:
from collections import defaultdict
import json
def merge_leads_by_domain(raw_records: list[dict]) -> list[dict]:
grouped = {}
for rec in raw_records:
domain = rec.get("domain") or rec.get("username") or rec.get("sourceUrl")
if not domain:
continue
if domain not in grouped:
grouped[domain] = {
"businessName": rec.get("businessName") or rec.get("fullName"),
"domain": domain,
"website": rec.get("website") or rec.get("sourceUrl"),
"address": rec.get("address") or rec.get("location"),
"rating": rec.get("rating"),
"emails": set(rec.get("emails") or []),
"phones": set(rec.get("phones") or []),
"linkedin": rec.get("linkedin"),
"twitter": rec.get("twitter"),
"facebook": rec.get("facebook"),
"instagram": rec.get("instagram"),
}
else:
entry = grouped[domain]
entry["emails"].update(rec.get("emails") or [])
entry["phones"].update(rec.get("phones") or [])
for field in ("address", "rating", "linkedin", "twitter", "facebook", "instagram"):
if not entry.get(field) and rec.get(field):
entry[field] = rec[field]
merged = []
for domain, data in grouped.items():
data["emails"] = sorted(data["emails"])
data["phones"] = sorted(data["phones"])
data["leadScore"] = (
(50 if data["emails"] else 0)
+ (25 if data["phones"] else 0)
+ (15 if data["linkedin"] else 0)
+ (10 if data["address"] else 0)
)
merged.append(data)
return sorted(merged, key=lambda x: x["leadScore"], reverse=True)
---
🌐 Try Our Live Apify Actors, RapidAPI Endpoints & Standalone Tools
Whether you prefer serverless cloud APIs or standalone offline Python + Excel VBA tools, all of our scrapers are live and ready to test:
1. Apify Store Actors (Pay-Per-Result, No Monthly Commitment)
- 🌍 Google Maps & Local Business Lead, Email, Phone & Social Extractor
- 💻 GitHub Developer, Contributor & Stargazer B2B Lead Scraper
- 🏢 B2B Website Email, Phone & Social Media Contact Scraper
- 🎬 YouTube Video Transcript, Metadata & Channel Email Scraper
- 💬 Reddit Subreddit, Search & Thread to Clean JSON & AI Markdown Scraper
- 📄 Website to Clean Markdown & Text Scraper for AI, LLM & RAG
- 🛍️ Shopify Store Products, Prices & B2B Contact Email Scraper
- 🛒 WooCommerce & WordPress Store Product, Price & Contact Scraper
- 🔍 Website SEO Audit, Meta Tags, Headings & Contact Email Scraper
2. RapidAPI Hub REST APIs (Free Tier BASIC $0.00/mo)
- B2B Website Email, Phone & Social Scraper API
- Website to Clean Markdown AI, LLM & RAG Converter API
- Shopify Store Product, Price & Contact Scraper API
- WooCommerce & WordPress Product, Price & Contact Spy API
- Website Sitemap, SEO Audit & Contact Scraper API
3. Instant Download Full Source Code & Excel VBA Suites (Payhip / Gumroad)
- 📦 Python B2B Lead Scraper & Excel VBA Suite Pro ($14.99)
- 📦 Shopify Competitor Price & Product Spy Pro ($12.99)
- 📦 Ultimate Excel VBA Automation Pack: 10 One-Click Macros ($9.99)
Need a custom web scraper, anti-bot bypass pipeline, or automated CRM integration built for your team? Message me on Fiverr (@tungnguyen84)!
Mentions
How to Build an Automated B2B Lead Generation Pipeline: Scraping Google Maps, Company Websites & GitHub Contributors with Node.js & Python Building a reliable B2B lead generation pipeline in 2026 requires more than scraping a single directory. Local business directories and Google Maps listings give you business names, categories, street addresses, ratings, and phone numbers — but they almost never expose verified decision-maker email addresses or LinkedIn / X (Twitter) profiles directly on the listing card. Similarly, when selling developer tools (DevTools), SaaS infrastructure, or technical recruiting services, your highest-converting prospects aren't sitting on generic business directories — they are actively shipping code on GitHub. In this deep-dive engineering guide, we will build a complete, multi-source B2B Lead Generation & Contact Enrichment Pipeline using Node.js (@crawlee/cheerio) and Python (requests, pandas). We will cover: 1. Stage 1 — Local Business & Google Maps Lead Extraction: Parsing application/ld+json (LocalBusiness, Organization, ProfessionalService), microdata, phone numbers, and aggregate ratings while automatically following official company domains (/contact, /about, /locations) to uncover hidden business emails. 2. Stage 2 — Technical & DevTools Lead Extraction from GitHub: Extracting open-source contributors, repository owners, public emails, personal blogs, X/Twitter handles, and company affiliations without hitting GitHub's strict 60-requests-per-hour unauthenticated REST rate limit. 3. Stage 3 — Lead Deduplication, Email Verification & CRM Export: Filtering out false-positive emails (sentry.io, wixpress.com, asset filenames), normalizing phone numbers to E.164-compatible formats, and exporting clean CSV/Excel datasets. If you want to skip hosting your own crawlers and call these pipelines immediately via REST API, you can run our production Actors directly on the Apify Store: - 🌍 Google Maps & Local Business Lead, Email, Phone & Social Extractor - 💻 GitHub Developer, Contributor & Stargazer B2B Lead Scraper - 🏢 B2B Website Email, Phone & Social Media Contact Scraper --- Architecture Overview: Why Multi-Hop Enrichment Wins Most off-the-shelf scrapers fail on B2B lead generation for one of two reasons: 1. Single-page blindness: They only scrape the landing URL you pass in. If a company puts their hello@company.com or sales@company.com address on /contact-us or /impressum, a single-page scraper returns emails: []. 2. Regex noise: Naive regular expressions match CSS bundles (bundle@2x.png), error-tracking DSNs (12345@o1.ingest.sentry.io), or placeholder strings (user@example.com), polluting your cold outreach sequence and hurting domain deliverability. Here is the multi-hop enrichment architecture we use in production: [Input URLs: Google Maps / Directories / Company Domains / GitHub Repos] ├──> Layer 1: JSON-LD Parser (LocalBusiness, Organization, AggregateRating) ├──> Layer 2: Same-Domain Subpage Crawler (/contact, /about, /locations) └──> Layer 3: Sanitization & Lead Scoring (Filter telemetry, export CSV/JSON) --- Stage 1: Extracting Local Business Leads, Emails & Phones in Node.js When crawling local business websites discovered from Google Maps or regional directories, roughly 68% of SMB websites embed rich application/ld+json blocks generated by WordPress (Yoast / RankMath), Squarespace, Shopify, or Wix. These JSON-LD blocks often contain the exact street address, postal code, direct telephone number, and aggregate customer rating in clean machine-readable JSON. Below is the production Node.js handler using @crawlee/cheerio that combines JSON-LD schema extraction with automatic /contact subpage crawling: import { CheerioCrawler, createCheerioRouter } from '@crawlee/cheerio'; const EMAIL_REGEX = /[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}/g; const PHONE_REGEX = /(?:\+?\d{1,3}[\s.-]?)?\(?\d{3}\)?[\s.-]?\d{3}[\s.-]?\d{4}/g; const BAD_EMAIL_EXT = /\.(png|jpg|jpeg|gif|svg|webp|ico|css|js|woff|woff2|ttf|map)$/i; const BLOCKED_DOMAINS = ['example.com', 'sentry.io', 'wixpress.com', 'schema.org', 'domain.com']; export function parseJsonLdBusinesses($) { const businesses = []; $('script[type="application/ld+json"]').each((_, el) => { try { const raw = $(el).contents().text().trim(); if (!raw) return; const parsed = JSON.parse(raw); const items = Array.isArray(parsed) ? parsed : (parsed['@graph'] || [parsed]); for (const item of items) { if (item && typeof item === 'object') { businesses.push(item); } } } catch { // Ignore malformed JSON-LD blocks on broken templates } }); return businesses; } export const router = createCheerioRouter(); router.addDefaultHandler(async ({ enqueueLinks, request, $, body, log }) => { const loadedUrl = request.loadedUrl || request.url; const domain = new URL(loadedUrl).hostname.replace(/^www\./, ''); const html = typeof body === 'string' ? body : body.toString('utf-8'); // Automatically follow high-value contact & location pages on the same domain if (!request.userData.isSubPage) { await enqueueLinks({ strategy: 'same-hostname', globs: ['**/contact*', '**/about*', '**/locations*', '**/impressum*'], userData: { isSubPage: true }, }); } const emailSet = new Set(); const phoneSet = new Set(); // 1. Extract explicit mailto: and tel: links $('a[href^="mailto:"]').each((_, el) => { const email = ($(el).attr('href') || '') .replace(/^mailto:/i, '') .split('?')[0] .trim() .toLowerCase(); if (email.includes('@') && !BAD_EMAIL_EXT.test(email)) { emailSet.add(email); } }); $('a[href^="tel:"]').each((_, el) => { const phone = ($(el).attr('href') || '').replace(/^tel:/i, '').trim(); if (phone.length >= 7) phoneSet.add(phone); }); // 2. Scan raw HTML for inline emails with strict false-positive filtering for (const match of html.matchAll(EMAIL_REGEX)) { const candidate = match[0].toLowerCase(); const isBlocked = BLOCKED_DOMAINS.some((d) => candidate.includes(d)); if (!BAD_EMAIL_EXT.test(candidate) && !isBlocked) { emailSet.add(candidate); } } // 3. Scan visible body text for phone numbers (10 to 15 digits) for (const match of ($('body').text() || '').matchAll(PHONE_REGEX)) { const rawPhone = match[0].trim(); const digitCount = rawPhone.replace(/\D/g, '').length; if (digitCount >= 10 && digitCount <= 15) { phoneSet.add(rawPhone); } } // 4. Enrich from Schema.org LocalBusiness / Organization JSON-LD const ldItems = parseJsonLdBusinesses($); let businessName = $('meta[property="og:site_name"]').attr('content') || $('h1').first().text().trim() || $('title').first().text().trim() || domain; let address = $('[itemprop="address"], address').first().text().replace(/\s+/g, ' ').trim() || null; let rating = null; let reviewCount = null; let category = null; for (const ld of ldItems) { if (ld.name && businessName === domain) businessName = ld.name; if (ld.telephone) phoneSet.add(String(ld.telephone)); if (ld.email) emailSet.add(String(ld.email).replace(/^mailto:/i, '').toLowerCase()); if (ld['@type'] && !category) { category = Array.isArray(ld['@type']) ? ld['@type'].join(', ') : String(ld['@type']); } if (ld.address && typeof ld.address === 'object') { const addrParts = [ ld.address.streetAddress, ld.address.addressLocality, ld.address.addressRegion, ld.address.postalCode, ld.address.addressCountry, ].filter(Boolean); if (addrParts.length > 0) address = addrParts.join(', '); } if (ld.aggregateRating && typeof ld.aggregateRating === 'object') { rating = ld.aggregateRating.ratingValue ?? rating; reviewCount = ld.aggregateRating.reviewCount ?? ld.aggregateRating.ratingCount ?? reviewCount; } } // 5. Extract Social Media Profiles const findSocial = (regex) => { let matched = null; $('a[href]').each((_, el) => { const href = $(el).attr('href') || ''; if (!matched && regex.test(href)) matched = href; }); return matched; }; const record = { businessName: String(businessName).replace(/\s+/g, ' ').trim(), category: category || 'LocalBusiness', domain, sourceUrl: loadedUrl, address, rating, reviewCount, emails: Array.from(emailSet), phones: Array.from(phoneSet).slice(0, 10), linkedin: findSocial(/linkedin\.com\/(company|in)\//i), facebook: findSocial(/facebook\.com\/(?!sharer)/i), instagram: findSocial(/instagram\.com\//i), twitter: findSocial(/(twitter|x)\.com\/(?!intent)/i), tiktok: findSocial(/tiktok\.com\/@/i), scrapedAt: new Date().toISOString(), }; log.info(`[Lead Extracted] ${record.businessName} -> ${record.emails.length} emails, ${record.phones.length} phones`); }); Calling the Deployed Google Maps & Local Business Lead Extractor via Python Instead of maintaining Node.js infrastructure and proxy pools yourself, you can invoke the cloud Actor tungnguyen84/google-maps-lead-email-phone-extractor directly from Python: import requests import pandas as pd APIFY_TOKEN = "YOUR_APIFY_API_TOKEN" ACTOR_URL = ( "https://api.apify.com/v2/acts/" "tungnguyen84~google-maps-lead-email-phone-extractor/" f"run-sync-get-dataset-items?token={APIFY_TOKEN}" ) payload = { "startUrls": [ {"url": "https://apify.com/contact"}, {"url": "https://crawlee.dev"} ], "enrichCompanyWebsites": True, "maxBusinesses": 50 } response = requests.post(ACTOR_URL, json=payload, timeout=180) response.raise_for_status() leads = response.json() df = pd.DataFrame(leads) print(df[["businessName", "domain", "emails", "phones", "linkedin"]]) df.to_csv("enriched_b2b_leads.csv", index=False) --- Stage 2: Scraping GitHub Developers & Contributors for DevTools B2B Outreach If you are recruiting senior engineers or selling developer-focused SaaS (CI/CD, observability, AI coding assistants, cloud databases, security scanners), scraping generic business directories won't help. You need active open-source contributors and maintainers from relevant GitHub repositories. However, GitHub's official REST API limits unauthenticated requests to just 60 requests per hour, which breaks almost immediately when enriching a repository with 200 contributors. By parsing GitHub's semantic HTML microformats (span.p-name, div.p-note, span.p-org, span.p-label, li[itemprop="url"]) with @crawlee/cheerio, we can extract contributor profiles, company affiliations, personal websites, X/Twitter handles, and public emails at high speed without burning API quota: // Extracting developer lead data from a GitHub profile DOM const username = new URL(loadedUrl).pathname.split('/').filter(Boolean)[0]; const fullName = $('span.p-name, [itemprop="name"]').first().text().trim() || username; const bio = $('div.p-note, [data-bio-text]').first().text().trim() || ''; const company = $('span.p-org, [itemprop="worksFor"]').first().text().trim() || null; const location = $('span.p-label, [itemprop="homeLocation"]').first().text().trim() || null; const website = $('li[itemprop="url"] a, a.Link--primary[rel*="me"]').first().attr('href') || null; const followers = $('a[href$="?tab=followers"] span.text-bold').first().text().trim() || null; Running the GitHub Developer & Contributor Scraper via cURL or Python Our live Actor GitHub Developer, Contributor & Stargazer B2B Lead Scraper accepts both GitHub repository URLs (automatically discovering and enqueuing contributor profiles) and direct developer profile URLs: curl -X POST "https://api.apify.com/v2/acts/tungnguyen84~github-developer-contributor-lead-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "targetUrls": [ { "url": "https://github.com/tungnguyen84" }, { "url": "https://github.com/apify/crawlee" } ], "maxProfiles": 25 }' --- Stage 3: Deduplicating & Scoring Multi-Page Leads in Python When your crawler visits both https://company.com and https://company.com/contact, it outputs one record per page. Before importing into HubSpot, Pipedrive, or Apollo, you should merge records by domain so each company occupies a single row with all discovered emails, phones, and social links combined: from collections import defaultdict import json def merge_leads_by_domain(raw_records: list[dict]) -> list[dict]: grouped = {} for rec in raw_records: domain = rec.get("domain") or rec.get("username") or rec.get("sourceUrl") if not domain: continue if domain not in grouped: grouped[domain] = { "businessName": rec.get("businessName") or rec.get("fullName"), "domain": domain, "website": rec.get("website") or rec.get("sourceUrl"), "address": rec.get("address") or rec.get("location"), "rating": rec.get("rating"), "emails": set(rec.get("emails") or []), "phones": set(rec.get("phones") or []), "linkedin": rec.get("linkedin"), "twitter": rec.get("twitter"), "facebook": rec.get("facebook"), "instagram": rec.get("instagram"), } else: entry = grouped[domain] entry["emails"].update(rec.get("emails") or []) entry["phones"].update(rec.get("phones") or []) for field in ("address", "rating", "linkedin", "twitter", "facebook", "instagram"): if not entry.get(field) and rec.get(field): entry[field] = rec[field] merged = [] for domain, data in grouped.items(): data["emails"] = sorted(data["emails"]) data["phones"] = sorted(data["phones"]) data["leadScore"] = ( (50 if data["emails"] else 0) + (25 if data["phones"] else 0) + (15 if data["linkedin"] else 0) + (10 if data["address"] else 0) ) merged.append(data) return sorted(merged, key=lambda x: x["leadScore"], reverse=True) --- 🌐 Try Our Live Apify Actors, RapidAPI Endpoints & Standalone Tools Whether you prefer serverless cloud APIs or standalone offline Python + Excel VBA tools, all of our scrapers are live and ready to test: 1. Apify Store Actors (Pay-Per-Result, No Monthly Commitment) - 🌍 Google Maps & Local Business Lead, Email, Phone & Social Extractor - 💻 GitHub Developer, Contributor & Stargazer B2B Lead Scraper - 🏢 B2B Website Email, Phone & Social Media Contact Scraper - 🎬 YouTube Video Transcript, Metadata & Channel Email Scraper - 💬 Reddit Subreddit, Search & Thread to Clean JSON & AI Markdown Scraper - 📄 Website to Clean Markdown & Text Scraper for AI, LLM & RAG - 🛍️ Shopify Store Products, Prices & B2B Contact Email Scraper - 🛒 WooCommerce & WordPress Store Product, Price & Contact Scraper - 🔍 Website SEO Audit, Meta Tags, Headings & Contact Email Scraper 2. RapidAPI Hub REST APIs (Free Tier BASIC $0.00/mo) - B2B Website Email, Phone & Social Scraper API - Website to Clean Markdown AI, LLM & RAG Converter API - Shopify Store Product, Price & Contact Scraper API - WooCommerce & WordPress Product, Price & Contact Spy API - Website Sitemap, SEO Audit & Contact Scraper API 3. Instant Download Full Source Code & Excel VBA Suites (Payhip / Gumroad) - 📦 Python B2B Lead Scraper & Excel VBA Suite Pro ($14.99) - 📦 Shopify Competitor Price & Product Spy Pro ($12.99) - 📦 Ultimate Excel VBA Automation Pack: 10 One-Click Macros ($9.99) Need a custom web scraper, anti-bot bypass pipeline, or automated CRM integration built for your team? Message me on Fiverr (@tungnguyen84)!
5. apify/crawlee Framework profesional con todo incluido: rotación de proxies, reintentos automáticos, browser fingerprinting, colas de trabajo. Lo que cobran las empresas de scraping por miles al mes. Aquí gratis.
Copying data from web pages by hand? 🕷️ Crawlee helps build JavaScript and TypeScript crawlers, with browser automation and structured extraction. Useful for permitted data collection. 🌐 apify/crawlee #WebScraping #浇朋友 #蓝朋友 #互关 #fo
写爬虫，总被链接队列和浏览器操作折腾？🕸️ Crawlee 提供爬取队列、浏览器自动化和数据存储。 适合采集获准访问的公开数据。🧰 apify/crawlee #WebScraping #Automation #浇朋友 #蓝朋友 #互关 #fo
Modern open-source crawler Works with BeautifulSoup apify/crawlee apify/crawlee-python
5. apify/crawlee Framework profesional con todo incluido: rotación de proxies, reintentos automáticos, browser fingerprinting, colas de trabajo. Lo que cobran las empresas de scraping por miles al mes. Aquí gratis.