Enrich a list of company domains with emails, socials and tech stack (Python)
You have a column of company domains, from a CRM export, a conference attendee list, or a list of companies that are hiring. You want a few useful fields for each one: the real company name, a one-line description, a contact email, LinkedIn and other social profiles, and where their careers page is.
Paid enrichment APIs exist, but much of this is published on the company's own website, often in machine-readable form. This post shows how to get it yourself, what makes it harder than it looks, and a hosted option.
Where the data lives
- <title> and <meta name="description">. The tagline, almost always present.
- schema.org JSON-LD. Many sites embed an Organization block with name, logo, address, foundingDate and sameAs (their social profiles). This is the cleanest source when it exists.
- Links. mailto: and tel: links, social profile links in the footer, and links to /careers or /jobs.
- Other pages. Contact, about and imprint pages often hold the email, phone and address that the homepage doesn't show.
A homepage-only version
import json, re, requests # pip install requests beautifulsoup4
from bs4 import BeautifulSoup
from urllib.parse import urljoin, urlparse
SOCIAL = {"linkedin.com": "linkedin", "x.com": "twitter", "twitter.com": "twitter",
"facebook.com": "facebook", "instagram.com": "instagram",
"youtube.com": "youtube", "github.com": "github", "tiktok.com": "tiktok"}
def enrich(domain):
url = f"https://{domain}/"
try:
r = requests.get(url, timeout=20, headers={"User-Agent": "Mozilla/5.0 (compatible; enrich-demo)"})
except requests.RequestException as e:
return {"domain": domain, "status": "unreachable", "error": type(e).__name__}
soup = BeautifulSoup(r.text, "html.parser")
row = {"domain": domain, "status": "ok", "httpStatus": r.status_code,
"title": soup.title.get_text(strip=True) if soup.title else None,
"description": (soup.find("meta", attrs={"name": "description"}) or {}).get("content"),
"emails": set(), "phones": set(), "socials": {}, "careersPage": None}
# schema.org JSON-LD often has the cleanest name, logo and social profiles
for s in soup.find_all("script", type="application/ld+json"):
try:
data = json.loads(s.string or "")
except ValueError:
continue
for node in data if isinstance(data, list) else data.get("@graph", [data]):
if isinstance(node, dict) and node.get("@type") in ("Organization", "Corporation"):
row["name"] = node.get("name")
for same in node.get("sameAs", []):
host = urlparse(same).netloc.removeprefix("www.")
if host in SOCIAL:
row["socials"].setdefault(SOCIAL[host], same)
for a in soup.find_all("a", href=True):
href = a["href"].strip()
if href.startswith("mailto:"):
row["emails"].add(href[7:].split("?")[0].lower())
elif href.startswith("tel:"):
row["phones"].add(re.sub(r"[^\d+]", "", href[4:]))
else:
full = urljoin(url, href)
host = urlparse(full).netloc.removeprefix("www.")
if host in SOCIAL and urlparse(full).path.strip("/"):
row["socials"].setdefault(SOCIAL[host], full)
elif host.endswith(domain) and re.search(r"/(careers|jobs)/?$", urlparse(full).path):
row["careersPage"] = row["careersPage"] or full
row["emails"], row["phones"] = sorted(row["emails"]), sorted(row["phones"])
return row
for d in ["zapier.com", "hubspot.com", "this-domain-does-not-exist-xyz123.com"]:
print(enrich(d))
Output (trimmed):
{'domain': 'zapier.com', 'status': 'ok', 'httpStatus': 200, 'title': 'Zapier: Automate AI Workflows, Agents, and Apps', 'emails': [], 'phones': [], 'socials': {'facebook': 'https://facebook.com/ZapierApp', 'github': 'https://github.com/zapier', 'linkedin': 'https://linkedin.com/company/zapier', 'instagram': 'https://instagram.com/zapier', 'youtube': 'https://youtube.com/@ZapierApp', 'twitter': 'https://x.com/zapier'}, ...}
{'domain': 'hubspot.com', 'status': 'ok', 'httpStatus': 200, 'title': 'HubSpot | Software & Tools for your Business - Homepage', 'emails': [], 'phones': [], 'socials': {'linkedin': 'https://www.linkedin.com/company/hubspot', 'twitter': 'https://x.com/HubSpot', ...}}
{'domain': 'this-domain-does-not-exist-xyz123.com', 'status': 'unreachable', 'error': 'SSLError'}
Social profiles and descriptions come through well. Emails and phones are empty, because neither homepage has a mailto: or tel: link.
The gotchas
1. Contacts are rarely on the homepage. You need to find and fetch the contact, about, imprint and careers pages, which means detecting those links in several languages (/kontakt, /impressum, /mentions-legales) and capping how many pages you fetch per site.
2. Social links are noisy. Footers link to share buttons (facebook.com/sharer), the site's own app pages, or the CMS vendor's profile. Ignore share/intent URLs and links without a path, and prefer sameAs from JSON-LD when it's present.
3. Emails are often obfuscated. name [at] company [dot] com, Cloudflare's email protection, or emails only in JavaScript. A plain mailto: scan misses those, and a loose regex picks up image@2x.png.
4. "Unreachable" has many causes. A dead domain didn't fail on DNS here; it failed with an SSLError (some resolvers return a parking page for unknown names). Timeouts, HTTP 403 from bot protection, and redirects to a different domain all need their own status, so you can tell "company gone" from "try again with a proxy".
5. JSON-LD is messy. It can be a list, a dict with @graph, @type can be a list, and some sites ship invalid JSON. Parse defensively, as the script does.
6. Tech stack detection is its own project. Spotting Contentful, Google Tag Manager or Shopify means matching script URLs, meta tags, cookies and headers against a signature list that you have to keep up to date.
If you'd rather not maintain it
Disclosure: I packaged this as a hosted tool, Company Website Enrichment on Apify, with the source on GitHub. It visits the homepage plus up to three contact, about, imprint or careers pages per site (maxPagesPerSite, default 4), and returns one row per company. It detects 130+ technologies and identifies the company's job board (Greenhouse, Lever, Ashby and others).
Input:
{"websites": ["stripe.com", "hubspot.com", "zapier.com", "this-domain-does-not-exist-xyz123.com"]}
The Zapier row (trimmed):
{
"domain": "zapier.com",
"status": "ok",
"companyName": "Zapier",
"legalName": "Zapier Inc",
"emails": ["recruiting@zapier.com"],
"address": {"street": "548 Market St. #62411", "city": "San Francisco", "region": "CA", "postalCode": "94104", "country": "US"},
"socials": {
"linkedin": "https://www.linkedin.com/company/zapier",
"twitter": "https://x.com/zapier",
"github": "https://github.com/zapier"
},
"contactPageUrl": "https://zapier.com/l/contact-sales",
"careersPageUrl": "https://zapier.com/jobs",
"jobBoard": {"ats": "ashby", "slug": "zapier"},
"technologies": [
{"name": "Contentful", "category": "CMS"},
{"name": "Google Tag Manager", "category": "Tag manager"},
{"name": "Next.js", "category": "JavaScript framework"},
{"name": "Vercel", "category": "Hosting"}
],
"foundingDate": "2011",
"pagesFetched": ["https://zapier.com/", "https://zapier.com/l/contact-sales", "https://zapier.com/legal", "https://zapier.com/jobs"]
}
Compare it with the homepage-only script: the email and address came from the extra pages, and the job board was detected from the careers page. The dead domain comes back as "status": "unreachable" with "error": "Website could not be reached", and the run summary counts rows by status ({"ok": 3, "unreachable": 1}).
Call it from Python, or chain it after another scraper
import os
from apify_client import ApifyClient # pip install apify-client
client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("swillz/company-website-enrichment").call(
run_input={"websites": ["stripe.com", "hubspot.com", "zapier.com"]},
max_total_charge_usd=0.05,
)
for row in client.dataset(run.default_dataset_id).iterate_items():
print(row["domain"], row["status"], row.get("emails"), row.get("socials", {}).get("linkedin"))
(On 1.x versions of the client, use run["defaultDatasetId"].)
Instead of websites, you can pass datasetId from another run, such as a Google Maps, job or lead scraper. The tool auto-detects the website field (website, websiteUrl, companyUrl and similar), or you can set websiteField (dot paths like company.url work). Duplicate domains are processed once.
Let an AI agent enrich a list (MCP)
{
"mcpServers": {
"apify": { "url": "https://mcp.apify.com?tools=swillz/company-website-enrichment" }
}
}
Then:
> "For these 10 domains, find the company name, LinkedIn page, a contact email and what CMS they use. Mark any that are unreachable."
The agent passes the list as websites. Each site takes a few seconds, so batches of 10–20 fit within the MCP call-actor wait. For larger lists, start the run and read the dataset when it finishes.
Wrap-up
- Start with JSON-LD Organization, meta tags and footer links; they're cheap and fairly clean.
- Contacts and addresses usually live on contact, about or imprint pages, not the homepage.
- Filter social share links, de-obfuscate emails, and give each failure its own status.
- Keep a per-row status, so you can tell a dead domain from a blocked one.
Mentions
Enrich a list of company domains with emails, socials and tech stack (Python) You have a column of company domains, from a CRM export, a conference attendee list, or a list of companies that are hiring. You want a few useful fields for each one: the real company name, a one-line description, a contact email, LinkedIn and other social profiles, and where their careers page is. Paid enrichment APIs exist, but much of this is published on the company's own website, often in machine-readable form. This post shows how to get it yourself, what makes it harder than it looks, and a hosted option. Where the data lives - <title> and <meta name="description">. The tagline, almost always present. - schema.org JSON-LD. Many sites embed an Organization block with name, logo, address, foundingDate and sameAs (their social profiles). This is the cleanest source when it exists. - Links. mailto: and tel: links, social profile links in the footer, and links to /careers or /jobs. - Other pages. Contact, about and imprint pages often hold the email, phone and address that the homepage doesn't show. A homepage-only version import json, re, requests # pip install requests beautifulsoup4 from bs4 import BeautifulSoup from urllib.parse import urljoin, urlparse SOCIAL = {"linkedin.com": "linkedin", "x.com": "twitter", "twitter.com": "twitter", "facebook.com": "facebook", "instagram.com": "instagram", "youtube.com": "youtube", "github.com": "github", "tiktok.com": "tiktok"} def enrich(domain): url = f"https://{domain}/" try: r = requests.get(url, timeout=20, headers={"User-Agent": "Mozilla/5.0 (compatible; enrich-demo)"}) except requests.RequestException as e: return {"domain": domain, "status": "unreachable", "error": type(e).__name__} soup = BeautifulSoup(r.text, "html.parser") row = {"domain": domain, "status": "ok", "httpStatus": r.status_code, "title": soup.title.get_text(strip=True) if soup.title else None, "description": (soup.find("meta", attrs={"name": "description"}) or {}).get("content"), "emails": set(), "phones": set(), "socials": {}, "careersPage": None} # schema.org JSON-LD often has the cleanest name, logo and social profiles for s in soup.find_all("script", type="application/ld+json"): try: data = json.loads(s.string or "") except ValueError: continue for node in data if isinstance(data, list) else data.get("@graph", [data]): if isinstance(node, dict) and node.get("@type") in ("Organization", "Corporation"): row["name"] = node.get("name") for same in node.get("sameAs", []): host = urlparse(same).netloc.removeprefix("www.") if host in SOCIAL: row["socials"].setdefault(SOCIAL[host], same) for a in soup.find_all("a", href=True): href = a["href"].strip() if href.startswith("mailto:"): row["emails"].add(href[7:].split("?")[0].lower()) elif href.startswith("tel:"): row["phones"].add(re.sub(r"[^\d+]", "", href[4:])) else: full = urljoin(url, href) host = urlparse(full).netloc.removeprefix("www.") if host in SOCIAL and urlparse(full).path.strip("/"): row["socials"].setdefault(SOCIAL[host], full) elif host.endswith(domain) and re.search(r"/(careers|jobs)/?$", urlparse(full).path): row["careersPage"] = row["careersPage"] or full row["emails"], row["phones"] = sorted(row["emails"]), sorted(row["phones"]) return row for d in ["zapier.com", "hubspot.com", "this-domain-does-not-exist-xyz123.com"]: print(enrich(d)) Output (trimmed): {'domain': 'zapier.com', 'status': 'ok', 'httpStatus': 200, 'title': 'Zapier: Automate AI Workflows, Agents, and Apps', 'emails': [], 'phones': [], 'socials': {'facebook': 'https://facebook.com/ZapierApp', 'github': 'https://github.com/zapier', 'linkedin': 'https://linkedin.com/company/zapier', 'instagram': 'https://instagram.com/zapier', 'youtube': 'https://youtube.com/@ZapierApp', 'twitter': 'https://x.com/zapier'}, ...} {'domain': 'hubspot.com', 'status': 'ok', 'httpStatus': 200, 'title': 'HubSpot | Software & Tools for your Business - Homepage', 'emails': [], 'phones': [], 'socials': {'linkedin': 'https://www.linkedin.com/company/hubspot', 'twitter': 'https://x.com/HubSpot', ...}} {'domain': 'this-domain-does-not-exist-xyz123.com', 'status': 'unreachable', 'error': 'SSLError'} Social profiles and descriptions come through well. Emails and phones are empty, because neither homepage has a mailto: or tel: link. The gotchas 1. Contacts are rarely on the homepage. You need to find and fetch the contact, about, imprint and careers pages, which means detecting those links in several languages (/kontakt, /impressum, /mentions-legales) and capping how many pages you fetch per site. 2. Social links are noisy. Footers link to share buttons (facebook.com/sharer), the site's own app pages, or the CMS vendor's profile. Ignore share/intent URLs and links without a path, and prefer sameAs from JSON-LD when it's present. 3. Emails are often obfuscated. name [at] company [dot] com, Cloudflare's email protection, or emails only in JavaScript. A plain mailto: scan misses those, and a loose regex picks up image@2x.png. 4. "Unreachable" has many causes. A dead domain didn't fail on DNS here; it failed with an SSLError (some resolvers return a parking page for unknown names). Timeouts, HTTP 403 from bot protection, and redirects to a different domain all need their own status, so you can tell "company gone" from "try again with a proxy". 5. JSON-LD is messy. It can be a list, a dict with @graph, @type can be a list, and some sites ship invalid JSON. Parse defensively, as the script does. 6. Tech stack detection is its own project. Spotting Contentful, Google Tag Manager or Shopify means matching script URLs, meta tags, cookies and headers against a signature list that you have to keep up to date. If you'd rather not maintain it Disclosure: I packaged this as a hosted tool, Company Website Enrichment on Apify, with the source on GitHub. It visits the homepage plus up to three contact, about, imprint or careers pages per site (maxPagesPerSite, default 4), and returns one row per company. It detects 130+ technologies and identifies the company's job board (Greenhouse, Lever, Ashby and others). Input: {"websites": ["stripe.com", "hubspot.com", "zapier.com", "this-domain-does-not-exist-xyz123.com"]} The Zapier row (trimmed): { "domain": "zapier.com", "status": "ok", "companyName": "Zapier", "legalName": "Zapier Inc", "emails": ["recruiting@zapier.com"], "address": {"street": "548 Market St. #62411", "city": "San Francisco", "region": "CA", "postalCode": "94104", "country": "US"}, "socials": { "linkedin": "https://www.linkedin.com/company/zapier", "twitter": "https://x.com/zapier", "github": "https://github.com/zapier" }, "contactPageUrl": "https://zapier.com/l/contact-sales", "careersPageUrl": "https://zapier.com/jobs", "jobBoard": {"ats": "ashby", "slug": "zapier"}, "technologies": [ {"name": "Contentful", "category": "CMS"}, {"name": "Google Tag Manager", "category": "Tag manager"}, {"name": "Next.js", "category": "JavaScript framework"}, {"name": "Vercel", "category": "Hosting"} ], "foundingDate": "2011", "pagesFetched": ["https://zapier.com/", "https://zapier.com/l/contact-sales", "https://zapier.com/legal", "https://zapier.com/jobs"] } Compare it with the homepage-only script: the email and address came from the extra pages, and the job board was detected from the careers page. The dead domain comes back as "status": "unreachable" with "error": "Website could not be reached", and the run summary counts rows by status ({"ok": 3, "unreachable": 1}). Call it from Python, or chain it after another scraper import os from apify_client import ApifyClient # pip install apify-client client = ApifyClient(os.environ["APIFY_TOKEN"]) run = client.actor("swillz/company-website-enrichment").call( run_input={"websites": ["stripe.com", "hubspot.com", "zapier.com"]}, max_total_charge_usd=0.05, ) for row in client.dataset(run.default_dataset_id).iterate_items(): print(row["domain"], row["status"], row.get("emails"), row.get("socials", {}).get("linkedin")) (On 1.x versions of the client, use run["defaultDatasetId"].) Instead of websites, you can pass datasetId from another run, such as a Google Maps, job or lead scraper. The tool auto-detects the website field (website, websiteUrl, companyUrl and similar), or you can set websiteField (dot paths like company.url work). Duplicate domains are processed once. Let an AI agent enrich a list (MCP) { "mcpServers": { "apify": { "url": "https://mcp.apify.com?tools=swillz/company-website-enrichment" } } } Then: > "For these 10 domains, find the company name, LinkedIn page, a contact email and what CMS they use. Mark any that are unreachable." The agent passes the list as websites. Each site takes a few seconds, so batches of 10–20 fit within the MCP call-actor wait. For larger lists, start the run and read the dataset when it finishes. Wrap-up - Start with JSON-LD Organization, meta tags and footer links; they're cheap and fairly clean. - Contacts and addresses usually live on contact, about or imprint pages, not the homepage. - Filter social share links, de-obfuscate emails, and give each failure its own status. - Keep a per-row status, so you can tell a dead domain from a blocked one.
💯 you shouldn't need another hire to chase unpaid invoices. But a Zap can do it. Simply ask Claude or your Grok Bot to: "Build me this zap automation with Zapier MCP: github.com/zapier/community-skills/tree/main/skil…" t.co
Brad is right; nobody has time to reply to every Google review. But a Zap can do it. I turned his problem into a Next Gen Zap. Simply ask Claude or your Grok Bot to: "Build me this zap automation with Zapier MCP: github.com/zapier/community-skills/tree/main/skil…" t.co
Speed is everything for a service business. So I turned @MosesD02 request into a Next Gen Zap. Ask Claude or your Grok Bot to: "Build me this zap automation with Zapier MCP: github.com/zapier/community-skills/tree/main/skil…" t.co