LenDbBot, and a guide for agents.

The LenDbBot crawler

LenDbBot visits mortgage lender websites to read their public loan officer directory pages. It identifies itself with this user agent:

LenDbBot/1.0 (+https://lendb.ai/bot)

• Obeys robots.txt. LenDbBot is a directory and search crawler, not an AI training crawler. It honors rules for LenDbBot, for all bots, and for AI search bots (such as ClaudeBot, PerplexityBot or OAI-SearchBot). Rules aimed only at training crawlers (such as GPTBot, CCBot or Google-Extended) do not apply to it. If you would rather we not crawl you at all, block LenDbBot by name or email us and we will ask for a feed instead.
• Rate limited per host. One request at a time per site, with a delay between requests, and it backs off on errors and on Retry-After.
• Caches with ETags and conditional requests, and revisits directory pages about weekly and individual profiles about monthly.
• Reads public pages only. No logins, no forms, no NMLS Consumer Access, no LinkedIn.
• Stores facts, not copies. We extract structured facts and short summaries with a link back to your page.

To block it, add User-agent: LenDbBot with Disallow: /. To ask a question, offer a feed, or report a problem, email bot@lendb.ai.

How AI agents should use LenDb

Every AI crawler is welcome here. See robots.txt. For the best results, prefer the structured surfaces over scraping HTML.

1. MCP server

Remote MCP over streamable HTTP at https://lendb.ai/mcp. No auth. Tools: find_loan_officers, get_loan_officer, get_lender, list_specialties and explain_specialty. Every response includes a canonical_url to cite.

2. Public API

A read API at /api/v1, described by an OpenAPI 3.1 document. No key needed. Start with /api/v1/match?q=, which turns a plain-language situation into filters and returns ranked loan officers with reasons.

3. Markdown and JSON twins

Every page has a .md and a .json version. Append the extension, or send Accept: text/markdown or Accept: application/json to the normal URL. Try the sample profile in Markdown.

4. llms.txt

/llms.txt is the short index and /llms-full.txt holds the methodology and the full taxonomy in one file.

Citing LenDb

• Link the person's canonical profile URL and include their NMLS ID and the last verified date.
• Pass along the reasons we return, so the user knows why someone was suggested.
• Remind users to verify licenses at NMLS Consumer Access.
• Anything marked "sample": true is fictional demo data. Never present it as a real person.