Extract links
Every link on a page resolved to an absolute URL, de-duplicated, with its anchor text and rel, and split into internal and external. What a crawling agent needs to decide where to go next, without downloading and parsing the HTML itself.
POST https://api.bottrunk.com/s/extract-links
Call it
# once, on the machine your agent runs on npx bottrunk-mcp wallet # prints the address; send it 0.3 ALGO, then USDC claude mcp add bottrunk -- npx -y bottrunk-mcp # then ask your agent for it — the tool is `bottrunk_extract_links`
# Ask. You get a 402 with the payment requirements.
curl https://api.bottrunk.com/s/extract-links \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","limit":100}'
# Pay and retry. Any x402 client does this for you.
curl https://api.bottrunk.com/s/extract-links \
-H "PAYMENT-SIGNATURE: <signed USDC transfer>" \
-H "Content-Type: application/json" \
-d '{"url":"https://example.com","limit":100}'
from x402.clients import requests as x402_requests
session = x402_requests.x402_http_adapter(account) # your agent's Algorand account
r = session.post("https://api.bottrunk.com/s/extract-links", json={"url":"https://example.com","limit":100})
print(r.json())
print(r.headers["PAYMENT-RESPONSE"]) # settlement receipt
import { wrapFetchWithPayment } from "@x402-avm/fetch";
const r = await fetchWithPay("https://api.bottrunk.com/s/extract-links", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({"url":"https://example.com","limit":100}),
});
console.log(await r.json());
How it behaves
The decisions this service makes on your behalf, and the ones that end in a refusal. A 4xx is never settled, so a refusal costs you nothing.
| Size and time | Pages over 2 MB are refused with page too large. Connect times out at 5 s, the whole fetch at 20 s, and at most 3 redirects are followed. |
| Who we look like | Requests go out as BotTrunk/0.1 (+https://bottrunk.com/docs). Sites that block unknown agents will block this one. |
| Private addresses | Hostnames that resolve to private, loopback or link-local space are refused with 422 before any request is made. Do not point this at localhost or an intranet. |
| Method | POST only. A GET on the same path returns this page. |
Input
| url | string | Public http(s) URL. |
| same_host | boolean | Only links on the same host. Default false. |
| limit | integer | Maximum links to return (1–500). Default 500. |
Output
| links | array | Objects with url, text, rel and internal. |
| total | integer | Links found after de-duplication. |
| internal | integer | How many are on the same host. |
| external | integer | How many point elsewhere. |