How to send bulk HTTP requests to an API or webhook, one per row
Short answer
Build one request per row from a URL template, keep within the API's rate limit, retry timeouts, 429s and 5xx errors with backoff, and save each status and response. Hookvolley on Apify does that for GET, POST, PUT, PATCH and DELETE at $0.0008 per request answered; requests that never get a response are free.
- Choose the source: a list of URLs, a list of individual requests, or a dataset with one request per row.
- Write the URL or body template with {{field}}, {{field|url}} or {{json}} (the whole row).
- Put API keys in headers and set maxRequestsPerSecondPerHost to stay inside the API's limit.
- For webhooks, set method POST and batchSize to send rows as JSON arrays.
- Use extractFields (for example data.price) to pull values out of each JSON response.
Copy-paste code
Python (pip install apify-client): one GET per ID
import os
from apify_client import ApifyClient
ids = [1, 2, 3]
client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("cybermax/bulk-http-requests").call(run_input={
"urls": [f"https://jsonplaceholder.typicode.com/posts/{i}" for i in ids],
"extractFields": ["title"],
"maxRequestsPerSecondPerHost": 5,
})
for r in client.dataset(run["defaultDatasetId"]).iterate_items():
print(r["url"], r.get("status"), r.get("extracted"), r.get("error"))JavaScript / Node (npm i apify-client): scraped rows to a webhook in batches
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('cybermax/bulk-http-requests').call({
datasetId: 'YOUR_DATASET_ID',
method: 'POST',
urlTemplate: 'https://hooks.example.com/abc',
batchSize: 50,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.map((r) => r.status));No code: n8n, Make or Zapier
n8n: HTTP Request node (or the Apify node, action "Run an Actor and get dataset")
Method: POST
URL: https://api.apify.com/v2/acts/cybermax~bulk-http-requests/run-sync-get-dataset-items
Auth: Header Auth -> Authorization: Bearer <your Apify token>
Body: JSON {"urls": {{ $json.endpoints }}, "headers": {"Authorization": "Bearer YOUR_KEY"}}
Output: one n8n item per request (status, body, extracted, durationMs)
Make: Apify app -> "Run an Actor" (Actor: cybermax/bulk-http-requests, wait for finish = yes)
-> Apify "Get Dataset Items" (dataset = the run's defaultDatasetId) -> your next module
Zapier: Apify app -> action "Run Actor" (cybermax/bulk-http-requests, input JSON as above)
-> trigger "Finished Actor Run" -> "Fetch Dataset Items" -> your app
Runs longer than 5 minutes: start with POST /v2/acts/cybermax~bulk-http-requests/runs and read the dataset when the run finishes.From an AI agent (MCP)
# Any MCP client (Claude Desktop, Claude Code, Cursor, VS Code): Apify's hosted MCP server
{
"mcpServers": {
"bulk-http-requests": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://mcp.apify.com/?tools=cybermax/bulk-http-requests",
"--header", "Authorization: Bearer ${APIFY_TOKEN}"],
"env": { "APIFY_TOKEN": "<your Apify token>" }
}
}
}
# The agent sees one tool, cybermax--bulk-http-requests, with the Actor's input schema.| Method | URL | Status | Extracted |
|---|---|---|---|
| GET | jsonplaceholder.typicode.com/posts/1 | 200 | title |
| GET | jsonplaceholder.typicode.com/users/1 | 200 | name: Leanne Graham |
| POST | postman-echo.com/post | 200 | json.hello: world |
| GET | jsonplaceholder.typicode.com/nope | 404 | |
| GET | 169.254.169.254 (cloud metadata) | blocked (not charged) |
Source: Hookvolley (Apify Actor cybermax/bulk-http-requests) README, real Apify run on 10 Nov 2026
Retries and rate limits that an HTTP module lacks
Network errors, timeouts, 408, 425, 429 and 5xx answers are retried with exponential backoff, honouring Retry-After, and a per-host rate limit keeps the run inside the API's quota.
Push scraped rows into your stack
Chain it after any scraper in Apify Integrations with datasetId set to the run's dataset, method POST and a batchSize, and every row lands in your CRM, Slack, Airtable or backend.
Watch an API for changes
With monitor on and a monitorKey, scheduled runs return only requests whose status or body changed; unchanged responses are free.
Good to know
Price on Apify: $0.0008 per request answered ($0.80 per 1,000), no start fee. A 404 or 500 counts as answered; requests with no response, invalid inputs and blocked addresses are free. Apify's free plan includes $5 of usage each month.
Private, loopback and cloud-metadata addresses are refused, also after redirects. Call only APIs and sites you are allowed to, within their terms and rate limits.
Bouncefern: Which of your URLs break, bounce or loop
$0.001 per URL, no start feeOnly need status codes and redirects? Bouncefern checks thousands of URLs for broken links, redirects and loops at $0.001 per URL.
Runs on your own Apify account, pay per result; failed results are not charged.Questions
Is a 404 charged?
Yes, the API answered. Requests that never get a response are free.
Where do API keys go?
In headers, for example Authorization. They stay in your run's input on your own Apify account.
How many requests per run?
Up to 100,000, with up to 20 at once.
Can an AI agent use it?
Yes. Apify's MCP server exposes it as cybermax--bulk-http-requests for Claude, Cursor and other MCP clients.
Free: the Free API Starter Kit
Copy-paste calls for 4 APIs with free daily tiers (CVE priority, VIN recalls, US duty rates, email checks), then 5 short emails on KEV/EPSS, MCP and free data.