You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Constraint: The MCP server does not expose firecrawl_batch_scrape or firecrawl_check_batch_status.
Rejected: Adding MCP batch tools now | expanding tool count needs product/experiment approval.
Confidence: high
Scope-risk: narrow
Directive: Keep README and runtime-visible tool descriptions limited to registered MCP tools.
Tested: npm run build; GitHub build check passed; independent re-review approved.
Not-tested: Full manual MCP trace not run.
Copy file name to clipboardExpand all lines: README.md
+20-89Lines changed: 20 additions & 89 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -324,22 +324,20 @@ These configurations control:
324
324
- Warning at 1000 credits remaining
325
325
- Critical alert at 100 credits remaining
326
326
327
-
### Rate Limiting and Batch Processing
327
+
### Rate Limiting
328
328
329
-
The server utilizes Firecrawl's built-in rate limiting and batch processing capabilities:
329
+
The server uses Firecrawl's built-in rate limiting:
330
330
331
331
- Automatic rate limit handling with exponential backoff
332
-
- Efficient parallel processing for batch operations
333
332
- Smart request queuing and throttling
334
333
- Automatic retries for transient errors
335
334
336
335
## How to Choose a Tool
337
336
338
337
Use this guide to select the right tool for your task:
339
338
340
-
-**If you know the exact URL(s) you want:**
341
-
- For one: use **scrape** (with JSON format for structured data)
342
-
- For many: use **batch_scrape**
339
+
-**If you know the exact URL you want:** use **scrape** (with JSON format for structured data)
340
+
-**If you have multiple known URLs:** call **scrape** for each URL. If you specifically need one bulk API operation, use the Firecrawl API batch endpoint outside MCP.
343
341
-**If you need to discover URLs on a site:** use **map**
344
342
-**If you want to search the web for info:** use **search**
345
343
-**If you need complex research across multiple unknown sources:** use **agent**
@@ -352,15 +350,14 @@ Use this guide to select the right tool for your task:
When using `scrape` or `batch_scrape`, choose the right format:
360
+
When using `scrape`, choose the right format:
364
361
365
362
-**JSON format (recommended for most cases):** Use when you need specific data from a page. Define a schema based on what you need to extract. This keeps responses small and avoids context window overflow.
366
363
-**Markdown format (use sparingly):** Only when you genuinely need the full page content, such as reading an entire article for summarization or analyzing page structure.
@@ -377,12 +374,12 @@ Scrape content from a single URL with advanced options.
377
374
378
375
**Not recommended for:**
379
376
380
-
- Extracting content from multiple pages (use batch_scrape for known URLs, or map + batch_scrape to discover URLs first, or crawl for full page content)
377
+
- Extracting content from multiple pages (use repeated scrape calls for known URLs, or map + scrape to discover URLs first, or crawl for full page content)
381
378
- When you're unsure which page contains the information (use search)
382
379
383
380
**Common mistakes:**
384
381
385
-
-Using scrape for a list of URLs (use batch_scrape instead).
382
+
-Passing a list of URLs to one scrape call. Call scrape once per URL in MCP. If you specifically need one bulk API operation, use the Firecrawl API batch endpoint outside MCP.
386
383
- Using markdown format by default (use JSON format to extract only what you need).
387
384
388
385
**Choosing the right format:**
@@ -452,72 +449,7 @@ Scrape content from a single URL with advanced options.
452
449
453
450
- JSON structured data, markdown, branding profile, or other formats as specified.
Sends structured feedback on a previous `firecrawl_search` result. The first feedback per search id refunds 1 credit and improves Firecrawl's search quality. Idempotent per search id.
605
537
@@ -641,7 +573,7 @@ Sends structured feedback on a previous `firecrawl_search` result. The first fee
Starts an asynchronous crawl job on a website and extract content from all pages.
684
616
@@ -689,10 +621,10 @@ Starts an asynchronous crawl job on a website and extract content from all pages
689
621
**Not recommended for:**
690
622
691
623
- Extracting content from a single page (use scrape)
692
-
- When token limits are a concern (use map + batch_scrape)
624
+
- When token limits are a concern (use map + scrape for tighter control)
693
625
- When you need fast results (crawling can be slow)
694
626
695
-
**Warning:** Crawl responses can be very large and may exceed token limits. Limit the crawl depth and number of pages, or use map + batch_scrape for better control.
627
+
**Warning:** Crawl responses can be very large and may exceed token limits. Limit the crawl depth and number of pages, or use map + scrape for tighter control.
696
628
697
629
**Common mistakes:**
698
630
@@ -734,7 +666,7 @@ Starts an asynchronous crawl job on a website and extract content from all pages
734
666
}
735
667
```
736
668
737
-
### 7. Check Crawl Status (`firecrawl_check_crawl_status`)
669
+
### 5. Check Crawl Status (`firecrawl_check_crawl_status`)
738
670
739
671
Check the status of a crawl job.
740
672
@@ -751,7 +683,7 @@ Check the status of a crawl job.
751
683
752
684
- Response includes the status of the crawl job:
753
685
754
-
### 8. Extract Tool (`firecrawl_extract`)
686
+
### 6. Extract Tool (`firecrawl_extract`)
755
687
756
688
Extract structured information from web pages using LLM capabilities. Supports both cloud AI and self-hosted LLM extraction.
757
689
@@ -824,7 +756,7 @@ When using a self-hosted instance, the extraction will use your configured LLM.
824
756
}
825
757
```
826
758
827
-
### 9. Agent Tool (`firecrawl_agent`)
759
+
### 7. Agent Tool (`firecrawl_agent`)
828
760
829
761
Autonomous web research agent. This is a separate AI agent layer that independently browses the internet, searches for information, navigates through pages, and extracts structured data based on your query.
830
762
@@ -905,7 +837,7 @@ Then poll with `firecrawl_agent_status` using the returned job ID.
905
837
906
838
- Job ID for status checking. Use `firecrawl_agent_status` to poll for results.
907
839
908
-
### 10. Check Agent Status (`firecrawl_agent_status`)
840
+
### 8. Check Agent Status (`firecrawl_agent_status`)
909
841
910
842
Check the status of an agent job and retrieve results when complete. Use this to poll for results after starting an agent.
911
843
@@ -926,7 +858,7 @@ Check the status of an agent job and retrieve results when complete. Use this to
926
858
-`completed`: Research finished - response includes the extracted data
927
859
-`failed`: An error occurred
928
860
929
-
### 11. Monitor Tools (`firecrawl_monitor_*`)
861
+
### 9. Monitor Tools (`firecrawl_monitor_*`)
930
862
931
863
Create and manage recurring page monitors. Monitors run scheduled scrapes or crawls, diff each result against the last retained snapshot, and can notify by webhook or email.
932
864
@@ -1009,7 +941,6 @@ Example log messages:
1009
941
```
1010
942
[INFO] Firecrawl MCP Server initialized successfully
1011
943
[INFO] Starting scrape for URL: https://example.com
1012
-
[INFO] Batch operation queued with ID: batch_1
1013
944
[WARNING] Credit usage has reached warning threshold
Copy file name to clipboardExpand all lines: src/index.ts
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -1832,8 +1832,8 @@ server.addTool({
1832
1832
Starts a crawl job on a website and extracts content from all pages.
1833
1833
1834
1834
**Best for:** Extracting content from multiple related pages, when you need comprehensive coverage.
1835
-
**Not recommended for:** Extracting content from a single page (use scrape); when token limits are a concern (use map + batch_scrape); when you need fast results (crawling can be slow).
1836
-
**Warning:** Crawl responses can be very large and may exceed token limits. Limit the crawl depth and number of pages, or use map + batch_scrape for better control.
1835
+
**Not recommended for:** Extracting content from a single page (use scrape); when token limits are a concern (use map + scrape for tighter control); when you need fast results (crawling can be slow).
1836
+
**Warning:** Crawl responses can be very large and may exceed token limits. Limit the crawl depth and number of pages, or use map + scrape for tighter control.
1837
1837
**Common mistakes:** Setting limit or maxDiscoveryDepth too high (causes token overflow) or too low (causes missing pages); using crawl for a single page (use scrape instead). Using a /* wildcard is not recommended.
1838
1838
**Prompt Example:** "Get all blog posts from the first two levels of example.com/blog."
0 commit comments