@@ -213,37 +213,12 @@ Hosted Firecrawl can issue OAuth **access tokens** (`fco_…`) via the authoriza
213213
214214Use ** access** tokens (` fco_… ` ) only. Refresh tokens (` fcr_… ` ) must be exchanged at the token endpoint, not passed to the scrape/search API.
215215
216- #### Optional Configuration
217-
218- ##### Retry Configuration
219-
220- - ` FIRECRAWL_RETRY_MAX_ATTEMPTS ` : Maximum number of retry attempts (default: 3)
221- - ` FIRECRAWL_RETRY_INITIAL_DELAY ` : Initial delay in milliseconds before first retry (default: 1000)
222- - ` FIRECRAWL_RETRY_MAX_DELAY ` : Maximum delay in milliseconds between retries (default: 10000)
223- - ` FIRECRAWL_RETRY_BACKOFF_FACTOR ` : Exponential backoff multiplier (default: 2)
224-
225- ##### Credit Usage Monitoring
226-
227- - ` FIRECRAWL_CREDIT_WARNING_THRESHOLD ` : Credit usage warning threshold (default: 1000)
228- - ` FIRECRAWL_CREDIT_CRITICAL_THRESHOLD ` : Credit usage critical threshold (default: 100)
229-
230216### Configuration Examples
231217
232- For cloud API usage with custom retry and credit monitoring :
218+ For cloud API usage:
233219
234220``` bash
235- # Required for cloud API
236221export FIRECRAWL_API_KEY=your-api-key
237-
238- # Optional retry configuration
239- export FIRECRAWL_RETRY_MAX_ATTEMPTS=5 # Increase max retry attempts
240- export FIRECRAWL_RETRY_INITIAL_DELAY=2000 # Start with 2s delay
241- export FIRECRAWL_RETRY_MAX_DELAY=30000 # Maximum 30s delay
242- export FIRECRAWL_RETRY_BACKOFF_FACTOR=3 # More aggressive backoff
243-
244- # Optional credit monitoring
245- export FIRECRAWL_CREDIT_WARNING_THRESHOLD=2000 # Warning at 2000 credits
246- export FIRECRAWL_CREDIT_CRITICAL_THRESHOLD=500 # Critical at 500 credits
247222```
248223
249224For self-hosted instance:
@@ -254,10 +229,6 @@ export FIRECRAWL_API_URL=https://firecrawl.your-domain.com
254229
255230# Optional authentication for self-hosted
256231export FIRECRAWL_API_KEY=your-api-key # If your instance requires auth
257-
258- # Custom retry configuration
259- export FIRECRAWL_RETRY_MAX_ATTEMPTS=10
260- export FIRECRAWL_RETRY_INITIAL_DELAY=500 # Start with faster retries
261232```
262233
263234### Usage with Claude Desktop
@@ -271,67 +242,13 @@ Add this to your `claude_desktop_config.json`:
271242 "command" : " npx" ,
272243 "args" : [" -y" , " firecrawl-mcp" ],
273244 "env" : {
274- "FIRECRAWL_API_KEY" : " YOUR_API_KEY_HERE" ,
275-
276- "FIRECRAWL_RETRY_MAX_ATTEMPTS" : " 5" ,
277- "FIRECRAWL_RETRY_INITIAL_DELAY" : " 2000" ,
278- "FIRECRAWL_RETRY_MAX_DELAY" : " 30000" ,
279- "FIRECRAWL_RETRY_BACKOFF_FACTOR" : " 3" ,
280-
281- "FIRECRAWL_CREDIT_WARNING_THRESHOLD" : " 2000" ,
282- "FIRECRAWL_CREDIT_CRITICAL_THRESHOLD" : " 500"
245+ "FIRECRAWL_API_KEY" : " YOUR_API_KEY_HERE"
283246 }
284247 }
285248 }
286249}
287250```
288251
289- ### System Configuration
290-
291- The server includes several configurable parameters that can be set via environment variables. Here are the default values if not configured:
292-
293- ``` typescript
294- const CONFIG = {
295- retry: {
296- maxAttempts: 3 , // Number of retry attempts for rate-limited requests
297- initialDelay: 1000 , // Initial delay before first retry (in milliseconds)
298- maxDelay: 10000 , // Maximum delay between retries (in milliseconds)
299- backoffFactor: 2 , // Multiplier for exponential backoff
300- },
301- credit: {
302- warningThreshold: 1000 , // Warn when credit usage reaches this level
303- criticalThreshold: 100 , // Critical alert when credit usage reaches this level
304- },
305- };
306- ```
307-
308- These configurations control:
309-
310- 1 . ** Retry Behavior**
311-
312- - Automatically retries failed requests due to rate limits
313- - Uses exponential backoff to avoid overwhelming the API
314- - Example: With default settings, retries will be attempted at:
315- - 1st retry: 1 second delay
316- - 2nd retry: 2 seconds delay
317- - 3rd retry: 4 seconds delay (capped at maxDelay)
318-
319- 2 . ** Credit Usage Monitoring**
320- - Tracks API credit consumption for cloud API usage
321- - Provides warnings at specified thresholds
322- - Helps prevent unexpected service interruption
323- - Example: With default settings:
324- - Warning at 1000 credits remaining
325- - Critical alert at 100 credits remaining
326-
327- ### Rate Limiting
328-
329- The server uses Firecrawl's built-in rate limiting:
330-
331- - Automatic rate limit handling with exponential backoff
332- - Smart request queuing and throttling
333- - Automatic retries for transient errors
334-
335252## How to Choose a Tool
336253
337254Use this guide to select the right tool for your task:
@@ -1005,7 +922,6 @@ The server includes comprehensive logging:
1005922
1006923- Operation status and progress
1007924- Performance metrics
1008- - Credit usage monitoring
1009925- Rate limit tracking
1010926- Error conditions
1011927
@@ -1014,18 +930,15 @@ Example log messages:
1014930```
1015931[INFO] Firecrawl MCP Server initialized successfully
1016932[INFO] Starting scrape for URL: https://example.com
1017- [WARNING] Credit usage has reached warning threshold
1018- [ERROR] Rate limit exceeded, retrying in 2s...
933+ [ERROR] Rate limit exceeded
1019934```
1020935
1021936## Error Handling
1022937
1023938The server provides robust error handling:
1024939
1025- - Automatic retries for transient errors
1026- - Rate limit handling with backoff
940+ - API rate-limit errors surfaced to the MCP client
1027941- Detailed error messages
1028- - Credit usage warnings
1029942- Network resilience
1030943
1031944Example error response:
@@ -1035,7 +948,7 @@ Example error response:
1035948 "content" : [
1036949 {
1037950 "type" : " text" ,
1038- "text" : " Error: Rate limit exceeded. Retrying in 2 seconds... "
951+ "text" : " Error: Rate limit exceeded"
1039952 }
1040953 ],
1041954 "isError" : true
0 commit comments