feat: implement cookie priority hierarchy (URL param > env var > file) for Facebook and eBay scrapers

This commit is contained in:
2026-01-23 15:32:07 -05:00
parent df0c528535
commit cf9784a565
8 changed files with 262 additions and 85 deletions

View File

@@ -1,24 +1,33 @@
# Facebook Marketplace Cookies Setup
# Marketplace Cookies Setup
To use the Facebook Marketplace scraper, you need to provide valid Facebook session cookies.
Both Facebook Marketplace and eBay require valid session cookies to bypass bot detection and access listings.
## Option 1: Cookies File (`facebook.json`)
## Cookie Priority Hierarchy
1. Log into Facebook in your browser
2. Open Developer Tools → Network tab
3. Visit facebook.com/marketplace (ensure you're logged in)
4. Look for any marketplace-related requests in the Network tab
5. Export cookies from the browser's Application/Storage → Cookies section
6. Save the cookies as a JSON array to `facebook.json`
All scrapers follow this priority order (highest to lowest):
1. **URL Parameter** - Passed directly in API/MCP request (overrides all)
2. **Environment Variable** - Set as `FACEBOOK_COOKIE` or `EBAY_COOKIE`
3. **Cookie File** - Stored in `facebook.json` or `ebay.json` (fallback)
The `facebook.json` file should contain Facebook session cookies, particularly:
---
## Facebook Marketplace (`facebook.json`)
### Required Cookies
- `c_user`: Your Facebook user ID
- `xs`: Facebook session token
- `fr`: Facebook request token
- `datr`: Data attribution token
- `sb`: Session browser token
Example structure:
### Setup Methods
**Method 1: Cookie File (Lowest Priority)**
1. Log into Facebook in your browser
2. Open Developer Tools → Application/Storage → Cookies
3. Export cookies as JSON array to `facebook.json`
Example `facebook.json`:
```json
[
{
@@ -27,26 +36,59 @@ Example structure:
"domain": ".facebook.com",
"path": "/",
"secure": true
},
// ... other cookies
}
]
```
## Option 2: URL Parameter
You can pass cookies directly via the `cookies` URL parameter:
**Method 2: Environment Variable**
```bash
export FACEBOOK_COOKIE='c_user=123; xs=token; fr=request'
```
GET /api/facebook?q=laptop&cookies=[{"name":"c_user","value":"123","domain":".facebook.com",...}]
**Method 3: URL Parameter (Highest Priority)**
```
GET /api/facebook?q=laptop&cookies=[{"name":"c_user","value":"123",...}]
```
---
## eBay (`ebay.json`)
eBay has aggressive bot detection that blocks requests without valid session cookies.
### Setup Methods
**Method 1: Cookie File (Lowest Priority)**
1. Log into eBay in your browser
2. Open Developer Tools → Network tab
3. Visit ebay.ca and inspect any request headers
4. Copy the full `Cookie` header value
5. Save as plain text to `ebay.json` (see `ebay.json.example`)
Example `ebay.json`:
```
s=VALUE; ds2=VALUE; ebay=VALUE; dp1=VALUE; nonsession=VALUE
```
**Method 2: Environment Variable**
```bash
export EBAY_COOKIE='s=VALUE; ds2=VALUE; ebay=VALUE'
```
**Method 3: URL Parameter (Highest Priority)**
```
GET /api/ebay?q=laptop&cookies=s=VALUE;ds2=VALUE;ebay=VALUE
```
---
## Important Notes
- Cookies must be from an active Facebook session
- Cookies expire, so you may need to refresh them periodically
- Never share real cookies or commit them to version control
- Facebook may block automated scraping even with valid cookies
- Cookies must be from active browser sessions
- Cookies expire and need periodic refresh
- **NEVER** commit real cookies to version control
- Platforms may still block automated scraping despite valid cookies
## Security
The cookies file is intentionally left out of version control for security reasons.</content>
All `*.json` files in this directory are git-ignored for security.</content>

View File

@@ -0,0 +1 @@
s=YOUR_VALUE; ds2=YOUR_VALUE; ebay=YOUR_VALUE; dp1=YOUR_VALUE; nonsession=YOUR_VALUE