chore: format markdown
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
This commit is contained in:
@@ -1,53 +1,64 @@
|
||||
# Cookie Env-Only Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use
|
||||
> superpowers:subagent-driven-development (recommended) or superpowers:executing-plans
|
||||
> to implement this plan task-by-task.
|
||||
> Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Remove cookie files and request-provided cookie overrides so all authenticated marketplace scraping reads raw `Cookie` header strings only from environment variables.
|
||||
**Goal:** Remove cookie files and request-provided cookie overrides so all authenticated
|
||||
marketplace scraping reads raw `Cookie` header strings only from environment variables.
|
||||
|
||||
**Architecture:** Collapse shared cookie loading to a single env-var reader in `packages/core/src/utils/cookies.ts`, then tighten Facebook and eBay core signatures to stop accepting request/file cookie inputs. Update the API and MCP adapters so they no longer advertise or forward cookie parameters, and rewrite docs/tests to match the env-only contract.
|
||||
**Architecture:** Collapse shared cookie loading to a single env-var reader in
|
||||
`packages/core/src/utils/cookies.ts`, then tighten Facebook and eBay core signatures to
|
||||
stop accepting request/file cookie inputs.
|
||||
Update the API and MCP adapters so they no longer advertise or forward cookie
|
||||
parameters, and rewrite docs/tests to match the env-only contract.
|
||||
|
||||
**Tech Stack:** Bun, TypeScript, Bun test, Biome, workspace package exports
|
||||
|
||||
---
|
||||
* * *
|
||||
|
||||
## File Map
|
||||
|
||||
- Modify: `packages/core/src/utils/cookies.ts`
|
||||
Purpose: remove JSON/file/request-source loading and keep env-only cookie parsing/formatting.
|
||||
- Modify: `packages/core/src/scrapers/facebook.ts`
|
||||
Purpose: drop `cookiesSource` / `cookiePath` arguments and env-only error text.
|
||||
- Modify: `packages/core/src/scrapers/ebay.ts`
|
||||
Purpose: remove `opts.cookies` request override and use env-only cookie loading.
|
||||
- Modify: `packages/core/src/index.ts`
|
||||
Purpose: keep exports aligned with tightened core signatures.
|
||||
- Modify: `packages/core/test/facebook-core.test.ts`
|
||||
Purpose: replace missing-file coverage with env-only auth tests.
|
||||
- Create: `packages/core/test/ebay-core.test.ts`
|
||||
Purpose: add dedicated eBay auth regression coverage instead of mixing it into Facebook tests.
|
||||
- Modify: `packages/api-server/src/routes/facebook.ts`
|
||||
Purpose: stop parsing/forwarding `cookies` query params.
|
||||
- Modify: `packages/api-server/src/routes/ebay.ts`
|
||||
Purpose: stop parsing/forwarding `cookies` query params.
|
||||
- Create: `packages/api-server/test/routes.test.ts`
|
||||
Purpose: verify Facebook/eBay routes ignore cookie query params and still call core correctly.
|
||||
- Modify: `packages/mcp-server/src/protocol/tools.ts`
|
||||
Purpose: remove Facebook/eBay cookie tool inputs and descriptions.
|
||||
- Modify: `packages/mcp-server/src/protocol/handler.ts`
|
||||
Purpose: stop mapping removed cookie tool inputs into API URLs.
|
||||
- Create: `packages/mcp-server/test/protocol.test.ts`
|
||||
Purpose: verify tool schemas and handler URL building no longer include Facebook/eBay cookie fields.
|
||||
- Modify: `cookies/AGENTS.md`
|
||||
Purpose: document env vars as the only supported cookie input.
|
||||
- Modify: `packages/core/src/utils/cookies.ts` Purpose: remove JSON/file/request-source
|
||||
loading and keep env-only cookie parsing/formatting.
|
||||
- Modify: `packages/core/src/scrapers/facebook.ts` Purpose: drop `cookiesSource` /
|
||||
`cookiePath` arguments and env-only error text.
|
||||
- Modify: `packages/core/src/scrapers/ebay.ts` Purpose: remove `opts.cookies` request
|
||||
override and use env-only cookie loading.
|
||||
- Modify: `packages/core/src/index.ts` Purpose: keep exports aligned with tightened core
|
||||
signatures.
|
||||
- Modify: `packages/core/test/facebook-core.test.ts` Purpose: replace missing-file
|
||||
coverage with env-only auth tests.
|
||||
- Create: `packages/core/test/ebay-core.test.ts` Purpose: add dedicated eBay auth
|
||||
regression coverage instead of mixing it into Facebook tests.
|
||||
- Modify: `packages/api-server/src/routes/facebook.ts` Purpose: stop parsing/forwarding
|
||||
`cookies` query params.
|
||||
- Modify: `packages/api-server/src/routes/ebay.ts` Purpose: stop parsing/forwarding
|
||||
`cookies` query params.
|
||||
- Create: `packages/api-server/test/routes.test.ts` Purpose: verify Facebook/eBay routes
|
||||
ignore cookie query params and still call core correctly.
|
||||
- Modify: `packages/mcp-server/src/protocol/tools.ts` Purpose: remove Facebook/eBay
|
||||
cookie tool inputs and descriptions.
|
||||
- Modify: `packages/mcp-server/src/protocol/handler.ts` Purpose: stop mapping removed
|
||||
cookie tool inputs into API URLs.
|
||||
- Create: `packages/mcp-server/test/protocol.test.ts` Purpose: verify tool schemas and
|
||||
handler URL building no longer include Facebook/eBay cookie fields.
|
||||
- Modify: `cookies/AGENTS.md` Purpose: document env vars as the only supported cookie
|
||||
input.
|
||||
|
||||
### Task 1: Lock core cookie utilities to env-only loading
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `packages/core/src/utils/cookies.ts:19-227`
|
||||
|
||||
- Test: `packages/core/test/facebook-core.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Add or replace the auth-source test block in `packages/core/test/facebook-core.test.ts` with env-only expectations:
|
||||
Add or replace the auth-source test block in `packages/core/test/facebook-core.test.ts`
|
||||
with env-only expectations:
|
||||
|
||||
```ts
|
||||
test("should load Facebook cookies from FACEBOOK_COOKIE env var", async () => {
|
||||
@@ -85,12 +96,14 @@ test("should reject missing Facebook auth env var", async () => {
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `bun test packages/core/test/facebook-core.test.ts`
|
||||
Expected: FAIL because the current implementation still allows missing env values to fall through to file/request-based behavior and does not emit the new env-only error.
|
||||
Run: `bun test packages/core/test/facebook-core.test.ts` Expected: FAIL because the
|
||||
current implementation still allows missing env values to fall through to
|
||||
file/request-based behavior and does not emit the new env-only error.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
Replace the multi-source loader in `packages/core/src/utils/cookies.ts` with an env-only loader. The target shape is:
|
||||
Replace the multi-source loader in `packages/core/src/utils/cookies.ts` with an env-only
|
||||
loader. The target shape is:
|
||||
|
||||
```ts
|
||||
export interface CookieConfig {
|
||||
@@ -129,8 +142,8 @@ Delete the now-dead helpers and types that exist only for JSON/file/request load
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `bun test packages/core/test/facebook-core.test.ts`
|
||||
Expected: PASS for the new env-only tests.
|
||||
Run: `bun test packages/core/test/facebook-core.test.ts` Expected: PASS for the new
|
||||
env-only tests.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
@@ -142,10 +155,15 @@ git commit -m "refactor: make cookie loading env-only"
|
||||
### Task 2: Tighten Facebook core APIs to the new contract
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `packages/core/src/scrapers/facebook.ts:23-29`
|
||||
|
||||
- Modify: `packages/core/src/scrapers/facebook.ts:214-228`
|
||||
|
||||
- Modify: `packages/core/src/scrapers/facebook.ts:823-929`
|
||||
|
||||
- Modify: `packages/core/src/index.ts:5-15`
|
||||
|
||||
- Test: `packages/core/test/facebook-core.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
@@ -171,8 +189,9 @@ test("should fail Facebook item fetch when FACEBOOK_COOKIE is unset", async () =
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `bun test packages/core/test/facebook-core.test.ts`
|
||||
Expected: FAIL because the current function signatures and error text still mention parameter/file-based auth paths.
|
||||
Run: `bun test packages/core/test/facebook-core.test.ts` Expected: FAIL because the
|
||||
current function signatures and error text still mention parameter/file-based auth
|
||||
paths.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
@@ -206,12 +225,14 @@ console.warn(
|
||||
);
|
||||
```
|
||||
|
||||
Remove the extra cookie arguments from `fetchFacebookItem(...)` and keep `packages/core/src/index.ts` exporting the tightened functions without the old parameter contract.
|
||||
Remove the extra cookie arguments from `fetchFacebookItem(...)` and keep
|
||||
`packages/core/src/index.ts` exporting the tightened functions without the old parameter
|
||||
contract.
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `bun test packages/core/test/facebook-core.test.ts`
|
||||
Expected: PASS with the new env-only Facebook API surface.
|
||||
Run: `bun test packages/core/test/facebook-core.test.ts` Expected: PASS with the new
|
||||
env-only Facebook API surface.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
@@ -223,8 +244,11 @@ git commit -m "refactor: remove facebook cookie overrides"
|
||||
### Task 3: Tighten eBay core APIs to env-only auth
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `packages/core/src/scrapers/ebay.ts:9-15`
|
||||
|
||||
- Modify: `packages/core/src/scrapers/ebay.ts:337-389`
|
||||
|
||||
- Create: `packages/core/test/ebay-core.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
@@ -249,8 +273,8 @@ test("should warn and continue without eBay cookies when EBAY_COOKIE is unset",
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `bun test packages/core/test/ebay-core.test.ts`
|
||||
Expected: FAIL because `loadEbayCookies` still accepts request overrides and mentions file/json sources.
|
||||
Run: `bun test packages/core/test/ebay-core.test.ts` Expected: FAIL because
|
||||
`loadEbayCookies` still accepts request overrides and mentions file/json sources.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
@@ -276,12 +300,13 @@ async function loadEbayCookies(): Promise<string | undefined> {
|
||||
}
|
||||
```
|
||||
|
||||
Then remove `cookies` from `fetchEbayItems(..., opts)` and the destructuring that feeds it into `loadEbayCookies()`.
|
||||
Then remove `cookies` from `fetchEbayItems(..., opts)` and the destructuring that feeds
|
||||
it into `loadEbayCookies()`.
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `bun test packages/core/test/ebay-core.test.ts`
|
||||
Expected: PASS for the eBay env-only regression coverage.
|
||||
Run: `bun test packages/core/test/ebay-core.test.ts` Expected: PASS for the eBay
|
||||
env-only regression coverage.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
@@ -293,13 +318,17 @@ git commit -m "refactor: make ebay auth env-only"
|
||||
### Task 4: Remove cookie query parameters from the API adapter
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `packages/api-server/src/routes/facebook.ts:3-33`
|
||||
|
||||
- Modify: `packages/api-server/src/routes/ebay.ts:3-52`
|
||||
|
||||
- Create: `packages/api-server/test/routes.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Create `packages/api-server/test/routes.test.ts` and mock `@marketplace-scrapers/core` so the route contract is explicit:
|
||||
Create `packages/api-server/test/routes.test.ts` and mock `@marketplace-scrapers/core`
|
||||
so the route contract is explicit:
|
||||
|
||||
```ts
|
||||
import { afterEach, describe, expect, mock, test } from "bun:test";
|
||||
@@ -347,8 +376,9 @@ test("ebayRoute ignores cookies query parameter", async () => {
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `bun test packages/api-server/test/routes.test.ts`
|
||||
Expected: FAIL because the current routes still parse `reqUrl.searchParams.get("cookies")` and forward it downstream.
|
||||
Run: `bun test packages/api-server/test/routes.test.ts` Expected: FAIL because the
|
||||
current routes still parse `reqUrl.searchParams.get("cookies")` and forward it
|
||||
downstream.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
@@ -383,8 +413,8 @@ const items = await fetchEbayItems(SEARCH_QUERY, 1, {
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `bun test packages/api-server/test/routes.test.ts`
|
||||
Expected: PASS for route coverage and no remaining adapter references to `cookies` for Facebook/eBay.
|
||||
Run: `bun test packages/api-server/test/routes.test.ts` Expected: PASS for route
|
||||
coverage and no remaining adapter references to `cookies` for Facebook/eBay.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
@@ -396,13 +426,17 @@ git commit -m "refactor: remove api cookie query overrides"
|
||||
### Task 5: Remove cookie inputs from MCP tool schemas and request mapping
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `packages/mcp-server/src/protocol/tools.ts:65-148`
|
||||
|
||||
- Modify: `packages/mcp-server/src/protocol/handler.ts:154-211`
|
||||
|
||||
- Create: `packages/mcp-server/test/protocol.test.ts`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Create `packages/mcp-server/test/protocol.test.ts` with schema and URL-building assertions:
|
||||
Create `packages/mcp-server/test/protocol.test.ts` with schema and URL-building
|
||||
assertions:
|
||||
|
||||
```ts
|
||||
import { expect, mock, test } from "bun:test";
|
||||
@@ -445,8 +479,8 @@ expect(calledUrl).not.toContain("cookies=");
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `bun test packages/mcp-server/test/protocol.test.ts`
|
||||
Expected: FAIL because the current MCP schema and handler still expose and forward those inputs.
|
||||
Run: `bun test packages/mcp-server/test/protocol.test.ts` Expected: FAIL because the
|
||||
current MCP schema and handler still expose and forward those inputs.
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
@@ -465,12 +499,13 @@ Delete the Facebook/eBay cookie tool properties and handler mapping:
|
||||
// if (args.cookies) params.append("cookies", args.cookies);
|
||||
```
|
||||
|
||||
Leave Kijiji alone; this plan only changes Facebook/eBay env-only auth paths defined by the approved spec.
|
||||
Leave Kijiji alone; this plan only changes Facebook/eBay env-only auth paths defined by
|
||||
the approved spec.
|
||||
|
||||
- [ ] **Step 4: Run test to verify it passes**
|
||||
|
||||
Run: `bun test packages/mcp-server/test/protocol.test.ts`
|
||||
Expected: PASS with MCP definitions and handler mapping in sync.
|
||||
Run: `bun test packages/mcp-server/test/protocol.test.ts` Expected: PASS with MCP
|
||||
definitions and handler mapping in sync.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
@@ -482,12 +517,16 @@ git commit -m "refactor: remove mcp cookie parameters"
|
||||
### Task 6: Rewrite cookie documentation and run full verification
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `cookies/AGENTS.md:9-85`
|
||||
- Modify: `docs/superpowers/specs/2026-04-21-cookie-env-only-design.md` only if implementation reveals a spec mismatch
|
||||
|
||||
- Modify: `docs/superpowers/specs/2026-04-21-cookie-env-only-design.md` only if
|
||||
implementation reveals a spec mismatch
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Treat docs drift as a contract failure. Capture the required state before editing:
|
||||
Treat docs drift as a contract failure.
|
||||
Capture the required state before editing:
|
||||
|
||||
```md
|
||||
- Cookie setup docs mention env vars only for Facebook and eBay
|
||||
@@ -497,14 +536,14 @@ Treat docs drift as a contract failure. Capture the required state before editin
|
||||
|
||||
- [ ] **Step 2: Run verification to prove current docs are stale**
|
||||
|
||||
Run: `rg -n "facebook\.json|ebay\.json|cookies=" cookies/AGENTS.md`
|
||||
Expected: matches found
|
||||
Run: `rg -n "facebook\.json|ebay\.json|cookies=" cookies/AGENTS.md` Expected: matches
|
||||
found
|
||||
|
||||
- [ ] **Step 3: Write minimal implementation**
|
||||
|
||||
Rewrite the cookie setup doc so Facebook and eBay each show only env-var setup:
|
||||
|
||||
```md
|
||||
````md
|
||||
## Cookie Configuration
|
||||
|
||||
All supported authenticated scrapers read cookies only from environment variables.
|
||||
@@ -513,14 +552,14 @@ All supported authenticated scrapers read cookies only from environment variable
|
||||
|
||||
```bash
|
||||
export FACEBOOK_COOKIE='c_user=123; xs=token; fr=request'
|
||||
```
|
||||
````
|
||||
|
||||
### eBay
|
||||
|
||||
```bash
|
||||
export EBAY_COOKIE='s=VALUE; ds2=VALUE; ebay=VALUE'
|
||||
```
|
||||
```
|
||||
````
|
||||
|
||||
Remove the file-based and request-parameter sections entirely.
|
||||
|
||||
@@ -534,10 +573,14 @@ Expected: all commands pass
|
||||
```bash
|
||||
git add cookies/AGENTS.md docs/superpowers/specs/2026-04-21-cookie-env-only-design.md
|
||||
git commit -m "docs: align cookie setup with env-only auth"
|
||||
```
|
||||
````
|
||||
|
||||
## Self-Review
|
||||
|
||||
- Spec coverage check: shared cookie utils, Facebook, eBay, API adapter, MCP adapter, tests, and docs each have explicit tasks.
|
||||
- Placeholder scan: concrete test files are now named for eBay core, API routes, and MCP protocol coverage.
|
||||
- Type consistency check: `ensureCookies(config)` is the single shared loader name used across Tasks 1-3, and Facebook/eBay route signatures stay aligned with the core changes.
|
||||
- Spec coverage check: shared cookie utils, Facebook, eBay, API adapter, MCP adapter,
|
||||
tests, and docs each have explicit tasks.
|
||||
- Placeholder scan: concrete test files are now named for eBay core, API routes, and MCP
|
||||
protocol coverage.
|
||||
- Type consistency check: `ensureCookies(config)` is the single shared loader name used
|
||||
across Tasks 1-3, and Facebook/eBay route signatures stay aligned with the core
|
||||
changes.
|
||||
|
||||
Reference in New Issue
Block a user