ca-marketplace-scraper/docs/superpowers/plans/2026-04-21-cookie-env-only.md

# Cookie Env-Only Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Remove cookie files and request-provided cookie overrides so all authenticated marketplace scraping reads raw `Cookie` header strings only from environment variables.

**Architecture:** Collapse shared cookie loading to a single env-var reader in `packages/core/src/utils/cookies.ts`, then tighten Facebook and eBay core signatures to stop accepting request/file cookie inputs. Update the API and MCP adapters so they no longer advertise or forward cookie parameters, and rewrite docs/tests to match the env-only contract.

**Tech Stack:** Bun, TypeScript, Bun test, Biome, workspace package exports

---

## File Map

- Modify: `packages/core/src/utils/cookies.ts`
  Purpose: remove JSON/file/request-source loading and keep env-only cookie parsing/formatting.
- Modify: `packages/core/src/scrapers/facebook.ts`
  Purpose: drop `cookiesSource` / `cookiePath` arguments and env-only error text.
- Modify: `packages/core/src/scrapers/ebay.ts`
  Purpose: remove `opts.cookies` request override and use env-only cookie loading.
- Modify: `packages/core/src/index.ts`
  Purpose: keep exports aligned with tightened core signatures.
- Modify: `packages/core/test/facebook-core.test.ts`
  Purpose: replace missing-file coverage with env-only auth tests.
- Create: `packages/core/test/ebay-core.test.ts`
  Purpose: add dedicated eBay auth regression coverage instead of mixing it into Facebook tests.
- Modify: `packages/api-server/src/routes/facebook.ts`
  Purpose: stop parsing/forwarding `cookies` query params.
- Modify: `packages/api-server/src/routes/ebay.ts`
  Purpose: stop parsing/forwarding `cookies` query params.
- Create: `packages/api-server/test/routes.test.ts`
  Purpose: verify Facebook/eBay routes ignore cookie query params and still call core correctly.
- Modify: `packages/mcp-server/src/protocol/tools.ts`
  Purpose: remove Facebook/eBay cookie tool inputs and descriptions.
- Modify: `packages/mcp-server/src/protocol/handler.ts`
  Purpose: stop mapping removed cookie tool inputs into API URLs.
- Create: `packages/mcp-server/test/protocol.test.ts`
  Purpose: verify tool schemas and handler URL building no longer include Facebook/eBay cookie fields.
- Modify: `cookies/AGENTS.md`
  Purpose: document env vars as the only supported cookie input.

### Task 1: Lock core cookie utilities to env-only loading

**Files:**
- Modify: `packages/core/src/utils/cookies.ts:19-227`
- Test: `packages/core/test/facebook-core.test.ts`

- [ ] **Step 1: Write the failing test**

Add or replace the auth-source test block in `packages/core/test/facebook-core.test.ts` with env-only expectations:

```ts
test("should load Facebook cookies from FACEBOOK_COOKIE env var", async () => {
  const previous = process.env.FACEBOOK_COOKIE;
  process.env.FACEBOOK_COOKIE = "c_user=123; xs=abc";

  try {
    const cookies = await ensureFacebookCookies();

    expect(cookies.map((cookie) => cookie.name)).toEqual(["c_user", "xs"]);
  } finally {
    if (previous === undefined) {
      delete process.env.FACEBOOK_COOKIE;
    } else {
      process.env.FACEBOOK_COOKIE = previous;
    }
  }
});

test("should reject missing Facebook auth env var", async () => {
  const previous = process.env.FACEBOOK_COOKIE;
  delete process.env.FACEBOOK_COOKIE;

  try {
    await expect(ensureFacebookCookies()).rejects.toThrow(
      "Provide cookies via FACEBOOK_COOKIE environment variable as a raw Cookie header string",
    );
  } finally {
    if (previous !== undefined) {
      process.env.FACEBOOK_COOKIE = previous;
    }
  }
});
```

- [ ] **Step 2: Run test to verify it fails**

Run: `bun test packages/core/test/facebook-core.test.ts`
Expected: FAIL because the current implementation still allows missing env values to fall through to file/request-based behavior and does not emit the new env-only error.

- [ ] **Step 3: Write minimal implementation**

Replace the multi-source loader in `packages/core/src/utils/cookies.ts` with an env-only loader. The target shape is:

```ts
export interface CookieConfig {
  name: string;
  domain: string;
  envVar: string;
}

export async function ensureCookies(config: CookieConfig): Promise<Cookie[]> {
  const envValue = process.env[config.envVar];
  const cookies = parseCookieString(envValue ?? "", config.domain);

  if (cookies.length > 0) {
    console.log(
      `Loaded ${cookies.length} ${config.name} cookies from ${config.envVar} env var`,
    );
    return cookies;
  }

  throw new Error(
    `No valid ${config.name} cookies found. Provide cookies via ${config.envVar} environment variable as a raw Cookie header string.`,
  );
}
```

Delete the now-dead helpers and types that exist only for JSON/file/request loading:

```ts
// Remove:
// - parseJsonCookies
// - parseCookiesAuto
// - loadCookiesFromFile
// - loadCookiesOptional
// - CookieConfig.filePath
```

- [ ] **Step 4: Run test to verify it passes**

Run: `bun test packages/core/test/facebook-core.test.ts`
Expected: PASS for the new env-only tests.

- [ ] **Step 5: Commit**

```bash
git add packages/core/src/utils/cookies.ts packages/core/test/facebook-core.test.ts
git commit -m "refactor: make cookie loading env-only"
```

### Task 2: Tighten Facebook core APIs to the new contract

**Files:**
- Modify: `packages/core/src/scrapers/facebook.ts:23-29`
- Modify: `packages/core/src/scrapers/facebook.ts:214-228`
- Modify: `packages/core/src/scrapers/facebook.ts:823-929`
- Modify: `packages/core/src/index.ts:5-15`
- Test: `packages/core/test/facebook-core.test.ts`

- [ ] **Step 1: Write the failing test**

Add a focused test proving Facebook item fetch now depends only on env auth:

```ts
test("should fail Facebook item fetch when FACEBOOK_COOKIE is unset", async () => {
  const previous = process.env.FACEBOOK_COOKIE;
  delete process.env.FACEBOOK_COOKIE;

  try {
    await expect(fetchFacebookItem("123")).rejects.toThrow(
      "Provide cookies via FACEBOOK_COOKIE environment variable as a raw Cookie header string",
    );
  } finally {
    if (previous !== undefined) {
      process.env.FACEBOOK_COOKIE = previous;
    }
  }
});
```

- [ ] **Step 2: Run test to verify it fails**

Run: `bun test packages/core/test/facebook-core.test.ts`
Expected: FAIL because the current function signatures and error text still mention parameter/file-based auth paths.

- [ ] **Step 3: Write minimal implementation**

Tighten the Facebook signatures and messages:

```ts
const FACEBOOK_COOKIE_CONFIG: CookieConfig = {
  name: "Facebook",
  domain: ".facebook.com",
  envVar: "FACEBOOK_COOKIE",
};

export async function ensureFacebookCookies(): Promise<Cookie[]> {
  return ensureCookies(FACEBOOK_COOKIE_CONFIG);
}

export default async function fetchFacebookItems(
  SEARCH_QUERY: string,
  REQUESTS_PER_SECOND = 1,
  LOCATION = "toronto",
  MAX_ITEMS = 25,
) {
  const cookies = await ensureFacebookCookies();
```

Also change the stale auth warnings:

```ts
console.warn(
  "This might indicate invalid or expired cookies. Update FACEBOOK_COOKIE with a fresh raw Cookie header string.",
);
```

Remove the extra cookie arguments from `fetchFacebookItem(...)` and keep `packages/core/src/index.ts` exporting the tightened functions without the old parameter contract.

- [ ] **Step 4: Run test to verify it passes**

Run: `bun test packages/core/test/facebook-core.test.ts`
Expected: PASS with the new env-only Facebook API surface.

- [ ] **Step 5: Commit**

```bash
git add packages/core/src/scrapers/facebook.ts packages/core/src/index.ts packages/core/test/facebook-core.test.ts
git commit -m "refactor: remove facebook cookie overrides"
```

### Task 3: Tighten eBay core APIs to env-only auth

**Files:**
- Modify: `packages/core/src/scrapers/ebay.ts:9-15`
- Modify: `packages/core/src/scrapers/ebay.ts:337-389`
- Create: `packages/core/test/ebay-core.test.ts`

- [ ] **Step 1: Write the failing test**

Create `packages/core/test/ebay-core.test.ts` with a dedicated auth regression test:

```ts
test("should warn and continue without eBay cookies when EBAY_COOKIE is unset", async () => {
  const previous = process.env.EBAY_COOKIE;
  delete process.env.EBAY_COOKIE;

  try {
    const cookies = await loadEbayCookies();
    expect(cookies).toBeUndefined();
  } finally {
    if (previous !== undefined) {
      process.env.EBAY_COOKIE = previous;
    }
  }
});
```

- [ ] **Step 2: Run test to verify it fails**

Run: `bun test packages/core/test/ebay-core.test.ts`
Expected: FAIL because `loadEbayCookies` still accepts request overrides and mentions file/json sources.

- [ ] **Step 3: Write minimal implementation**

Remove file/request branches from the eBay cookie path:

```ts
const EBAY_COOKIE_CONFIG: CookieConfig = {
  name: "eBay",
  domain: ".ebay.ca",
  envVar: "EBAY_COOKIE",
};

async function loadEbayCookies(): Promise<string | undefined> {
  try {
    const cookies = await ensureCookies(EBAY_COOKIE_CONFIG);
    return formatCookiesForHeader(cookies, "www.ebay.ca");
  } catch {
    console.warn(
      "No valid eBay cookies found in EBAY_COOKIE. eBay may block requests without a raw Cookie header string.",
    );
    return undefined;
  }
}
```

Then remove `cookies` from `fetchEbayItems(..., opts)` and the destructuring that feeds it into `loadEbayCookies()`.

- [ ] **Step 4: Run test to verify it passes**

Run: `bun test packages/core/test/ebay-core.test.ts`
Expected: PASS for the eBay env-only regression coverage.

- [ ] **Step 5: Commit**

```bash
git add packages/core/src/scrapers/ebay.ts packages/core/test/ebay-core.test.ts
git commit -m "refactor: make ebay auth env-only"
```

### Task 4: Remove cookie query parameters from the API adapter

**Files:**
- Modify: `packages/api-server/src/routes/facebook.ts:3-33`
- Modify: `packages/api-server/src/routes/ebay.ts:3-52`
- Create: `packages/api-server/test/routes.test.ts`

- [ ] **Step 1: Write the failing test**

Create `packages/api-server/test/routes.test.ts` and mock `@marketplace-scrapers/core` so the route contract is explicit:

```ts
import { afterEach, describe, expect, mock, test } from "bun:test";

const fetchFacebookItems = mock(() => Promise.resolve([{ title: "item" }]));
const fetchEbayItems = mock(() => Promise.resolve([{ title: "item" }]));

mock.module("@marketplace-scrapers/core", () => ({
  fetchFacebookItems,
  fetchEbayItems,
}));

import { ebayRoute } from "../src/routes/ebay";
import { facebookRoute } from "../src/routes/facebook";

afterEach(() => {
  fetchFacebookItems.mockReset();
  fetchEbayItems.mockReset();
});

test("facebookRoute ignores cookies query parameter", async () => {
  await facebookRoute(
    new Request("http://localhost/api/facebook?q=laptop&location=toronto&maxItems=3&cookies=c_user=1"),
  );

  expect(fetchFacebookItems).toHaveBeenCalledWith("laptop", 1, "toronto", 3);
});

test("ebayRoute ignores cookies query parameter", async () => {
  await ebayRoute(
    new Request("http://localhost/api/ebay?q=laptop&cookies=s%3D1&buyItNowOnly=true"),
  );

  expect(fetchEbayItems).toHaveBeenCalledWith("laptop", 1, {
    minPrice: undefined,
    maxPrice: undefined,
    strictMode: false,
    exclusions: [],
    keywords: ["laptop"],
    buyItNowOnly: true,
    canadaOnly: true,
  });
});
```

- [ ] **Step 2: Run test to verify it fails**

Run: `bun test packages/api-server/test/routes.test.ts`
Expected: FAIL because the current routes still parse `reqUrl.searchParams.get("cookies")` and forward it downstream.

- [ ] **Step 3: Write minimal implementation**

Delete the cookie query-parameter handling from both routes:

```ts
// packages/api-server/src/routes/facebook.ts
/**
 * GET /api/facebook?q={query}&location={location}
 * Search Facebook Marketplace for listings
 */

const items = await fetchFacebookItems(SEARCH_QUERY, 1, LOCATION, maxItems);
```

```ts
// packages/api-server/src/routes/ebay.ts
/**
 * GET /api/ebay?q={query}&minPrice={minPrice}&maxPrice={maxPrice}&strictMode={strictMode}&exclusions={exclusions}&keywords={keywords}&buyItNowOnly={buyItNowOnly}&canadaOnly={canadaOnly}
 */

const items = await fetchEbayItems(SEARCH_QUERY, 1, {
  minPrice,
  maxPrice,
  strictMode,
  exclusions,
  keywords,
  buyItNowOnly,
  canadaOnly,
});
```

- [ ] **Step 4: Run test to verify it passes**

Run: `bun test packages/api-server/test/routes.test.ts`
Expected: PASS for route coverage and no remaining adapter references to `cookies` for Facebook/eBay.

- [ ] **Step 5: Commit**

```bash
git add packages/api-server/src/routes/facebook.ts packages/api-server/src/routes/ebay.ts packages/api-server/test/routes.test.ts
git commit -m "refactor: remove api cookie query overrides"
```

### Task 5: Remove cookie inputs from MCP tool schemas and request mapping

**Files:**
- Modify: `packages/mcp-server/src/protocol/tools.ts:65-148`
- Modify: `packages/mcp-server/src/protocol/handler.ts:154-211`
- Create: `packages/mcp-server/test/protocol.test.ts`

- [ ] **Step 1: Write the failing test**

Create `packages/mcp-server/test/protocol.test.ts` with schema and URL-building assertions:

```ts
import { expect, mock, test } from "bun:test";

import { TOOLS } from "../src/protocol/tools";
import { handleJsonRpcRequest } from "../src/protocol/handler";

const searchFacebookTool = TOOLS.find((tool) => tool.name === "search_facebook");
const searchEbayTool = TOOLS.find((tool) => tool.name === "search_ebay");

expect(searchFacebookTool.inputSchema.properties).not.toHaveProperty("cookiesSource");
expect(searchEbayTool.inputSchema.properties).not.toHaveProperty("cookies");
```

And handler URL construction should omit cookie params:

```ts
const fetchMock = mock(() =>
  Promise.resolve(new Response(JSON.stringify([]), { status: 200 })),
);

global.fetch = fetchMock as typeof fetch;

await handleJsonRpcRequest(
  new Request("http://localhost", {
    method: "POST",
    body: JSON.stringify({
      jsonrpc: "2.0",
      id: 1,
      method: "tools/call",
      params: { name: "search_facebook", arguments: { query: "laptop" } },
    }),
  }),
);

const calledUrl = fetchMock.mock.calls[0]?.[0] as string;
expect(calledUrl).toContain("/facebook?q=laptop");
expect(calledUrl).not.toContain("cookies=");
```

- [ ] **Step 2: Run test to verify it fails**

Run: `bun test packages/mcp-server/test/protocol.test.ts`
Expected: FAIL because the current MCP schema and handler still expose and forward those inputs.

- [ ] **Step 3: Write minimal implementation**

Delete the Facebook/eBay cookie tool properties and handler mapping:

```ts
// tools.ts
// Remove `cookiesSource` from search_facebook
// Remove `cookies` from search_ebay
```

```ts
// handler.ts
// Remove:
// if (args.cookiesSource) params.append("cookies", args.cookiesSource);
// if (args.cookies) params.append("cookies", args.cookies);
```

Leave Kijiji alone; this plan only changes Facebook/eBay env-only auth paths defined by the approved spec.

- [ ] **Step 4: Run test to verify it passes**

Run: `bun test packages/mcp-server/test/protocol.test.ts`
Expected: PASS with MCP definitions and handler mapping in sync.

- [ ] **Step 5: Commit**

```bash
git add packages/mcp-server/src/protocol/tools.ts packages/mcp-server/src/protocol/handler.ts packages/mcp-server/test/protocol.test.ts
git commit -m "refactor: remove mcp cookie parameters"
```

### Task 6: Rewrite cookie documentation and run full verification

**Files:**
- Modify: `cookies/AGENTS.md:9-85`
- Modify: `docs/superpowers/specs/2026-04-21-cookie-env-only-design.md` only if implementation reveals a spec mismatch

- [ ] **Step 1: Write the failing test**

Treat docs drift as a contract failure. Capture the required state before editing:

```md
- Cookie setup docs mention env vars only for Facebook and eBay
- No examples remain that show `cookies=` request params
- No examples remain that show `facebook.json` or `ebay.json`
```

- [ ] **Step 2: Run verification to prove current docs are stale**

Run: `rg -n "facebook\.json|ebay\.json|cookies=" cookies/AGENTS.md`
Expected: matches found

- [ ] **Step 3: Write minimal implementation**

Rewrite the cookie setup doc so Facebook and eBay each show only env-var setup:

```md
## Cookie Configuration

All supported authenticated scrapers read cookies only from environment variables.

### Facebook Marketplace

```bash
export FACEBOOK_COOKIE='c_user=123; xs=token; fr=request'
```

### eBay

```bash
export EBAY_COOKIE='s=VALUE; ds2=VALUE; ebay=VALUE'
```
```

Remove the file-based and request-parameter sections entirely.

- [ ] **Step 4: Run full verification**

Run: `bun test && bun run ci && bun run build`
Expected: all commands pass

- [ ] **Step 5: Commit**

```bash
git add cookies/AGENTS.md docs/superpowers/specs/2026-04-21-cookie-env-only-design.md
git commit -m "docs: align cookie setup with env-only auth"
```

## Self-Review

- Spec coverage check: shared cookie utils, Facebook, eBay, API adapter, MCP adapter, tests, and docs each have explicit tasks.
- Placeholder scan: concrete test files are now named for eBay core, API routes, and MCP protocol coverage.
- Type consistency check: `ensureCookies(config)` is the single shared loader name used across Tasks 1-3, and Facebook/eBay route signatures stay aligned with the core changes.