ca-marketplace-scraper/docs/superpowers/plans/2026-04-21-cookie-env-only.md

Cookie Env-Only Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Remove cookie files and request-provided cookie overrides so all authenticated marketplace scraping reads raw Cookie header strings only from environment variables.

Architecture: Collapse shared cookie loading to a single env-var reader in packages/core/src/utils/cookies.ts, then tighten Facebook and eBay core signatures to stop accepting request/file cookie inputs. Update the API and MCP adapters so they no longer advertise or forward cookie parameters, and rewrite docs/tests to match the env-only contract.

Tech Stack: Bun, TypeScript, Bun test, Biome, workspace package exports

---

File Map

• Modify: packages/core/src/utils/cookies.ts Purpose: remove JSON/file/request-source loading and keep env-only cookie parsing/formatting.
• Modify: packages/core/src/scrapers/facebook.ts Purpose: drop cookiesSource / cookiePath arguments and env-only error text.
• Modify: packages/core/src/scrapers/ebay.ts Purpose: remove opts.cookies request override and use env-only cookie loading.
• Modify: packages/core/src/index.ts Purpose: keep exports aligned with tightened core signatures.
• Modify: packages/core/test/facebook-core.test.ts Purpose: replace missing-file coverage with env-only auth tests.
• Create: packages/core/test/ebay-core.test.ts Purpose: add dedicated eBay auth regression coverage instead of mixing it into Facebook tests.
• Modify: packages/api-server/src/routes/facebook.ts Purpose: stop parsing/forwarding cookies query params.
• Modify: packages/api-server/src/routes/ebay.ts Purpose: stop parsing/forwarding cookies query params.
• Create: packages/api-server/test/routes.test.ts Purpose: verify Facebook/eBay routes ignore cookie query params and still call core correctly.
• Modify: packages/mcp-server/src/protocol/tools.ts Purpose: remove Facebook/eBay cookie tool inputs and descriptions.
• Modify: packages/mcp-server/src/protocol/handler.ts Purpose: stop mapping removed cookie tool inputs into API URLs.
• Create: packages/mcp-server/test/protocol.test.ts Purpose: verify tool schemas and handler URL building no longer include Facebook/eBay cookie fields.
• Modify: cookies/AGENTS.md Purpose: document env vars as the only supported cookie input.

Task 1: Lock core cookie utilities to env-only loading

Files:

• Modify: packages/core/src/utils/cookies.ts:19-227

• Test: packages/core/test/facebook-core.test.ts

• Step 1: Write the failing test

Add or replace the auth-source test block in packages/core/test/facebook-core.test.ts with env-only expectations:

test("should load Facebook cookies from FACEBOOK_COOKIE env var", async () => {
  const previous = process.env.FACEBOOK_COOKIE;
  process.env.FACEBOOK_COOKIE = "c_user=123; xs=abc";

  try {
    const cookies = await ensureFacebookCookies();

    expect(cookies.map((cookie) => cookie.name)).toEqual(["c_user", "xs"]);
  } finally {
    if (previous === undefined) {
      delete process.env.FACEBOOK_COOKIE;
    } else {
      process.env.FACEBOOK_COOKIE = previous;
    }
  }
});

test("should reject missing Facebook auth env var", async () => {
  const previous = process.env.FACEBOOK_COOKIE;
  delete process.env.FACEBOOK_COOKIE;

  try {
    await expect(ensureFacebookCookies()).rejects.toThrow(
      "Provide cookies via FACEBOOK_COOKIE environment variable as a raw Cookie header string",
    );
  } finally {
    if (previous !== undefined) {
      process.env.FACEBOOK_COOKIE = previous;
    }
  }
});

• Step 2: Run test to verify it fails

Run: bun test packages/core/test/facebook-core.test.ts Expected: FAIL because the current implementation still allows missing env values to fall through to file/request-based behavior and does not emit the new env-only error.

• Step 3: Write minimal implementation

Replace the multi-source loader in packages/core/src/utils/cookies.ts with an env-only loader. The target shape is:

export interface CookieConfig {
  name: string;
  domain: string;
  envVar: string;
}

export async function ensureCookies(config: CookieConfig): Promise<Cookie[]> {
  const envValue = process.env[config.envVar];
  const cookies = parseCookieString(envValue ?? "", config.domain);

  if (cookies.length > 0) {
    console.log(
      `Loaded ${cookies.length} ${config.name} cookies from ${config.envVar} env var`,
    );
    return cookies;
  }

  throw new Error(
    `No valid ${config.name} cookies found. Provide cookies via ${config.envVar} environment variable as a raw Cookie header string.`,
  );
}

Delete the now-dead helpers and types that exist only for JSON/file/request loading:

// Remove:
// - parseJsonCookies
// - parseCookiesAuto
// - loadCookiesFromFile
// - loadCookiesOptional
// - CookieConfig.filePath

• Step 4: Run test to verify it passes

Run: bun test packages/core/test/facebook-core.test.ts Expected: PASS for the new env-only tests.

• Step 5: Commit

git add packages/core/src/utils/cookies.ts packages/core/test/facebook-core.test.ts
git commit -m "refactor: make cookie loading env-only"

Task 2: Tighten Facebook core APIs to the new contract

Files:

• Modify: packages/core/src/scrapers/facebook.ts:23-29

• Modify: packages/core/src/scrapers/facebook.ts:214-228

• Modify: packages/core/src/scrapers/facebook.ts:823-929

• Modify: packages/core/src/index.ts:5-15

• Test: packages/core/test/facebook-core.test.ts

• Step 1: Write the failing test

Add a focused test proving Facebook item fetch now depends only on env auth:

test("should fail Facebook item fetch when FACEBOOK_COOKIE is unset", async () => {
  const previous = process.env.FACEBOOK_COOKIE;
  delete process.env.FACEBOOK_COOKIE;

  try {
    await expect(fetchFacebookItem("123")).rejects.toThrow(
      "Provide cookies via FACEBOOK_COOKIE environment variable as a raw Cookie header string",
    );
  } finally {
    if (previous !== undefined) {
      process.env.FACEBOOK_COOKIE = previous;
    }
  }
});

• Step 2: Run test to verify it fails

Run: bun test packages/core/test/facebook-core.test.ts Expected: FAIL because the current function signatures and error text still mention parameter/file-based auth paths.

• Step 3: Write minimal implementation

Tighten the Facebook signatures and messages:

const FACEBOOK_COOKIE_CONFIG: CookieConfig = {
  name: "Facebook",
  domain: ".facebook.com",
  envVar: "FACEBOOK_COOKIE",
};

export async function ensureFacebookCookies(): Promise<Cookie[]> {
  return ensureCookies(FACEBOOK_COOKIE_CONFIG);
}

export default async function fetchFacebookItems(
  SEARCH_QUERY: string,
  REQUESTS_PER_SECOND = 1,
  LOCATION = "toronto",
  MAX_ITEMS = 25,
) {
  const cookies = await ensureFacebookCookies();

Also change the stale auth warnings:

console.warn(
  "This might indicate invalid or expired cookies. Update FACEBOOK_COOKIE with a fresh raw Cookie header string.",
);

Remove the extra cookie arguments from fetchFacebookItem(...) and keep packages/core/src/index.ts exporting the tightened functions without the old parameter contract.

• Step 4: Run test to verify it passes

Run: bun test packages/core/test/facebook-core.test.ts Expected: PASS with the new env-only Facebook API surface.

• Step 5: Commit

git add packages/core/src/scrapers/facebook.ts packages/core/src/index.ts packages/core/test/facebook-core.test.ts
git commit -m "refactor: remove facebook cookie overrides"

Task 3: Tighten eBay core APIs to env-only auth

Files:

• Modify: packages/core/src/scrapers/ebay.ts:9-15

• Modify: packages/core/src/scrapers/ebay.ts:337-389

• Create: packages/core/test/ebay-core.test.ts

• Step 1: Write the failing test

Create packages/core/test/ebay-core.test.ts with a dedicated auth regression test:

test("should warn and continue without eBay cookies when EBAY_COOKIE is unset", async () => {
  const previous = process.env.EBAY_COOKIE;
  delete process.env.EBAY_COOKIE;

  try {
    const cookies = await loadEbayCookies();
    expect(cookies).toBeUndefined();
  } finally {
    if (previous !== undefined) {
      process.env.EBAY_COOKIE = previous;
    }
  }
});

• Step 2: Run test to verify it fails

Run: bun test packages/core/test/ebay-core.test.ts Expected: FAIL because loadEbayCookies still accepts request overrides and mentions file/json sources.

• Step 3: Write minimal implementation

Remove file/request branches from the eBay cookie path:

const EBAY_COOKIE_CONFIG: CookieConfig = {
  name: "eBay",
  domain: ".ebay.ca",
  envVar: "EBAY_COOKIE",
};

async function loadEbayCookies(): Promise<string | undefined> {
  try {
    const cookies = await ensureCookies(EBAY_COOKIE_CONFIG);
    return formatCookiesForHeader(cookies, "www.ebay.ca");
  } catch {
    console.warn(
      "No valid eBay cookies found in EBAY_COOKIE. eBay may block requests without a raw Cookie header string.",
    );
    return undefined;
  }
}

Then remove cookies from fetchEbayItems(..., opts) and the destructuring that feeds it into loadEbayCookies().

• Step 4: Run test to verify it passes

Run: bun test packages/core/test/ebay-core.test.ts Expected: PASS for the eBay env-only regression coverage.

• Step 5: Commit

git add packages/core/src/scrapers/ebay.ts packages/core/test/ebay-core.test.ts
git commit -m "refactor: make ebay auth env-only"

Task 4: Remove cookie query parameters from the API adapter

Files:

• Modify: packages/api-server/src/routes/facebook.ts:3-33

• Modify: packages/api-server/src/routes/ebay.ts:3-52

• Create: packages/api-server/test/routes.test.ts

• Step 1: Write the failing test

Create packages/api-server/test/routes.test.ts and mock @marketplace-scrapers/core so the route contract is explicit:

import { afterEach, describe, expect, mock, test } from "bun:test";

const fetchFacebookItems = mock(() => Promise.resolve([{ title: "item" }]));
const fetchEbayItems = mock(() => Promise.resolve([{ title: "item" }]));

mock.module("@marketplace-scrapers/core", () => ({
  fetchFacebookItems,
  fetchEbayItems,
}));

import { ebayRoute } from "../src/routes/ebay";
import { facebookRoute } from "../src/routes/facebook";

afterEach(() => {
  fetchFacebookItems.mockReset();
  fetchEbayItems.mockReset();
});

test("facebookRoute ignores cookies query parameter", async () => {
  await facebookRoute(
    new Request("http://localhost/api/facebook?q=laptop&location=toronto&maxItems=3&cookies=c_user=1"),
  );

  expect(fetchFacebookItems).toHaveBeenCalledWith("laptop", 1, "toronto", 3);
});

test("ebayRoute ignores cookies query parameter", async () => {
  await ebayRoute(
    new Request("http://localhost/api/ebay?q=laptop&cookies=s%3D1&buyItNowOnly=true"),
  );

  expect(fetchEbayItems).toHaveBeenCalledWith("laptop", 1, {
    minPrice: undefined,
    maxPrice: undefined,
    strictMode: false,
    exclusions: [],
    keywords: ["laptop"],
    buyItNowOnly: true,
    canadaOnly: true,
  });
});

• Step 2: Run test to verify it fails

Run: bun test packages/api-server/test/routes.test.ts Expected: FAIL because the current routes still parse reqUrl.searchParams.get("cookies") and forward it downstream.

• Step 3: Write minimal implementation

Delete the cookie query-parameter handling from both routes:

// packages/api-server/src/routes/facebook.ts
/**
 * GET /api/facebook?q={query}&location={location}
 * Search Facebook Marketplace for listings
 */

const items = await fetchFacebookItems(SEARCH_QUERY, 1, LOCATION, maxItems);

// packages/api-server/src/routes/ebay.ts
/**
 * GET /api/ebay?q={query}&minPrice={minPrice}&maxPrice={maxPrice}&strictMode={strictMode}&exclusions={exclusions}&keywords={keywords}&buyItNowOnly={buyItNowOnly}&canadaOnly={canadaOnly}
 */

const items = await fetchEbayItems(SEARCH_QUERY, 1, {
  minPrice,
  maxPrice,
  strictMode,
  exclusions,
  keywords,
  buyItNowOnly,
  canadaOnly,
});

• Step 4: Run test to verify it passes

Run: bun test packages/api-server/test/routes.test.ts Expected: PASS for route coverage and no remaining adapter references to cookies for Facebook/eBay.

• Step 5: Commit

git add packages/api-server/src/routes/facebook.ts packages/api-server/src/routes/ebay.ts packages/api-server/test/routes.test.ts
git commit -m "refactor: remove api cookie query overrides"

Task 5: Remove cookie inputs from MCP tool schemas and request mapping

Files:

• Modify: packages/mcp-server/src/protocol/tools.ts:65-148

• Modify: packages/mcp-server/src/protocol/handler.ts:154-211

• Create: packages/mcp-server/test/protocol.test.ts

• Step 1: Write the failing test

Create packages/mcp-server/test/protocol.test.ts with schema and URL-building assertions:

import { expect, mock, test } from "bun:test";

import { TOOLS } from "../src/protocol/tools";
import { handleJsonRpcRequest } from "../src/protocol/handler";

const searchFacebookTool = TOOLS.find((tool) => tool.name === "search_facebook");
const searchEbayTool = TOOLS.find((tool) => tool.name === "search_ebay");

expect(searchFacebookTool.inputSchema.properties).not.toHaveProperty("cookiesSource");
expect(searchEbayTool.inputSchema.properties).not.toHaveProperty("cookies");

And handler URL construction should omit cookie params:

const fetchMock = mock(() =>
  Promise.resolve(new Response(JSON.stringify([]), { status: 200 })),
);

global.fetch = fetchMock as typeof fetch;

await handleJsonRpcRequest(
  new Request("http://localhost", {
    method: "POST",
    body: JSON.stringify({
      jsonrpc: "2.0",
      id: 1,
      method: "tools/call",
      params: { name: "search_facebook", arguments: { query: "laptop" } },
    }),
  }),
);

const calledUrl = fetchMock.mock.calls[0]?.[0] as string;
expect(calledUrl).toContain("/facebook?q=laptop");
expect(calledUrl).not.toContain("cookies=");

• Step 2: Run test to verify it fails

Run: bun test packages/mcp-server/test/protocol.test.ts Expected: FAIL because the current MCP schema and handler still expose and forward those inputs.

• Step 3: Write minimal implementation

Delete the Facebook/eBay cookie tool properties and handler mapping:

// tools.ts
// Remove `cookiesSource` from search_facebook
// Remove `cookies` from search_ebay

// handler.ts
// Remove:
// if (args.cookiesSource) params.append("cookies", args.cookiesSource);
// if (args.cookies) params.append("cookies", args.cookies);

Leave Kijiji alone; this plan only changes Facebook/eBay env-only auth paths defined by the approved spec.

• Step 4: Run test to verify it passes

Run: bun test packages/mcp-server/test/protocol.test.ts Expected: PASS with MCP definitions and handler mapping in sync.

• Step 5: Commit

git add packages/mcp-server/src/protocol/tools.ts packages/mcp-server/src/protocol/handler.ts packages/mcp-server/test/protocol.test.ts
git commit -m "refactor: remove mcp cookie parameters"

Task 6: Rewrite cookie documentation and run full verification

Files:

• Modify: cookies/AGENTS.md:9-85

• Modify: docs/superpowers/specs/2026-04-21-cookie-env-only-design.md only if implementation reveals a spec mismatch

• Step 1: Write the failing test

Treat docs drift as a contract failure. Capture the required state before editing:

- Cookie setup docs mention env vars only for Facebook and eBay
- No examples remain that show `cookies=` request params
- No examples remain that show `facebook.json` or `ebay.json`

• Step 2: Run verification to prove current docs are stale

Run: rg -n "facebook\.json|ebay\.json|cookies=" cookies/AGENTS.md Expected: matches found

• Step 3: Write minimal implementation

Rewrite the cookie setup doc so Facebook and eBay each show only env-var setup:

## Cookie Configuration

All supported authenticated scrapers read cookies only from environment variables.

### Facebook Marketplace

```bash
export FACEBOOK_COOKIE='c_user=123; xs=token; fr=request'

eBay

export EBAY_COOKIE='s=VALUE; ds2=VALUE; ebay=VALUE'

Remove the file-based and request-parameter sections entirely.

- [ ] **Step 4: Run full verification**

Run: `bun test && bun run ci && bun run build`
Expected: all commands pass

- [ ] **Step 5: Commit**

```bash
git add cookies/AGENTS.md docs/superpowers/specs/2026-04-21-cookie-env-only-design.md
git commit -m "docs: align cookie setup with env-only auth"

Self-Review

• Spec coverage check: shared cookie utils, Facebook, eBay, API adapter, MCP adapter, tests, and docs each have explicit tasks.
• Placeholder scan: concrete test files are now named for eBay core, API routes, and MCP protocol coverage.
• Type consistency check: ensureCookies(config) is the single shared loader name used across Tasks 1-3, and Facebook/eBay route signatures stay aligned with the core changes.