# ca-marketplace-scraper

## Repo Shape

- Bun workspace monorepo with packages under `packages/*`.
- `packages/core`: scraper behavior, parsing, result types, cookie handling, HTTP helpers.
- `packages/api-server`: Bun HTTP adapter exposing `/api/*` routes over core.
- `packages/mcp-server`: MCP/JSON-RPC adapter that proxies to the API server.
- `cookies/`: local cookie docs/examples only. Treat real cookie files as secrets.
- `dist/`, `node_modules/`, `.turbo/`, `.direnv/`, `.devenv/`: generated/vendor/cache. Do not edit.

## Commands

- Install: `bun install`
- Lint/format/typecheck: `bun run ci`
- Build all packages: `bun run build`
- Build bundled runtime output: `bun run build:all`
- Run tests: `bun test packages/core/test packages/api-server/test packages/mcp-server/test`
- API dev server: `bun run --cwd packages/api-server dev`
- MCP dev server: `bun run --cwd packages/mcp-server dev`

## Boundaries

- Marketplace behavior belongs in `packages/core`, not adapter packages.
- HTTP route code should parse request input, call core, and map status/errors.
- MCP code should define tools, validate JSON-RPC flow, and map tool args to API URLs.
- Keep API query params and MCP tool args in sync.
- Shared public surface for scraper code is `packages/core/src/index.ts`; update exports deliberately.

## Invariants

- Cookie precedence in core helpers: explicit/request cookie string before environment variable.
- Tests must be deterministic and offline. Mock `fetch`; do not hit live marketplace endpoints.
- Use Bun and Bun-native APIs. Do not add Node-specific tooling unless already required.
- Biome and strict TypeScript are contract. Fix code; do not relax config.

## Verification

- Core changes: `bun test packages/core/test && bun run ci`
- Adapter-only changes: relevant package build plus `bun run ci`
- Cross-package contract changes: `bun test packages/core/test packages/api-server/test packages/mcp-server/test && bun run ci && bun run build`

## Gotchas

- `bunfig.toml` points test root at `./do-not-run-tests-from-root`; pass package test paths explicitly.
- Root `build` cleans `dist`, then Turbo emits bundles for API and MCP.
- `scripts/start.sh` launches `dist/api/index.js` and `dist/mcp/index.js`.
- Package `tsconfig.json` files override root `include`; shared ambient declarations under root `types/` must be included from each package that typechecks cross-package source.