39 lines
1.7 KiB
Markdown
39 lines
1.7 KiB
Markdown
# ca-marketplace-scraper
|
|
|
|
## Repo Shape
|
|
|
|
- Bun workspace monorepo.
|
|
- `packages/core`: scraper logic, parsing, shared cookie/http/format helpers, and the only checked-in tests.
|
|
- `packages/api-server`: Bun HTTP adapter exposing `/api/*` routes.
|
|
- `packages/mcp-server`: MCP JSON-RPC adapter that proxies to the API server.
|
|
- `dist/`: build output. Do not edit generated files here.
|
|
- `cookies/`: local cookie examples and docs. Never commit real session cookies.
|
|
|
|
## Commands
|
|
|
|
- Install: `bun install`
|
|
- Lint/format check: `bun run ci`
|
|
- Build everything: `bun run build`
|
|
- Run tests: `bun test`
|
|
- API dev server: `bun run --cwd packages/api-server dev`
|
|
- MCP dev server: `bun run --cwd packages/mcp-server dev`
|
|
|
|
## Repo Conventions
|
|
|
|
- Keep marketplace scraping behavior in `packages/core`. `api-server` and `mcp-server` stay thin adapters.
|
|
- Preserve cookie precedence everywhere: request parameter > environment variable > cookie file.
|
|
- Shared public surface for scraper code is `packages/core/src/index.ts`. Update exports deliberately.
|
|
- Tests should stay deterministic and offline. Mock `fetch`; do not hit live marketplace endpoints.
|
|
- Use Bun and Bun-native APIs in this repo. Do not introduce Node-specific tooling unless already required.
|
|
- Biome and strict TypeScript are part of the contract. Fix code to satisfy them; do not relax config.
|
|
|
|
## Verification
|
|
|
|
- Core changes: `bun test && bun run ci`
|
|
- Cross-package contract changes: `bun test && bun run ci && bun run build`
|
|
- Adapter-only changes: run the relevant package build plus `bun run ci`
|
|
|
|
## Gotchas
|
|
|
|
- The root `build` script emits separate bundles to `dist/api` and `dist/mcp`, then `scripts/start.sh` launches both.
|