From cb5e1e62d219afca43e3e39e6995320bd28addc4 Mon Sep 17 00:00:00 2001 From: Dmytro Stanchiev Date: Wed, 22 Apr 2026 17:51:07 -0400 Subject: [PATCH] docs: add unstable listing mode design --- ...2026-04-22-unstable-listing-mode-design.md | 148 ++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-22-unstable-listing-mode-design.md diff --git a/docs/superpowers/specs/2026-04-22-unstable-listing-mode-design.md b/docs/superpowers/specs/2026-04-22-unstable-listing-mode-design.md new file mode 100644 index 0000000..7d979aa --- /dev/null +++ b/docs/superpowers/specs/2026-04-22-unstable-listing-mode-design.md @@ -0,0 +1,148 @@ +# Unstable Listing Mode Design + +## Summary + +Add an optional shared result mode across Facebook, eBay, and Kijiji that moves suspiciously cheap listings out of the main results into a separate `unstableResults` bucket. +Listings are considered unstable when their price is more than 20% below the median price of the scraper's priced search results. + +## Goals + +- Support the same optional unstable-listing mode across all scrapers. +- Keep current default scraper and route behavior unchanged unless the mode is enabled. +- Hide unstable listings from the main results while still returning them separately. +- Implement the rule once in shared core code instead of duplicating marketplace-specific logic. +- Document the option in MCP tool descriptions so callers can discover it. + +## Non-Goals + +- Adding marketplace-specific thresholds or heuristics. +- Re-ranking results beyond splitting stable and unstable buckets. +- Classifying free, missing-price, or invalid-price listings as unstable. +- Changing unrelated scraper parsing behavior. + +## Current State + +`packages/core` currently returns plain arrays from scraper search functions. +`packages/api-server` forwards those scraper results directly from marketplace routes. +`packages/mcp-server` documents search tools per marketplace, but does not expose or describe any result-stability mode. + +There is no shared result-classification utility today. +Price filtering exists in some scrapers, but not a cross-marketplace median-based split. + +## Chosen Approach + +Use a shared core utility plus per-route and per-tool opt-in. + +The shared utility will accept parsed listings, compute the median from valid positive prices, and split the data into `results` and `unstableResults`. +Each scraper will opt into that utility when the caller enables unstable-listing mode. +API routes and MCP tools will expose the same optional mode so the feature is consistently available everywhere scraper search is surfaced. + +This keeps the heuristic centralized, minimizes duplicated logic, and preserves existing consumers by leaving the default path unchanged. + +## Design + +### Shared Core Classification + +Add a shared utility in `packages/core` for listing stability classification. + +Responsibilities: + +- accept parsed listing arrays with `listingPrice.cents` +- ignore listings whose price is missing, non-numeric, or non-positive when computing the median +- compute the median price from valid priced listings +- classify listings as unstable when `listingPrice.cents < median * 0.8` +- return an object with: + - `results`: listings that remain in the main bucket + - `unstableResults`: listings moved out of the main bucket + +Listings excluded from median computation because their price is missing or non-positive remain in `results` unchanged. + +### Scraper Integration + +Facebook, eBay, and Kijiji search entrypoints will gain the same optional mode flag. + +Default behavior: + +- return the current plain array result shape + +Opt-in behavior: + +- run the shared classification utility after parsing search results +- classify before final result limiting so unstable items do not consume main-result slots +- return an object shaped like: + +```ts +{ + results: ListingDetails[]; + unstableResults: ListingDetails[]; +} +``` + +Each scraper will use its existing concrete listing subtype for these arrays. + +### API Surface + +Marketplace API routes will expose an optional query parameter for unstable-listing mode. + +Requirements: + +- keep existing route responses unchanged when the parameter is absent or false +- when enabled, return the object payload with `results` and `unstableResults` +- use the same semantics across Facebook, eBay, and Kijiji routes + +The exact parameter name should be consistent across routes and intentionally describe the behavior, for example `unstableFilter=true`. + +### MCP Surface + +Marketplace MCP tools will expose the same optional mode as an input field. + +Tool descriptions should explicitly document: + +- that the option is optional +- that it moves listings priced more than 20% below the median into `unstableResults` +- that enabling it changes the response shape from a plain list to an object with `results` and `unstableResults` +- that the behavior is available for Facebook, eBay, and Kijiji search tools + +The wording should be aligned across all three tools so the feature reads as one shared capability. + +### Error Handling + +The unstable-listing mode should be best-effort and non-failing. + +- If there are no valid positive prices, return all listings in `results` and an empty `unstableResults` array. +- If there is only one valid priced listing, do not classify it as unstable. +- Parsing failures remain governed by existing scraper behavior; the classification layer should not introduce new scraper-specific errors. + +### Testing Strategy + +Follow TDD. +Start with shared utility tests, then wire the option through scraper and route tests. + +Coverage targets: + +1. Median calculation for odd-sized valid price sets. +2. Median calculation for even-sized valid price sets. +3. Strict cutoff behavior where only listings with `price < median * 0.8` move to `unstableResults`. +4. Missing, invalid, zero, or negative prices are excluded from median computation and remain in `results`. +5. Default scraper behavior still returns plain arrays when the option is disabled. +6. Enabled scraper behavior returns `{ results, unstableResults }` for Facebook, eBay, and Kijiji. +7. API routes preserve existing response shapes by default and switch to the object payload only when enabled. +8. MCP tool metadata documents the new optional mode for all three marketplace search tools. + +Verification target after implementation: + +- `bun test packages/core/test` +- `bun test packages/api-server/test` +- `bun test packages/mcp-server/test` if MCP metadata tests exist or are added +- `bun run ci` + +## Risks + +- The optional mode introduces a union return shape for scraper callers, which can ripple into downstream TypeScript signatures. +- Applying classification before final limiting changes which items appear in the main bucket compared with a naive post-limit split. +- Kijiji and eBay may have different mixes of priced and unpriced results, so excluding non-positive prices from the median must remain explicit and tested. + +## Rollout Notes + +Land the shared classifier, scraper wiring, route wiring, tests, and MCP description updates together. +That avoids a partial rollout where the feature exists in one surface but is undocumented or inconsistent elsewhere.