ca-marketplace-scraper/docs/superpowers/specs/2026-04-22-unstable-listing-mode-design.md

Unstable Listing Mode Design

Summary

Add an optional shared result mode across Facebook, eBay, and Kijiji that moves suspiciously cheap listings out of the main results into a separate unstableResults bucket. Listings are considered unstable when their price is more than 20% below the median price of the scraper's priced search results.

Goals

• Support the same optional unstable-listing mode across all scrapers.
• Keep current default scraper and route behavior unchanged unless the mode is enabled.
• Hide unstable listings from the main results while still returning them separately.
• Implement the rule once in shared core code instead of duplicating marketplace-specific logic.
• Document the option in MCP tool descriptions so callers can discover it.

Non-Goals

• Adding marketplace-specific thresholds or heuristics.
• Re-ranking results beyond splitting stable and unstable buckets.
• Classifying free, missing-price, or invalid-price listings as unstable.
• Changing unrelated scraper parsing behavior.

Current State

packages/core currently returns plain arrays from scraper search functions. packages/api-server forwards those scraper results directly from marketplace routes. packages/mcp-server documents search tools per marketplace, but does not expose or describe any result-stability mode.

There is no shared result-classification utility today. Price filtering exists in some scrapers, but not a cross-marketplace median-based split.

Chosen Approach

Use a shared core utility plus per-route and per-tool opt-in.

The shared utility will accept parsed listings, compute the median from valid positive prices, and split the data into results and unstableResults. Each scraper will opt into that utility when the caller enables unstable-listing mode. API routes and MCP tools will expose the same optional mode so the feature is consistently available everywhere scraper search is surfaced.

This keeps the heuristic centralized, minimizes duplicated logic, and preserves existing consumers by leaving the default path unchanged.

Design

Shared Core Classification

Add a shared utility in packages/core for listing stability classification.

Responsibilities:

• accept parsed listing arrays with listingPrice.cents
• ignore listings whose price is missing, non-numeric, or non-positive when computing the median
• compute the median price from valid priced listings
• classify listings as unstable when listingPrice.cents < median * 0.8
• return an object with:
  • results: listings that remain in the main bucket
  • unstableResults: listings moved out of the main bucket

Listings excluded from median computation because their price is missing or non-positive remain in results unchanged.

Scraper Integration

Facebook, eBay, and Kijiji search entrypoints will gain the same optional mode flag.

Default behavior:

• return the current plain array result shape

Opt-in behavior:

• run the shared classification utility after parsing search results
• classify before final result limiting so unstable items do not consume main-result slots
• return an object shaped like:

{
  results: ListingDetails[];
  unstableResults: ListingDetails[];
}

Each scraper will use its existing concrete listing subtype for these arrays.

API Surface

Marketplace API routes will expose an optional query parameter for unstable-listing mode.

Requirements:

• keep existing route responses unchanged when the parameter is absent or false
• when enabled, return the object payload with results and unstableResults
• use the same semantics across Facebook, eBay, and Kijiji routes

The exact parameter name should be consistent across routes and intentionally describe the behavior, for example unstableFilter=true.

MCP Surface

Marketplace MCP tools will expose the same optional mode as an input field.

Tool descriptions should explicitly document:

• that the option is optional
• that it moves listings priced more than 20% below the median into unstableResults
• that enabling it changes the response shape from a plain list to an object with results and unstableResults
• that the behavior is available for Facebook, eBay, and Kijiji search tools

The wording should be aligned across all three tools so the feature reads as one shared capability.

Error Handling

The unstable-listing mode should be best-effort and non-failing.

• If there are no valid positive prices, return all listings in results and an empty unstableResults array.
• If there is only one valid priced listing, do not classify it as unstable.
• Parsing failures remain governed by existing scraper behavior; the classification layer should not introduce new scraper-specific errors.

Testing Strategy

Follow TDD. Start with shared utility tests, then wire the option through scraper and route tests.

Coverage targets:

1. Median calculation for odd-sized valid price sets.
2. Median calculation for even-sized valid price sets.
3. Strict cutoff behavior where only listings with price < median * 0.8 move to unstableResults.
4. Missing, invalid, zero, or negative prices are excluded from median computation and remain in results.
5. Default scraper behavior still returns plain arrays when the option is disabled.
6. Enabled scraper behavior returns { results, unstableResults } for Facebook, eBay, and Kijiji.
7. API routes preserve existing response shapes by default and switch to the object payload only when enabled.
8. MCP tool metadata documents the new optional mode for all three marketplace search tools.

Verification target after implementation:

• bun test packages/core/test
• bun test packages/api-server/test
• bun test packages/mcp-server/test if MCP metadata tests exist or are added
• bun run ci

Risks

• The optional mode introduces a union return shape for scraper callers, which can ripple into downstream TypeScript signatures.
• Applying classification before final limiting changes which items appear in the main bucket compared with a naive post-limit split.
• Kijiji and eBay may have different mixes of priced and unpriced results, so excluding non-positive prices from the median must remain explicit and tested.

Rollout Notes

Land the shared classifier, scraper wiring, route wiring, tests, and MCP description updates together. That avoids a partial rollout where the feature exists in one surface but is undocumented or inconsistent elsewhere.