old4ever/ca-marketplace-scraper
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
164 Commits 1 Branch 0 Tags
0873df7e829ee963ff20883f5d77174c68573fdf
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
old4ever 0873df7e82 chore: merge code-smell-cleanup 2026-04-30 21:08:34 -04:00
old4ever 24e0a8266e Revert "test: preload core fetch guard" 
This reverts commit 28b3267b7d.
 2026-04-30 20:58:06 -04:00
old4ever db173aef1b Revert "chore: add sentinel file for bun test root" 
This reverts commit d1cd028f34.
 2026-04-30 20:58:06 -04:00
old4ever d1cd028f34 chore: add sentinel file for bun test root 2026-04-30 20:56:14 -04:00
old4ever 28b3267b7d test: preload core fetch guard 2026-04-30 20:53:31 -04:00
old4ever c0dda57f64 test: require explicit fetch mocks 2026-04-30 20:51:13 -04:00
old4ever 31866de787 refactor: clean kijiji scraper internals 2026-04-30 20:48:15 -04:00
old4ever 9c4c347933 feat: ebay splashui challenge solver 
argon2id pow → /challengesvc/answer → chlgref cookie
warm homepage for akamai cookies, detect 307 redirect,
solve + retry transparently in fetchEbayItems flow
 2026-04-30 20:44:37 -04:00
old4ever 53eafe6d4c chore: agent-browser skills path env 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-30 20:44:05 -04:00
old4ever 84f17fbdfd chore: ebay parser fix 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-30 16:56:55 -04:00
old4ever 3a722a2d11 chore: agent-browser vars 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-30 16:56:44 -04:00
old4ever f95b974c7e fix: harden shared http helper 2026-04-29 21:09:10 -04:00
old4ever f5339cadf1 style: format shared http refactor 2026-04-29 21:05:36 -04:00
old4ever 5d86a4e54d fix: preserve ebay rate-limit fallback 2026-04-29 14:52:08 -04:00
old4ever 82e7abc057 fix: keep shared http refactor in scope 2026-04-29 14:48:47 -04:00
old4ever 6e50ebf901 refactor: share scraper http fetching 2026-04-29 13:14:20 -04:00
old4ever 5ecb645ee3 docs: smell cleanup plan 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-29 13:09:38 -04:00
old4ever 82e12283de docs: surface Kijiji AND-matching behavior in tool, API, and MCP responses 
Kijiji zero-result queries (e.g. 'macbook air m1 apple silicon') are
confusing because the failure mode is non-obvious. Surface the root
cause everywhere the caller can see it:
- MCP tool description warns about AND-matching and gives a concrete
  before/after example
- API 404 body includes the actionable hint via emptySearchResponse(hint)
- Core scraper logs the built URL and tip on page-1 zero results
- MCP handler unwraps the API message field so the hint reaches the LLM
 2026-04-29 13:06:31 -04:00
old4ever 22eb65d4a2 refactor: share mcp api calls 2026-04-29 05:37:24 -04:00
old4ever abdd39d65c fix: complete ebay integer validation test coverage 2026-04-29 00:56:37 -04:00
old4ever 3e4e35c9ae fix: tighten route integer parsing and test coverage 2026-04-29 00:32:23 -04:00
old4ever 3ea6ee3938 fix: strictly parse route integers 2026-04-29 00:12:26 -04:00
old4ever d178f9c9cb fix: remove cookie query forwarding 2026-04-28 23:52:45 -04:00
old4ever 9cbba9ba13 chore: ignore local worktrees 2026-04-28 23:08:04 -04:00
old4ever b6aaec0b65 chore: update ruler docs 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-28 22:29:12 -04:00
old4ever 11dce39428 fix(core): parse Kijiji StandardListing records 2026-04-28 21:57:10 -04:00
old4ever 2a5701aeb9 test: quiet and speed up test runs 2026-04-28 21:45:06 -04:00
old4ever c6c44a0914 fix(api): preserve unstable buckets 2026-04-28 21:34:47 -04:00
old4ever 3fe5fdb63f fix(core): handle partial listing data 2026-04-28 21:34:45 -04:00
old4ever 7966073bf8 fix(core): prefer explicit cookie source 2026-04-28 21:34:40 -04:00
old4ever df2635d92f chore: prepend typecheck command before biome ci 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-28 20:11:43 -04:00
old4ever ddadc7d5ae chore: add bun types to global tsconfig 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-28 20:07:05 -04:00
old4ever d77a006ded chore: ignore .turbo cache dirs 2026-04-28 19:56:39 -04:00
old4ever 56b2198df1 chore: fix turbo build outputs path to match actual dist location 2026-04-28 19:56:29 -04:00
old4ever 63716272c5 chore: slim per-package tsconfigs to extend root 2026-04-28 19:55:59 -04:00
old4ever 1d21c66945 chore: use exports field and catalog refs in all packages 2026-04-28 19:55:37 -04:00
old4ever f2f78225f3 chore: add workspace catalog and turbo to root package.json 2026-04-28 19:54:46 -04:00
old4ever 43d15fce5f chore: add shared root tsconfig.json 2026-04-28 19:53:58 -04:00
old4ever fef2f1968a chore: add bunfig.toml and turbo.json 2026-04-28 19:53:47 -04:00
old4ever 01081f6b2e docs: add opencode monorepo config adoption implementation plan 2026-04-28 19:52:28 -04:00
old4ever d10d5305a3 docs: add opencode monorepo config adoption design spec 2026-04-28 19:50:51 -04:00
old4ever bf393eacae chore: setup typecheck scripts for each package 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-28 19:37:18 -04:00
old4ever 79bb249603 chore: replace any cast by asserting tool schema property types 
Tightens the type assertion for the `unstableFilter` schema property in tests to ensure correct structural checking of its `type` and `description` fields.
 2026-04-28 19:24:39 -04:00
old4ever 957e0f137b chore: biome lint and formatting 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-28 19:21:16 -04:00
old4ever 49e90d45f8 docs: expose unstable mode in mcp tools 2026-04-28 19:03:42 -04:00
old4ever b6456047a6 feat: add maxItems support to ebay scraper 2026-04-27 10:56:23 -04:00
old4ever 02b3f805b2 fix: use explicit conditional calls and validate negative params 2026-04-27 10:46:06 -04:00
old4ever a1af5d2630 fix: align ebay route with spec and validate params 2026-04-27 09:56:39 -04:00
old4ever 77b9fc9934 fix: validate route params and reduce duplication 2026-04-27 09:45:47 -04:00
old4ever a802035ca4 fix: correct empty-result and maxItems handling in routes 2026-04-27 09:34:08 -04:00
old4ever 974190de6b fix: preserve maxItems limit in unstable mode 2026-04-27 08:57:48 -04:00
old4ever 3c38232cd5 feat: expose unstable mode in api routes 2026-04-27 02:49:35 -04:00
old4ever 224e83ac4c fix: correct ebay title filtering and type contracts 2026-04-27 02:04:48 -04:00
old4ever b73faa35da fix: respect scraper pacing details 2026-04-27 00:13:42 -04:00
old4ever 0f77155c8d fix: align marketplace price filter parsing 2026-04-23 11:14:57 -04:00
old4ever 10c2856bf6 fix: tighten item price and pacing behavior 2026-04-23 10:59:33 -04:00
old4ever 9c8643086a fix: refine scraper output behavior 2026-04-23 10:43:38 -04:00
old4ever 244a88e63c fix: harden scraper price parsing 2026-04-23 10:31:08 -04:00
old4ever 807849e257 fix: expose ebay unstable mode typing 2026-04-23 05:47:50 -04:00
old4ever eb37e8814e fix: preserve free results and request pacing 2026-04-23 05:40:42 -04:00
old4ever 13c0fec305 fix: tighten scraper type contracts 2026-04-23 05:28:46 -04:00
old4ever 08d59ab497 fix: tighten ebay result parsing 2026-04-23 05:13:40 -04:00
old4ever 0a0723a560 fix: respect filtered result sets in unstable mode 2026-04-23 05:03:26 -04:00
old4ever 881c2ddf8c fix: finalize scraper unstable mode integration 2026-04-23 00:20:21 -04:00
old4ever 55faee7dd5 fix: cover scraper pricing edge cases 2026-04-22 23:54:07 -04:00
old4ever b5e14e686a fix: tighten scraper edge case handling 2026-04-22 23:46:52 -04:00
old4ever 6f9d4db419 fix: tighten scraper parsing behavior 2026-04-22 23:41:08 -04:00
old4ever 08edfa8097 fix: align scraper unstable mode behavior 2026-04-22 23:36:00 -04:00
old4ever c7fc8352ac fix: preserve default scraper result contracts 2026-04-22 23:30:17 -04:00
old4ever 1ee41fb346 feat: add unstable mode to scraper results 2026-04-22 23:23:31 -04:00
old4ever 8141de5b4b feat: add shared unstable listing classifier 2026-04-22 17:56:26 -04:00
old4ever f8975fa91d docs: add unstable listing mode plan 2026-04-22 17:53:45 -04:00
old4ever cb5e1e62d2 docs: add unstable listing mode design 2026-04-22 17:51:07 -04:00
old4ever 9070f76412 refactor: handle facebook route-aware failure states 2026-04-22 11:48:47 -04:00
old4ever 7ddc96dfdf refactor: add facebook html fallbacks 2026-04-22 11:36:47 -04:00
old4ever 63ca006696 refactor: rewrite facebook item parser for comet bootstrap 2026-04-22 02:44:17 -04:00
old4ever c90ee54cc1 refactor: rewrite facebook search parser for comet bootstrap 2026-04-22 02:32:55 -04:00
old4ever cfd7619737 refactor: add facebook bootstrap candidate extraction 2026-04-21 23:46:00 -04:00
old4ever b072599bc6 refactor: add facebook response classification 2026-04-21 23:31:45 -04:00
old4ever 2617afc62f docs: add facebook comet rewrite plan 2026-04-21 23:06:03 -04:00
old4ever ba889a1f9d docs: add facebook comet rewrite design 2026-04-21 23:02:47 -04:00
old4ever 45cff20377 docs: update cookies agents.md 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-21 22:05:49 -04:00
old4ever b6e9501448 docs: align cookie setup with env-only auth 2026-04-21 21:53:42 -04:00
old4ever d65d81dbd1 refactor: remove mcp cookie parameters 2026-04-21 21:48:34 -04:00
old4ever 1a2c0cf6b8 refactor: remove api cookie query overrides 2026-04-21 21:47:37 -04:00
old4ever 918ee92441 refactor: make ebay auth env-only 2026-04-21 21:46:40 -04:00
old4ever a7a5eca7ad refactor: remove facebook cookie overrides 2026-04-21 21:45:42 -04:00
old4ever 847ce28590 refactor: make cookie loading env-only 2026-04-21 21:44:12 -04:00
old4ever 7b4b656868 chore: import order 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-21 21:07:07 -04:00
old4ever e144dcabeb fix: accept nullable marketplace prices in formatter 2026-04-21 21:01:53 -04:00
old4ever 651d54b837 fix: respect custom facebook cookie path 2026-04-21 21:01:53 -04:00
old4ever 2231603692 chore: fix lint config 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-21 20:51:28 -04:00
old4ever 86191e7a45 chore: deep-init 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-21 20:47:00 -04:00
old4ever c58d614948 chore: devenv lock update 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-21 20:37:55 -04:00
old4ever 7cf21546e2 chore: ai agent config 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-04-21 20:37:55 -04:00
old4ever ffc4a2c5c5 chore: useragent 2026-04-21 19:45:29 -04:00
old4everandClaude Opus 4.5 e4ab145d70 feat: add cookie support to kijiji scraper 
Add optional cookie parameter to bypass bot detection (403 errors).
Cookies can be provided via parameter, KIJIJI_COOKIE env var, or
cookies/kijiji.json file. Supports both JSON array and string formats.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-23 19:29:13 -05:00
old4everandClaude Opus 4.5 1dce0392e3 refactor: use shared cookie utility in ebay scraper 
Replace inline cookie loading with shared utility functions.
Now supports both JSON array and cookie string formats.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-23 19:29:02 -05:00
old4everandClaude Opus 4.5 251fcbb7d9 refactor: use shared cookie utility in facebook scraper 
Replace inline cookie parsing with shared utility functions.
Maintains backward compatibility with existing exports.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-23 19:28:52 -05:00
old4everandClaude Opus 4.5 9bc57d6b54 refactor: add shared cookie utility to core package 
Move cookie parsing logic to a dedicated utility module that can be
shared across all scrapers. Supports both JSON array and cookie string
formats for all input sources (parameter, env var, file).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
 2026-01-23 19:28:44 -05:00