f3839aba54
fix: increase kijiji rate limit to 4 rps
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2026-01-23 13:59:47 -05:00
90b98bfb09
chore: testing mcp server
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2026-01-23 13:59:28 -05:00
eb6705df0f
feat: add 60-second timeouts to MCP request handlers for reliability
2026-01-23 13:59:28 -05:00
72525609ed
fix: set idle timeout to 255 seconds in MCP server to prevent premature shutdown
2026-01-23 13:59:28 -05:00
8b0a65860c
chore: add imports for linkedom and delay utils in ebay scraper
2026-01-23 13:10:44 -05:00
f9b1c7e096
fix: remove eslint-disable directives
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2026-01-23 13:08:38 -05:00
9edc74cbeb
chore: local dev scripts
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2026-01-23 13:07:01 -05:00
ee0fca826d
style: fix formatting in MCP server
2026-01-23 11:56:54 -05:00
f7372612fb
test: fix formatting in test setup
2026-01-23 11:56:51 -05:00
bce126664e
test: remove unused imports in Kijiji utils tests
2026-01-23 11:56:47 -05:00
8cbf11538e
test: fix formatting and remove unused HttpError import in Kijiji tests
2026-01-23 11:56:44 -05:00
79f47fdaef
test: remove unused import in Facebook integration tests
2026-01-23 11:56:41 -05:00
de5069bf2b
test: fix unused variable in Facebook core tests
2026-01-23 11:56:38 -05:00
637f1a4e75
fix: resolve biome lint errors and warnings
2026-01-23 10:33:15 -05:00
441ff436c4
feat(mcp): extend Kijiji tool with filtering parameters
2026-01-23 09:55:37 -05:00
1f53ec912a
feat(mcp): add search options to Kijiji and eBay tools
2026-01-23 09:55:21 -05:00
053efd815b
feat(api/kijiji): add filtering and pagination parameters
2026-01-23 09:54:30 -05:00
d619fa5d77
feat(api/facebook): add maxItems parameter support
2026-01-23 09:53:51 -05:00
050fd0adba
feat(api/ebay): add maxItems parameter and error handling
2026-01-23 09:53:00 -05:00
7b106c91ce
style: format ebay scraper with consistent indentation
2026-01-23 09:52:25 -05:00
6e0487f8f3
style: format api-server index with consistent indentation
2026-01-23 09:52:22 -05:00
da23ca1c3f
chore: agents update
2026-01-23 00:52:35 -05:00
c35aae4c95
chore: biome auto-fixes
2026-01-23 00:52:26 -05:00
02162c02f5
chore: biome init
2026-01-23 00:52:10 -05:00
50d56201af
feat: port upstream scraper improvements to monorepo
...
Kijiji improvements:
- Add error classes: NetworkError, ParseError, RateLimitError, ValidationError
- Add exponential backoff with jitter for retries
- Add request timeout (30s abort)
- Add pagination support (SearchOptions.maxPages)
- Add location/category mappings and resolution functions
- Add enhanced DetailedListing interface with images, seller info, attributes
- Add GraphQL client for seller details
Facebook improvements:
- Add parseFacebookCookieString() for parsing cookie strings
- Add ensureFacebookCookies() with env var fallback
- Add extractFacebookItemData() with multiple extraction paths
- Add fetchFacebookItem() for individual item fetching
- Add extraction metrics and API stability monitoring
- Add vehicle-specific field extraction
- Improve error handling with specific guidance for auth errors
Shared utilities:
- Update http.ts with new error classes and improved fetchHtml
Documentation:
- Port KIJIJI.md, FMARKETPLACE.md, AGENTS.md from upstream
Tests:
- Port kijiji-core, kijiji-integration, kijiji-utils tests
- Port facebook-core, facebook-integration tests
- Add test setup file
Scripts:
- Port parse-facebook-cookies.ts script
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com >
2026-01-23 00:34:50 -05:00
497c7995a2
feat: ebay 'buy it now' and 'canada only' filters support
2025-12-17 14:38:52 -05:00
083b862552
fix healthcheck
2025-12-17 13:58:18 -05:00
0a32094e93
feat: adapt Dockerfile for monorepo structure
2025-12-13 20:54:32 -05:00
a66b5b2362
migrate to monorepo?
2025-12-13 20:31:10 -05:00
7da6408d7a
fix: add target to a bun build command
2025-10-03 09:06:02 -04:00
3863d3139e
chore: update docker compose
2025-10-03 09:06:02 -04:00
11dbf29f93
chore: update README.md
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2025-10-03 08:59:13 -04:00
2e2c81b7e5
chore: update package.json
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2025-10-03 08:56:03 -04:00
ea0a69ccd6
feat(docker): enable containerization of marketplace scraper
...
Introduce Dockerfile with production build using Bun, .dockerignore for efficient builds, and docker-compose.yml for easy local and prod deployment on port 3000.
2025-10-03 08:44:51 -04:00
fa7ac59c45
feat: ebay parser
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2025-10-02 13:52:29 -04:00
8c52efe5e7
feat(facebook): parse additional listing details like status, images, and seller info
...
Enhance Facebook scraping to extract listing status (ACTIVE/SOLD/PENDING/HIDDEN), primary image/video URLs, seller name/ID, category ID, and delivery options, improving response completeness.
2025-10-02 12:03:59 -04:00
dcd0da29a2
feat: facebook scraping
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2025-10-02 11:50:35 -04:00
ee09162faa
chore: add claude code dev dependency
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2025-10-02 10:39:49 -04:00
ddf7639854
fix: add newlines to debug logging due to overlap with the progress bar
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2025-09-18 17:12:40 -04:00
d42c74331e
feat: add requests progress bar
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2025-09-18 17:12:10 -04:00
49a32bf6b4
fix: bun's --hot option doesn't like modules
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2025-09-18 17:11:31 -04:00
3166b5a95f
chore: remove redundant logging
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2025-09-18 17:11:09 -04:00
cb1fb2bae6
feat: add unidecode dependency and slugify function for improved Kijiji search
...
This commit introduces the `unidecode` library to handle non-ASCII characters in search queries. A `slugify` function is implemented to improve Kijiji search URL generation by creating more user-friendly and reliable URLs. It converts the user query to a URL friendly string.
It also addresses issues in Kijiji scraping: improves listing filtering logic to ensure only valid listings with prices are added and enhances error handling to provide more informative messages and prevent process termination due to HTTP errors during listing detail retrieval.
Additionally, refactors the Kijiji search query URL, leveraging the new `slugify` function for enhanced URL generation using the search query. Added debugging logs for better traceability.
2025-09-18 11:35:10 -04:00
add93dc6bd
feat: increase requests per second for Kijiji search
2025-09-18 11:33:47 -04:00
99848d1852
build: configure absolute imports using @ alias
...
Configures TypeScript path aliases for cleaner and more maintainable imports.
2025-09-18 11:33:06 -04:00
d35e37f4c2
feat: implement Kijiji scraping API endpoint
...
This commit introduces a new API endpoint that allows users to search for items on Kijiji.
The endpoint accepts a search query as a header or query parameter and returns a JSON response containing the search results. It also handles cases where no query is provided or no results are found, returning appropriate error responses.
2025-09-17 22:58:11 -04:00
2e9547ebaa
fix: search GTA only
...
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev >
2025-09-17 22:57:42 -04:00
46a8ac92cf
feat: extract Kijiji scraping logic into reusable function
...
This commit extracts the Kijiji scraping functionality into a reusable function `fetchKijijiItems`. This allows for easier integration into other parts of the application and improves code modularity. The function accepts search query, requests per second, and base URL as parameters, enabling customizable scraping.
2025-09-17 22:03:24 -04:00
b7a61423c3
refactor: improve Kijiji scraping architecture
...
Introduces types, utilities, and a dedicated HTTP client for improved structure and maintainability. Includes rate limit handling, retry logic, and error handling.
2025-09-17 20:45:57 -04:00
23dc9cae4c
feat: scrape listing details from Kijiji ads
2025-09-17 20:43:07 -04:00
