old4ever/ca-marketplace-scraper
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
48 Commits 1 Branch 0 Tags
d8542eb8f7122028ebec4dd56afd78bce82d8d1a
Commit Graph
48 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
old4ever d8542eb8f7 feat: parse Facebook marketplace item details and test exports 2026-01-22 19:54:44 -05:00
old4ever 0a114cf323 feat: extract individual Facebook marketplace items 2026-01-22 19:54:14 -05:00
old4ever 5f7de1167e fix: add currency style and USD to formatCentsToCurrency 2026-01-22 19:53:53 -05:00
old4ever 9edafc88c8 feat: add extraction monitoring and metrics logging 2026-01-22 19:52:39 -05:00
old4ever 5871644e8b refactor: improve search extraction with edge case handling 2026-01-22 19:52:09 -05:00
old4ever d5d050013e feat: add Facebook cookie parsing and auto-loading 2026-01-22 19:51:35 -05:00
old4ever ff56a29171 feat: add cookiePath parameter to loadFacebookCookies 2026-01-22 19:51:18 -05:00
old4ever 6a36214528 feat: add FacebookMarketplaceItem interface 2026-01-22 19:48:41 -05:00
old4ever 7af1be3977 feat: improve Cookie interface type safety 2026-01-22 19:47:37 -05:00
old4ever 844e566b57 feat: add Facebook cookie parser script 2026-01-22 19:35:47 -05:00
old4ever b3be32835a test: add Facebook marketplace test suite 2026-01-22 19:35:38 -05:00
old4ever baa34eefdf chore: agent.md 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-01-22 16:22:06 -05:00
old4ever 9011ab4793 feat: fmarketplace docs 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-01-22 01:02:36 -05:00
old4ever aae0ce90b8 feat: update index with new kijiji api 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-01-22 00:40:28 -05:00
old4ever daa61c25d8 test: kijiji scraper 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-01-22 00:25:26 -05:00
old4ever 87aa31cf1b feat: update kijiji scraper 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-01-22 00:25:19 -05:00
old4ever bdf504ba37 feat: testing setup 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-01-22 00:25:10 -05:00
old4ever 589af630fa feat: kijiji api findings 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-01-22 00:06:31 -05:00
old4ever 8ae42d5630 chore: prep for opencode 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2026-01-21 23:50:00 -05:00
old4ever 7da6408d7a fix: add target to a bun build command 2025-10-03 09:06:02 -04:00
old4ever 3863d3139e chore: update docker compose 2025-10-03 09:06:02 -04:00
old4ever 11dbf29f93 chore: update README.md 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-10-03 08:59:13 -04:00
old4ever 2e2c81b7e5 chore: update package.json 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-10-03 08:56:03 -04:00
old4ever ea0a69ccd6 feat(docker): enable containerization of marketplace scraper 
Introduce Dockerfile with production build using Bun, .dockerignore for efficient builds, and docker-compose.yml for easy local and prod deployment on port 3000.
 2025-10-03 08:44:51 -04:00
old4ever fa7ac59c45 feat: ebay parser 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-10-02 13:52:29 -04:00
old4ever 8c52efe5e7 feat(facebook): parse additional listing details like status, images, and seller info 
Enhance Facebook scraping to extract listing status (ACTIVE/SOLD/PENDING/HIDDEN), primary image/video URLs, seller name/ID, category ID, and delivery options, improving response completeness.
 2025-10-02 12:03:59 -04:00
old4ever dcd0da29a2 feat: facebook scraping 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-10-02 11:50:35 -04:00
old4ever ee09162faa chore: add claude code dev dependency 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-10-02 10:39:49 -04:00
old4ever ddf7639854 fix: add newlines to debug logging due to overlap with the progress bar 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-09-18 17:12:40 -04:00
old4ever d42c74331e feat: add requests progress bar 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-09-18 17:12:10 -04:00
old4ever 49a32bf6b4 fix: bun's --hot option doesn't like modules 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-09-18 17:11:31 -04:00
old4ever 3166b5a95f chore: remove redundant logging 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-09-18 17:11:09 -04:00
old4ever cb1fb2bae6 feat: add unidecode dependency and slugify function for improved Kijiji search 
This commit introduces the `unidecode` library to handle non-ASCII characters in search queries. A `slugify` function is implemented to improve Kijiji search URL generation by creating more user-friendly and reliable URLs. It converts the user query to a URL friendly string.

It also addresses issues in Kijiji scraping: improves listing filtering logic to ensure only valid listings with prices are added and enhances error handling to provide more informative messages and prevent process termination due to HTTP errors during listing detail retrieval.

Additionally, refactors the Kijiji search query URL, leveraging the new `slugify` function for enhanced URL generation using the search query. Added debugging logs for better traceability.
 2025-09-18 11:35:10 -04:00
old4ever add93dc6bd feat: increase requests per second for Kijiji search 2025-09-18 11:33:47 -04:00
old4ever 99848d1852 build: configure absolute imports using @ alias 
Configures TypeScript path aliases for cleaner and more maintainable imports.
 2025-09-18 11:33:06 -04:00
old4ever d35e37f4c2 feat: implement Kijiji scraping API endpoint 
This commit introduces a new API endpoint that allows users to search for items on Kijiji.
The endpoint accepts a search query as a header or query parameter and returns a JSON response containing the search results. It also handles cases where no query is provided or no results are found, returning appropriate error responses.
 2025-09-17 22:58:11 -04:00
old4ever 2e9547ebaa fix: search GTA only 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-09-17 22:57:42 -04:00
old4ever 46a8ac92cf feat: extract Kijiji scraping logic into reusable function 
This commit extracts the Kijiji scraping functionality into a reusable function `fetchKijijiItems`. This allows for easier integration into other parts of the application and improves code modularity. The function accepts search query, requests per second, and base URL as parameters, enabling customizable scraping.
 2025-09-17 22:03:24 -04:00
old4ever b7a61423c3 refactor: improve Kijiji scraping architecture 
Introduces types, utilities, and a dedicated HTTP client for improved structure and maintainability.  Includes rate limit handling, retry logic, and error handling.
 2025-09-17 20:45:57 -04:00
old4ever 23dc9cae4c feat: scrape listing details from Kijiji ads 2025-09-17 20:43:07 -04:00
old4ever 257edf394c style: specify source of apollo state interface 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-09-17 19:00:13 -04:00
old4ever 63683593e4 feat: add kijiji search parser 
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
 2025-09-17 18:58:57 -04:00
old4ever c09650b9f4 build: configure build and development scripts 2025-09-17 18:54:19 -04:00
old4ever 030e3c302d feat: integrate linkedom for DOM manipulation 
Adds `linkedom` as a dependency for parsing HTML and manipulating the DOM. This enables scraping capabilities.
 2025-09-17 18:48:23 -04:00
old4ever 3fccbb7a46 minor tweaks 2025-08-14 14:49:41 -04:00
old4ever e0cf995769 init bun 2025-07-29 22:31:17 -04:00
old4ever e013ce5a46 install bun 2025-07-29 22:30:40 -04:00
old4ever cc0bb000f2 init devenv 2025-07-29 22:22:17 -04:00