Commit Graph

28 Commits

Author SHA1 Message Date
01dd52bf58 chore: update docker compose 2025-10-03 08:59:13 -04:00
11dbf29f93 chore: update README.md
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-10-03 08:59:13 -04:00
2e2c81b7e5 chore: update package.json
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-10-03 08:56:03 -04:00
ea0a69ccd6 feat(docker): enable containerization of marketplace scraper
Introduce Dockerfile with production build using Bun, .dockerignore for efficient builds, and docker-compose.yml for easy local and prod deployment on port 3000.
2025-10-03 08:44:51 -04:00
fa7ac59c45 feat: ebay parser
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-10-02 13:52:29 -04:00
8c52efe5e7 feat(facebook): parse additional listing details like status, images, and seller info
Enhance Facebook scraping to extract listing status (ACTIVE/SOLD/PENDING/HIDDEN), primary image/video URLs, seller name/ID, category ID, and delivery options, improving response completeness.
2025-10-02 12:03:59 -04:00
dcd0da29a2 feat: facebook scraping
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-10-02 11:50:35 -04:00
ee09162faa chore: add claude code dev dependency
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-10-02 10:39:49 -04:00
ddf7639854 fix: add newlines to debug logging due to overlap with the progress bar
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-09-18 17:12:40 -04:00
d42c74331e feat: add requests progress bar
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-09-18 17:12:10 -04:00
49a32bf6b4 fix: bun's --hot option doesn't like modules
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-09-18 17:11:31 -04:00
3166b5a95f chore: remove redundant logging
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-09-18 17:11:09 -04:00
cb1fb2bae6 feat: add unidecode dependency and slugify function for improved Kijiji search
This commit introduces the `unidecode` library to handle non-ASCII characters in search queries. A `slugify` function is implemented to improve Kijiji search URL generation by creating more user-friendly and reliable URLs. It converts the user query to a URL friendly string.

It also addresses issues in Kijiji scraping: improves listing filtering logic to ensure only valid listings with prices are added and enhances error handling to provide more informative messages and prevent process termination due to HTTP errors during listing detail retrieval.

Additionally, refactors the Kijiji search query URL, leveraging the new `slugify` function for enhanced URL generation using the search query. Added debugging logs for better traceability.
2025-09-18 11:35:10 -04:00
add93dc6bd feat: increase requests per second for Kijiji search 2025-09-18 11:33:47 -04:00
99848d1852 build: configure absolute imports using @ alias
Configures TypeScript path aliases for cleaner and more maintainable imports.
2025-09-18 11:33:06 -04:00
d35e37f4c2 feat: implement Kijiji scraping API endpoint
This commit introduces a new API endpoint that allows users to search for items on Kijiji.
The endpoint accepts a search query as a header or query parameter and returns a JSON response containing the search results. It also handles cases where no query is provided or no results are found, returning appropriate error responses.
2025-09-17 22:58:11 -04:00
2e9547ebaa fix: search GTA only
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-09-17 22:57:42 -04:00
46a8ac92cf feat: extract Kijiji scraping logic into reusable function
This commit extracts the Kijiji scraping functionality into a reusable function `fetchKijijiItems`. This allows for easier integration into other parts of the application and improves code modularity. The function accepts search query, requests per second, and base URL as parameters, enabling customizable scraping.
2025-09-17 22:03:24 -04:00
b7a61423c3 refactor: improve Kijiji scraping architecture
Introduces types, utilities, and a dedicated HTTP client for improved structure and maintainability.  Includes rate limit handling, retry logic, and error handling.
2025-09-17 20:45:57 -04:00
23dc9cae4c feat: scrape listing details from Kijiji ads 2025-09-17 20:43:07 -04:00
257edf394c style: specify source of apollo state interface
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-09-17 19:00:13 -04:00
63683593e4 feat: add kijiji search parser
Signed-off-by: Dmytro Stanchiev <git@dmytros.dev>
2025-09-17 18:58:57 -04:00
c09650b9f4 build: configure build and development scripts 2025-09-17 18:54:19 -04:00
030e3c302d feat: integrate linkedom for DOM manipulation
Adds `linkedom` as a dependency for parsing HTML and manipulating the DOM. This enables scraping capabilities.
2025-09-17 18:48:23 -04:00
3fccbb7a46 minor tweaks 2025-08-14 14:49:41 -04:00
e0cf995769 init bun 2025-07-29 22:31:17 -04:00
e013ce5a46 install bun 2025-07-29 22:30:40 -04:00
cc0bb000f2 init devenv 2025-07-29 22:22:17 -04:00