feat: add unidecode dependency and slugify function for improved Kijiji search
This commit introduces the `unidecode` library to handle non-ASCII characters in search queries. A `slugify` function is implemented to improve Kijiji search URL generation by creating more user-friendly and reliable URLs. It converts the user query to a URL friendly string. It also addresses issues in Kijiji scraping: improves listing filtering logic to ensure only valid listings with prices are added and enhances error handling to provide more informative messages and prevent process termination due to HTTP errors during listing detail retrieval. Additionally, refactors the Kijiji search query URL, leveraging the new `slugify` function for enhanced URL generation using the search query. Added debugging logs for better traceability.
This commit is contained in:
@@ -9,12 +9,14 @@
|
||||
"type": "module",
|
||||
"private": true,
|
||||
"devDependencies": {
|
||||
"@types/bun": "latest"
|
||||
"@types/bun": "latest",
|
||||
"@types/unidecode": "^1.1.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"typescript": "^5"
|
||||
},
|
||||
"dependencies": {
|
||||
"linkedom": "^0.18.12"
|
||||
"linkedom": "^0.18.12",
|
||||
"unidecode": "^1.1.0"
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user