Crawling Amazon for Review Text
I am looking to better understand how to use the "Crawl Web" operator to pull review text from Amazon.
I have looked through a few posts but nothing seems to be getting at exactly what I am looking for. The goal would be to use Amazon's link structure to scan all of the reviews for a given product.
Below is the basic link structure for getting reviews for item "B019XFKM3". The only thing that needs to be changed or looped within the link is ..."paging_btm_1?" and "reviews&pageNumber=1". When changing the numbers to 1, 2, 3.... we would be able to scan that page of reviews.
How would I be able to set this up using the "Web crawl" operator and further, how would I be able to just pull the review text and star rating?
Any help would be greatly appreciated.