Look for an API for the particular scraping you want (such as rankings for keywords).
Then use an appropriate language to decode what the API gives you. If it gives you JSON or CSV, then Perl and PHP are excellent. Use the programming language to massage the data, then build a bulk INSERT or a CSV file (for LOAD DATA) and insert the stuff into an InnoDB table.
If you cannot find a suitable API, but you can find suitable web pages, then Perl may be the best for parsing. Look in CPAN for a suitable library to help you; there will be several (some better than others).
manpreet![Tuteehub forum best answer]()
Best Answer
2 years ago
mates i need to know what is the best programming technology is best for web scraping from dynamic sites like Google search,bing search,Social media sites etc hope you get my point.
Want something is highly scalable and low resource taker also.
Also waste majority of developers community?
Modern language with best combination of DATABASE also i was thinking for MYSQL InnoDB? As we need to store the scraped data and present.
Cause we have been using PHP with MYSQL which is slow working at scrapping.
Let me know thanks please.
Regards