Which real-time technology/technique is the best for running a heavy loop every 1 minute?

manpreet Best Answer 3 years ago

I'm attempting to create a procedure that will be running on the server every 1 min(more or less). I know I could achieve it with a cronjob but I'm concerned, let's say I have about 1000 tasks (1000 users that the system would need to check every 1 min), wouldn't it kill the server?

This system is supposed to sync data from google adwords API and do something with it. for example it should read a campaign from google and every 1000 impressions or clicks it should do something. So obviously I need to keep running a connection to adwords api to see the stats on real time. Imagine this procedure needs to run with 1000 registered users.

What technology should I use in my case when I need to run a heavy loop every 1 min? Thanks a lot, Oron

manpreet 3 years ago

Ideally, if you're using a distributed workflow, you are better off servicing multiple users this way.