Which technology ( SQL/NoSQL) to use for real-time data aggregation
General Tech Technology & Software 2 years ago
Posted on 16 Aug 2022, this text provides information on Technology & Software related to General Tech. Please note that while accuracy is prioritized, the data presented might not be entirely correct or up-to-date. This information is offered for general knowledge and informational purposes only, and should not be considered as a substitute for professional advice.
No matter what stage you're at in your education or career, TuteeHub will help you reach the next level that you're aiming for. Simply,Choose a subject/topic and get started in self-paced practice sessions to improve your knowledge and scores.
Which operating system you favour and why?
General Tech 10 Answers
What are the most popular tech portals in India?
General Tech 7 Answers
What are best technologies available today for education / aiding learning?
General Tech 3 Answers
What professional training certifications are most marketable?
General Tech 9 Answers
manpreet![Tuteehub forum best answer]()
Best Answer
2 years ago
I need to design a near real-time system where documents">documents ( with fields:id,keywords,timestamp ) are getting added to the system. The requirement is to get top-k keywords from the documents">documents added to the system in last x minutes. The typical document addition rate is around 100 documents">documents/sec, which may increase in the future ( hence technology should be horizontally scalable ).
I am thinking of using solr-facets ( with sharding ) to generate the top-k keywords, where I am a bit concerned about the high writes/sec for solr. Another option is to use Cassandra, but not sure how it will scale for range queries ( to compute aggregates ), as OrderPreservingPartitioner could make it difficult to distribute the load.