Speak now
Please Wait Image Converting Into Text...
Embark on a journey of knowledge! Take the quiz and earn valuable credits.
Challenge yourself and boost your learning! Start the quiz now to earn credits.
Unlock your potential! Begin the quiz, answer questions, and accumulate credits along the way.
General Tech Technology & Software 2 years ago
Posted on 16 Aug 2022, this text provides information on Technology & Software related to General Tech. Please note that while accuracy is prioritized, the data presented might not be entirely correct or up-to-date. This information is offered for general knowledge and informational purposes only, and should not be considered as a substitute for professional advice.
Turn Your Knowledge into Earnings.
I am stuck looking for a suitable database technology to suit our application. I am stuck between going cloud-based or keeping it local.
We have a service that receives a log stream for thousands of devices. We have to store every log in order to do queries and analytics on that data. We do about 4-6 GB a day.
We currently have a local machine running SQL Server Enterprise with daily partitions. But the database has reached 1TB now and queries have become very slow and when running big queries it starts affecting the service that is continuously inserting in to the DB.
So we are stuck between spending a fair amount of money on upgrading the local machine vs moving it to the cloud. However, when looking at cloud solutions, they seem to get very expensive when running at such levels of performance.
I am also open to moving away from SQL Server. I quite like the idea of Azure table storage and how it does easy partitioning. Is there possibly a similar technology that we can run on a local machine?
So my question is: what are my options here?
Considering the volumes you are discussing moving it to the cloud isn't going to be magic fix.
"There is no such thing as a free lunch"
A computer still has to process that volume, and that comes at a cost. If you are asking a 3rd party vendor to provide that theyre not just covering costs, but trying to make a profit too.
Changing to a different db platform can also have a lot of hidden costs. Even though licensing might be "free", there can be additional costs in maintenance, integration and compatibility. Even learning a new system is an overhead.
The spec you mention is not a big machine by server standards. I would be confident saying thats undersized for the volumes you describe.
What performance tuning have you done already? What are the bottlenecks? RAM/CPU/IO?
How much faster does it need to be? How fast is fast enough? Do you need a 20% improvement? 50% 100% You might easily get that with more RAM and some SSD's.
If you cant add more resources, is there anyway you can make the server do less work?
Archiving: Could you move old data off to a separate db. Or even better could you aggregate the data and purge the raw records?
No matter what stage you're at in your education or career, TuteeHub will help you reach the next level that you're aiming for. Simply,Choose a subject/topic and get started in self-paced practice sessions to improve your knowledge and scores.
General Tech 9 Answers
General Tech 7 Answers
General Tech 3 Answers
General Tech 2 Answers
Ready to take your education and career to the next level? Register today and join our growing community of learners and professionals.