How does google protect its javascripts against being executed by bots?

Web Technologies Web Development 3 years ago

2.76K 1 0 0 0

_x000D_ _x000D_ I want to make certain areas of my website accessible to browser users only, and not accessible to automated multithread bots. I am doing this to protect my data, and this is one of the measures I plan to use. I thought of requiring all the requests to contain a pair of numbers, which must be computed with javascript, which is going to be obfuscated and obscured. But, after that, there's nothing that should stop a bot writer from grabbing a javascrpt bot engine like htmlunit and just executing the javascripts. Google seems to had solved this problem somehow. I remember that most times I run HtmlUnit on a page with adwords enabled, the script engine crashed. Yet, in real browsers, the pages work just fine. There must be a trick to it. Can anyone point me to a good info on how I could do this on my page?

User submissions are the sole responsibility of contributors, with TuteeHUB disclaiming liability for accuracy, copyrights, or consequences of use; content is for informational purposes only and not professional advice.

Answers (1)

Post Answer
profilepic.png
manpreet Tuteehub forum best answer Best Answer 3 years ago
_x000D_ Based on your requirements, I'd implement some type of authentication. Of course, with respect to how big and known your user base is. Some options are: Have data included from javascript file or ajax call. Most bots won't scrape javascript-included text off of a webpage. Maybe load a blank page, give a short delay (2-3 seconds), and then load the content. That way, most (?) bots will simply detect a blank page. HTTP Authentication would require that users enter a username and password, but it's very simple to manage. Captcha User accounts using maybe php. This may be more overhead than you want though.
0 views
0 shares

No matter what stage you're at in your education or career, TuteeHUB will help you reach the next level that you're aiming for. Simply,Choose a subject/topic and get started in self-paced practice sessions to improve your knowledge and scores.

Similar Forum


Q

Unity hosting a custom piece of code on own website [on hold]

_x000D_ _x000D_ I was wondering how would you be able to host an "API" on your own website that you...
Q

I am needing advice on a web bot [on hold]

_x000D_ _x000D_ I want to code a bot that has a GUI or CLI doesn't really matter yet but a GUI would...
Q

date and time picker, that can take a max and min absolute time, compatible with react [on hold]

_x000D_ _x000D_ I have an issue with scheduling things in a web platform (react.js), and need a date...

Important Web Technologies Links