Detecting spammers with artificially generated target class

manpreet Best Answer 2 years ago

I have a bunch of data related to my company's incoming and outgoing calls, and I'm trying to detect spam callers in order to blacklist them. The features I've extracted from the data are a bit simple right now:

• Caller Number: Who is calling
• N_Unique: The amount of different phones called
• N_Answered: The amount of times the call was connected
• N_Missed: The amount of times the call was ignored/missed
• Avg_Duration: The average duration answered calls lasted

Initially my superior sent me the dataset with a field called "IsSpammer", which was binary, and after a very simple Random Forest (1000 trees, default scikit-learn everything else) I got a 100% accuracy even after cross-validation. I took a closer look at the data and realized anyone who called more than 20 different numbers inside the company or had average call duration less than 20 seconds where flagged as spammers (which may be true), but then I asked supervisor how this field was generated, and he told me he set it under that rule the random forest discovered.

I then realized every ML problem I've faced came from a Kaggle contest that gave me a clear field to predict, and a training set that had "correct" information on the fields to predict. I don't know how to proceed with ML if I don't have the information of Spammer/NotSpammer, since apparently if I come up with a clever (or not) rule, then the best I can hope to discover is that rule I'm using in the first place... I'm not sure if getting more features and making and obscure rule using all of them would help. Obviously I lack experience in this, so I figured I would ask.

Is this a problem that could benefit from using machine learning?

manpreet 2 years ago