Evaluate a ranking algorithm [closed]

manpreet Best Answer 2 years ago

I'm working on an algorithm to rank nodes in a directed graph. My goal is to detect the most important node bases on its centricity on the graph.

My problem is: How to evaluate and judge the correction of my algorithm? How can I have a ground truth? I can compare my results with PageRank or HIST algorithms. However, they are not a ground truth.

Any suggestion?

manpreet 2 years ago

The correctness cannot be evaluated because there is no ground truth.

The algorithm performs a mathematically well-defined computation. Every computer / person in the world will come to the same result when given the same inputs.

The concept of 'importance' is a not-so-well-defined human / social variable which could vary due to population demographics like country, user age groups, gender, race, time, economics etc. There could be multiple meanings and results for this concept based on whom you ask and when.

It is, in general, tough to correlate a mathematical concept with a social concept. Hence the correctness can only be guessed by sociological means.

As you say, you could compare your model to other mathematical models such as Pagerank and they also correspond to an approximation of importance.