roksana.evaluation package

Submodules

roksana.evaluation.evaluator module

class roksana.evaluation.evaluator.Evaluator(search_method_before, search_method_after, k_values: List[int] = [5, 10, 20])[source]

Bases: object

Evaluator class to assess the impact of attack methods on search strategies.

__init__(search_method_before, search_method_after, k_values: List[int] = [5, 10, 20])[source]

Initialize the Evaluator.

Parameters:

  • search_method_before – Instance of SearchMethod before attack.

  • search_method_after – Instance of SearchMethod after attack.

  • k_values (List[int], optional) – List of k values for Hit@k and Recall@k. Defaults to [5, 10, 20].

evaluate(queries: List[int], gold_sets: List[List[int]], results_dir: str = 'results', filename: str = 'evaluation_results.csv') → None[source]

Perform evaluation on the given queries and save the results.

Parameters:

  • queries (List[int]) – List of query node indices.

  • gold_sets (List[List[int]]) – List of gold sets corresponding to each query.

  • results_dir (str, optional) – Directory to save the results file. Defaults to ‘results’.

  • filename (str, optional) – Name of the results file. Defaults to ‘evaluation_results.csv’.

get_all_results() → List[Dict[str, Any]][source]

Retrieve all evaluation results.

Returns:

List of evaluation result dictionaries.

Return type:

List[Dict[str, Any]]

roksana.evaluation.metrics module

roksana.evaluation.metrics.demotion_value(before_attack_rank: int, after_attack_rank: int) → int[source]

Calculate the Demotion Value metric.

Parameters:

  • before_attack_rank (int) – The rank of the target node before the attack.

  • after_attack_rank (int) – The rank of the target node after the attack.

Returns:

Difference in rank (after_attack_rank - before_attack_rank).

A positive value indicates demotion.

Return type:

int

roksana.evaluation.metrics.hit_at_k(retrieved: List[int], gold_set: List[int], k: int) → float[source]

Calculate Hit@k metric.

Parameters:

  • retrieved (List[int]) – List of retrieved node indices.

  • gold_set (List[int]) – List of gold node indices.

  • k (int) – The k in Hit@k.

Returns:

Hit@k value (1 if at least one gold node is in the top-k, else 0).

Return type:

float

roksana.evaluation.metrics.recall_at_k(retrieved: List[int], gold_set: List[int], k: int) → float[source]

Calculate Recall@k metric.

Parameters:

  • retrieved (List[int]) – List of retrieved node indices.

  • gold_set (List[int]) – List of gold node indices.

  • k (int) – The k in Recall@k.

Returns:

Recall@k value.

Return type:

float

roksana.evaluation.utils module

roksana.evaluation.utils.save_results_to_csv(results: List[Dict[str, Any]], filepath: str) → None[source]

Save evaluation results to a CSV file.

Parameters:

  • results (List[Dict[str, Any]]) – List of evaluation result dictionaries.

  • filepath (str) – Path to the CSV file where results will be saved.

roksana.evaluation.utils.save_results_to_json(results: List[Dict[str, Any]], filepath: str) → None[source]

Save evaluation results to a JSON file.

Parameters:

  • results (List[Dict[str, Any]]) – List of evaluation result dictionaries.

  • filepath (str) – Path to the JSON file where results will be saved.

roksana.evaluation.utils.save_results_to_pickle(results: List[Dict[str, Any]], filepath: str) → None[source]

Save evaluation results to a Pickle file.

Parameters:

  • results (List[Dict[str, Any]]) – List of evaluation result dictionaries.

  • filepath (str) – Path to the Pickle file where results will be saved.

roksana.evaluation.utils.tensor_to_serializable(obj: Any) → Any[source]

Recursively convert Tensor objects in a data structure to serializable types.

Parameters:

obj – The data structure containing potentially Tensors.

Returns:

The data structure with Tensors converted to lists or scalars.

Module contents

class roksana.evaluation.Evaluator(search_method_before, search_method_after, k_values: List[int] = [5, 10, 20])[source]

Bases: object

Evaluator class to assess the impact of attack methods on search strategies.

__init__(search_method_before, search_method_after, k_values: List[int] = [5, 10, 20])[source]

Initialize the Evaluator.

Parameters:

  • search_method_before – Instance of SearchMethod before attack.

  • search_method_after – Instance of SearchMethod after attack.

  • k_values (List[int], optional) – List of k values for Hit@k and Recall@k. Defaults to [5, 10, 20].

evaluate(queries: List[int], gold_sets: List[List[int]], results_dir: str = 'results', filename: str = 'evaluation_results.csv') → None[source]

Perform evaluation on the given queries and save the results.

Parameters:

  • queries (List[int]) – List of query node indices.

  • gold_sets (List[List[int]]) – List of gold sets corresponding to each query.

  • results_dir (str, optional) – Directory to save the results file. Defaults to ‘results’.

  • filename (str, optional) – Name of the results file. Defaults to ‘evaluation_results.csv’.

get_all_results() → List[Dict[str, Any]][source]

Retrieve all evaluation results.

Returns:

List of evaluation result dictionaries.

Return type:

List[Dict[str, Any]]

roksana.evaluation.demotion_value(before_attack_rank: int, after_attack_rank: int) → int[source]

Calculate the Demotion Value metric.

Parameters:

  • before_attack_rank (int) – The rank of the target node before the attack.

  • after_attack_rank (int) – The rank of the target node after the attack.

Returns:

Difference in rank (after_attack_rank - before_attack_rank).

A positive value indicates demotion.

Return type:

int

roksana.evaluation.hit_at_k(retrieved: List[int], gold_set: List[int], k: int) → float[source]

Calculate Hit@k metric.

Parameters:

  • retrieved (List[int]) – List of retrieved node indices.

  • gold_set (List[int]) – List of gold node indices.

  • k (int) – The k in Hit@k.

Returns:

Hit@k value (1 if at least one gold node is in the top-k, else 0).

Return type:

float

roksana.evaluation.recall_at_k(retrieved: List[int], gold_set: List[int], k: int) → float[source]

Calculate Recall@k metric.

Parameters:

  • retrieved (List[int]) – List of retrieved node indices.

  • gold_set (List[int]) – List of gold node indices.

  • k (int) – The k in Recall@k.

Returns:

Recall@k value.

Return type:

float

roksana.evaluation.save_results_to_csv(results: List[Dict[str, Any]], filepath: str) → None[source]

Save evaluation results to a CSV file.

Parameters:

  • results (List[Dict[str, Any]]) – List of evaluation result dictionaries.

  • filepath (str) – Path to the CSV file where results will be saved.

roksana.evaluation.save_results_to_json(results: List[Dict[str, Any]], filepath: str) → None[source]

Save evaluation results to a JSON file.

Parameters:

  • results (List[Dict[str, Any]]) – List of evaluation result dictionaries.

  • filepath (str) – Path to the JSON file where results will be saved.

roksana.evaluation.save_results_to_pickle(results: List[Dict[str, Any]], filepath: str) → None[source]

Save evaluation results to a Pickle file.

Parameters:

  • results (List[Dict[str, Any]]) – List of evaluation result dictionaries.

  • filepath (str) – Path to the Pickle file where results will be saved.