graphtester.evaluate

graphtester.evaluate(dataset: Dataset, ignore_node_features: bool = False, ignore_edge_features: bool = True, additional_features: List[str] | None = None, metrics: List[str] | _Metric | None = None, iterations: int = 3, transformer: bool = False) → EvaluationResult

Evaluate a dataset.

This method analyzes how “hard” a dataset is to classify from 1-WL perspective. It does so by calculating the 1-WL hashes of each node and graph in the dataset and then estimating bounds of the accuracy of a predictor that uses these hashes as features.

The analysis admits additional structural features, which are computed for each graph and added as node and/or edge feature.

Parameters:

datasetDataset

The dataset to evaluate.

ignore_node_featuresbool, optional

Whether to ignore the existing node and edge features of the graphs, by default False

ignore_edge_featuresbool, optional

Whether to ignore the existing edge features of the graphs, by default True

additional_featuresList[str], optional

Additional structural features to estimate and evaluate the dataset with. If None (default), only the existing features of the graphs are used.

metricsList[str], optional

The metrics to use for evaluation. If None (default), all metrics are used. Currently, the following metrics are supported: - “identifiability”: 1-WL-identifiability - “upper_bound_accuracy”: Upper bound accuracy - “upper_bound_accuracy_node”: Upper bound accuracy (node labels)

iterationsint, optional

The number of iterations to run 1-WL for, by default 3

transformerbool, optional

Whether to assume transformer-style node labels, by default False. If True, the graph labels are appended to each node label before estimating a node or link prediction tasks’ bounds. Note that this has no effect for graph prediction tasks.

From a theoretical perspective, this is equivalent to the way a graph transformer would predict a node or edge label since each node can attend to all other nodesin the graph, consequently understanding the graph structure from a 1-WL perspective. GNNs however often does not make use of a readout layer for node or edge prediction tasks, which is why this option is disabled by default.

Returns:

resultEvaluationResult

The evaluation results object.