美文网首页
Bengio等人:benchmarking-gnns

Bengio等人:benchmarking-gnns

作者: Valar_Morghulis | 来源:发表于2022-06-24 14:35 被阅读0次

Benchmarking Graph Neural Networks

Bengio等人在2020年3月份的时候对各种流行的图神经网络做了一些评测。

最近(2022年5月)对这一评测做了一些更新。

https://arxiv.org/pdf/2003.00982.pdf

https://github.com/graphdeeplearning/benchmarking-gnns

In the last few years, graph neural networks (GNNs) have become the standard toolkit for analyzing and learning from data on graphs. This emerging field has witnessed an extensive growth of promising techniques that have been applied with success to computer science, mathematics, biology, physics and chemistry. But for any successful field to become mainstream and reliable, benchmarks must be developed to quantify progress. This led us in March 2020 to release a benchmark framework that i) comprises of a diverse collection of mathematical and real-world graphs, ii) enables fair model comparison with the same parameter budget to identify key architectures, iii) has an open-source, easy-to-use and reproducible code infrastructure, and iv) is flexible for researchers to experiment with new theoretical ideas. As of May 2022, the GitHub repository has reached 1,800 stars and 339 forks, which demonstrates the utility of the proposed open-source framework through the wide usage by the GNN community. In this paper, we present an updated version of our benchmark with a concise presentation of the aforementioned framework characteristics, an additional medium-sized molecular dataset AQSOL, similar to the popular ZINC, but with a real-world measured chemical target, and discuss how this framework can be leveraged to explore new GNN designs and insights. As a proof of value of our benchmark, we study the case of graph positional encoding (PE) in GNNs, which was introduced with this benchmark and has since spurred interest of exploring more powerful PE for Transformers and GNNs in a robust experimental setting.

在过去几年中,图形神经网络(GNNs)已成为分析和学习图形数据的标准工具包。这一新兴领域见证了有希望的技术的广泛发展,这些技术已成功应用于计算机科学、数学、生物学、物理和化学。但要使任何成功的领域成为主流和可靠的领域,必须制定基准来量化进展。这导致我们于2020年3月发布了一个基准框架,该框架i)由各种数学和现实世界的图表组成,ii)能够与相同的参数预算进行公平的模型比较,以确定关键架构,iii)具有开源、易于使用和可复制的代码基础架构,以及iv)研究人员可以灵活地尝试新的理论想法。截至2022年5月,GitHub存储库已达到1800个stars和339个forks,这证明了所提议的开源框架在GNN社区的广泛使用中的效用。在本文中,我们介绍了我们的基准测试的更新版本,其中简要介绍了上述框架特征,另外一个中等规模的分子数据集AQSOL,类似于流行的ZINC,但具有实际测量的化学目标,并讨论了如何利用此框架探索新的GNN设计和见解。为了证明我们的基准测试的价值,我们研究了GNNs中的图形位置编码(PE)案例,该案例是在该基准测试中引入的,并激发了在稳健的实验环境中探索Transformers和GNN更强大PE的兴趣。

Gated Graph ConvNet (GatedGCN) (Bresson and Laurent, 2017) 

GatedGCN considers residual connections, batch normalization and edge gates (Marcheggiani and Titov, 2017) to design another anisotropic variant of GCN. The authors propose to explicitly update edge features along with node features:

Gated Graph ConvNet (GatedGCN) (Bresson and Laurent, 2017)GatedGCN考虑了残差连接、批量归一化和连边门(edge gates)(Marcheggiani和Titov,2017)来设计GCN的另一种各向异性变体。作者建议显式更新边特征和节点特征:

h_{i}^{\ell+1} = h_{i}^{\ell} + \text{ReLU} \Big( \text{BN} \Big( U^{\ell} h_{i}^{\ell} + \sum_{j \in \mathcal{N}_i} e_{ij}^{\ell} \odot V^{\ell} h_{j}^{\ell} \Big)\Big)

其中U^{\ell}, V^{\ell} \in \mathbb{R}^{d \times d}\odot是Hadamard积,连边门e_{ij}^{\ell}定义为:

e_{ij}^{\ell} = \frac{\sigma(\hat e_{ij}^{\ell})}{\sum_{j’ \in \mathcal{N}_i} \sigma(\hat e_{ij’}^{\ell}) + \varepsilon }    (17)

\hat e_{ij}^{\ell} = \hat e_{ij}^{\ell-1} + \text{ReLU} \Big( \text{BN} \big( A^{\ell} h_{i}^{\ell-1} + B^{\ell} h_{j}^{\ell-1} + C^{\ell} \hat e_{ij}^{\ell-1} \big) \Big)    (18)

其中\sigma 是激活函数,\varepsilon 是数值稳定性的小固定常数,A^\ell, B^\ell, C^\ell \in \mathbb{R}^{d \times d},如图9。请注意,公式(17)可以看作是一个软注意过程,与标准的稀疏注意机制有关 (Bahdanau et al., 2014)。与其他各向异性GNN不同,GatedGCN体系结构明确维护了每层的边缘特征\hat e_{ij},following Bresson和Laurent(2019);Joshi等人(2019年)。

GatedGCN的来源:

Residual Gated Graph ConvNets

作者:Xavier Bresson, Thomas Laurent

https://arxiv.org/abs/1711.07553

Graph-structured data such as social networks, functional brain networks, gene regulatory networks, communications networks have brought the interest in generalizing deep learning techniques to graph domains. In this paper, we are interested to design neural networks for graphs with variable length in order to solve learning problems such as vertex classification, graph classification, graph regression, and graph generative tasks. Most existing works have focused on recurrent neural networks (RNNs) to learn meaningful representations of graphs, and more recently new convolutional neural networks (ConvNets) have been introduced. In this work, we want to compare rigorously these two fundamental families of architectures to solve graph learning tasks. We review existing graph RNN and ConvNet architectures, and propose natural extension of LSTM and ConvNet to graphs with arbitrary size. Then, we design a set of analytically controlled experiments on two basic graph problems, i.e. subgraph matching and graph clustering, to test the different architectures. Numerical results show that the proposed graph ConvNets are 3-17% more accurate and 1.5-4x faster than graph RNNs. Graph ConvNets are also 36% more accurate than variational (non-learning) techniques. Finally, the most effective graph ConvNet architecture uses gated edges and residuality. Residuality plays an essential role to learn multi-layer architectures as they provide a 10% gain of performance.

社交网络、功能性大脑网络、基因调控网络、通信网络等图形结构数据带来了将深度学习技术推广到图形领域的兴趣。在本文中,我们有兴趣为变长图设计神经网络,以解决诸如顶点分类、图分类、图回归和图生成任务等学习问题。现有的大多数工作都集中于递归神经网络(RNN)来学习图的有意义表示,最近又引入了新的卷积神经网络(CONVNET)。在这项工作中,我们希望严格比较这两个基本体系结构家族,以解决图形学习任务。我们回顾了现有的图RNN和ConvNet体系结构,并提出了LSTM和ConvNet对任意大小图的自然扩展。然后,我们针对两个基本的图问题,即子图匹配和图聚类,设计了一组分析控制实验来测试不同的体系结构。数值结果表明,与图RNNs相比,该图CONVNET的精度提高了3-17%,速度提高了1.5-4倍。图形转换网络也比变分(非学习)技术的精度高36%。最后,最有效的graph ConvNet体系结构使用了选通边和残差。剩余性在学习多层体系结构方面起着至关重要的作用,因为它们可以提供10%的性能增益。

相关文章

网友评论

      本文标题:Bengio等人:benchmarking-gnns

      本文链接:https://www.haomeiwen.com/subject/yygvvrtx.html