Topological Deep Learning: Going Beyond Graph Data

Topological deep learning is a rapidly growing field that pertains to the development of deep learning models for data supported on topological domains such as simplicial complexes, cell complexes, and hypergraphs, which generalize many domains encountered in scientific computations. In this paper, we present a unifying deep learning framework built upon a richer data structure that includes widely adopted topological domains. Specifically, we first introduce combinatorial complexes, a novel type of topological domain. Combinatorial complexes can be seen as generalizations of graphs that maintain certain desirable properties. Similar to hypergraphs, combinatorial complexes impose no constraints on the set of relations. In addition, combinatorial complexes permit the construction of hierarchical higher-order relations, analogous to those found in simplicial and cell complexes. Thus, combinatorial complexes generalize and combine useful traits of both hypergraphs and cell complexes, which have emerged as two promising abstractions that facilitate the generalization of graph neural networks to topological spaces. Second, building upon combinatorial complexes and their rich combinatorial and algebraic structure, we develop a general class of message-passing combinatorial complex neural networks (CCNNs), focusing primarily on attention-based CCNNs. We characterize permutation and orientation equivariances of CCNNs, and discuss pooling and unpooling operations within CCNNs in detail. Third, we evaluate the performance of CCNNs on tasks related to mesh shape analysis and graph learning. Our experiments demonstrate that CCNNs have competitive performance as compared to state-of-the-art deep learning models specifically tailored to the same tasks. Our findings demonstrate the advantages of incorporating higher-order relations into deep learning models in different applications.

Mustafa Hajij, Ghada Zamzmi, Theodore Papamarkou, Nina Miolane, Aldo Guzmán-Sáenz, Karthikeyan Natesan Ramamurthy, Tolga Birdal, Tamal K. Dey, Soham Mukherjee, Shreyas N. Samaga, Neal Livesay, Robin Walters, Paul Rosen, and Michael T. Schaub. Topological Deep Learning: Going Beyond Graph Data. arXiv Preprint, 2023.

@article{hajij2023tdl,
  title = {Topological Deep Learning: Going Beyond Graph Data},
  author = {Hajij, Mustafa and Zamzmi, Ghada and Papamarkou, Theodore and Miolane, Nina
    and Guzmán-Sáenz, Aldo and Ramamurthy, Karthikeyan Natesan and Birdal, Tolga and Dey,
    Tamal K. and Mukherjee, Soham and Samaga, Shreyas N. and Livesay, Neal and Walters,
    Robin and Rosen, Paul and Schaub, Michael T.},
  journal = {arXiv Preprint},
  year = {2023},
  abstract = {Topological deep learning is a rapidly growing field that pertains to the
    development of deep learning models for data supported on topological domains such as
    simplicial complexes, cell complexes, and hypergraphs, which generalize many domains
    encountered in scientific computations. In this paper, we present a unifying deep
    learning framework built upon a richer data structure that includes widely adopted
    topological domains. Specifically, we first introduce combinatorial complexes, a novel
    type of topological domain. Combinatorial complexes can be seen as generalizations of
    graphs that maintain certain desirable properties. Similar to hypergraphs, combinatorial
    complexes impose no constraints on the set of relations. In addition, combinatorial
    complexes permit the construction of hierarchical higher-order relations, analogous to
    those found in simplicial and cell complexes. Thus, combinatorial complexes generalize
    and combine useful traits of both hypergraphs and cell complexes, which have emerged as
    two promising abstractions that facilitate the generalization of graph neural networks
    to topological spaces. Second, building upon combinatorial complexes and their rich
    combinatorial and algebraic structure, we develop a general class of message-passing
    combinatorial complex neural networks (CCNNs), focusing primarily on attention-based
    CCNNs. We characterize permutation and orientation equivariances of CCNNs, and discuss
    pooling and unpooling operations within CCNNs in detail. Third, we evaluate the
    performance of CCNNs on tasks related to mesh shape analysis and graph learning. Our
    experiments demonstrate that CCNNs have competitive performance as compared to
    state-of-the-art deep learning models specifically tailored to the same tasks. Our
    findings demonstrate the advantages of incorporating higher-order relations into deep
    learning models in different applications.}
}