mlsanityData to 7 Oct 2026

SymSeqBench: a unified framework for the generation and analysis of rule-based symbolic sequences and datasets

arXiv 2512.24977 ↗First posted 31 Dec 2025q-bio.NC cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Sequential structure is a key feature of multiple domains of natural cognition and behavior, such as language, movement and decision-making. Likewise, it is also a central property of tasks to which we would like to apply artificial intelligence. It is therefore of great importance to develop frameworks that allow us to evaluate sequence learning and processing in a domain agnostic fashion, whilst simultaneously providing a link to formal theories of computation and computability. To address this need, we introduce two complementary software tools: SymSeq, designed to rigorously generate and analyze structured symbolic sequences, and SeqBench, a comprehensive benchmark suite of rule-based sequence processing tasks to evaluate the performance of artificial learning systems in cognitively relevant domains. In combination, SymSeqBench offers versatility in investigating sequential structure across diverse knowledge domains, including experimental psycholinguistics, cognitive psychology, behavioral analysis, neuromorphic computing and artificial intelligence. Due to its basis in Formal Language Theory (FLT), SymSeqBench provides researchers in multiple domains with a convenient and practical way to apply the concepts of FLT to conceptualize and standardize their experiments, thus advancing our understanding of cognition and behavior through shared computational frameworks and formalisms. The tool is modular, openly available and accessible to the research community.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 165 references that are in the map. First posted on arXiv 31 Dec 2025.

Builds on

39 of 165 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  4. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  5. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  6. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  7. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  8. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
  9. 2014Neural Turing Machinescited by 630
  10. 2014Memory Networkscited by 354
  11. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
  12. 2023Multimodal Deep Learningcited by 257
27 more
  1. 2021Training Spiking Neural Networks Using Lessons From Deep Learningcited by 251
  2. 2022Neural Networks and the Chomsky Hierarchycited by 214
  3. 2018Markerless tracking of user-defined features with deep learningcited by 203
  4. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 163
  5. 2019The Heidelberg spiking datasets for the systematic evaluation of spiking neural networkscited by 135
  6. 2024In-Context Language Learning: Architectures and Algorithmscited by 86
  7. 2024A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Modelscited by 41
  8. 2022SuperAnimal pretrained pose estimation models for behavioral analysiscited by 38
  9. 2016Criticality in Formal Languages and Statistical Physicscited by 33
  10. 2017Theory of Deep Learning III: explaining the non-overfitting puzzlecited by 32
  11. 2021A comparison of LSTM and GRU networks for learning symbolic sequencescited by 24
  12. 2025Line Goes Up? Inherent Limitations of Benchmarks for Evaluating Large Language Modelscited by 13
  13. 2024Advancing Spatio-Temporal Processing in Spiking Neural Networks through Adaptationcited by 12
  14. 2022Spiking Neural Networks for event-based action recognition: A new task to understand their advantagecited by 9
  15. 2021Metastable dynamics of neural circuits and networkscited by 5
  16. 2021Sequence learning, prediction, and replay in networks of spiking neuronscited by 5
  17. 2017Re-run, Repeat, Reproduce, Reuse, Replicate: Transforming Code into Scientific Contributionscited by 3
  18. 2020Review of Machine-Learning Methods for RNA Secondary Structure Predictioncited by 3
  19. 2023Decoding the Enigma: Benchmarking Humans and AIs on the Many Facets of Working Memorycited by 3
  20. 2024On the generalization capacity of neural networks during generic multimodal reasoningcited by 3
  21. 2024Cognition is All You Need -- The Next Layer of AI Above Large Language Modelscited by 3
  22. 2025Neuromorphic Sequential Arena: A Benchmark for Neuromorphic Temporal Processingcited by 3
  23. 2022Coherent noise enables probabilistic sequence replay in spiking neuronal networkscited by 2
  24. 2022Characterizing the structure of mouse behavior using Motion Sequencingcited by 2
  25. 2025NeuroMorse: A Temporally Structured Dataset For Neuromorphic Computingcited by 2
  26. 2014AI Evaluation: past, present and futurecited by 1
  27. 2016Closed loop interactions between spiking neural network and robotic simulators based on MUSIC and ROScited by 1

Built on by

Not cited yet.