mlsanityData to 7 Oct 2026

FLAME: A small language model for spreadsheet formulas

arXiv 2301.13779 ↗First posted 31 Jan 2023cs.PL cs.AI cs.SEcited by 15

Not on the map. Why?

Abstract (arXiv)

Spreadsheets are a vital tool for end-user data management. Using large language models for formula authoring assistance in these environments can be difficult, as these models are expensive to train and challenging to deploy due to their size (up to billions of parameters). We present FLAME, a transformer-based model trained exclusively on Excel formulas that leverages domain insights to achieve competitive performance while being substantially smaller (60M parameters) and training on two orders of magnitude less data. We curate a training dataset using sketch deduplication, introduce an Excel-specific formula tokenizer, and use domain-specific versions of masked span prediction and noisy auto-encoding as pre-training objectives. We evaluate FLAME on formula repair, formula completion, and similarity-based formula retrieval. FLAME can outperform much larger models, such as the Davinci (175B) and Cushman (12B) variants of Codex and CodeT5 (220M), in 10 of 14 evaluation settings for the repair and completion tasks. For formula retrieval, FLAME outperforms CodeT5, CodeBERT, and GraphCodeBERT.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 35 of its 80 references that are in the map. First posted on arXiv 31 Jan 2023.

Builds on

35 of 80 references.

References · 35

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  10. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  11. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  12. 2018Hierarchical Neural Story Generationcited by 1,182
23 more
  1. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  2. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  3. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
  4. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  5. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
  6. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  7. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  8. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  9. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  10. 2021TAPEX: Table Pre-training via Learning a Neural SQL Executorcited by 233
  11. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  12. 2020Pre-training Tasks for Embedding-based Large-scale Retrievalcited by 153
  13. 2018The Adverse Effects of Code Duplication in Machine Learning Models of Codecited by 123
  14. 2022Automating Code Review Activities by Large-Scale Pre-trainingcited by 98
  15. 2020Graph-based, Self-Supervised Program Repair from Diagnostic Feedbackcited by 93
  16. 2021Break-It-Fix-It: Unsupervised Learning for Program Repaircited by 71
  17. 2022Repair Is Nearly Generation: Multilingual Program Repair with LLMscited by 68
  18. 2022Table Pre-training: A Survey on Model Architectures, Pre-training Objectives, and Downstream Taskscited by 42
  19. 2021SpreadsheetCoder: Formula Prediction from Semi-structured Contextcited by 41
  20. 2018How Much Does Tokenization Affect Neural Machine Translation?cited by 26
  21. 2021Learning to Complete Code with Sketchescited by 23
  22. 2021Multi-modal Program Inference: a Marriage of Pre-trainedLanguage Models and Component-based Synthesiscited by 12
  23. 2022Neurosymbolic Repair for Low-Code Formula Languagescited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 7; 2026: 1.

15 papers.

Sort:

By area:

  1. 2023Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflowcited by 76
  2. 2023SheetCopilot: Bringing Software Productivity to the Next Level through Large Language Modelscited by 53
  3. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  4. 2024SpreadsheetLLM: Encoding Spreadsheets for Large Language Modelscited by 15
  5. 2026SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflowscited by 7
  6. 2024Data Analysis in the Era of Generative AIcited by 6
  7. 2025TableTalk: Scaffolding Spreadsheet Development with a Language Agentcited by 5
  8. 2023Knowledge Base Question Answering for Space Debris Queriescited by 3
  9. 2025Formula-R1: Incentivizing LLM Reasoning over Complex Tables with Numerical Computation via Formula-Driven Reinforcement Learningcited by 3
  10. 2025Diffusion is a code repair operator and generatorcited by 3
  11. 2025Small Language Models (SLMs) Can Still Pack a Punch: A survey (updated 2026)cited by 2
  12. 2025SODBench: A Large Language Model Approach to Documenting Spreadsheet Operationscited by 1
3 more
  1. 2025AdaGradSelect: An adaptive gradient-guided layer selection method for efficient fine-tuning of SLMscited by 1
  2. 2024An Empirical Study of Validating Synthetic Data for Formula Generationcited by 0
  3. 2025Benchmark Dataset Generation and Evaluation for Excel Formula Repair with LLMscited by 0