mlsanityData to 14 Oct 2026

MobQA: A Benchmark Dataset for Semantic Understanding of Human Mobility Data through Question Answering

arXiv 2508.11163 ↗First posted 15 Aug 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper presents MobQA, a benchmark dataset designed to evaluate the semantic understanding capabilities of large language models (LLMs) for human mobility data through natural language question answering. While existing models excel at predicting human movement patterns, it remains unobvious how much they can interpret the underlying reasons or semantic meaning of those patterns. MobQA provides a comprehensive evaluation framework for LLMs to answer questions about diverse human GPS trajectories spanning daily to weekly granularities. It comprises 5,800 high-quality question-answer pairs across three complementary question types: factual retrieval (precise data extraction), multiple-choice reasoning (semantic inference), and free-form explanation (interpretive description), which all require spatial, temporal, and semantic reasoning. Our evaluation of major LLMs reveals strong performance on factual retrieval but significant limitations in semantic reasoning and explanation question answering, with trajectory length substantially impacting model effectiveness. These findings demonstrate the achievements and limitations of state-of-the-art LLMs for semantic mobility understanding.footnoteMobQA dataset is available at https://github.com/CyberAgentAILab/mobqa.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 41 of its 92 references that are in the map. First posted on arXiv 15 Aug 2025.

Builds on

41 of 92 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  6. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  7. 2024A Survey on LLM-as-a-Judgecited by 1,423
  8. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  9. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  10. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  11. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 254
  12. 2020A Survey of Evaluation Metrics Used for NLG Systemscited by 164
29 more
  1. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 159
  2. 2019TVQA+: Spatio-Temporal Grounding for Video Question Answeringcited by 154
  3. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 151
  4. 2019"Going on a vacation" takes longer than "Going for a walk": A Study of Temporal Commonsense Understandingcited by 123
  5. 2021AGQA: A Benchmark for Compositional Spatio-Temporal Reasoningcited by 120
  6. 2024Large Language Models Can Learn Temporal Reasoningcited by 111
  7. 2024Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generationcited by 100
  8. 2023Where Would I Go Next? Large Language Models as Human Mobility Predictorscited by 93
  9. 2020A Survey on Deep Learning for Human Mobilitycited by 88
  10. 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 88
  11. 2022Video Question Answering: Datasets, Algorithms and Challengescited by 84
  12. 2021SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoningcited by 77
  13. 2020TORQUE: A Reading Comprehension Dataset of Temporal Ordering Questionscited by 65
  14. 2020Temporal Reasoning on Implicit Events from Distant Supervisioncited by 58
  15. 2024Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Modelscited by 57
  16. 2015Gated Feedback Recurrent Neural Networkscited by 54
  17. 2024HARGPT: Are LLMs Zero-Shot Human Activity Recognizers?cited by 54
  18. 2020A Survey on Machine Reading Comprehension Systemscited by 41
  19. 2023Neural Retrievers are Biased Towards LLM-Generated Contentcited by 38
  20. 2022Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoningcited by 34
  21. 2023Exploring Large Language Models for Human Mobility Prediction under Public Eventscited by 34
  22. 2023Motion Question Answering via Modular Motion Programscited by 27
  23. 2023TempTabQA: Temporal Question Answering for Semi-Structured Tablescited by 27
  24. 2016Extracting Temporal and Causal Relations between Eventscited by 16
  25. 2024Future Directions in Human Mobility Sciencecited by 16
  26. 2024By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Promptingcited by 12
  27. 2024MobilityGPT: Enhanced Human Mobility Modeling with a GPT modelcited by 11
  28. 2023Human Mobility Question Answering (Vision Paper)cited by 1
  29. 2024Text2Traj2Text: Learning-by-Synthesis Framework for Contextual Captioning of Human Movement Trajectoriescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Do LLMs Know Your Neighborhood? Auditing LLM Priors for Neighborhood-Level Mobility Prediction and Structural Alignmentcited by 0