mlsanityData to 7 Oct 2026

CLEAR: Cue Learning using Evolution for Accurate Recognition Applied to Sustainability Data Extraction

arXiv 2501.18504 ↗First posted 30 Jan 2025cs.CV cs.AI cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Model (LLM) image recognition is a powerful tool for extracting data from images, but accuracy depends on providing sufficient cues in the prompt - requiring a domain expert for specialized tasks. We introduce Cue Learning using Evolution for Accurate Recognition (CLEAR), which uses a combination of LLMs and evolutionary computation to generate and optimize cues such that recognition of specialized features in images is improved. It achieves this by auto-generating a novel domain-specific representation and then using it to optimize suitable textual cues with a genetic algorithm. We apply CLEAR to the real-world task of identifying sustainability data from interior and exterior images of buildings. We investigate the effects of using a variable-length representation compared to fixed-length and show how LLM consistency can be improved by refactoring from categorical to real-valued estimates. We show that CLEAR enables higher accuracy compared to expert human recognition and human-authored prompts in every task with error rates improved by up to two orders of magnitude and an ablation study evincing solution concision.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 35 references that are in the map. First posted on arXiv 30 Jan 2025.

Builds on

17 of 35 references.

References · 17

  1. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  2. 2023Large Language Models as Optimizerscited by 830
  3. 2023ChartLlama: A Multimodal LLM for Chart Understanding and Generationcited by 158
  4. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 129
  5. 2023Large Language Models as Evolutionary Optimizerscited by 122
  6. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 119
  7. 2023LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimizationcited by 86
  8. 2017Dealing with Integer-valued Variables in Bayesian Optimization with Gaussian Processescited by 65
  9. 2023MarioGPT: Open-Ended Text2Level Generation through Large Language Modelscited by 63
  10. 2022Investigating Prompt Engineering in Diffusion Modelscited by 58
  11. 2022A Taxonomy of Prompt Modifiers for Text-To-Image Generationcited by 54
  12. 2022Best Prompts for Text-to-Image Models and How to Find Themcited by 35
5 more
  1. 2024LLM Guided Evolution -- The Automation of Models Advancing Modelscited by 20
  2. 2024EIVEN: Efficient Implicit Attribute Value Extraction using Multimodal LLMcited by 12
  3. 2024SCAPE: Searching Conceptual Architecture Prompts using Evolutioncited by 4
  4. 2024Address-Specific Sustainable Accommodation Choice Through Real-World Data Integrationcited by 2
  5. 2024Automated Real-World Sustainability Data Generation from Images of Buildingscited by 2

Built on by

Not cited yet.