Papers

New on arXiv

Thu 13 Jun 2024

387 papers

Robustness of Structured Data Extraction from In-plane Rotated Documents using Multi-Modal Large Language Models (LLM)Multimodal LLMs / vision-language models → Document understanding and multimodal RAG
ResearchArena: Benchmarking Large Language Models' Ability to Collect and Organize Information as Research AgentsLLM agents and tool use → Web, search, and research agents
Multi-Modal Retrieval For Large Language Model Based Speech RecognitionMultimodal LLMs / vision-language models → Document understanding and multimodal RAG
Multimodal Large Language Models with Fusion Low Rank Adaptation for Device Directed Speech DetectionMultimodal LLMs / vision-language models → MLLM training and visual instruction tuning
Turns Out I'm Not Real: Towards Robust Detection of AI-Generated VideosDiffusion models → Video generation
Online Bandit Learning with Offline Preference Data for Improved RLHFLLM alignment / RLHF / preference optimisation → Reward modeling and RLHF
Improving Consistency Models with Generator-Augmented FlowsDiffusion models → Diffusion foundations and sampling
QMamba: On First Exploration of Vision Mamba for Image Quality AssessmentState-space sequence models (S4, Mamba) → Vision Mamba backbones
CleanDiffuser: An Easy-to-use Modularized Library for Diffusion Models in Decision MakingDiffusion models → Diffusion foundations and sampling
FADE: Towards Fairness-aware Generation for Domain Generalization via Classifier-Guided Score-based Diffusion ModelsDiffusion models → Image generation, text-to-image and editing
Language-driven Grasp DetectionDiffusion models → Diffusion and flow policies for robotics
SeMOPO: Learning High-quality Model and Policy from Low-quality Offline Visual DatasetsOffline RL and imitation learning from data → Offline RL algorithms
Is Diffusion Model Safe? Severe Data Leakage via Gradient-Guided Diffusion ModelDiffusion models → Robustness, privacy and adversarial aspects
GPT-ology, Computational Models, Silicon Sampling: How should we think about LLMs in Cognitive Science?LLM reasoning → Reasoning benchmarks and evaluation
SViTT-Ego: A Sparse Video-Text Transformer for Egocentric VideoMultimodal LLMs / vision-language models → Embodied and 3D vision-language models
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video UnderstandingMultimodal LLMs / vision-language models → Video understanding with MLLMs
Rethinking Score Distillation as a Bridge Between Image DistributionsDiffusion models → 3D, point cloud and scene generation
Alleviating Distortion in Image Generation via Multi-Resolution Diffusion Models and Time-Dependent Layer NormalizationDiffusion models → Diffusion foundations and sampling
Interpreting the Weight Space of Customized Diffusion ModelsDiffusion models → Diffusion foundations and sampling
MuirBench: A Comprehensive Benchmark for Robust Multi-image UnderstandingMultimodal LLMs / vision-language models → MLLM training and visual instruction tuning
Data Attribution for Text-to-Image Models by Unlearning Synthesized ImagesLLM safety and security → Vision-language and text-to-image safety
ConsistDreamer: 3D-Consistent 2D Diffusion for High-Fidelity Scene EditingDiffusion models → 3D, point cloud and scene generation
Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language ModelsLLM reasoning → Multimodal reasoning +1
Instruct 4D-to-4D: Editing 4D Scenes as Pseudo-3D Scenes Using 2D DiffusionDiffusion models → 3D, point cloud and scene generation
Yo'LLaVA: Your Personalized Language and Vision AssistantMultimodal LLMs / vision-language models → MLLM training and visual instruction tuning
Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and AlgorithmsLLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment
A More Practical Approach to Machine UnlearningLLM safety and security → Machine unlearning
Sagiri: Low Dynamic Range Image Enhancement with Generative Diffusion PriorDiffusion models → Inverse problems and image restoration
Towards Vision-Language Geo-Foundation Model: A SurveyMultimodal LLMs / vision-language models → Embodied and 3D vision-language models
CLIPAway: Harmonizing Focused Embeddings for Removing Objects via Diffusion ModelsDiffusion models → 3D, point cloud and scene generation
Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMsMultimodal LLMs / vision-language models → Video understanding with MLLMs
Understanding Hallucinations in Diffusion Models through Mode InterpolationDiffusion models → Medical imaging and segmentation
Advancing Graph Generation through Beta DiffusionDiffusion models → Flow matching, time series and science
Enhancing Domain Adaptation through Prompt Gradient AlignmentMultimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP
Is Value Learning Really the Main Bottleneck in Offline RL?Offline RL and imitation learning from data → Offline RL algorithms
REVS: Unlearning Sensitive Information in Language Models via Rank Editing in the Vocabulary SpaceLLM safety and security → Machine unlearning
Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMsLLM safety and security → Jailbreaking
JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language ModelsLLM safety and security → Jailbreaking
Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language ModelMultimodal LLMs / vision-language models → Medical vision-language models
Transformers meet Neural Algorithmic ReasonersLLM reasoning → Reasoning benchmarks and evaluation
AlignMMBench: Evaluating Chinese Multimodal Alignment in Large Vision-Language ModelsMultimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
StableMaterials: Enhancing Diversity in Material Generation via Semi-Supervised LearningDiffusion models → 3D, point cloud and scene generation
Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion ModelsDiffusion models → 3D, point cloud and scene generation
Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language ModelsLLM safety and security → Jailbreaking
FlowAVSE: Efficient Audio-Visual Speech Enhancement with Conditional Flow MatchingDiffusion models → Audio, speech and music
Unpacking DPO and PPO: Disentangling Best Practices for Learning from Preference FeedbackLLM reasoning → Reinforcement learning for reasoning +1
Position: Towards Bidirectional Human-AI AlignmentLLM alignment / RLHF / preference optimisation → Human values and AI feedback alignment
Generative Inverse Design of Crystal Structures via Diffusion Models with TransformersDiffusion models → Flow matching, time series and science
MirrorCheck: Efficient Adversarial Defense for Vision-Language ModelsMultimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
OpenVLA: An Open-Source Vision-Language-Action ModelVision-language-action models → Generalist VLA models and benchmarks
Comparison Visual Instruction TuningLLM reasoning → Multimodal reasoning
On Softmax Direct Preference Optimization for RecommendationLLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG
Self-Training for Sample-Efficient Active Learning for Text Classification with Pre-Trained Language ModelsSynthetic data and self-improvement for LLMs → Self-training for few-shot NLP
GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled ReasoningLLM reasoning → Reasoning for agents, planning and code +2
Towards Effective Evaluations and Comparisons for LLM Unlearning MethodsLLM safety and security → Machine unlearning
ReMI: A Dataset for Reasoning with Multiple ImagesLLM reasoning → Multimodal reasoning
Potion: Towards Poison UnlearningLLM safety and security → Backdoor and data poisoning attacks
Test of Time: A Benchmark for Evaluating LLMs on Temporal ReasoningLLM reasoning → Knowledge- and tool-augmented reasoning
EMMA: Your Text-to-Image Diffusion Model Can Secretly Accept Multi-Modal PromptsDiffusion models → Image generation, text-to-image and editing
Complex Image-Generative Diffusion Transformer for Audio DenoisingDiffusion models → Audio, speech and music
Diffusion Gaussian Mixture Audio DenoiseDiffusion models → Audio, speech and music
Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMsLLM reasoning → Prompting and chain-of-thought reasoning +1
MMRel: Benchmarking Relation Understanding in Multi-Modal Large Language ModelsMultimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Imitation Learning-based Direct Visual Servoing using the Large Projection FormulationOffline RL and imitation learning from data → Robot manipulation imitation learning
INS-MMBench: A Comprehensive Benchmark for Evaluating LVLMs' Performance in InsuranceMultimodal LLMs / vision-language models → Chart understanding and anomaly detection
DiffPoGAN: Diffusion Policies with Generative Adversarial Networks for Offline Reinforcement LearningDiffusion models → Diffusion and flow policies for robotics +1
Operator-Informed Score Matching for Markov Diffusion ModelsDiffusion models → Diffusion foundations and sampling
Are we making progress in unlearning? Findings from the first NeurIPS unlearning competitionLLM safety and security → Machine unlearning
Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning?LLM reasoning → Knowledge- and tool-augmented reasoning
FairCoT: Enhancing Fairness in Text-to-Image Generation via Chain of Thought Reasoning with Multimodal Large Language ModelsLLM reasoning → Multimodal reasoning +1
Dispelling the Mirage of Progress in Offline MARL through Standardised Baselines and EvaluationOffline RL and imitation learning from data → Offline multi-agent RL
FacEnhance: Facial Expression Enhancing with Recurrent DDPMsDiffusion models → Audio, speech and music
Multi-Agent Collaboration via Cross-Team OrchestrationLLM agents and tool use → General LLM agent frameworks and benchmarks
Preserving Identity with Variational Score for General-purpose 3D EditingDiffusion models → 3D, point cloud and scene generation
Step-by-Step Diffusion: An Elementary TutorialDiffusion models → Diffusion foundations and sampling
CIMRL: Combining IMitation and Reinforcement Learning for Safe Autonomous DrivingEnd-to-end autonomous driving → Imitation-learning end-to-end planners +1
Cognitively Inspired Energy-Based World ModelsWorld models → World models for language agents
Trajectory Planning for Autonomous Driving in Unstructured Scenarios Based on Graph Neural Network and Numerical OptimizationEnd-to-end autonomous driving → Imitation-learning end-to-end planners
COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video EditingDiffusion models → Video generation
ContraSolver: Self-Alignment of Language Models by Resolving Internal Preference ContradictionsLLM alignment / RLHF / preference optimisation → LLM preference optimization methods
A Dual Approach to Imitation Learning from Observations with Offline DatasetsOffline RL and imitation learning from data → Imitation learning methods
FouRA: Fourier Low Rank AdaptationDiffusion models → Image generation, text-to-image and editing
StructuralSleight: Automated Jailbreak Attacks on Large Language Models Utilizing Uncommon Text-Organization StructuresLLM safety and security → Jailbreaking
StreamBench: Towards Benchmarking Continuous Improvement of Language AgentsLLM agents and tool use → General LLM agent frameworks and benchmarks
RL-JACK: Reinforcement Learning-powered Black-box Jailbreaking Attack against LLMsLLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks +1
mOSCAR: A Large-scale Multilingual and Multimodal Document-level CorpusMultimodal LLMs / vision-language models → MLLM training and visual instruction tuning
When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided SearchLLM safety and security → Jailbreaking
VLind-Bench: Measuring Language Priors in Large Vision-Language ModelsMultimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
ChatISA: A Prompt-Engineered, In-House Multi-Modal Generative AI Chatbot for Information Systems Education
On when is Reservoir Computing with Cellular Automata Beneficial?
A Document-based Knowledge Discovery with Microservices Architecture
MoleculeCLA: Rethinking Molecular Benchmark via Computational Ligand-Target Binding Analysis
Modelling the 5G Energy Consumption using Real-world Data: Energy Fingerprint is All You Need
Advanced Multimodal Deep Learning Architecture for Image-Text Matching
Bioptic B1: A Target-Agnostic Potency-Based Small Molecules Search Engine
Opening the Black Box: predicting the trainability of deep neural networks with reconstruction entropy
How Out-of-Distribution Detection Learning Theory Enhances Transformer: Learnability and Reliability
The Significance of Latent Data Divergence in Predicting System Degradation
T-JEPA: A Joint-Embedding Predictive Architecture for Trajectory Similarity Computation
The Promise of Analog Deep Learning: Recent Advances, Challenges and Opportunities

All 387