# When Fusion Helps and When It Breaks: View-Aligned Robustness in Same-Source Financial Imaging

arXiv: https://arxiv.org/abs/2602.11020 · HTML: https://mlsanity.com/paper/2602.11020/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 11 Feb 2026.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Imaging Time-Series to Improve Classification and Imputation](https://mlsanity.com/paper/1506.00327/) (2015)
- [Adversarial Attacks on Deep Neural Networks for Time Series Classification](https://mlsanity.com/paper/1903.07054/) (2019)
- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [Defending Multimodal Fusion Models against Single-Source Adversaries](https://mlsanity.com/paper/2206.12714/) (2022)
- [LastingBench: Defend Benchmarks Against Knowledge Leakage](https://mlsanity.com/paper/2506.21614/) (2025)
- [Financial Trading Model with Stock Bar Chart Image Time Series with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1903.04610/) (2019)
- [Encoding Candlesticks as Images for Patterns Classification Using Convolutional Neural Networks](https://mlsanity.com/paper/1901.05237/) (2019)
- [Adversarial Robustness of Deep Convolutional Candlestick Learner](https://mlsanity.com/paper/2006.03686/) (2020)

## Abstract (arXiv)

We study same-source multi-view learning and adversarial robustness for next-day direction prediction using two deterministic, window-aligned image views derived from the same time series: an OHLCV-rendered chart (ohlcv) and a technical-indicator matrix (indic). To control label ambiguity from near-zero moves, we use an ex-post minimum-movement threshold min_move (tau) based on realized absolute next-day return, defining an offline benchmark on the subset where the absolute next-day return is at least tau. Under leakage-resistant time-block splits with embargo, we compare early fusion (channel stacking) and dual-encoder late fusion with optional cross-branch consistency. We then evaluate pixel-space L-infinity evasion attacks (FGSM/PGD) under view-constrained and joint threat models. We find that fusion is regime dependent: early fusion can suffer negative transfer under noisier settings, whereas late fusion is a more reliable default once labels stabilize. Robustness degrades sharply under tiny budgets with stable view-dependent vulnerabilities; late fusion often helps under view-constrained attacks, but joint perturbations remain challenging.