Pretraining on Call Graphs: When Binary Analysis Tasks Profit From Context

Binary function embedding models are trained to encode the semantics of binary code in such a way that they can be generalized to a variety of reverse engineering tasks, such as binary code search, vulnerability detection, or malware classification. While many models only take the function in question as contextual input, there have been successful attempts to improve function embeddings by leveraging information from the call graph. In this study, we dissect the implications of these embedding refinements. We conduct experiments using a range of graph-based models on the embeddings generated by two state-of-the-art binary function embedding models. Integrating inter-procedural context, we show that improvements on binary code similarity detection (BCSD) will not necessarily generalize to downstream tasks, neither of semantic nor of syntactic nature. More generally, we find that optimizing for semantic similarity tasks correlates with worse performance on syntactic tasks. By conducting an explanatory analysis on the dataset, we find that the call graph-based enhancements significantly enhance the robustness of embeddings, particularly in scenarios where the initial models struggle. Furthermore, we observe that the added context is more beneficial for namespace-related functions than for those focused on individual logic, confirming that the call graph can be leveraged most effectively in context-dependent scenarios.