IEEE

On the Effectiveness of Automatic Code Generation for Synthetic Dataset Creation

Abstract: This paper compares synthetic and real-world code datasets for machine learning applications in cybersecurity by examining the relationships between machine code and Low-Level Virtual ...
IEEE

Model-Agnostic Empirical Evaluation of Test-Driven Prompt Engineering on Improving Accuracy and Efficiency in Large Language Models Python Code Generation

Abstract: Although Large Language Models (LLMs) are widely adopted for code generation, the generated code can be semantically incorrect, requiring iterations of evaluation and refinement. Test-driven ...
GitHub

Rankify: A Comprehensive Python Toolkit for Retrieval, Re-Ranking, and Retrieval-Augmented Generation

[2025-06-01] Many thanks to @aherzinger for implementing and refactoring the Generator and RAG models. [2025-05-30] Huge thanks to @baraayusry for implementing the Online Retriever using CrawAI and ...
GitHub

DeepCode: Open Agentic Coding

We evaluate DeepCode on the PaperBench benchmark (released by OpenAI), a rigorous testbed requiring AI agents to independently reproduce 20 ICML 2024 papers from scratch. The benchmark comprises 8,316 ...