Publications

Research, in the open.

Papers, code and the ideas behind our projects. Search by title, author or research topic.

110 papers
2026AI for Chip Design NeurIPS Workshop

Beyond Equivalence: Task-Level Verification Unlocks Architectural Optimizations by LLM Agents

Felix Arnold, Ryan Amaudruz, Renzo Andri, Lukas Cavigelli

LLM InferenceEfficient AI
Abstract

Automated RTL optimization is conventionally verified by logical equivalence. The optimized design must reproduce the reference's cycle-by-cycle I/O behavior, which fixes the schedule and the memory access pattern. We relax the verification contract to the ask level, where the only requirement is correctness of the output data for given input data, leaving the cycle schedule, memory access order, and internal architecture free. On six benchmarks ranging from matrix multiplication to a RISC-V ALU, an LLM optimization agent improves the baseline RTL design under both formulations. We compare the achieved cost using cycle-aware metrics: area-runtime product and its energy-weighted extension, where energy is the measured circuit energy for a given workload. Equivalence-constrained optimization reduces cost by up to 57%, while task-level freedom reaches up to 82%. The measured workload energy decreases by up to 31% under equivalence and up to 90% with task-level freedom. A further analysis on the matrix-multiplication benchmark weights the cost by data-movement count and shows that an exponent on the movement term acts as a dial over the accelerator design space, steering the optimal architecture from a single multiplier through operand-reuse tiles to a fully buffered parallel array.

2026SLM-Agents: NeurIPS Workshop

Knowing the Questions before the Exam: Enabling Selective Memory in Linear Attention Models

Sam Isac Aegerter, Lorenz Mueller, Jiawei Zhuang, Lukas Cavigelli

LLM InferenceEfficient AI
Abstract

Linear-attention language models compress their input into a fixed-size state and while processing tokens must decide what information to retain. Such compressed attention models are particularly attractive for long form generation as in agentic workloads. An advance hint or question, as is available in many practical agentic tasks, should facilitate this selection. We test this hypothesis on multi-document question answering by placing the question before or after contexts containing relevant and irrelevant information. Contrary to expectation, both a recurrent model (RWKV-7 7.2B) and a hybrid model (Qwen3.5 0.8B) recall less when the question appears first. Low-rank adaptation on question-first data reverses this result, enabling selective retention at lengths far beyond fine-tuning and even beyond the pretrained context window. Fine-tuning makes state writing, erasing, and decay depend on question relevance, a behavior we call selective memory. Varying question reliability during training controls the trade-off between selectivity and breadth. Thus, linear-attention architectures support selective memory, but targeted adaptation is required to enable it.

2026AIWILD@NeurIPS

Pretext: Defeating Malicious Skill Detection Frameworks for AI Agents

Tobias Kaiser, Aritra Dhar

Efficient AI
Abstract

Skills extend agent capability by injecting instructions and information into the context, and are widely used by agents such as OpenClaw and Claude Code. Prior work shows third-party marketplaces host malicious skills that give attackers direct influence over the victim's agent. The emerging defense scans skills before installation, pairing deterministic static checks with an LLM-based semantic judge, as in NVIDIA's SkillSpector. We show that such defenses fall to an attacker who knows the detector. Our white-box LLM attacker, Pretext, iteratively crafts skills that evade detection while still delivering the payload and performing the benign task: moving the payload from code into natural language leaves static analysis inert, while framing it as the skill's legitimate purpose and splitting instructions across files keeps the LLM stage below its blocking threshold. Across three open-source models, Pretext achieves up to 97% and 77% against a frozen detector and a co-adaptive one, respectively, revealing major gaps in current skill scanners.

2026AIWILD@NeurIPS

RAG-Pull: Turning Retrieval into a Code-Injection Channel via Invisible Unicode Perturbations

Aritra Dhar, Vasilije Stambolic, Lukas Cavigelli

SecurityEfficient AI
Abstract

Retrieval-Augmented Generation (RAG) increases the reliability and trustworthiness of the LLM response and reduces hallucination by eliminating the need for model retraining. It does so by adding external data into the LLM's context. We develop a new class of black-box attack, RAG-Pull, that inserts hidden UTF characters into queries or external code repositories, redirecting retrieval toward malicious code, thereby breaking the models' safety alignment. We observe that query and code perturbations alone can shift retrieval toward attacker-controlled snippets, while combined query-and-target perturbations achieve near-perfect success. Once retrieved, these snippets introduce exploitable vulnerabilities such as remote code execution and SQL injection. RAG-Pull's minimal perturbations can alter the model's safety alignment and increase preference towards unsafe code, therefore opening up a new class of attacks on LLMs.

Read the source
2026AI for Chip Design NeurIPS Workshop

RTLScout: Joint Agentic Code and Synthesis Optimization for Efficient Digital Circuits

Felix Arnold, Ryan Amaudruz, Dimitrios Tsaras, Renzo Andri, Lukas Cavigelli

Efficient AI
Abstract

We present RTLScout, an autonomous system that combines LLM-driven agentic design with logic synthesis optimization and arithmetic architecture selection. An LLM agent iteratively writes, evaluates, and refines RTL designs, guided by delay and area feedback from Yosys and OpenROAD. The agent writes Spire, a Python-embedded HDL we introduce, in which optimization intent is expressed locally in the source, selecting logic-synthesis or arithmetic-architecture optimizations per subcircuit. The four-phase pipeline relies entirely on open-source EDA tools and an open-weights LLM. On an IEEE-754-compliant 16-bit floating-point multiplier with subnormal support, RTLScout reduces area by 34% and delay by 38% relative to a starting design and outperforms a commercial-tool reference design on the ASAP7 technology. We show that agentic RTL rewriting and synthesis optimization are complementary, with neither alone reaching the result of the full pipeline. On 14 RTLRewriter benchmarks, the Spire-based pipeline achieves 16.3% lower mean per-case Yosys cell count than an otherwise identical Verilog pipeline.

2026ICPP

Elasticity in Parallel Sparse Triangular Solve

Raphael Sebastian Steiner, Christos Konstantinos Matzoros, Pál András Papp, Toni Boehnlein, Albert-Jan N. Yzelman

Parallel ComputingSystems Software
Abstract

We introduce stale synchronous parallel as a mode of execution in parallel sparse triangular linear system solve and present a general directed-acyclic-graph scheduler capable of producing such schedules. Stale-synchronous-parallel schedules allow the overlap of synchronisation and compute which results in a geometric-mean speed-up of 7-30% of our scheduler, ElasticDivide, over state-of-the-art synchronous scheduler GrowLocal on an ARM machine using 48 cores. On an x86 machine using 48 cores, we report geometric-mean speed-ups of 19-60% over SpMP.

Read the source
2026NeurIPS

Approaching I/O-optimality for Approximate Attention

Pál András Papp, Aleksandros Sobczyk, Anastasios Zouzias

LLM InferenceSystems Software
Abstract

We revisit the I/O complexity of attention in large language models. Given query-key-value matrices Q,K,V∈ℝ^(n× d), and a machine with fast memory size M, the goal is to compute the "attention matrix" A=softmax(Q K ^(T)/√(d)) V with the minimal number of data transfers between fast and slow memory. Existing methods in the literature, most notably FlashAttention and its variants, incur an I/O cost that depends quadratically on n, while a trivial lower bound only requires Ω(nd) I/O's to read the inputs and write the output. In this work, we present a technique for computing attention where the I/O cost only depends almost-linearly on n in most parameter regimes. This is achieved by developing I/O-efficient algorithms inspired by the recent approximate attention framework of Alman and Song. We also prove corresponding lower bounds in each parameter regime to show that our algorithms are indeed close to I/O-optimal.

2026NeurIPS

Diff3R: Feed-forward 3D Gaussian Splatting with Uncertainty Aware Differentiable Optimization

Yueh-Cheng Liu, Jozef Hladký, Matthias Nießner, Angela Dai

RenderingGeometryGraphicsEfficient AI
Abstract

Recent advances in 3D Gaussian Splatting (3DGS) present two main directions: feed-forward models offer fast inference in sparse-view settings, while per-scene optimization yields high-quality renderings but is computationally expensive. To combine the benefits of both, we introduce Diff3R, a novel framework that explicitly bridges feed-forward prediction and test-time optimization. By incorporating a differentiable 3DGS optimization layer directly into the training loop, our network learns to predict an optimal initialization for test-time optimization rather than a conventional zero-shot result. To overcome the computational cost of backpropagating through the optimization steps, we propose computing gradients via the Implicit Function Theorem and a scalable, matrix-free PCG solver tailored for 3DGS optimization. Additionally, we incorporate a data-driven uncertainty model into the optimization process by adaptively controlling how much the parameters are allowed to change during optimization. This approach effectively mitigates overfitting in under-constrained regions and increases robustness against input outliers. Since our proposed optimization layer is model-agnostic, we show that it can be seamlessly integrated into existing feed-forward 3DGS architectures for both pose-given and pose-free methods, providing improvements for test-time optimization.

2026NeurIPS

Fast and Stable Triangular Inversion for Delta-Rule Linear Transformers

Aleksandros Sobczyk, Gioele Gottardo, Christos K. Matzoros, Mirko De Vita, Filip Skogh, Anastasios Zouzias, Jiawei Zhuang

Parallel ComputingSystems Software
Abstract

Linear attention has emerged as a cornerstone for efficient long-context architectures, as evidenced by its integration into state-of-the-art open-source models including Qwen3.5/3.6, Kimi Linear, and RWKV-7. Models that incorporate linear attention layers with the so-called Delta-Rule involve the inversion of triangular matrices as a core sub-routine. This operation often forms a performance bottleneck, and, due to its high-sensitivity to numerical errors, it can significantly deteriorate end-to-end model accuracy if it is not carefully implemented. This work provides a systematic analysis of both direct and iterative triangular inversion algorithms, targeting methods that are rich in matrix products, and, therefore, have the potential to efficiently utilize modern hardware. To that end, our analysis covers a broad spectrum of mathematical and practical aspects, with a heavy focus on numerical stability, computational complexity, and, ultimately, hardware efficiency and practical considerations. We provide a rigorous experimental evaluation to verify these properties in practical scenarios, and in low-precision floating-point representations, highlighting the strengths and limitations of each method. Performance benchmarks on NPUs reveal up to 4.3× speed-up against the state-of-the-art implementations of SGLang for triangular matrix inversion, leading to significant performance improvements on the entire layer level, while maintaining full end-to-end model accuracy.

2026NeurIPS

GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction

Katharina Schmid, Nicolas von Lützow, Jozef Hladký, Angela Dai, Matthias Nießner

GeometryGraphicsEfficient AI
Abstract

We introduce a new approach to high-fidelity 3D scene reconstruction from multi-view RGB images that tightly couples reconstruction with a strong generative 3D prior. We cast scene reconstruction as conditional 3D generation over a set of spatially-localized, overlapping chunks that together tile the scene, scaling generation to large scene extents. Crucially, we inherit the fidelity and completeness of state-of-the-art generative shape models -- we use Trellis.2 as an example -- which we generalize to the scene level. To this end, we propose a projection-based conditioning mechanism that lifts posed multi-view image features into a coherent 3D representation aligned with the generative model, independent of view ordering and spatially anchored to the scene, yielding high-fidelity, multi-view consistent generated geometry. This enables lifting the strong object-level prior of Trellis.2 to multi-view, scene-scale generation, producing faithful, editable PBR mesh reconstructions of indoor environments. As a result, we obtain high-fidelity results that outperform cutting-edge reconstruction methods by 16%.

2026arXiv

GitScholar: A Dataset for Predicting AI Research Impact from GitHub Engagement

Emilien Guandalino, Lorenz K. Müller, Beatrice Alessandra Motetti, Konstantin Berestizshevsky, Lukas Cavigelli

Research DiscoveryGraph Neural Networks
Abstract summary

GitScholar connects AI papers, citations and timestamped GitHub activity to study early signals of research impact. The dataset links approximately 444,000 repositories with more than 558,000 AI arXiv papers. Experiments investigate whether repository engagement improves the identification of papers that later attract substantial citation attention, beyond academic information alone. Its temporal structure supports retrospective evaluation using information available at a given date.

Read the source
2026FMCAD

Beyond Lemma Sharing - Novel Parallelization Strategies for Property Directed Reachability

Verner Vlačić

Parallel ComputingSystems Software
Abstract

Property Directed Reachability (PDR) is a commonly used technique for automated hardware model checking, yet efficiently parallelizing it remains a significant challenge. Existing approaches, such as lemma sharing, often suffer from limited scalability as processor counts increase. In this work, we present two novel sharing-based parallelization strategies, preemptive propagation and ARPOS, and compare their performance with classical lemma sharing. To this end, we develop an asynchronous MPI-based message passing framework for the state-of-the-art rIC3 hardware model checker. Experimental results on the 2025 Hardware Model Checking Competition benchmark demonstrate that our preemptive propagation strategy yields a significant performance boost over classical lemma sharing.