The AAD Tool Selection Challenge

Antoine Savine: "The main challenge faced by global investment banks today is a computational one"

The benefits of Automated Adjoint Differentiation (AAD) libraries are clear: fast XVA risk, model calibration, hedging, and Live Risk are only some use cases difficult to achieve without a modern AAD tool. Many financial institutions have implemented an AAD solution—using in-house resources, open-source, or vendor products—and now experience both benefits and limitations of their chosen approach.

Computation of risks is a primary factor driving operational costs—whether via manual differentiation, automatic differentiation, or bump-and-revalue. Organizations employ teams of expensive quants to enable and support their solutions, while an average compute bill for a Tier 2 bank exceeds $10M per year. Even a 10% reduction is significant.

AAD is desired for numerical stability and performance, making the right tool critical to competitive advantage. There are currently three approaches for implementing AAD: tape-based, code transformation, and code generation. Each methodology has distinct advantages and disadvantages.

Three types of automatic differentiation tools

Three Types of Automatic Differentiation Tools

Approach 1: Tape-Based Automatic Differentiation

Current Status: Most commonly used AAD approach in production today

How It Works

Operator Overloading (OO) captures elementary operations while executing analytics. All mathematical operations (the computational graph) are recorded on a data structure called the 'tape'. The tape is then processed backwards to compute all risks using the Adjoint Differentiation method.
Tape-based AAD architecture showing recording and backward pass

Tape-based AAD architecture showing recording and backward pass

Performance Characteristics

Although the OO tape-based approach is faster than bump-and-revalue, limitations stem from original software design:

  • Sparse vectorization and multi-threading usage retained and multiplied
  • Replicative implications affect each Monte Carlo iteration
  • Adjoint factor: 2-5x (computing adjoints takes 2-5x the original function time)

Key Limitation

  • All computations are linearized—all loops unrolled
  • Tape size depends on number of elementary operations
  • More memory required to store tape data structure
  • Memory access is slow and expensive

When Tape-Based AAD Works Well

  • Simple models with few operations
  • Infrequent execution (tape build overhead acceptable)
  • Organizations with existing tape-based implementations
  • Quick prototyping and testing

When Tape-Based AAD Struggles

  • Complex Monte Carlo simulations (large tapes)
  • High-frequency calculations (tape rebuild overhead)
  • Memory-constrained environments
  • Multi-threaded scenarios (tape management complexity)
  • Second-order Greeks (cannot accelerate)

Approach 2: Code Transformation

Current Status: Theoretical appeal but limited practical adoption

How It Works

Both the function and its adjoint version are generated automatically at compilation stage (ahead of execution). It's similar to manual differentiation—developing an adjoint function manually—but done programmatically.
Source code transformation AAD showing compilation-time adjoint generation

Source code transformation AAD showing compilation-time adjoint generation

Critical Limitation

  • While the approach can work for localized projects
  • It does not scale at all for production systems
  • Does not present a feasible solution for real-life quant software environments

Why Code Transformation Fails in Practice

  • Requires source code access to all components
  • Complex build systems become unmanageable
  • Third-party library integration nearly impossible
  • Maintenance burden for evolving codebases
  • Compilation time becomes prohibitive for large systems

Not recommended for production financial software due to scalability limitations, despite theoretical performance advantages.

Approach 3: Code Generation AAD

Current Status: Novel approach with significant advantages, limited awareness

How It Works

This novel method was introduced by Dmitri Goloubentsev and Evgeny Lakshtanov in Wilmott magazine (2019). It generates adjoint code at runtime during first execution by redefining elementary operations using operator overloading. Key Innovation: When the function executes for the first time, the system collects the sequence of operations (accounting for branching) then generates code to replicate the original function and compute adjoints using the Chain Rule for arbitrary inputs.
Code generation AAD showing runtime kernel generation

Code generation AAD showing runtime kernel generation

Performance Advantages

This generated code processes arbitrary inputs (random numbers or market rates) and is used in all subsequent iterations. It delivers maximized performance as it uses information available at runtime and can apply additional optimizations.

Key Consideration

  • Compilation done once per task (trade/portfolio)
  • Doesn't depend on number of iterations
  • But must regenerate for configuration changes (new trade, date change, portfolio amendment)
  • For real-life quant/risk systems, compilation time is part of overall execution

Code Generation AAD Can:

  • Speed up both original function AND sensitivities
  • Accelerate second-order Greeks and scenario risk
  • Tape-based AAD can only accelerate 1st order risk

Minimizing compilation time is key to this solution. Using off-the-shelf compilers like LLVM or C++ simply cannot deliver the performance needed for practical production use.

MatLogica AADC: Production-Ready Code Generation

For code generation AAD, compilation time is part of overall execution time. Using an off-the-shelf compiler is sufficient to test potential in principle, but it can't be used in practice.

The MatLogica Innovation

MatLogica's AADC uses a fundamentally different approach: Operator overloading generates machine code in streaming mode directly. During initial execution of user analytics, AADC generates binary kernels on-the-fly for both original and adjoint functions.

Critical Differentiators

  • Generation process highly optimized for compilation speed
  • Does not rely on external compiler
  • Configuration changes (new trades, portfolio amendments) processed on-the-fly
  • Makes code generation a practical AAD solution

Performance Optimization

Beyond fast compilation, generated kernels deliver exceptional performance:

  • Vectorized to native AVX2 or AVX512 architecture
  • Process 4 samples (AVX2) or 8 samples (AVX512) in parallel
  • AVX512 approximately 1.7x faster than AVX2
  • Multi-thread safe by design
  • Results in speedups of 100x including compilation time
AADC just-in-time compilation architecture with direct machine code generation

AADC just-in-time compilation architecture with direct machine code generation

Comprehensive Comparison and Conclusions

A modern AAD tool is must-have for quantitative and risk management software. The table below summarizes key practical aspects of the automatic differentiation tools we've discussed.

Comparison table: Tape-based vs Code Transformation vs Code Generation AAD

Comprehensive AAD Tools Comparison: Performance, Usability, Memory

Performance Benchmarks

In this analysis, we've examined the inner workings of each automatic differentiation approach. With code transformation being nearly impossible in practice, and considering all aspects, MatLogica AADC stands out in terms of performance.

Quantified Performance Results

  • When compared to tape-based Adept, AADC is 23x faster for XVA pricing and Greeks on a single AVX512 CPU core
  • With secure out-of-the-box multithreading, speedups scale almost linearly with number of cores
  • Speedups of 100x can be easily achieved on existing hardware
  • Organizations already using some form of AAD will benefit with 5-20x speedup on a single core

Business Impact

With very manageable changes to quant libraries, organizations achieve:

Cost Savings

  • Impressive compute bill reductions
  • 100x speedup on existing hardware
  • No new infrastructure investment needed
  • Reduced cloud costs

Operational Benefits

  • Reduced turnaround for new model development
  • Better numerical stability
  • Initial results in weeks
  • Easy integration path

Strategic Advantages

MatLogica AADC is a modern AAD tool that acts as an abstraction layer delivering optimal performance and ease of use.

  • Ability to compute risks faster enables running more what-if scenarios and extra backtesting
  • Enables better decision-making
  • Resulting Live Risk capability permits new trading opportunities to be seized before the competition

Decision Framework

Scenario Recommended Approach Rationale
Simple models, infrequent execution Tape-Based AAD Overhead acceptable for simple cases
Academic/research projects Code Transformation Small scale, controlled environment
Production quant systems Code Generation (AADC) Best performance, scalability, ROI
High-frequency risk calculations Code Generation (AADC) Low latency, high throughput
Monte Carlo-heavy workloads Code Generation (AADC) Vectorization, multi-threading
Organizations with existing AAD Upgrade to AADC 5-20x additional speedup

Ready to Benchmark AADC Against Your Current Solution?

See the performance difference on your actual production code

Schedule Benchmark Contact Us

Related topics: AAD comparison tape-based vs code generation, automatic differentiation tool selection, operator overloading AAD performance, code transformation AAD scalability, AADC vs Adept benchmark, XVA AAD performance, adjoint factor comparison, memory usage AAD tools, AVX2 AVX512 vectorization AAD, multi-threading automatic differentiation, JIT compiler AAD, production AAD implementation