Autodiff may in some cases compute unexpected results that could be interpreted as incorrect derivatives. These pitfalls occur systematically across multiple tools and approaches, making automated detection difficult.
We show that these problems are caused if autodiff is applied after approximating a function in a way that changes its derivatives, often while lowering the program to a different level of abstraction. We illustrate this problem with known examples such as discretizations, polynomial approximations, and lookup tables. We also discuss the challenge that Enzyme faces unlike most other autodiff tools, because it differentiates code at a different level of abstraction than what programmers write.