The analyzer was built to go off syntax nodes. This (AFAICT) meant that the SemanticModel had to be recalculated for every single invocation.
If you don't know what the above means: it basically means the compiler has to re-analyze the entire file.
Fix this by moving it to an operation analyzer so the compiler can properly cache the semantic model.
521e7981bc