I am reviving an old post, but I have the same issue.
My Mac is running macOS 15.6.1, and I have a Focusrite Red8pre (same issue with a Focusrite 4i4). For timing reference, I use the card's hardware loopback and send the L channel to channel 25. I haven't managed to find a buffer setting that...