Was there any advantage to having it lean like that?
Not this one, but newer forts were built with angled walls to help protect against canon balls and the like.
In defensive terms, no, not really. They had to build it like this because these aren’t really walls per se, it’s just brick lining on the outside of an earthen mound, and mounds are, well, mound-shaped. https://gomadnomad.com/wp-content/uploads/2013/02/Bukharas-Ark-Palace.jpg
You’d think there’d be a reason beyond construction requirements, though—otherwise someone in the past 1,500 years would have replaced it with a more conventional wall.
Are we sure they built it like that 1500 years ago? Churches can suck down in the span of only a few hundred years.
You’d think it would lean the other way to make bit harder to climb.
Edit: or this could be a view from the inside. Or maybe the goal is to keep people in rather than out.
Edit again: none of these things seem true according to Wikipedia. It’s curved inwards and it houses the rich, so it seems to just be aesthetics.