Hmm, well that certainly is some interesting piece of code. Maybe it works (too late to check now), but especially the last “else” case looks wrong. Where is the non-rotated case? Also the comments don’t seem to match with the code.
There’s also the thing that Tiled does not only support rotation, but also flipping. In fact, the three bits used to store the orientation of the tile are only storing flipping information. Horizontal, vertical and anti-diagonal flipping.
Now, an anti-diagonal flip is the same as rotating 90 degrees clockwise and then doing vertical flip. So in order to encode a 90-degree rotation, Tiled sets the anti-diagonal flip bit, and the vertical flip bit (the result is 90-degree rotate + vertical flip + vertical flip, and since the latter two negate each other, you’re left with only the 90-degree rotation).
Btw those numbers look a lot easier on the eye in hex notation:
const int FlippedHorizontallyFlag = 0x80000000;
const int FlippedVerticallyFlag = 0x40000000;
const int FlippedAntiDiagonallyFlag = 0x20000000;
3221225472 is
0xC0000000 is
0x40000000 | 0x80000000 is
FlippedVerticallyFlag | FlippedHorizontallyFlag
(and flipping both vertically and horizontally is indeed the same as an 180 degree rotation)