So in this design, every cell can have a south-facing wall, and every cell can have an east-facing wall. I’ve separated the walls into two layers, one for the south walls, and one for the east walls. Tiled is rendering the two layers in two passes, so that the east walls are always on top - which looks wrong, as you can see. I understand the problem, but I don’t know how to fix it. If I only use one layer for all the walls, the render order problem goes away, but then I have a different problem: some squares have both a south wall and an east wall, and I can’t put two tiles in one layer. I don’t know how to make this work out.
You’d need to have the back walls (upper left and upper right) on the lower layer, and the front walls (lower left and lower right) on the upper layer.
That arrangement doesn’t work. See, the frontmost tile of the building - the tile closest to the camera - it has two walls, a south-facing wall and an east-facing wall. Those two walls segments need to be in different layers.
With this simple one-room building, it is possible to find an ad-hoc arrangement that works. But as the building gets larger - more rooms - it gets harder and harder to find an ad-hoc arrangement that works. I would prefer a systematic way to lay out the tiles into layers, something that always works.
For arbitrary layouts, you should be able to have 3 layers, two that are used as I described above, and an overflow/exceptions layer for the tricky scenarios which contains only those tiles that would clash with the middle layer.
With this approach, some of your walls would be split between multiple layers. If you want a solution where each wall is always on one layer, you’d need more layers. The exact number would depend on your room arrangement, as some are easier to do with 2-3 layers than others.
From what I’ve seen, games requiring complex room arrangements usually use more complex tilesets that include T-junctions and the like.