I have the following set of 47 tiles, which happen to be a complete “blob” tileset as suggested in the Wang brush documentation.
The terrain brush fundamentally cannot handle these. Terrain is defined by corners, which means the maximum number of distinct tiles is 2^4 = 16, whereas I have 47 distinct tiles. (There is obviously no way, for example, to encode the 1×1 tile in a terrain definition.) But that’s not a big surprise — terrain support is designed for transitions, whereas these are edges.
I was really excited about the Wang support, both because I thought it might solve this problem, and because it would even be able to draw random variations (e.g., stick an occasional decoration in where an otherwise solid black tile would go). Unfortunately, even though this is a cromulent Wang set, it still doesn’t work. The setup looks like this:
(I’m not sure whether the green is helpful here or not, but I can’t have a set of only 1 color and there doesn’t seem to be a Wang eraser, so I suspect I need it.)
If I try to draw on an empty layer with the Wang brush, absolutely nothing happens. If I use the Wang brush to draw on existing terrain, I can only poke corner holes by using the green corner, otherwise nothing happens:
If I use the regular brush in Wang mode, I get a tile arrangement that is technically correct, but not quite what I was going for:
I think the problem is that I don’t have “maximal” pattern coverage. If I start with an empty layer and try to draw either corners or edges, I’d end up with tiles that have no Wang colorings except for either one or more corners or one or more edges. But this “blob” arrangement has no tiles like that, and given the aesthetic, it can’t have tiles like that. If two tiles touch at a corner, then either they must touch at the adjacent edges as well, or they aren’t actually connected — after all, there’d be nowhere for the art to go. I can’t encode this rule in Tiled, and since it ignores any coloring attempt that has no valid tile — I assume because the colors are referenced from the tile and not actually stored separately in the map — there’s no way to change a tile from “completely uncolored” to “colored on a corner and its adjacent edges”.
I believe this would work if Tiled had explicit support for the blob rule — that a colored corner requires colored edges — but I don’t know the right way to fit that into the existing and highly generic Wang implementation. For one, it only makes sense if edges and corners use the same set of colors, whereas the two sets are independent right now. More awkwardly, it only works in this particular case if “nothing” is an explicit color — otherwise, given my example setup above, there’s no reason that red corners should force red edges but green corners should not.
But even then, I’m not sure how I’d draw the 1×1 tile using the Wang brush. Perhaps the core problem is that these tiles are ultimately defined by their centers, which is the one place in a tile that neither terrain nor Wang understand.
If it helps, the one tool I’ve used that can actually do this is Godot, which recently got an “autotile” feature. It’s not documented at all, and the configuration UI is really clumsy, but it’s basically magic once it’s set up. You assign tile connections as a bitmask (here set to “3×3 minimal”, but there are also “2×2” and “3×3” modes):
And then you just draw.
I would love to be able to do this in Tiled, and in fact I think Tiled could do it better — I also have a variant of these tiles with grass on top, which Godot’s bitmask can’t express, but which Tiled’s multiple Wang colors potentially could.