The artefacts are being caused by the fact the tileset is incomplete for the situations you’re trying to depict (mainly, transitions to transparency/empty). For example, when you paint an L-shaped bunch of sand surrounded by emptiness, there’s no valid tile for Tiled to put in the crook of the L, so is tries to modify surrounding tiles to make the overall shape work using a randomly-chosen solution. Sometimes that solution is making sand-edges everywhere (hence the black lines), sometimes that solution is an explosion of grass.
Also, you’ve labelled the dirt tiles underneath the bunch of sand tiles as “all dirt”, causing them to be used as variants of the basic dirt tile, which looks wrong. Unlabel those, they’re for special cases I think.
I recommend filling your map with sand or some other terrain so there’s no transparency, and then painting other terrains on top of that, that will avoid the problem with missing many transitions to transparency.
Keep in mind though that you may still have wonky results sometimes because your tileset is incomplete.
- there are no diagonal corner transitions
- there are no L-shaped corners for transparency.
- there are no three-way transitions, so for example, you can’t easily have a Dirt path in the Sand with Pink branching off it or right next to it, because there aren’t Dirt-Sand-Pink tiles.
- not every terrain transitions to every other terrain, and the terrain tool isn’t a full-on constraint solver so it won’t modify everything to make your desired multi-tier transitions work, you’ll have to think about that yourself.
- (this is probably not a complete list)
If you’re conscious of these missing tiles, you can work around them, but if you want complete freeform terrain painting, you’d need to add them.