I’m a bit confused about how the tile collision editor works. I tried it myself and I noticed that an ObjectGroup is added to the edited tile in the TileSet. However, when I add that tile to a TileLayer, nothing else is saved in the map. Is that ObjectGroup supposed to be “inflated” by a collision engine based on the tile properties? What properties are valid for an ObjectGroup within a Tile? Is it valid to have custom properties on it or color (edited by hand in the resulting .json map file)? What would be Tiled’s expected behavior in this case?
Yes. The collision data is stored per-tile and Tiled won’t duplicate it in the map. The best example of how it can be used that I know of is Tiled2Unity, which can merge these collision shapes by layer or on other conditions and create combined optimized collision meshes out of it.
Tiled allows you to set any properties that are valid for regular object layers. Which ones are valid to use in the context of tile collision info depends entirely on your game / engine / pipeline. One thing that Tiled does not allow is placing tile objects in object layers used for collision information, since they would not be able to use a global tile ID because they are not always saved in the context of a particular map (could be an external tileset).
You can add custom properties on either the object layer or its objects in Tiled, and as long as you keep the syntax valid you can of course also do this by editing the file directly.
Maybe you were using an older version of Tiled where you couldn’t access the properties on the object layer itself? In the current version, the properties window displays the properties of the object layer when you have no actual object selected.