Some time ago I had an idea to create a game object for terrain tiles as well, because it was easier to handle game messages, damage to terrain etc. But it was kind of a mistake which I'm now going to fix with another rewrite (or refactoring, how you want to call it). At this point even rewriting doesn't feel like a problem, because it's mostly rearranging source code to more logical units.
The new plan is a terrain map with Tile class tiles which is a more complex way to use an integer to show which terrain tile is in that place. With that you also need fov and automap data and so on. Then, when a terrain is changed (most often when damaged) a terrain game object is created in that place. This is going to solve another mistake I made when I removed location data from some game objects. It became incredibly annoying to work with objects that had a location and others which didn't.
I think it's important to rewrite when you feel something is not right. The irony in this case is that I got it almost right the first time, but then made a decision to rewrite it because the strange problems I had (and still have) with the game message routine. Game messages shouldn't affect the way terrain map routines are implemented at all and it's an important lesson. Each part of the system should work more or less independently at the internal level and external connections should not rely on that.