Back in some years I ran into trouble with a large scale data-driven system developed for the dungeon generation. It seems like data-driving works best in local, isolated contex just like many other things in programming. Since then I've been breaking the data-driven system into level theme classes, using only some data-driven parts.
The good thing is that most sub-systems can be run directly from theme classes with some minor changes in the way data is passed into them.
In the past I used to think that level generation is complex, but in reality it's quite simple. As always the main point is first making a decision what the level actually contains, rather than thinking about doing it later. When you say "I'm going to do it later" you are in trouble.
Most of the building blocks are ready and about half of the themes have a basic structure. There is even a simple street generation routine for town levels where I made an observation that by storing and comparing distances from previous locations you can space the things without having to do it manually. However after all these years it's hilarious when you proceed to the next level you realize there is more to it.
I'm going to be happy if the basic structures of all themes are ready before the end of this year, because after that it's much easier to add the rest with sub-systems that mostly work by filling in the rest of the free space.