Nice work!
I have thought the same! If the optimization passes were done “on” the UGens, instead of “in” the UGens, it would be much easier to reason about. Whenever I’m looking at the current graph optimization code, my head immediately begins to hurt 
Perhaps this could be its own topic
IMO it would warrant a new topic. Also because this thread is already quite long.