Well then I dunno.
It certainly slows down a little bit for me also, but it isn't graphical or RAM lag, it seems to just be part of the programming of the game itself.
My PC is a beast and can run stuff like Bioshock and COD4 at top levels so the slowdown I see in this game simply cannot be due to my PC and can only be due to a substandard program and/or a poor programming/graphical choice by the developers. It fades out one floor to fade into another (pretty, but quite slow and a bad design choice in a hectic time management game). But the slowdown was never too terrible and never really got in my way.
If it's not RAM and not your video card, then your problem goes beyond my quite meager nerd skills. Do you have a hundred thousand programs running in the background that you don't even know about? People who bitorrent music files often do...
Aside from that, I'm at a loss.