It really is. I'm not sure if it's an ability of the mansion itself or of the queen, but every event is based off someone's memories... and it seems like once she gets the idea or if you've been there long enough, she can strip you of your memories completely. I know several people who have lost memories already and there are some words I know I can't say anymore. I wish I knew which, though...

[She's reeling.] That's terrible!