When Common Sense Tells You Not To