Black mark

Black mark is an informal term for something in someone's life that goes against the person and which is remembered as such.