Are there people whose constitution doesn't suit perfume?

I usually enjoy sampling perfumes and have many that I want to own, so I have bought and tried several. However, the scent I smelled during the sampling doesn't come through on me. I like the same perfume scents I catch on other people on the street, but for some reason, it feels like my own scent is a bit different or has changed. Is it possible to have a body type that doesn't respond well to perfume?