Not all paprika is made with bell pepper - most of what I see isn't, so that may be a difference in your reaction to some versus others. I like my jalepenos roasted (or chipotle - same pepper, but smoked, as I recall) and the red ones also have a better taste (less chlorophyll flavor).
As for habaneros, my last batch of chili just might put you in the grave (or keep you on the crapper wishing you were dead): I made a chili paste with fresh, roasted habaneros - nothing removed except the stems - and combined with some chipotles, adobo and chili pequines, plus a few other things.
My base paste is made with dried/reconstituted chilis of varying flavor and heat profiles, but seeded first. Makes for a medium heat paste - the second paste puts the spurs to it.