While I do think having such magickal abilities is a blessing, it’s not much fun if it is causing you to suffer 
Have you perhaps looked into ways to protect yourself from the energies of others? There are magickal methods of shielding oneself from outside energy- things such as veiling or looking into Empath protection spellwork are both popular options.
I hope you can stay safe and be well, my friend! 
