Here is the research paper that @SilverBear mentioned above. According to the abstract, selenite can be an irritant or cause dermatitis.
However I found a topic in the forum of Mindat (one of the most valued resources for gemologists and rock lovers) and some of the answers seem to imply that there could be something else on the selenite that is causing the reaction.
There doesn’t seem to be a lot of information online about selenite allergies. Maybe you can try your own experiment and share your findings here! I recently did an experiment at home to see if selenite dissolves in water.