YAY!
Bring on the magickal theory! Goodness I love these discussions- they really work the brain in such a fun way 

After sitting and pondering on this for a few minutes, I think my answer is going to be that- yes, in general, protection spells “stack”. But I don’t think they stack like a stack of pancakes with one directly on top of the other
. I think they layer on in way where different types of protection spells cover different “holes” or openings in someone’s defenses 
The Protective Alarm Analogy: As an analogy, most homes have a fire alarm (hopefully more than one). But then they also have, as either a separate or as a 2-in-1 device, carbon monoxide detectors. Some homes also have burglar alarms- whether it is an actual alarm system or just a very aggressive chihuahua are different types of anti-burglar protection, but ultimately serve the same purpose

Although they are all protective features, each one serves a different purpose. Some people may feel safe only when they have all of them active and in place. Other people may feel comfortable at ease with only one (or, for those living on the edge, no protection at all).
Cancelling Out: If I can continue with the alarm analogy, the protective alarms, like spells, wouldn’t cancel each other out- having a fire alarm doesn’t negate the burglar alarm- and it’s okay to have multiple types within each area of protection. You might (I would say: should) put a fire alarm in each room of the house, just as you could put a protection spell in each area of your home.
In this same sense, I think you can cast (random examples here) a travel protection spell, protection against the evil eye, and an empath shielding spell all the same time and have little to no overlap. A general protection spell would overlap everything, but would be strengthened by the more specific, targeted spells 
“Over-Protection”: In terms of over-protection, I don’t personally think it’s something the vast majority of witches would need to worry about- I think it would be really tough to over-protect. I do think there is a certain point where any additional spells are going to just be “spares”- in place but never used. Like filling a room from top to bottom with just fire alarms, it’s not really “negative” per say, but it does become waste of money, time, and effort.
In the same way, casting the exact same type of protective spell over and over is a waste of energy- it’s essentially just making “spare” protection spells in an area that is already fully protected 
As for how to tell if an area is already fully protected, intuition and how you feel are going to be the best judge of safety (unless you have a condition such as anxiety or paranoia that makes it difficult to tell).
I wish spells came with an expiration date! That sure would make it easier for us to know when casting

It does seem to be trial and error when it comes to working out when to re-do a protection spell (or any long-term spell, for that matter). I’ve found that when I am actively under “attack” (stressful situations) the protection spellwork needs to renewed more frequently than when I have it in place “just because”.
In general, I’ve found tangible spell products (spell jars, bags, pouches, written or drawn sigils, etc) tend to last significantly longer than just spoken or candle spells.
Full disclaimer: everything above is just my personal thoughts on the matter. I should also mention that I am someone who uses a TON of protective spellwork, so I’m definitely biased when it comes to this haha.
I had a ton of fun thinking this one through and I imagine it is going to be on my mind all day- thank you so much, @anon87969570, for this absolutely delicious session of magickal theory! It’s my favorite

Can’t wait to hear what other coven members think about protective spellwork too! 

Blessed be!