Different practitioners will take different stances on this one. I’ve heard some witches say that you must have the physical herb or item in front of you, and I respect that that is what works for them in their practice.
Here in the physical world, there are indeed many benefits to using the things around us. However, there is no escaping that we are bound to the Laws of Physics in the physical world. If you want to cast magick that breaks or goes beyonds the limits of the physical world, it can be done- by entering other worlds, realms, and planes of existance.
I can go on and on about astral travel, lucid dreaming, the powers of meditation and visualization- but for the sake of answering this question, I’ll say TLDR: there are magickal possibilities and benefits both in the physical world and also in “inner” or other worlds that are non-physical.
So to answer the question, yes - I personally think it is absolutely possible to visit your inner temple and do very meaningful and powerful spellwork within that space 
Absolutely the space can change! Your inner mind is limited only by your own mind- anything you can think of you can conjure within that space. The possibilities of mental work are truly astounding 
As you go through life, it is only natural (and healthy!) to change and grow. As you develop, it makes sense that your inner sacred space will change and grow with you 
That’s another really great question, Danni! I actually wrote a lot about this (leaving the circle, forgetting to close it, etc) in this Guide on Closing Circles 
Another resource you might find helpful is the Circle Casting FAQ on Spells8.
Yes, energy buildup is normal in certain types of meditation! I personally see it as being a similar process to Reiki- a type of energy healing process.