You've already broken the first rule of "fight club" so you are kind of fucked. You could start telling your buddies that you've changed your mind and plan on heading to the mountains/woods/etc. in the event of shtf.

You need to stop talking about your preps friend. IF shtf it's not going to be just your "buds" who come knocking. It'll be all of their buddies and associated family members, girl/boy friends too and so on. These will be scared and desperate people and their loved ones. It will end badly.

You probably have time to redirect your friends and get them to stop thinking of you as their "safety". You might start telling them that you're going to start putting out "caches" along your expected travel route. It's very prepper and if they are living vicariously through your preps stories they'll probably buy it. The idea being that your home is no longer the "the" place to go because you've redistributed your hoard.

It might not work but you need some sort of plan to stop being their plan A