Everyone thinks I'm an idiot, how do I deal with these people?

Do it. Just make sure you're sure about it and that you'll be well accommodated and safe. It's your life, don't let others tell you how to live it.