See that there is no self that can go mad. Who is it that's going mad? You could say it's your mind, but who is in control of this mind -- who is the "your" in "your mind?" Or in your question, who is the "I" that wants to stop "myself?"
When you said that "my brain always has a retort," who is the "my" referring to? If there is no controller of your brain, then one doesn't even have a mind, and therefore the mind doesn't exist. Since Solipsism, according to Wikipedia, is "...the idea that one's own mind is all that exists," the idea fails because there is no you that even has a mind.
The reason we think we possesses a mind is because of our ability to recall past events and plan for the future. This creates the illusion that there is some unchanging part of us, which gives rise to "I" trying to fix "me," or in the case of Solipsism, an "I" believing that he has a mind ("me"), and that that mind is all that exists. In reality, there is no mind, no self, no other -- just experience -- just the eternal present moment.