First of all, do not feel ashamed of your efforts to learn English. You are trying, and you should praise yourself for being a courageous learner
Second, it's going to take lots and lots of repetition (repeat, repeat, again and again) to learn. So be patient with yourself and try to make it fun if you can. Watch a TV series and write down new interesting vocabulary words or phrases. Grab a children's book and learn easy words and grammar first. Once you have mastered those, move on to a more difficult level.
Your fear of speaking will only worsen your efforts to learn - I know this from years of Spanish that I've forgotten most of. When I could only speak Spanish in class, I felt panic sometimes. If possible, maybe you need a new teacher who makes you feel comfortable? Learning should be relaxed and fun.