Hello
You give very few information, however it may come from someone / one of your parents (both?), who was insecure around you, during your childhood. Not necessarily someone who hurt you. An insecurity may be from someone who is for example : is not balanced, clumsy, etc..... This style of parents may generate an anxiety that the child keeps the whole life, because of he felt insecure with the person he/her lived. Later once he is an adult, he perceives people ( or most of them), like a ' danger '. Anyway the cause (whatever it is) is now became unconscious for you, that is why you don't know why you have this fear still ' nowadays '.
You can get rid of it by doing a work on youself with a Psychoanalyst**. He/her will let you talk like you want to (free association), and will help you to find the cause of your current ' problem'. Once the story is relived (with the anxiety that is linked), the symptom disappears. The symptom being the fear you feel currently.
If money is a problem, you need to know that most psychoanalysts adapt their prices in accordance with the income of people. Moreover you are totally free to search another if the first ones you found don't do that.
Best
** Only psychoanalysts deal with the unconscious and subconscious, that is why they are recommended for fix the problems whose the causes are unconscious for people. Psychologists / simple therapists and psychiatrists work only on the symptoms (what is is apparent) and it is not sufficient for remove forever the causes (so the symptoms).