Hello Pirate and welcome
My first thought is, yes you are worrying too much. You are (with the help of Dr. Google) overthinking and coming up with the worst case scenario.
I agree with the others in that if you are still worrying, then return to the GP and ask to be referred on the basis that you are a worrier and need the reassurance of further tests. Worrying will in itself, produce other problems, probably digestive so no point in setting yourself up for further ill health when a check would put your mind at ease.
Like Chris says, if you use google you'll convince yourself your dying of rabies of something (been there - done that!) so leave it alone until you have something specific to check out.
Three questions: You say the PSA came back "normal".
Did your GP actually give you the figure or was that just her opinion? It would help to know what that PSA was.
Did your GP do a physical examination (DRE - digital (finger up the bottom) rectal examination? IF so did she also say that was "normal"
Did the GP check your groin? Did she discount a hernia?
I would get a referral and wait for the outcome of that before you start telling yourself you have major concerns. If the hospital says it's worth checking via Biopsy and MRI then at least you will have been taken seriously.
We are good at listening though, so please come back if we can help anymore.