Do these things have to do with asperger's?

Yes! The worst is the being spoken to sternly, am I right? Wait, maybe being touched! There is no one reason that will make all the bad go away. You have a long road ahead of you. You will, however get to where you will be most comfortable. You can't help the above mentioned, so do NOT be told where your emotional state is! It will also take practice with girls. Maybe seek support groups, and therefore meet girls similar to you. Wish you well!