This question made me realize a situation in which I'm pretty horribly inept at reading people: 
When I'm physically/emotionally attracted to someone or romantically interested. It's a huge problem area. And yes it is confusing, and scary, and causes anxiety. It's the area of my life where I'd tend to make negative assumptions...maybe to protect myself from disappointment? I don't know.

It's only when it personally involves me though.
Check this out:
I have a crazy knack for reading romantic interactions that don't involve me! This is silly and odd, but I called the whole Jesse James/Sandra Bullock thing just from watching him look at her when she gave her Oscar acceptance speech. I blurted it out because it made me so sad, and all my friends watching the Oscars with me thought I was nuts and teased me about it.
Then, when his cheating scandal came out a week or so later, all those same friends were calling me and asking me what I thought their husbands/SOs thought of them. haha

I have had several times when I knew that people were secretly dating.
Give me just a few minutes watching people, and I know who is into who...and who isn't.
People have asked me how I know these things, and it's hard for me to break it down for them. When I try, they feel that it wasn't enough evidence. (But *shrug*, I'm right after all, which they'll admit.)

But when it involves me, or if I think someone is attractive, I totally blank the eff out. Combined with the fact that I get awkward around people I find attractive, and can't string together more than 2 coherent thoughts, well...let's just say that it can suck.