So after I followed Patrick’s advice, it seemed like we had everything working better(?) but then again, it was only faster sometimes…
After doing some tinkering and comparing between 4 different people, 2 of us with slow performance and 2 with normal performance, I stumbled on the answer… Patrick and I both had the “wassup” plugin enabled and the others didn’t. I wouldn’t have known that was it except I am hosting 2 blogs on this same machine and one responds faster than the other. And the fast one has a post with a bunch of pictures in it…
So we both disabled the “wassup” plugin and now we are both happier with the performance.