Abrev107,
First, hello. This is my first post here, and since I read your thread, I thought I'd make my introduction to you. I'm in primary flight training myself and feel we are very similar. I have logged about 40 hours though (I'd have to open the logbook and actually count to get an exact number). I have found riding in the back seat on occasion very helpful.
I teach horseback riding, hunter-jumpers, but not for shows, in case you're curious. I consider our goals in riding lessons to be the most practical, efficient, safe riders we can be. Riding horses and flying airplanes are very similar, only the horse doesn't get so far off the ground, take off and landing are much closer together, and the order in the pattern is different, setup, approach, take off, mid-flight, landing, departure. Ok so the only thing different is the departure after jumping a horse is after landing. If we could keep the horse in the air to depart the pattern, believe me, some of us would.
On to my flight training, which ties into riding instruction because I am teaching a few pilots how to keep horses between themselves and the ground. I haven't mentioned how much I love pilots. They (we, I also love that I can start including myself in that elite), have the unique perspective of seeing the world in ways nobody crawling around on the ground can, and we are in the unique position of having to rely totally on ourselves and our resources to keep us alive enough to walk away from the airplane (even if we have to sit in the airplane after shutdown until our knees quit shaking too bad to stand.)
I don't know when I quit squeaking out a weak, 'help ' on final, and changed to, "shhh, I can DO this." My cfi would always calmly say, "your doing fine." He also does tell me mistakes he makes, like calling the wrong field in sight, and realizing it when the airport didn't look quite right. I'm in the cross-country phase now, so getting lost is one of the latest concerns. I cried recently when I couldn't figure out for the life of me where 011 was on the DG and was drifting off course. The tears were frustration because I expect more out of myself and I'd already gotten shaken by mistakes earlier. I wasn't fully prepared for that flight anyway, and was disappointed in my performance.
Back to beginnings (forgive the tangents), I had that , "Oh Lord, THERE GOES THE GROUND." on take off, and, "Oh Lord, THERE'S THE GROUND, " throughout flights for quite a few hours. During demonstrated stalls, I alternated grabbing the glareshield, support structure by the windshield, the cfi's leg, and my seat. Once, I turned like I thought I was going to crawl in the back of the airplane. I'm in a Cessna 152. I'm within the weight limit of the baggage area, but wouldn't have been comfortable once I got back there, if I'd actually gotten all the way back there. I'm ready to ask my cfi if he'd do demo stalls again and even teach me a few of them, but only because I've ridden in the back seat of a couple of other airplanes to get used to a few scary maneuvers. I rode in the back seats of a couple of friends' T-6 Texans while they did aileron rolls, low-approaches, and break-aways into overhead approaches. After those, any maneuvers in the happy 152 seem tame. The point is, backseat riding can be a big help because I don't have to worry about anyone saying it's my turn to do the same maneuver, "you have the airplane." "Keep it. IIIIII don't want it."
Even after my first solo, I didn't want to solo again for a few weeks. I had to laugh while puttering around the pattern over and over when my cfi asked how about making the next one a full stop so I could go a few times without him sitting there being a distraction to me. I said, "I don't want to solo today," and he said, most incredulously, "hunh???" I just didn't feel in the groove and figured I'd be paying for his presence whether he's sitting next to me or standing looking up at me. I preferred him sitting next to me. He knows how to make me chuckle and said he was happy sitting up there until "they" shot us down. I started picturing someone else in the pattern or on the ground, perhaps at the hold-short, getting so annoyed with our little 152 taking up airspace that they'd actually blast us out of the sky. Good sense of humor, on both student and instructor's parts, is important to learning.
Connection between the student and cfi is very important too. I'm nervous with my cfi because I like him so much, I don't want to disappoint him, but I'm oddly at ease with him because he's such a calming spirit. I had to fly with a different one, a newly minted cfi. It was a surprise when I walked in and my own cfi couldn't be there, and had already asked her to fly with me. I should have followed my first instinct and either gone solo again, or gone home until I could reschedule with my own cfi. In a way, I felt like I knew more about the 152 than she did, though she'd done her primary in 152's. She had Archer numbers and the old PTS swirling in her head. She was very friendly, bubbly even, but I managed to feel like I had made a horrible mistake thinking I could ever possibly become a pilot. I alternated between feeling totally incompetant and wanting to say, "shut up and let me fly the d*** airplane. I was post solo then (had already been out to the practice area solo twice), and wished I'd just left her back at the ramp while I spent quality time alone with my metal winged steed. I do a lot of internal struggling too, so your posts ring close to home for me. I'll make an effort to do anything my cfi wants me to, but with the substitute, I resented the constant chatter. The only good I got out of it is that I realized I'm an impatient pilot with annoying passengers.
I don't know what will work for you to get over the anxiety. There should always be a certain amount of fear, or something. A pilot should never become totally comfortable or complacent. For me, taking time between lessons, even a few days (I was flying twice a week for a while), really dwelling over each flight and studying for the next one, helped. I hang out at the airport a lot. Go sit and watch airplanes in the pattern. Sit and listen to other pilots. I'm spending many hours flying my flight sim on my computer, fs 2002, multiplayer with other students from another website, and spending more hours solo in my flight sim going over emergency procedures. My landings after engine outs upon take off are strangely smoother than after normal approaches, go figure THAT one.
I've rambled on enough.
I'll just end saying that old cliche, "you're not alone."
). On the other hand, pilots in the overall sense accept each other, regardless of age. In other words, maybe you're not accepting your instructor because of his/her age. Being old, grey and having crow's feet from years of open-cockpit airmail flying is not a prerequisite to be a good flight instructor. 
But his took on a certain edge that I felt was inappropriate for someone with my experience. It set a tone for the entire (short) time I was at this company.
