How long are they?
My first one was about 5-6 hours long. At my current job the initial and recurrent checks take an entire shift. Maybe you are lucky enough to get out in 9 hours. Very in depth at my current location. At my previous gig it wasn't too intense, but not easy either.
Who administers them?
Certain people are usually designated to give comp checks. Generally somebody who didn't train you in, and somebody who has a lot of experience within the company. A dispatch supervisor, or a dispatcher designated by the company
If you administer them do you grade categories on a 1-5 kind of scale or is it a pass/fail system?
Both places I have been it was just a pass, moderate, or fail system.
Along those lines if someone is struggling with a particular topic, do you do coaching during the check or grade them for further training?
I personally have nor seen anyone bomb a topic so bad that somebody was failed right on the spot during their checks. I have seen them help you out in senarios where you dont fully understand what they are looking for, or if you are struggling in a topic. Usually they want you to know where to find the information. Its impossible to know everything. But have a good understanding of the basics and where to find the nessary information.
What types of questions and scenarios are brought up?
It depends on the carrier and type of operations going on really. I had everything from simple hub and spoke information, aircraft limitations, crew limitations, op spec exemptions, deice stuff to deriving alternate minimums, re-releases, and ETOPS stuff. It really depends. Usually they wont ask stuff that hasnt been covered in OTJ
As a joke though, the first question the examiner gave me on my comp check was to find out the serial number of right rear main tire and its corresponding tire pressue limit. Needless to say I was a little stunned, but after some thought it clicked to call MX control and find out the information. It was a little "hazing" joke they do, but it was a good way to break the ice. He said if you found that information, you will find other info easily
They are not there to fail you. A good one wants to asses your abilities and see your struggles. They will offer help if they feel confident in your abilities, and they do want you to succeed. Key is knowing where to find the information you don't know instead of trying to know everything.
Good luck if you are having an exam done