Probably the biggest problem is their limited ramp space. There's not much. There's more now than there was a year or more ago(the FBO moved to a bigger ramp area, but are now in a portable building). Plus, SUN is a one way in, one way out airport pretty much. You land to the North, depart to the South - due to the terrain in the area. It's also a day only airport for most aircraft.
Big difference: Sun Valley is basically a one way airport in the mountains well below radar coverage. If you want to fly a downwind you better be 3000 AGL or more; otherwise it's a 10 mile straight in. While you are on the approach, nobody can leave, unless you cancel; if you do, they shoot a G5 at you.