FlyChicaga:
Welcome to leadership, pal.
I don't teach in a formal school now, but in a previous life I did.
Given enough time, money & training, I believe that any reasonably intelligent person can be taught to fly; however, there are some people who--despite being able to pilot an aircraft--will never be able to exercise the judgment required to do it professionally with passengers or troops aboard.
Now, I instruct part-time in a couple of clubs at what most folks would call "a podunk grass strip in East Tennessee." I was also a flight instructor in the Marine Corps.
I have (& have had) some students who I know I can teach to fly on an occasional basis, but who could never hack it in the service or who could never be professional pilots. My wife (formerly a school teacher) laughingly calls them my "LD" (learning disabled) students.
It's not fun, but I level with these students about their progress in comparison to what I've come to know as a "normal" student. Despite having been a Marine, I still don't enjoy this; however, I believe they still respect me for this because they still patronize me & they still try their best--even if it will never be good enough for anything more than casual flying. I actually wish more people had work ethics like these folks.
There are times when your best--as an instructor or as a human being--just won't be enough. If you did the best that you could at the time, then there's nothing to be ashamed of, or regretful for. Put it behind you, & try to learn from it for the next time. You might also try to solicit advice from some of your mentors or peers who have been at it a bit longer. Actually, you're doing this right now. Kudos to you!
I, personally, don't think it's bad to care about your students. I believe that people of good character are able to sense your sincerity & will respond to it in a positive manner--even if you have to make a tough decision that adversely affects them. I also don't believe that you can give any customer the best service possible without knowing him.
Maybe your frustration with situations like this will never pass, but life is inherently unfair. The true test of character is how one deals with that fact. Are you going to stop doing your best because of one failure, or are you going to resolve to learn from it & try to do better in the future?