here is the EXACT methodology. perhaps a shortcut can be seen from this.
we will use ORD (41 58' 42.972" N x 87 54' 17.429" W) to LAX (33 56' 33.1301" N x 118 24' 29.068" W)
convert to decimals so for ex. 41 + (58 + (42.972/60))/60 = 41.9786033. It will be negative if S and W. Thus ORD (41.9786033, -87.904841) to LAX (33.9425361, -118.40807).
convert the decimals values to radians by multiplying them by PI/180. so for ex. 41.9786033 x PI / 180 = .73266484. Thus ORD (0.73266484, -1.5342289) to LAX (0.59240901, -2.066108).
calculate the SIN and COS of each latitude and longitude value.
The formula is:
ARCCOS (SIN(Lat1) SIN(Lat2) + COS(Lat1) COS (Lat2) COS(Long2-Long1)) x (180/PI) X 60
The 180/PI is to convert back to degrees and 60 is the number of nm per degree of lat/long.
For ORD to LAX the value is 1512.00272.
Basically you are measuring arc distance and that is where the math comes from. The radius of the earth is a constant and thus from the center of the earth to each lat/long coord is the radius of the earth plus its field elevation above MSL. then with two sides of the triangle known and the angle between the two calculated (the arc cosine value) the arc length can be calculated.
http://www.themathpage.com/aTrig/arc-length.htm
Your frequent flier miles will be credited.....