I believe the gem field on KOTG is the only one that is not worth mining...that one's just too far away and entails substantial risk to boot.
On another note, I loaded up RA1 for the first time in a good while. I sent my ore trucks to a gem field to mine, and the strangest thing happened...the gems disappeared when I mined them!