The yellow color in natural diamonds is caused by the presence of nitrogen within their crystal structure.
Based on scientific understanding, the fancy yellow color in natural diamonds comes from nitrogen within the crystal lattice. The way these groups of nitrogen atoms are arranged inside the diamond means that they absorb more light from the blue-end of the spectrum. When the blue light is absorbed, the remaining light that is transmitted or reflected back to our eyes is primarily yellow, and therefore give the stone its yellow appearance.
This selective absorption of blue light by nitrogen impurities is the fundamental reason behind the characteristic yellow hue seen in these natural gemstones.