The study investigates the impact of instruction tuning on the confidence calibration of large language models (LLMs), revealing significant degradation in calibration post-tuning. It introduces label smoothing as a promising solution to mitigate overconfidence during supervised fine-tuning, while also addressing challenges related to memory consumption in the computation of cross-entropy loss.