Data is the new oil for training large AI models. However, the "oil" generated by humans may run out someday or grow much slower than the speed of AI consuming them. Moreover, the human-created data are less controllable in terms of quality, opinions, format, style, etc., and may lead to biases or privacy concerns when used for model training. Can we leverage the power of Generative-AI and automatically create synthetic training data in a more efficient, controllable, and safe manner? Does continual learning on synthetic data lead to self-evolving AI? What risks should we expect? In this talk, I will present our recent works that aim to investigate whether and how synthetic data can be generated to improve large language models (LLMs) and computer vision models, especially when the real data is non-perfect. These works include Mosaic-IT (compositional data augmentation for instruction tuning), Selective Reflection Tuning (data generation via student-teacher interactions), DEBATunE (data generation by LLM debate), and Diffusion Curriculum (generative curriculum learning of images). These projects are led by my Ph.D. students Ming Li and Yijun Liang.