Imagine you trained an LLM on only pre-Anaximander texts, those from the 7th century BCE and before (assume that you could upload everything ever written or said to that point, so there would be enough training data). Imagine I then asked it to explain natural phenomena, like earthquakes and rain, and the shape of the earth, and the composition of the heavens. Would it be able to figure out natural laws, or would it explain things in terms of the gods' actions?
If an LLM were trained exclusively on texts from the 7th century BCE and before, its understanding of the world would be limited to the knowledge and beliefs of that time. In that era, explanations of natural phenomena were often based on mythology and the actions of gods, rather than on empirical observations and scientific principles.
As a result, when asked to explain natural phenomena like earthquakes, rain, the shape of the earth, and the composition of the heavens, the LLM would most likely provide explanations rooted in the myths and beliefs of that time. It might describe these phenomena as the results of divine intervention or the actions of gods, rather than offering scientific explanations based on natural laws.
While the LLM could still generate text that is coherent and contextually relevant, its ability to discover and explain natural laws would be limited by the knowledge available in the training data. To provide more accurate, scientifically grounded explanations, the model would need to be trained on a broader, more diverse range of texts, including those from later time periods that encompass the development of scientific thought and understanding.