LOL The trees and animals were bigger because the size of the PLANET! was bigger. Read about Tiamat, which were between Mars and Jupiter. You will discover why ancient Summerians called the Earth "the rock" and "what is left" ...
Where do I begin? Alex already pointed out your fallacy, but on the cosmological scale, let's assume there was a larger planet between Mars and Jupiter and Earth is just a remnant of it.
1. The events would play out over the course of billions of years. The time scale is just incomprehensibly long, too long for any civilization, let alone an individual, to recount the past events.
2. The scale of the event would be catastrophic. Assuming the planet broke up and Earth is just the largest piece, all life on the planet would be wiped out along with any remains of the past. The planet's surface would become molten for hundreds of millions of years.
3. The energy required to decelerate Earth from the asteroid belt to its current orbit is incredibly high.
4. At the distance to the asteroid belt, insolation is too low to sustain liquid water. On top of that, Sun's output is steadily increasing over millions of years, so billions of years ago, insolation would be even lower than today.
5. Even if most of the mass in the asteroid belt was accrued by the growing planets in the early Solar System, scientists estimate that the absolute maximum was about twice the mass of Earth. By the time that this process was nearly complete, and 1 Ceres and 4 Vesta formed, the asteroid belt was maybe 100 times less massive than Earth, more likely 200 times less massive.
Never mind that the effects of such a huge change would be readily evident on Earth and in the Solar System.
Alternatively, if the events played out over the course of a few decades, maybe centuries, so that they could be remembered by Sumerians, with some incredible forces involved that would neatly cut out Earth from "Tiamat", with some rapid cooling of the inner side that wouldn't destroy the atmosphere of the young Earth and kill all life on it, well, that would have left even more traces of such events, especially if they were so recent (going back mere thousands of years ago).