Measures of general cognitive ability (GCA) are highly stable from adolescence onward, particularly at the level of genetic influences. In contrast, measurement of GCA in early life (before 3 y old) is less reliable and less is known about the stability of GCA across this period, including its relation to adult GCA. Using data from the Colorado Longitudinal Twin study (N = 1,098), we examined the stability of GCA measures across 5 time-points (years 1 to 2, 3, 7, 16, and 29), including how an array of cognitive measures given at 7 and 9 mo relate to later GCA. We then examined the genetic and environmental stability of GCA across the first 30 y of life using complementary methods: twin analyses and polygenic scores (PGSs). Two infant cognition measures, object novelty and tester-rated task orientation, predicted GCA in adulthood (r = 0.16 and 0.18, respectively). Correlational analyses were consistent with a pattern of increasing stability across development for GCA measures between year 1 to 2 and adulthood (r = 0.39 to 0.85). Subsequent twin analyses revealed that 22% of variance in adulthood GCA was captured by genetic influences on GCA from year 3 or earlier, with an additional 10% explained by shared environmental influences on GCA at year 1 to 2. PGSs for adulthood GCA and educational attainment predicted GCA from 1 to 2 y onward (βs = 0.09 to 0.44) but not infant cognition. Findings suggest that genetic and environmental influences on GCA demonstrate considerable stability as early as age 3 y, but that measures of infant cognition are less predictive of later cognitive ability.