1850 census of Morgan Co., Ky, enumerated 5 Aug, lists on p. 26 as family #155:
Samuel Ramey, 27, laborer; Sarah, 25; Benjamin, 2; Sarah, 1; Susannah, 2/12. All born in Ky.

1860 census of Bath Co., Ky, enumerated 27 July, lists on p. 140 as family #1003:
Samuel Raimey, 35, farm hand; Sarah, 34; Benjamin, 14; Elizabeth, 12; Susanah, 10; Jesse, 9; John, 7; Missouri, 6; Harrison, 2. All born in Kentucky. Patrick Dorsey, 31, farmer, b. Ireland.

I cannot find this family in the 1870 census. It may be that they moved that summer, and were missed by the census takers in both the old and new precincts. From birth records, Jesse was only barely 8 (7 on the official census date of June 1), and John only 5, so Missouri is probably really only about 4.

1880 census of Greenup Co., Ky, Old Town precinct, enumerated :
Ramey, Sarah, 52; Harrison, son, 18; Fleming, son, 17; Georgiana, dau, 14. All born Ky Ky Ky.