In 1870 he was listed as age 22 with his sister Permelia and her husband Augustus McKenzie. I have not found him after that date. (There were other James H. Browns born 1845 to 1848 listed in Grant, Muhlenberg, Barren and Nelson counties, but they are all eliminated either due to appearing in those counties in 1850 through 1870 censuses, or having death certificates that list other parents.) So he probably either died before 1880, or moved out of Kentucky. With such a common name, it would be impossible to identify him if the former is true.