In the 1861 census, James and his brother William (then an 11 year old mason's labourer) were found lodging with labourer William Alford and family at 3 South Street, Ryde. The whereabouts of their parents at that time has not yet been established, though it is possible they were living in London. Interestingly, in the 1871 census, a widowed Mary Clark of the right age was found working as a nurse in St Anne, Soho, London. This was the birthplace Mary gave in the 1881 census, by which time she was back living in Ryde (working as a cook at Sheldon House, Belvidere Street). Likewise, at the time of the 1871 census, brother William was also found lodging nearby in London.

James Clark died tragically on 10th June 1870 along with two friends, Frederick Thurlow and Ebenezer Tilley, when their sailing boat capsized half a mile off Ryde Pier in breezy weather. The inquest returned a verdict of accidental death; the cause being attributed to the ignorance of the three young men in handling the boat. At the inquest, James' brother William identified the body and said that James lived in Ryde and was a photographer. No other evidence of James' photographic career has been found to date, so it is likely he worked at one of the several studios in the town.

James Clark was buried at Ryde Cemetery on 15th June 1870, and his mother was also buried in the same plot on her death in 1886.