Mary Catherine L'Hommedieu was born Mary Catherine Havens on September 25, 1765. Her father was Nicoll Havens of Shelter Island and her mother was Sarah Fosdick. She married Ezra L'Hommedieu of Southold on June 15, 1803. Ezra was a descendent of Nathaniel Sylvester, one of the original land owners on Shelter Island. Mary Catherine's sister Esther Sarah married Sylvester Dering son of Thomas Dering, another prominent Shelter Island resident. Ezra L'Hommedieu died September 27, 1811 leaving a sizable estate that included the Sylvester Manor to his wife. Mary Catherine died July 25, 1843. The L'Hommedieu estate was then inherited by Mary Catherine Gardiner and her husband Samuel S. Gardiner.

Samuel S. Gardiner married Mary Catherine L'Hommedieu, August 7, 1823. Samuel and Mary Gardiner lived on the family estate referred to by the family as Abbey Manor. In addition to managing the estate, Samuel was a practicing attorney. Samuel and Mary had three daughters: Mary L'Hommedieu, Phoebe Dalton and Francis Eliza. Mary Catherine died January 28, 1838. His second wife was Susan F. Mott whom he married in the mid 1840s. Samuel S. Gardiner died March 21, 1859. His estate was left to his the three daughters. Mary Gardiner Horsford's share was divided among her four daughters.

Mary Gardiner Horsford married Eben Norton Horsford August 4, 1847. She was born Mary L'Hommedieu Gardiner on September 26, 1824, daughter of Samuel and Mary Catherine Gardiner. Mary was privately tutored as a child and later attended the Albany Female Seminary where she met Eben. Mary's avocation was poetry. Many of her poems were printed in local newspapers and she eventually published a volume of poems entitled Indian Legends and Other Poems. Mary gave birth to four daughters: Lilian, Mary Katherine, Gertrude and Mary Gardiner. Mary died November 25, 1855, several months after the birth of her last child. She was buried at Shelter Island.

Phoebe Gardiner Horsford married Eben Norton Horsford July 22, 1857. She was born Phoebe Dayton Gardiner on August 13, 1826, daughter of Samuel and Mary Catherine Gardiner and sister of Mary Gardiner Horsford. Phoebe was also privately tutored as a child and attended the Albany Female Seminary. Phoebe was an accomplished seamstress. She gave birth to one daughter, Cornelia Conway Felton Horsford, on September 25, 1861. Phoebe died October 8, 1900 in Cambridge, Massachusetts.

Lilian Horsford Farlow was born Mary Leila Horsford, daughter of Eben and Mary Horsford, on September 18, 1848. She lived in the family home in Cambridge until her marriage to William G. Farlow on December 10, 1900. She died August 16, 1927.

Mary Catherine Horsford was the second daughter of Eben and Mary Horsford born on October 24, 1850. She was known to the family as Kate. She died in 1927.

Cornelia Conway Felton Horsford was the only daughter born to Eben and Phoebe Horsford. She was born on September 25, 1861 and was named for Cornelius Conway Felton president of Harvard at the time of her birth. She is commonly referred to as Nellie in family correspondence. Cornelia traveled extensively and lived for many years with her maiden sisters Lilian and Kate.