Agnes was the daughter of Thomas Ayloff but her mother is unknown. The date and place of her birth have not been found.
Her husband was sir John Brydges. They were married, but the date and place have not been found. Their seven known children were Giles (c1488-1557), Anthony (c1491-?), Ursula (c1495-c1530), George (c1497-?), Bridget (c1500-?), Elizabeth (c1503-?) and Winifred (c1507->1586).