The College Endowment Association was founded in 1890 and traces its roots to a Ladies' Art and Science class organized in 1874. The class was centered on lecturer Charles Farrar, who was affiliated with Milwaukee College which later became Milwaukee Downer College. Mrs. William Grant Fitch proposed a permanent organization to endow the school and secure funds. This was the beginning of the College Endowment Association (CEA).
Milwaukee historian John Gurda described the College Endowment Association in his history entitled Women Learning: A History of the College Endowment Association, 2005:
"It has no permanent faculty, no set curriculum, and no grades. 'Tuition' is a paltry $50 (now $70) a year. The group takes its name from an institution that no longer exists, and its student body consists largely of women who earned their last diplomas decades ago.
This most uncommon organization is the College Endowment Association. Founded in 1890, it is one of the Milwaukee area's oldest educational institutions – older, in fact, than nearly all the colleges and universities in the region. Its history is intimately connected with the history of women's higher education in both the state and the nation. Like every women's group, the College Endowment Association has changed with the world around it, but the organization maintains a remarkable fidelity to its original mission. From the nineteenth century to the twenty-first, the CEA's members have had one thing in common. They are women learning."