The Monterey Public Library faced several challenges in the decades following 1860.
In 1874, the Library finally moved to Colton Hall where, because of weak local government, the library was kept under lock and key. Soon thereafter, the library moved again to the newly built Monterey schoolhouse where in 1893 both the school and much of the Library's early collection were destroyed in a fire.
Then, in 1901, a women's literary society placed the library under its wing, soliciting book donations and holding fundraising events. The women volunteers were able to keep the library open two afternoons a week in various locations along Alvarado Street, including the old Wolter Adobe.
The fortunes of California's first public library began to change in 1906, when the Monterey Library Association turned over all its assets to the City of Monterey. Mrs. Ada Freitas donated property for a new Monterey Public Library and philanthropist Andrew Carnegie gave a grant for the new building, which opened at 425 Van Buren Street in 1911.
The new library was designed by well known California architect William Weeks in the Mission Revival style, reflecting Monterey's Spanish and Mexican roots.
For the next forty years, the Carnegie library building was the home of the public library. It featured separate reading rooms for adults and children and a basement smoking room with fireplace "for gentlemen only." The Monterey Daily Cypress predicted that the "new gentlemen's smoking room would be quite popular with workingmen because the entrance was situated so that a fellow could drop in for a read and a smoke without having to dress up."
Today, the Carnegie library building, which has been expanded and remodeled, is part of the Monterey Institute of International Studies' library.