Palmer Hall has been home to the mathematics department since its dedication in 1904. It was President Slocum's vision that brought the science building from conception in 1897, when the college had about 250 students and seven buildings, to completion in 1903. Although it did house the sciences, there were also rooms designated for Latin, Greek, Philosophy, French, German, and English. At the west end of the building, President Slocum had his office along with rooms for the trustees, treasurer, and dean. Much of the original building was classroom and laboratory space; office space was at a minimum with professors often simply using a desk in their classrooms.

Palmer Hall in 1917

The most enduring and endearing legend about Palmer Hall is that it was built to stop the streetcar line from bisecting the campus. William Stratton's streetcar company did make such a request of the city council, but it was turned down after President Slocum came to the rescue. Palmer Hall at that time was due to be positioned west of where it now stands. An "anonymous" gift of $100,000 (most probably from Palmer) and a suggestion to reposition the building resulted in the final position - blocking the streetcar line. (The streetcars subsequently jogged to Nevada and then back to Tejon.)

There are three floors in Palmer Hall (plus a small sub-basement and some finished attic space). On the middle floor (called the first floor) at the east end was a large chemistry laboratory. In the early 1960's, Palmer was renovated and chemistry moved to the new Olin Hall. An entrance was added to the east end of Palmer which bisected the old laboratory. On either side, office space was parceled out and became the center of the Mathematics Department. On the eve (1999) of another new science building, geology and psychology will move from Palmer Hall leaving Mathematics as the last of the Natural Sciences to remain in their historic home.

On July 3, 1986, Palmer Hall was added to the National Registry of Historic Buildings protecting its stately stature.