Where did the phrase "four-eyes" come from?

The term four-eyes, usually used as a derogatory reference to someone who wears glasses, dates to at least the 1870s, when a character bearing the name appeared in the best selling novel "The Iron Pirate" by Max Pemberton. However, the character does not wear glasses in this book.

This suggests the slang term either predated the novel or the name later evolved to refer to someone wearing glasses. The 1921 book "Roosevelt in the Badlands" by Hermann Hagedorn suggests the term was common by the 1880s, when future president Theodore Roosevelt acquired the nickname because he wore spectacles while working in the badlands of the Dakotas.