Hammer was born in Vienna, Austria to Karl and Maria (Fuhrmann) Hammer. He began his apprenticeship in architecture at the age of fifteen in the studio of Camillo Sitte, author of Der Staedte-Bau nach seinen kuenstlerischen Graundsaetzen. In 1898, he transferred to the Academy of Fine Arts Vienna, which he left ten years later.
Hammer produced his first type design, Hammer Uncial, in 1921. In 1922, he moved to Florence, Italy, where he set up a printing press. In 1929, he moved his printing operation into the Villa Santuccio in Florence and named it the Stamperia del Santuccio. The first book that was printed in this operation was Milton's Samson Agonistes (1931), using what would be known as his Samson Uncial type. Punches for the type were cut by Paul Koch, son of Rudolf Koch. Hammer moved to Kolbsheim in Alsace in 1934, where he designed and built a chapel on an estate for a friend.
From 1936 to 1939, Hammer lived in Vienna, where he served as professor at the Akademie der Bildenden Künste. In 1939, he fled the Nazis and immigrated with his first wife to the United States, leaving behind all his cutting and casting tools and most of his fonts. He taught at Wells College in Aurora, New York until 1948. Here, he produced American Uncial – the best known of his five typefaces.
In 1948, Hammer settled in Lexington, Kentucky and was artist-in-residence at Transylvania University, a post he held until retirement in 1953. While in Kentucky, Hammer was known for designing the official seal of Louisville, which was used until the city's city-county government merger in 2003. He also designed the seal for the University of Louisville, a portrait of Minerva, which has been used since the 1950s (although revised in the 1990s for better reproduction on screens).
Hammer built his wooden press in 1927 with the help of local Florentine craftsmen based on a press in the Laurentian Library; in 1960, the Laurentian's press was discovered to be a copy constructed in 1818. It was first used to print Samson Agonistes. When he closed his studio in 1933, the press was stored. In 1954, it was moved to the University of Kentucky where it has been in use by the King Library Press since 1959.
Hammer was first married to Rosl Rossbach, and together they had two children, Veronika and Jacob. Hammer married Carolyn Reading in 1955. Carolyn Reading Hammer founded the King Library Press in 1956 and later became the University of Kentucky Libraries' curator of rare books. Since 1943, Carolyn Reading and Amelia Buckley had been printing at their Bur Press and their Chandler & Price printing press was moved to Bullock Place in Lexington when Carolyn and Victor Hammer married. This press, together with the antique-style wooden press Victor used in Florence in the 1920s, were both moved to the basement of University of Kentucky's M. I.King Library.