Dorfsman was born in 1918 on the Lower East Side of Manhattan, and moved as a child to the Bronx. His parents were Jewish immigrants from Poland. Dorfsman attended Theodore Roosevelt High School, graduating in 1935. Dorfsman had wanted to attend New York University and study bacteriology there, but was unable to afford the tuition. He chose to attend Cooper Union, where he received a four year scholarship and graduated in 1939. Dorfsman served on Cooper Union’s board of directors for many years.

Dorfsman was hired in 1946 as art director for the CBS Radio Network. Dorfsman’s designs were described by The New York Times as featuring “clear typography, simple slogans and smart illustration”. After William Golden died in 1959, Dorfsman was named creative director of CBS Television. By 1964, he was selected as the director of design for all of CBS and was later promoted to senior vice president and creative director for marketing communications and design. In this role he maintained creative control over the network’s use of the CBS eye logo to its proprietary CBS Didot typeface. The Times credited the “cleverness and subtle beauty of his advertisements” with drawing viewers to the network’s news and entertainment programs.