Born in Vienna where he trained at the Gewerbeschule but when the Nazis came to power in the Anschluss he was forced to leave. His father later died in Auschwitz. On arrival in England he first worked as a labourer before being interned at the out-break of the Second War. On his release he attended Birmingham School of Art and on graduation took a job at a commercial art studio. He worked as a freelance illustrator until 1964 when he took up painting full time.

He participated in many one-man and group exhibitions and in 1981 received the £5,000 Hunting Group prize for the best watercolour of the year. His pictures can be found in many collections including the National Maritime Museum, the Science Museum and Trinity College, Cambridge (for whom he painted its Master R.A. Butler). The National Portrait Gallery commissioned a portrait of Sir Nikolaus Pevsner as well as what is probably Schwarz's most frequently seen work, a triple portrait done in 1984 of the trade union leaders Tom Jackson, Sid Weighell and Joe Gormley standing amidst the pigeons of Trafalgar Square, the picture now hangs in one of the NPG's 20th-century galleries.