He grew up as son of the amateur quartet player Robert Emil Madsen and first learned piano with Ruth Gaarder (1924-27). He then studied with Arild Sandvold (1928-30) and in a second study period made organ his main instrument (exam in 1936, the studies continued until 1940). Additionally, he studied piano with Reimar Riefling (1937-39), composition and counterpoint with Per Steenberg (1935-37) and instrumentation with Bjarne Brustad (1939-40). After further studies with Øivin Fjelstad he graduated as bandmaster (“kapellmester”) in 1943. While his duties as an organist already began in 1932, he started a conducting career in 1938, at the same time gaining attention as a composer. After the war he studied with Aaron Copland in Tanglewood and with Ernest White in New York.
