Devienne was born in Joinville, as the youngest of fourteen children of a saddlemaker. After receiving his first musical training as a choirboy in his hometown, he played in various Parisian ensembles as soloist and orchestra player. He studied the flute with Félix Rault; in 1780 he joined the household of Cardinal de Rohan. He was active in Paris as a flutist, bassoonist and composer, and played bassoon at the Paris Opera. He wrote successful operas in the 1790s, including Les visitandines (1792) which brought him much success.

He was also a member of the Military Band of the French Guard, where he was given the rank of sergeant with the duty of teaching the children of his colleagues in the military band in its Free School of Music. After the Revolutionary period, when the Free School became the National Institute of Music, later chartered as the Paris Conservatory in 1795, Devienne was appointed as flute professor (1795–1803); among his students was François René Gebauer. He wrote Méthode de Flûte Théorique et Pratique (1793), which was reprinted several times and did much to improve the level of French wind music in the late 18th century. Like Mozart and many other musicians, he joined the Freemasons and Concert de la Loge Olympique orchestra.

His output comprises approximately 300 instrumental works that are mostly written for wind instruments. There are a dozen flute concertos (plus two posthumously published works, one of them a flute arrangement of G. B. Viotti's violin concerto No 23), sinfonias for woodwinds, quartets and trios for different ensembles, 12 operas, 5 bassoon concertos, 6 bassoon sonatas and 6 oboe sonatas (Opp. 70 and 71). Devienne's compositions for flute, revived by Jean-Pierre Rampal in the 1960s, became well known among flutists. His collected work comprises extensive educational work, including the Méthode, and eight books of sonatas for flute or bassoon, a variety of chamber music, and no less than seventeen concertos. He became known in his day as the "Mozart of the Flute"