The Lady shown on the painting was traditionally identified as the Duchess of Bedford. Therefore the painting most probably was once in the possession of the Russell family, Dukes of Bedford at Woburn Abbey. However, after intensive research the sitter was identified as Jane, Duchess of Gordon, née Maxwell (1748/49–1812) wife of Alexander Gordon, 4th Duke of Gordon and mother of Lady Georgiana Gordon (1781–1853). Lady Georgiana Gordon married John Russell, 6th Duke of Bedford (1766–1839) in 1803. It can be assumed that this portrait of Jane, Duchess of Gordon came through her daughter Georgiana Russell, Duchess of Bedford, née Gordon into the possession of the Russell family. The similarity to other portraits of the Duchess of Gordon as well as the fact that she is only wearing one glove on her right hand indicates that the sitter is Jane Gordon, Duchess of Gordon, née Maxwell. When Jane was 14 years old she had a nasty accident in Edinburgh where she lost one finger on her right hand. Since that time she wore gloves whenever possible to hide the consequences of this accident.

Possibly in the late 19th century the portrait was put in an elaborate gilded Victorian frame and labelled: "Duchess of Bedford – Sir Thomas Gainsborough". An old label on the back of the painting shows that the portrait later came into the possession of Col. W. S. C. Curtis at Marley House, Battle, Sussex. The same label shows that in 1958 Col. W. S. C. Curtis had the painting cleaned, restored and revarnished by Stacy-Marks Ltd., Art Dealers, 120-122, Terminus Road, Eastbourne, Sussex. Obviously it was at that time when the portrait was attributed to Joshua Reynolds.

The fact that several friends and relatives of Jane, Duchess of Gordon, née Maxwell were portrayed by Daniel Gardner – including her daughter Lady Louisa Gordon (1776–1850) who married Charles, 2nd Marquess Cornwallis and she therefore consequently became the Marchioness Cornwallis – as well as the similarity to both composition and handling of a portrait by Gardner showing Lady Elizabeth Bridgeman (1755–1806) - in 1794 her husband Sir Henry Bridgeman was created Baron Bradford - indicates that this portrait might well be by Daniel Gardner (Reference: George Charles Williamson: Daniel Gardner, painter in pastel and gouache: A brief account of his life and works. John Lane, the bodley head, Vigo St., W, London 1921, appendix I, p. 133 → description of the portrait of Louisa, Marchioness Cornwallis, née Lady Louisa Gordon, daughter of Jane, Duchess of Gordon). Another painting, once in the possession of the Viscount Harcourt, showing two children in a landscape with a goat and a dog shows great similarity as well to both composition and handling to the portrait showing Jane, Duchess of Gordon. Especially to the girl in the foreground who holds her arms in the same positions like the Duchess of Gordon (Reference: George Charles Williamson: Daniel Gardner, painter in pastel and gouache: A brief account of his life and works. John Lane, the bodley head, Vigo St., W, London 1921, appendix I, plate 10). Furthermore, there is a great similarity in the way how Gardner painted the jackets, the cravats and the gloves of the riding habits of the three gentlemen in his portrait The Heathcote family on the hunt, and in the way how Gardner painted the jacket, the cravat and the glove of the riding habit of Jane, Duchess of Gordon. The portrait of the Heathcote family, which is on display in Montacute House, shows the Rev. William Heathcote (1772–1802), on horseback (son of the 3rd Baronet); Sir William Heathcote of Hursley, 3rd Baronet (1746–1819), holding his horse and whip; and Major Vincent Hawkins Gilbert, M.F.H. (Reference: George Charles Williamson: Daniel Gardner, painter in pastel and gouache: A brief account of his life and works. John Lane, the bodley head, Vigo St., W, London 1921, plate to face p. 28).

Since Daniel Gardner rarely signed his portraits it was quite often the case that his paintings were later attributed to his better known colleagues Joshua Reynolds or Thomas Gainsborough. However, this does not come as a surprise if one takes into account that Gardner worked with both, Joshua Reynolds as well as Thomas Gainsborough.