Preheat the oven to 350 degrees F. Line a 10 inch loaf tin with parchment or spray well with non-stick cooking spray.
In a large bowl, beat together ¾ cup of the brown butter and granulated sugar until fluffy and lightened in color, about 2-3 minutes. Beat in eggs until light and fluffy, then add 2 teaspoons vanilla extract and mix.
In a medium size bowl, whisk together flour, nutmeg, baking powder, and ½ teaspoon salt.
Mix in about ⅓ of the dry ingredient above into the creamed sugar mixture, mixing until just barely combined. Then mix in about ½ of the buttermilk. Add another ⅓ of dry ingredients, then ½ of buttermilk, then final ⅓ of dry ingredients, mixing between each until just barely incorporated. Be careful not to over mix.
Bake in preheated oven for 40-50 minutes, until loaf appears set and toothpick inserted in the center comes out clean. Remove from oven and let cool completely.
While the cake cools, make the glaze. Melt the remaining ¼ cup browned butter just until it becomes liquid again, then cool for a few minutes until no longer hot. To the melted brown butter, add powdered sugar, milk, 1 teaspoon vanilla, and salt, stirring until smooth.
Let cake cool completely, then spread glaze over the top.