Preheat oven to 350°. In a large bowl, cream butter and sugar until light and fluffy, 5-7 minutes. Beat in eggs. Beat in lime zest, juice and vanilla. Combine flour, baking powder and salt; gradually add to creamed mixture alternately with milk, beating well after each addition. Fold in walnuts.
Transfer to 2 greased 9x5-in. loaf pans. Bake 50-55 minutes or until a toothpick inserted in center comes out clean. Cool 10 minutes before removing from pans to wire racks.
Combine confectioners' sugar and enough lime juice to reach desired consistency; drizzle over warm bread. Cool completely.