Preheat oven to 325 degrees F (165 degrees C). Grease and flour a 10 inch tube pan.
Cream together 1cup butter or margarine, shortening, brown sugar, and 1 cup white sugar. Add eggs one at a time, beating well after each addition. Stir in 1 teaspoon vanilla.
Sift 2 1/2 cups of the flour and baking powder. Add alternately with milk to the creamed mixture. Stir 1/2 cup flour with the chopped pecans, then fold into the batter. Pour into prepared pan.
Bake in preheated oven for 60 to 90 minutes, or until a toothpick inserted into the center comes out clean. Let stand in pan 10 minutes, and then turn out onto a wire rack and cool completely.
To Make Frosting: In a small saucepan, beat 1 cup brown sugar, 1 cup white sugar, butter or margarine, and milk together. Bring to a rolling boil, and cook for one minute. Remove from heat, and stir in vanilla. Cool, then beat until thickened. Add a little milk if it too thick. Spread on cake.