Place oven racks in the upper-middle and lower-middle positions. Preheat oven to 350°F Line 2 baking sheets with parchment or Silpats.
Whisk flour, baking powder and salt together in a bowl.
In a large bowl (or a stand mixer with the paddle attachment), beat butter and sugar on medium speed until light and fluffy, 4-6 minutes. Beat in eggs one at a time, then vanilla and lemon. Stop and scrape down bowl and beaters as needed.
Reduce speed to low and add 1/3 of dry ingredients. When incorporated, add 1/2 of the milk. Add half the remaining dry ingredients, then the rest of the milk, then the last of the dry ingredients, pausing after each addition until it is incorporated.
Scoop 1/4-cup mounds of batter onto prepared sheets, spaced 2 apart (you should get about 6 on each sheet). Use a wet finger or the back of a wet spoon to smooth out the tops. Bake until edges are just turning light golden brown, about 15 minute At about 7 minutes, rotate each sheet 180° and swap racks.
Let cookies cool on racks 10 minutes, then transfer to wire racks until completely cool, at least 1 hour. Let sheets cool and replace parchment before preparing the next batch of cookies.
When all the cookies are done and cooled, prepare the icings. Bring the corn syrup and water to a boil over medium-high heat. Off heat, whisk in confectioner's sugar and vanilla until smooth. Remove half to a separate bowl; whisk melted chocolate and water into one bowl (add water as needed to achieve a spreadable consistency).
Place a wire rack over newspaper. Spread 2 tbsp of one icing (you may start with either one) on half of the flat side of each cookie. Let stand on rack until the icing is just set, about 15 minutes. Spread 2 tbsp of the other icing on the other half of each cookie (you can overlap them slightly), then let stand on the rack until icings are fully set, about 60 minutes.