Preheat the oven to 375°F (190°C). Line two baking sheets with parchment paper.
In a large mixing bowl, cream together the butter, brown sugar, and granulated sugar until light and fluffy.
Add the egg, egg yolk, and vanilla extract. Mix until fully incorporated.
Stir in the baking powder, baking soda, salt, cinnamon, and cornstarch.
Gradually add the flour and mix until a soft dough forms.
Scoop the dough into large 3-tablespoon portions and place them on the prepared baking sheets.
Using your thumb or the back of a spoon, create a shallow indentation in the center of each cookie.
In a separate bowl, combine the crumb topping ingredients until a crumbly mixture forms.
Fill each indentation generously with crumb topping.
Bake the Heavenly Coffee Cake Cookies for 9–12 minutes or until the edges are lightly golden.
Allow the cookies to cool completely on a wire rack.
Whisk together the powdered sugar, cinnamon, and milk until smooth.
Drizzle the glaze over the cooled cookies and serve.