Preheat the oven to 325°F (165°C). Grease and line an 8×8-inch baking pan, allowing parchment to hang over the sides.
Start by making the crumble. Combine the flour, cinnamon, and sugars in a bowl, then mix in the melted butter until the mixture resembles wet sand. Place the bowl in the refrigerator to chill.
For the banana cake, whisk together the flour, baking soda, and salt in a medium bowl. Set aside.
In a large mixing bowl, whisk together the melted butter and both sugars until smooth. Whisk in the sour cream, egg, vanilla extract, and mashed bananas.
Gently fold in the dry ingredients using a rubber spatula until just combined.
Spread half of the batter into the prepared pan. Mix together the cinnamon sugar ingredients and sprinkle evenly over the batter.
Cover with the remaining batter and gently spread it evenly. Remove the crumble from the refrigerator and sprinkle it evenly over the top.
Bake for 40–45 minutes, or until a toothpick inserted in the center comes out clean with a few moist crumbs. Allow the cake to cool for 30–40 minutes.
While the cake cools, whisk together the powdered sugar and milk to make the icing. The consistency should be thick but drizzleable. Add a little more milk if needed. Drizzle over the slightly warm cake and serve.