Preheat your oven to 350°F (175°C). In a medium bowl, whisk together the all-purpose flour, baking powder, baking soda, and salt. Set aside. In a large mixing bowl, cream the softened butter, brown sugar, and granulated sugar together using an electric mixer for about 2-3 minutes until light and fluffy.
Chill the dough by dividing it in half, flattening each half into a disk, wrapping in plastic wrap, and refrigerating for at least 30 minutes.
Roll out one disk of dough on a lightly floured surface to about 1/4 inch thick and cut into rectangles, approximately 3x4 inches. Place half of the rectangles on prepared baking sheets and spoon about 1 tablespoon of the brown sugar cinnamon filling onto the center of each.
Roll out the second disk of dough, cut into rectangles, and place on top of the filled ones. Use a fork to seal the edges.
Bake for 12-15 minutes until edges are lightly golden. Let cool on baking sheets for about 5 minutes before transferring to a wire rack.
Prepare the frosting by mixing powdered sugar with 2-3 tablespoons of milk until smooth.
Once cookies are cool, drizzle frosting over them and add sprinkles if desired.