Preheat oven to 425°F (220°C).
In a large bowl, combine 2 tablespoons olive oil, salt, balsamic vinegar, Dijon mustard, 1 tablespoon oregano, garlic, and shallot.
Add chicken thighs and toss until fully coated.
Slice one lemon and add it to the bowl along with the halved potatoes. Toss well to combine.
Spread everything evenly onto a sheet pan in a single layer. Place lemon slices on top of the chicken.
Roast for 30 minutes until chicken is cooked through and potatoes are tender.
While the chicken roasts, whisk remaining olive oil, lemon juice (2 tablespoons from second lemon), remaining oregano, and capers in a small bowl.
Remove sheet pan from oven and spoon the lemon-caper mixture over the chicken and potatoes.
Sprinkle feta over the pan and drizzle with tzatziki before serving.