Place chicken thighs in a shallow casserole dish. Season with salt and pepper on all sides. Toss with the olive oil, vinegar, lemon, oregano, garlic and bay leaf. Cover and marinate overnight in the refrigerator.
Preheat the oven to 375°F. Bring the chicken to room temperature. Set chicken skin side up in the casserole dish. Top each piece of chicken with a little brown sugar. Add the white wine to the pan, scatter the capers around the dish and crumble the feta over the top. Cover and bake for 30 minutes. Remove the foil and continue cooking for another 20 minutes until tender and beginning to brown.