Heat the oven to 175 C / 350 F. Line the bottom of a 23 cm / 9-inch round cake pan with parchment and rub the sides with 1 tsp olive oil. Slice the kumquats crosswise into 3 mm rounds and flick out the seeds with the tip of a knife.
Stir 100 g sugar and 120 ml water together in a 20 cm / 2-quart saucepan over medium heat until the sugar dissolves. Add the kumquat slices and simmer gently for 10 minutes, until the rinds turn translucent and the syrup thickens slightly.
Lift the slices out with a slotted spoon and arrange them in an overlapping single layer across the bottom of the pan. Spoon 1 tbsp of the syrup over them and set the rest of the syrup aside.
Whisk the eggs, 150 g sugar and orange zest in a large bowl for 2 minutes, until pale and thick. Whisk in the olive oil in a thin stream, then the milk and vanilla.
Sprinkle the flour, baking powder and salt over the wet mixture. Fold with a spatula until no dry streaks remain, then stop. Overmixing makes the crumb tough.
Pour the batter over the kumquats, smooth the top and tap the pan once on the counter. Bake on the middle rack for 40 minutes, until the top is deep golden and springs back and a toothpick comes out clean. If it shows wet crumbs, bake 5 minutes more.
Cool the cake in the pan on a rack for 15 minutes. Run a knife around the edge, invert onto a plate and peel off the parchment. Brush the reserved syrup over the warm top.