Preheat the oven to 350°. Grease and flour a bundt pan.
Put the kumquat halves in a food processor and pulse several times until they are the consistency of chunky applesauce. Add the vanilla. Pulse once or twice more. Set aside.
Cream together eggs and sugar until light and fluffy. Mix in the butter and buttermilk.
Whisk together flour, baking powder, and baking soda. Add ⅓ at a time to the wet ingredients and mix until combined after each addition.
Stir in the kumquat purèe. Pour into the prepared bundt pan and bake for 60 minutes or until a toothpick tester comes out clean.
Allow the cake to cool for 5 minutes before removing it from the pan. Whisk together the powdered sugar and cream or juice and pour on the top of the warm cake, allowing it to run down the sides. Zest an orange over glaze if you like.