Make the starter: Mix warm water, yeast, sugar, and 1/2 cup flour. Cover and let sit 15 minutes until bubbly.
Prepare the dough: In a large bowl, mix 3 1/2 cups flour, sugar, and salt. Make a well, add eggs, milk, vanilla, and starter. Stir to form a sticky dough.
Add butter and zest: Knead in softened butter gradually, then fold in orange and lemon zest. Knead 10 minutes until smooth.
First rise: Place dough in a greased bowl, cover, and let rise 2 hours until doubled.
Add fruits: Gently fold in raisins and candied citrus. Shape dough into a ball and place in a tall panettone mold.
Second rise: Cover and let rise 60–90 minutes until the dough reaches the top of the mold.
Bake: Preheat oven to 350°F (175°C). Brush with beaten egg and sprinkle sugar. Bake 35–40 minutes, tent with foil if browning too fast.
Cool: Let the panettone cool completely before slicing. Serve warm or at room temperature.