Preheat your oven to 350°F (175°C). Grease, flour, and line the bottoms of 3 8-inch cake pans with parchment paper.
Melt chunks of chocolate in boiling water, then let it cool.
In a large bowl, beat softened margarine and sugar until light and fluffy.
Add egg yolks and vanilla extract, then mix in the melted chocolate-water mixture until everything is well combined.
In a separate bowl, sift together flour, baking soda, and salt. Alternate adding this mixture with buttermilk to the wet ingredients, beating until smooth.
In another bowl, beat reserved egg whites until stiff peaks form, then gently fold them into the batter.
Pour the batter into prepared cake pans and bake for 35 to 40 minutes, or until a toothpick inserted into the center comes out clean.
While the cakes bake, prepare the frosting by cooking evaporated milk, sugar, egg yolks, margarine, and vanilla extract until thickened. Mix in coconut and pecans, then frost and fill the cakes once cooled.