The reason we replace the root of the heap with the rightmost node of the last level in the heap is so that we can ensure that our heap is still a complete binary tree after the deletion.
(From Wikipedia: "A **complete binary tree** is a binary tree in which every level, *except possibly the last*, is completely filled, and all nodes are as far left as possible.")

It is important for our heap to be a complete binary tree so that the depth of each node in our tree is [math]O(\lg n)[/math]. This upper bound is critical to attaining logarithmic time complexity for insert() and extractMin().

If we were to replace the root node with any other node and percolate that node down, the result would be that some path from the root to a leaf would be one node shorter. If a path besides the rightmost path is one node shorter, then our tree no longer is complete (the last row will have a hole somewhere in the middle). On the other hand, if we replace the root node with the rightmost node of the bottom level, then the last row has no holes among the leaves, so we still have a complete binary tree.

Suppose we tried to take a different approach. For example, we know that the new root node will ultimately be the smaller of its two children. To make the deletion process more efficient, we would want to replace the root with the smaller of the two children and then find some other node to fill the hole. However, to fill the hole, we create another hole. And to fill that hole, we create another hole, and so on. Even if we could find a good swapping pattern, it would still require that we eventually move the hole to take the place of the rightmost leaf on the bottom row. There's no other way to preserve the completeness of the binary tree without removing the rightmost leaf of the bottom row. Searching through the tree to find good candidates for replacements is going to be more inefficient than the percolate down process, and we have no guarantees about whether we can even find a more suitable chain of nodes to swap by searching through the tree.

If you think about it, replacing the root node with the rightmost node in the bottom level and then percolating that node down really isn't that bad. If we have 1,000,000 nodes in our tree, the node would have to percolate down no more than 20 levels deep ([math]\lg 1000000 \approx 20[/math]). That's a very low cost considering the number of nodes we're working with.</span>