I managed to fix the issue, I couldn't use jQueries remove() function, because that too stalled the browser, however you can do this.

$('.parentDiv').html('');
$('.parentDiv'.remove();

Just set the parent div's HTML to be empty and then remove it. I read something about remove() keeping it's contents available and that was why it froze up.

I've now spent days on this and tried a number of different approaches. It still crashes every browser without throwing any js errors.

This is all good and I have been able to have it all working fine, until it comes time to remove it if needed. The function below is called when the user clicks outside of the tinymce instance, and turns the instance back into the original Div. Before I was just hiding it and then removing the parent div, but it stalled the browser for 1minute or more.

$('body').click(function(){
    if(currentTinyMce != null){
        tinyMCE.execCommand('mceFocus', false, currentTinyMce);                   
        tinyMCE.execCommand('mceRemoveControl', false, currentTinyMce);
        currentTinyMce = null;
    }
});

currentTinyMce stores the id of the selected div. It seems to be the call to mceRemoveControl that is causing it, and it stalls just the same if I call tinyMce.remove()

If anyone has run into something similar, some help would be awesome.