I can say one thing - your multi-thread design is full of flaws First of all you access the same object from within more than one thread which is a Bad Thing (TM). Second of all all events for the serial object of yours will be processed in the main thread. I don't know if you actually use events there but based on what you observe, it is highly possible.

My first and most basic question is - are you sure you really need separate threads for the communication?