Message-Id: 202111241624
Period: 23.11.2021, starting 1:30 PM
Betroffen: Individual VMs
Auswirkung: Network interruptions of some VMware VMs

Since 1:30PM yesterday 23.11. a few VMs are subject to network interruptions. This is seen
as either repetitive short interrupts or as
complete network outage by the VM. From Linux the interrupts are logged by the kernel as
“tx hang”. We can provide a workaround for
individual VM network failures but we can not guarantee the success of this workaround.

As of now we can not single out the root cause of this error but are busy trying to find
it. Due to the nature of the error there is
currently no way neither from the VMware environment nor from dedicated VMs in the
networks to detect, reproduce or diagnose the error properly.

Update: We suspect the reason to be a bug in the firmware of the network cards. To test this we updated the firmware on some network cards to see if this fixes the issue.

Update:
Currently the interrupts seemed to have stopped, unfortunately we still haven’t found the
cause.
Together with VMware we are working on the analysis of the failures.