August 17, 2023

These strange network problems in a weak project (Episode 2: Weird image freeze)

To begin with, give a unified answer to reasons for first episode: for ease of understanding, network card can be considered as a manual transmission for a car. Usually, everyone uses an automatic transmission for automatic gear shifting, but, for example, when such situations arise like climbing, getting out of trouble or high altitude, we need to switch to manual transmission in order to better meet torque requirements of current conditions.

In first episode, maximum speed of network card of camera is 100M, and maximum speed of switch is 1000M. When both sides self-adapt, they do not match. Set NIC speed of both sides to 100M and full duplex While configurations of both sides are consistent, they match, so NIC is installed with manual network speed configuration option.

Let's start today's post

In 2016, when a newly built winery was inspecting an underperforming project, a strange problem arose. The situation looks like this

1. A total of 9 large monitoring screens are installed on site, and 9 large image monitoring screens often freeze.

These strange network problems in a weak project (Episode 2: Weird image freeze)

Monitor on a big screen

Secondly, client's technology interacts with monitoring manufacturer's technology to conduct investigations, and ideas of investigation are as follows:

When frozen

1. Video playback from surveillance cameras is normal - output: camera and DVR are normal.

2. The decode status of decoder is normal and packet loss of video stream has been detected using basic packet capture.

Because video stream packet loss is mostly caused by network packet loss, client wants me to troubleshoot network issues.

Third, to make troubleshooting easier, monitor manufacturer and I have refined method of transmitting video stream and organized it into pictures, which is easy to understand intuitively.

These strange network problems in a weak project (Episode 2: Weird image freeze)

Video streaming diagram

1. The camera encodes collected video image into a network signal and transmits it to DVR over network.

2. The DVR stores video image and transmits it to decoder over network.

3. The decoder finally decodes video image into digital information and displays it on a large monitoring screen via a video cable.

That is, if it's a network problem, it should appear in link between VCR and decoder, because VCR and decoder are on same switch.

When image freezes again

1. I have verified that switch's connected port utilization is 17% and there are no error messages in switch log.

2. To quickly determine if issue is with switch, we replaced this switch with another replacement switch of same model and result was same.

From this, we can conclude that switch should be normal, and utilization rate of gigabit port of switch is 17%, which means that current parallel stream of decoder is only about 170M.

The problem reappeared: network equipment was fine, I still suspected that it was a problem with decoder, but there was not enough evidence, so I just had to calm down and look for problem. According to common experience, if main cause of network packet loss is a network bottleneck, packet loss will occur, however, gigabit network card data stream is only over 170 megabytes, indicating that bottleneck is not on switch.

I had no idea, so I asked monitor manufacturer. The network bottleneck on device is sufficient. The manufacturer sent me a picture and I found reason.

These strange network problems in a weak project (Episode 2: Weird image freeze)

VCR input bandwidth and output bandwidth

Recall logic behind transmission of video stream mentioned above.

These strange network problems in a weak project (Episode 2: Weird image freeze)

The DVR transmits video stream to decoder

Output: When VCR transmits video to decoder, it will be limited by output bandwidth of VCR. , while parallel code stream of decoder is 170M., so there is a bottleneck, which leads to loss of video stream connected to decoder, which ultimately leads to problem of image hang.

Have any friends in industry encountered similar problems and how to solve this problem? You can chat in comments and we will answer solution to this problem in next issue.