Recently, I encountered two Fixed Wireless troubles within the span of a week. This is what they had in common:
- Both were very close to the serving tower.
- Both used equipment manufactured by RAD.
- Both were delivering bit rates of less than 10 MB upstream and downstream.
- Both had tech savvy customers that run speed tests often.
- Both were installed less than six months ago.
- The NOC errantly requested that the antenna be replaced to fix the trouble for both.
This is what was quite different between the two:
- Trouble #1 had been working at full rates (100 MB down and 9 MB up) but had recently degraded: Trouble 2 had never achieved the designed speeds.
- Trouble 1 was fed by a serving tower that had an uplink to another tower: Trouble 2 was fed by a serving tower that had a fiber uplink.
Working Blind?
Replacing equipment for no logical reason as part of a troubleshooting strategy has been an epidemic since the advent of DSL. Not sure what to do next, replace the router. Depending on the study you believe, between 60% and 75% of all routers said to be bad are completely functional. This practice has transferred with ease to Fixed Wireless.
There is still a router involved, so it is normally replaced first. Why give up on a perfectly fine bad practice? When that does not work, replacing the antenna has become popular. This is especially true for the NOC because they don’t have to find another antenna, climb a ladder in below zero Fahrenheit temperatures, remove the existing one, mount and tune the new one.
This next point is important to remember: No matter what you do or don’t do at the customer premise, if the uplink is bad, you can’t fix the problem by yourself.
Going against these best practices, we refused to replace the antenna both times. Why were we so confident that this would not fix anything? The RAD antennas can be accessed via Wi-Fi. You will probably have to power down the Power over Ethernet (PoE) device, but 3 or 4 minutes after you do, you will see a Wi-Fi option that begins with “RAD”. Log in to that WAP. You won’t have Internet access, but you don’t need it. Then you need to log in to the antenna.
Once in, you have access to all kinds of valuable data. Beyond seeing signal strength and frequency, you can run a speed test between the antenna and the tower. If you see the target rates or greater, there is a 95% chance there is nothing wrong with the antenna or the alignment.
The customer for Trouble #1 is a remote IT person completely dependent on his Fixed Wireless service to do his job. (See Figure 1.) He runs a speed test two to three times a week. He normally gets >90 MB down and >8 MB up. He called us when his downstream rates dropped to <10 MB. His upstream was not impacted significantly.
After logging into the antenna, we saw rates that matched what the customer was getting before the degradation. We called into the NOC and did the normal routine:
- Plugged directly into the router with an Ethernet cable to eliminate the Wi-Fi connection as the problem.
- Made sure nothing on-site was eating up the bandwidth. This is an important step. I have seen situations when the customer swore everything was disconnected and we found nine devices logged in through the Wi-Fi. Two of them were unidentified and were pulling 40 MB combined. Changing the Wi-Fi password fixed that trouble.
- Power cycle the modem. (For details on this, check out my August 2021 column on this very issue: https://isemag.com/2021/08/telecom-2021-network-troubleshooting-modems/.)
- We postponed replacing the router because the chipset shortage has made us short on routers.
- We refused to replace the antenna because of the readings we got when logged in.
At this point, the NOC basically said, “We can’t help you,” so we hung up. Then we called the fixed wireless engineers. This group verified our testing and agreed that replacing the router or antenna would fix nothing. They then checked the microwave uplink of the serving tower and the numbers looked bad. They reset the microwave link, and everything came back to the speeds for which the customer was paying. This next point is important to remember: No matter what you do or don’t do at the customer premise, if the uplink is bad, you can’t fix the problem by yourself.
Our second customer is a manufacturing plant. They were an existing DSL customer getting 25 MB down and 5 MB up. They called in about a bit rate upgrade and somebody realized that the building is a duffer’s pitching wedge distance from a new Fixed Wireless tower with no customers being fed. (See Figure 2.)
The service was installed many months ago, but it never worked above 10 MB down and up. The DSL service was still in place, so the service was used as a secondary source until the problem was fixed. This customer also required a fixed IP address which complicated the testing a bit. Six dispatches and three months later, we showed up.
Nearly everything was replaced by the first six technicians, but you can’t assume anything when troubleshooting. This is especially true when you are sent after others have tried to fix the problem.
We followed the same procedure we used for Trouble #1. The signal was consistently -52 dBm. The speeds were 110 MB down and 55 MB up. The NOC again could not help, so we called Fixed Wireless engineering. This tower was fed directly by fiber, so no microwave issue was possible. However, since this customer was the only customer on the tower, it could have been a dirty fiber connection. It was not. The light looked great, and the uplink was to spec.
The Fixed Wireless team had us replace the PoE with no change in performance. We were then asked to test the Cat 5 connections between the antenna and the PoE.
Our investigation uncovered that this installation was done by two technicians working together. The Cat 5 cable was new and was not stapled anywhere. Zip ties were used throughout the approximately 100 feet of cable. Testing on the antenna end was difficult because it was very high on the building above a wooden deck.
The connectors look good on both sides. Then we realized something.
The antenna side was wired Orange white – Orange – Blue white – Blue – Green white – Green – Brown white – Brown. The PoE side was wired Orange white – Orange – Green white – Blue – Blue white – Green – Brown white – Brown. Therefore, only 10 MB down and 10 MB up was possible. We replaced the connector on the antenna side and matched the wiring on the PoE side. The service immediately improved to 96 MB down and 9 MB up. Subsequent, contacts with the customer have shown no degradation. This service is now the primary Internet feed to the building with the old DSL set to be disconnected.
Replacing equipment for no logical reason as part of a troubleshooting strategy has been an epidemic since the advent of DSL. Not sure what to do next, replace the router.
What’s Old Is New Again
It is becoming obvious to me that the same basic troubleshooting strategy used for POTS, DSL and FTTP works for Fixed Wireless. You just need some basic understanding of the technology and equipment involved. Being able to log in to the antenna makes a huge difference and gives the field technician some baseline information that is not dependent on remote help.
I am NOT casting dispersions on the NOC personnel. I have been on that side of troubleshooting, and it is not easy. It is almost impossible when the personnel are not given the correct tools and/or training.
It is important to remember this when you are the technician sent to fix a problem no one else has been able to fix. So, act like you are the first one troubleshooting the problem and start from scratch. I have learned the hard way that even if Alexander Graham Bell himself was the last technician on-site, you should follow what your own testing and troubleshooting strategy tells you. The trouble is where you find it.
As always, reader feedback and content are the life’s blood of this column. Please share your experiences with the industry.
Network Maintenance Corner Columnist for ISE Magazine
Vernon May is the Chief Technologist and Founder of Vernon May Solutions. He is an expert in OSP and ISP Operations, and focuses on new technology introduction, from marketing and sales to design enhancement to training to product approval. Along with writing a column for ISE magazine, he also hosts seminars available throughout the country. For more information, call Vernon at 1.319.238.0285, email: [email protected], visit https://www.vmaysolutions.com/, and follow him on Twitter @Vernonmay13.
