Regular Internet outages when using bypassing Telus router

Reply
rf94z
Connector

Just wanted to get something on the forum that might be a benefit to others running into the situation I've run into with a number of Telus business and home clients. The problem occurs when you want to bypass the ActionTec router that Telus provides and use your own. The ActionTec can either be in bridge mode or removed from the circuit completely and firewall/router duties replaced by your own on-premises device. In our case, all the affected circuits had the ActionTec replaced by Sonicwall firewalls. The main symptom is that the connection appears to drop periodically and no traffic is received by your on-premises firewall from the Telus gateway. Monitoring the connection showed that the connection would go down every 37 minutes precisely from the time it was reset (a simple unplug of the Internet cable usually did the trick). If left on its own, the connection would resume 2 hours and 53 minutes later, and the cycle would continue. Telus support was of no help at all, unsurprisingly, but analyzing packet captures revealed unusual ARP (Address Resolution Protocol) behavior. Sonicwall support confirmed that the problem was a "non-standard implementation of ARP". When I contacted Telus, they eventually confirmed that ARP was disabled on the circuit and is disabled on every circuit that the ActionTec T3200H router is deployed on. After Telus enabled ARP on the circuit, the connection stabilized completely. ARP is an essential last hop protocol and should never be disabled but, for some reason, Telus chooses to disable it. Consequently, it causes drop-out problems in moderately more sophisticated home and business environments that need more functionality than the simple ActionTec modem provides. Note that Telus will enable ARP on a business service but will not on a residential service, so in that case you're stuck with the ActionTec modem. Normally this is not a problem, but if you need more sophisticated services such as a VPN, you're simply out of luck. The only solution for a home user is to upgrade your circuit to business and have Telus enable ARP. I suppose you could downgrade to residential without losing the ARP setting but haven't tried. Hope this saves someone from the hours of work I had to do to figure it out. Telus was of very little help, sadly but unsurprisingly.

BM1
Neighbour
Hey!

I’m having the same issues with the dropped connections. I just switched to Telus Business 1GB Fiber and the intermittent drops are driving me insane. To give you some background I’m using my own router (Ubiquiti Edge Router ER-4) to bypass the Telus router completely, this is for security and performance reasons. I’m plugged directly into the SFP port. Anyways, I followed your suggestion after spending hours explaining my issues to tech support, at first they had no clue what ARP was. They went ahead and enabled it (at least that’s what they tell me, I don't have much faith so far) and I’m still having issues!! Any suggestions other than to use the Telus router in bridge mode, because I will not use their hardware. Please let me know if you want to see my config files, I’m happy to post them if that will be useful.
rf94z
Connector

If using the ActionTec router in bridge mode works for you, I'd recommend you just go with it. The router is really just a dumb layer 2 device in that configuration. What sort of pattern are you seeing with dropouts? How long is the connection up before it drops? Have you run a packet capture on the circuit?

BM1
Neighbour
As I mentioned I can’t use it for security reasons. The connection is usually up for about two hours then drops for about 1-2 minutes. I’ll run one now, should I be looking for something in particular. It’s fine if your not sure, I’m just not able to use the Telus router.
rf94z
Connector

Look for a conversation between the Telus gateway and your device that might indicate a lack of response from one side or the other just before an outage. Not sure why you think there is a security issue with the ActionTec in bridge mode but if it's non-negotiable, just replace the ActionTec modem with your own small smart or dumb gigabit switch. I've seen an intermediate switch solve drop out issues on Telus circuits.

BM1
Neighbour
Thanks I’ll take a look. It’s not my choice to not use the Telus router, it’s my company policy. So as I mentioned twice before it’s not an option that I can use. Additionally, I know it’s possible without their router, I have seen it done before hence why posted on the forums. I made this post looking for a solution to my problem so thank you for the suggestion regarding the packets, I’ll check it out and see what I can find. Please don’t suggest I use their router when it’s not an option. In regards to your other suggestion, you recommend that I put a SFP switch before my Edge router?
BM1
Neighbour
Something like this?

TP-Link MC220L Gigabit Media Converter, 1000Mbps RJ45 to 1000Mbps SFP Slot Supporting MiniGBIC modules, Chassis mountable,Grey https://www.amazon.ca/dp/B003CFATL0/ref=cm_sw_r_cp_api_i_ZhcUEbZCB96Y0
rf94z
Connector

So you're trying to eliminate both the Telus ActionTec router and Lucent transceiver and go directly from the Ubiquiti router into their fibre network? Brave man. Totally unnecessary, but whatever. The TP-Link transceiver should work but you might want to try a different SFP module before you spend any money on something that may or may not work.

BM1
Neighbour
I’ll give both of those a shot and let you know what happens. I already ordered the TP-Link transceiver, Amazon should be delivering it tomorrow. Worst thing I return it, Amazon won’t give me any hassle.
NFtoBC
Community Power User
Community Power User
If it is company policy, aren’t they offering support towards a solution?
NFtoBC
If you find a post useful, please give the author a "Like"
Paul_Reid
Just Moved In

We're an IT company, and run into this issue all the time. It's maddening! We've fixed with a few times with MAC spoofing or having Telus add a static DHCP entry to their servers. In a few of those cases, neither of those worked, and Telus lost a customer over it.

 

Interesting about the ARP, and the small switch between Telus and the router ideas. We're currently fighting with a connection in Vernon, BC that the MAC spoof and static routes didn't help with. We'll have Telus enable ARP. We're putting in a small switch anyway, since the client wants to run a separate router for their automation and music stuff. Hopefully one of those does the trick, or Telus is going to lose another customer.

 

It's sad that Telus can't just follow standards. They're constantly screwing with stuff to force people to do things their way. The use of static DHCP entries for static IPs, for example. Why? Just give us static IP information, so we're not at the mercy of DHCP! Or the deliberate rejection of renewal of the current IP, forcing an IP change when renewing IPs - which is a violation of the DHCP standard - to make it difficult to run a server on a business connection without registering the MAC with them. That also messes with VPNs and other outgoing traffic that breaks when the IP changes. These things are just stupid and unnecessary.

 

Telus is so Mickey Mouse. Just consumer grade nonsense. If it wasn't for the fact that they're the only game in town for cheap fiber, I'd be all over other providers. I can't wait for other providers to be able to resell PureFiber. The second that happens, Telus is going to see a bunch of client migrate the next day.

 

rf94z
Connector

Telus does do things its own way, to the point of frustration. In their defense, they're deploying for the lowest common denominator and really aren't setup to handle more advanced small business or consumer setups, mainly by design. Keeps costs down and profits up.

Paul_Reid
Just Moved In

Just a quick update.

 

We finally got Telus' systems to be happy with the SonicWALL for our client.

 

Initially, we'd called Telus and had them enable ARP, which didn't help. We then put in a small HP OfficeConnect 5-Port switch between the Nokia and the SonicWALL. That didn't help either. We called Telus back, and got a different tech. It turned out the first guy didn't enable ARP, like he said he had. The second tech enabled it, and the connection has been stable ever since. ARP not being enabled was the issue.

 

ARP is required on a network for things to work, unless static ARP is being employed - which of course won't work if the router being communicated with hasn't been instructed to keep the ARP entry it makes for the SonicWALL. As soon as the dynamic ARP entry expires, traffic flow stops. This is just boneheaded on Telus' side. They do this seemingly randomly (though often) on the PureFiber network. There's just no reason to do this and generate support calls where the technician has no idea what the problem is, because they're not receiving training on a very common issue. That is very annoying.

 

At least we now know what the issue is, and can have Telus address it out of the gate from now on - so, thank you, rf94z. 🙂

rf94z
Connector

Glad it worked for you. Agreed, Telus support is pretty low grade, really hit and miss. For people like yourself that have a solid networking background, it's even worse because once you penetrate that first layer of support and get escalation, you hit a brick wall of bureaucracy where they're loathe to change anything despite the evidence you might have that Telus is the problem. Not my favourite provider for that reason but I do like the robustness of the service once it's working.

rf94z
Connector

Just went through another Telus Business Fibre deployment, bypassing the Telus router directly to a Sonicwall firewall. As part of the installation order, I requested that ARP be enabled on the circuit and was assured it was. Yet I had the same symptoms, ~37 minutes uptime followed by ~2 hours, 54 minutes downtime. Contacted Telus support five times, waited on hold probably 5 hours total. Each time I was told that ARP was enabled. It wasn't. I finally reached someone competent in the Fibre division who said it wasn't enabled and enabled it for me, which immediately fixed the problem. Of course, the process took two full days to play out. Bottom line: don't get fooled by Telus' protestations that ARP is enabled if you continue to see these symptoms. Insist that it isn't until you get someone competent who can fix it. Start by calling the Fibre support line at 888-310-2267. Refer them to this thread if you need to.

 

The bigger moral to all this is that, if you need support beyond the mundane level 1 issues, Telus will give you the worst technical support out there. Not just in the industry, but overall. I'd rather deal with the federal government. Bunch of employees who don't seem to want to help resolve your issue, just want to move on to the next easy call. Long holds, dropped calls, transferring around, finger-pointing, uneducated agents. If you have the opportunity to get a similar service from a competitive provider, do it, even if you have to pay more. Telus' glossy marketing veneer belies an incompetent beast below. They don't deserve your business.

dgris
Just Moved In

Props to you, rf94z, and to Paul Reid for taking the time to post this issue.

 

I had the exact same 37:00 up, 2:53:36 down problem with our Sonicwall router on a new fibre install.  Thanks to the information you guys posted I was able to point the Telus tech to the "enable the ARP port" solution and get it resolved quickly.  Fortunately a tier 2 tech answered the call had some past experience with similar issues - really cut down on the time spent finger pointing before we got down to having it fixed by tier 3.

 

One thing I'll add is that enabling the ARP port didn't immediately fix the connection and get me ping response.  They also had to power cycle the Nokia modem.

 

 

 

 

WatCan_Solutions
Just Moved In

Thank you for this extremely valuable info!! Are you taking the "naked" fibre SFP connection directly into the customer owned router (Sonicwall, in your case, I want to use Ubiquiti) or is it going to the Nokia equipment then ethernet to your Sonicwall?

rf94z
Connector

You may not have the problem with a Ubiquiti firewall. I've just used the Telus-provided Nokia fibre transceiver but there's no reason you can't eliminate it and use your own SFP port.

MB_YVR
Just Moved In

Thank you for this detailed post and thread. I am a residential fibre customer and have been suffering these disconnects for a couple of years now and finally was able to find a resolution once you pointed me towards the ARP issue.

 

The Sonicwall TZ400 has some "undocumented" options which include allowing for some less than secure behaviours. I enabled Open ARP Behaviour on the TZ400 that allows it to respond to ARP requests from 0.0.0.0 Depending on your use case this may or may not be a good idea. It's entirely possible that I do not fully understand the security implications of doing this either.

 

Further, I asked them for their Actiontec device but instead of putting it in bridge mode I just put it away and deployed its MAC address on the TZ-400 WAN interface. It's been over a week of a stable Fibre connection in over two years.

 

I did try using the Actiontec in bridge mode but it wasn't enough and I had to still configure the Sonicwall TZ400 to respond to ARP requests from 0.0.0.0 So I figured if that was the root cause then there was no need for this extra device. So I am back to SFP->Nokia ONT->TZ400->LAN etc