09 June 2015

612. Randomly Rebooting Router (E2500-AU v1.0 w/ TomatoUSB)

Rolling update:
* 24 June 2015: 7 days uptime with wifi working perfectly. Did reboot it last night because my work computer lost contact with the router somehow (connects via reverse tunnel). The issue with the Randomly Rebooting Router can be considered solved. Obviously, it's solved by crippling the router by turning off the 5 GHz band and tkip (the latter may not be related though).

* Submitted a bug report: http://tomato.groov.pl/?page_id=334&bugerator_nav=display&bug_project=1&issue=1833

* 16 June 2015 12:42 AEST.  The router has been up for two days and four hours (and counting) in spite of heavy use of our phones. Seems like turning off 5 GHz and/or switching from AES to TKIP has worked. A fair criticism is that I don't have much of a baseline to compare with when it comes to reboots, but subjectively there's a lot less swearing over crappy wifi the past two days.

* 14 June 2015 08:05 AEST. After two days of uptime when radio-silence was enforced, we turned our phones back on. The router rebooted later that night. Same thing happened the next night. After briefly putting dd-wrt on the router, I put tomatousb back on it, turned off 5 GHz and changed from AES to TKIP. The router has been up since 9.30 pm last night (10 h and counting)

Found this bug report: http://tomato.groov.pl/?page_id=334&bugerator_nav=display&bug_project=1&issue=1813

Also read this with interest: http://movingpackets.net/2013/11/18/linksys-e2500-deserves-no-airplay/

I've seen posts that find that dd-wrt doesn't have the randomly rebooting issue. dd-wrt doesn't support dual band, at least on e2500. I was surprised that v1 of the cisco linksys firmware had the same exact issue (random reboots when 5 GHz is on). It's all pointing in a specific direction.

Not sure why using the 5 GHz channel with my laptop doesn't trigger the reboots, but maybe they did -- but happened less frequently due to the lower number of 5 GHz capable devices prior to us getting the phones.

* 10 June 2015 16:24 AEST. Since turning off wifi on the Samsung Galaxy S4 phones (but using the two laptops and the tablet listed below) the router has stayed up for 24 hours 8 hours and 11 minutes, and counting. The night between Monday and Tuesday, when we were using our phones, the router rebooted at least twice.


This is another one of those posts that don't offer a solution, but rather states a problem. I'm doing this in the hope that others who are making similar observations as I am will see this and...well, feel slightly less alone at the very least. In the best case, someone will have a solution and offer it as a comment.

So, here's the issue: 
* I have a Linksys E2500-AU v1.0 that is running TomatoUSB (howto)
Tomato v1.28.0000 MIPSR2-128 K26 USB Max ======================================================== Welcome to the Linksys E2500 v1.0 [TomatoUSB] Uptime: 08:14:26 up 1 min Load average: 0.52, 0.18, 0.06 Mem usage: 28.4% (used 17.06 of 59.96 MB) WAN : 192.168.1.100/24 @ 58:6D:8F:D3:XX:XX LAN : 192.168.2.1/24 @ DHCP: 192.168.2.2 - 192.168.2.202 WL0 : volatile @ channel: AU13 @ 58:6D:8F:D3:XX:XX WL1 : volatile50 @ channel: AU153 @ 00:01:36:1F:XX:XX ========================================================
* It has a "Broadcom BCM5357 chip rev 2 pkg 8"

* For a long time it, and its predecessor (a WRT-54GL), were running just fine. The predecessor got replaced due to a fried power supply.

* Over the past six-seven months there have been issues with the wireless signal dropping. It isn't just the wireless transmission being stopped and restarted, but the router actually reboots (according to uptime).

* We used to have the following wireless devices: Fujitsu lifebook (v100?), Thinkpad SL410, Google Nexus One and a HTC Legend. At some point we also got a Samsung Galaxy Tab 2. This configuration was running for a few years.

* Coinciding roughly with the perceived start of the rebooting issue was me purchasing a Samsung Galaxy S4 (i9505).

* The issue got a lot worse recently.

* Recently my partner also got a Samsung Galaxy S4 (i9505).

* I have an almost identical router (v2) at work, and the current uptime is 140 days. I do connect very occasionally via wireless to it using my Samsung Galaxy S4. However, this router has a "Broadcom BCM5357 chip rev 1 pkg 8".

What seems to be happening:
The Samsung Galaxy S4 phones seem to be destabilising the router and causing reboots. No, wait, hear me out. It shouldn't happen, and the adage about 'correlation vs causation' may well be true in this case too, but there are precendents (apparently) when it comes to Intel wireless devices:

On http://www.linksysinfo.org/index.php?threads/tomatousb-keeps-resetting-the-router.33208/ from 2010
Hrm...routers used to spontaneously reboot when the wireless driver failed on Tomato as a result of an Intel (mobile) wireless driver bug on Windows. Maybe similar?
On http://www.linksysinfo.org/index.php?threads/random-reboots.21020/ from 2007
Currently using DD-wrt V24, it's been up for 25 days I can confirm it has something to do with the broadcom wireless drivers.

and
What kind of wireless device does your laptop have? Is it Intel 2100/2200 by any chance?
And in the end the thread concludes that it was due to users with Intel 2100/2200 cards.

The Samsung Galaxy S4 has a Qualcomm Snapdragon 600 APQ8064AB, with is a system-on-a-chip. The Nexus One and HTC Legend also had snapdragons, but obviously much older models. The Galaxy Tab 2 seems to have a Texas Instrument chip (TI OMAP 4430).

Could it be that the phones are causing the issues?

Luckily it's something that's reasonably easy to test, so I'm looking forward to reporting back in a couple of days (of enforced radio silence).

A different test will be to swap routers (but not power supplies) between work and home and see if the behaviour is location dependent. That will take a lot more effort though due to the very specific set-ups.

As the logs get erased on reboot I'm tracking the uptime from now on using autossh and logging from a work computer that's always on.


Some more:
Below is a post regarding iphones, and rebooting routers.

While that post is not related to 5GHz causing issues, it's got me thinking that as neither the Fujitsu, Galaxy Tab, Nexus One or HTC legend support 5 GHz but the Samsung Galaxy S4 phones do, the issue may be possibly related to that. There is obviously quite a lot of things to test.
ADDITIONAL INFORMATION: in the mean time we have been checking and elimination as well. We have been trying to connect certain wireless devices to the network through the DAP and something odd has come up. We have been trying mobile phones (smartphones) at first. My own telephone (Samsung Galaxy S3) seems to cause no troubles. With that phone connected for a whole day, internet connection does not fail once (the router does not reboot). I have been trying both 5Ghz and 2.4 Ghz. bands, both worked okay. One colleague also wanted to connect his Apple Iphone 3G (S?) to the network. I told him he could but this phone could not find the 5Ghz network, so I have switched back to 2.4Ghz again. The iPhone connected and within 5 minutes the connection interrupted. I have set the network back to 5Ghz (so the iPhone could no longer connect) and changed the network settings again. This morning I switched back to 2.4 with only my phone connected. Not a problem. This afternoon I let my colleague connect his phone again and disconnected my Samsung. Within 5 minutes the router started to reboot! After I had the iPhone disconnected again and let another colleague connect his phone, a Samsung Galaxy S(1). So far no problems.

Tomato Anon
Somehow the Spontaneously Rebooting Router doesn't show up here, while the stable one does: http://anon.groov.pl/index.php?country=Australia

Either way, the anon database is a great way of quickly finding out what Tomato version you can put on your router.

4 comments:

  1. Hello and thanks for all your posts. I'm having the same problem and I just turned off 5ghz to test your solution. I also reported the bug to shibby.

    It's a shame it's just bougth router. I had one e900 that cost half of e2500 and decided for the last only for 5ghz capabilites and tomato support.

    ReplyDelete
    Replies
    1. Thanks for the feedback.

      One thing that I haven't added above is that because I got a bit annoyed with the sshd going down (I have a reverse tunnel set up and I find myself having to do service sshd restart every now and again) I ended up flashing the router with the latest linksys firmware.
      After an evening of dropped PVN connections I quickly flashed tomato back on and my PVN connections were much more stable (still a bit crap because of Cisco Anyconnect being generally unreliable on linux).
      Either way, my wireless has been rock solid since I turned off 5 GHz.

      Delete
  2. Hello again.

    It's not random anymore. See my post on shibby's site.

    http://tomato.groov.pl/?page_id=334&bugerator_nav=display&bug_project=1&issue=1848

    ReplyDelete
    Replies
    1. Thanks for the info and for working out a reproducible way of triggering a reboot.
      In my case, we've never moved outside the router range (house is too small), so not sure how it applies. Could be that when we turn 'off' the phones they kill the wifi immediately.

      Delete