For several years, I had 2x Unifi U6 Lite Access Points and they worked great. I had a special Wi-Fi network for my phones and laptops with a number of settings enabled but then I upgrade to the U7 Lite and immediately started having issues where my phone would disconnect. I got frustrated enough to break out my handy tool box to figure what was going wrong.

As it turns out, proxy ARP was breaking DNS traffic. It was able to send DNS queries to my DNS server, but responses were getting stuck because it wasn’t able to send the response back.

ARP is necessary to translate an IP address into the Ethernet MAC Address that switches actually use to forward packet when on the same LAN.

First Attempt using airomon-ng

I first attempted to packet capture the wireless packets between my phone and access point to see what is actually causing the phone to think it’s disconnected. Using my linux laptop, I used airmon-ng to listen on my network channel.

1
2
3

airmon-ng check kill
airmon-ng start wlp1s0
airodump-ng wlp1s0mon --channel 155 --band a -w output.pcap

I had some difficulty figuring out what channel to use for this given it was an 80Mhz channel width and airodump only seemed to support up to a 40Mhz channel width. This command was able to capture some packets.

Decrypting WPA3

Now that I have a Wireshark packet capture, I tried to decrypt it in Wireshark. Wireshark supports decryption if you have the encryption key, but WPA3 makes this more difficult because each device has it’s own encryption key so you need the “Pairwise Master Key (PMK)” from either the AP or the device. It’s not easy to get it out of Android, but luckily my AP has SSH access.

I SSH’d into the AP and ran:

1
2
3

ps | grep hostapd

kill {pid} && hostapd -S -g /var/run/hostapd/global -P /var/run/host -d K

I reconnected my phone to Wi-Fi. At some point the logs contain this:

1

WPA: PMK - hexdump(len=32): bb 6e 4a 93 7b 2e e2 29 8b d3 29 24 02 6e b0 0e f4 d5 99 d6 f6 d2 06 60 a1 e1 5f 53 01 2c 37 17

To load this into Wireshark, go to Edit > Preferences > Protocols > IEEE 802.11 > Decryption Keys Edit. Add a WPA-PSK:

1

bb6e4a937b2ee2298bd32924026eb00ef4d599d6f6d20660a1e15f53012c3717

In theory, Wireshark should now be able to decrypt the Wi-Fi packets. However, what I noticed is that airodump-ng did not seem to dump all the data packets, possible due to a limitation in my Intel Wi-Fi card.

A second look at the HostAPd logs

I went back to log at the logs on the access point and just looked at the logs being generated:

1

tail -f /var/log/messages

I saw something interesting every time the phone would report the Wi-Fi doesn’t work. It was sending DNS queries to my pi-hole server and the AP was detecting that there was no response back.

1
2
3
4
5
6

kern.warn kernel: [STA_TRACKER] DNS request timed out; [STA: 12:f3:85:dd:92:95][QUERY: play.googleapis.com.] [DNS_SERVER :192.168.6.2] [TXN_ID 81e7] [SRCPORT 8571]
kern.warn kernel: [STA_TRACKER] DNS request timed out; [STA: 12:f3:85:dd:92:95][QUERY: play.googleapis.com.] [DNS_SERVER :192.168.6.2] [TXN_ID 81e7] [SRCPORT 8571]
kern.warn kernel: [STA_TRACKER] DNS request timed out; [STA: 12:f3:85:dd:92:95][QUERY: connectivitycheck.gstatic.com.] [DNS_SERVER :192.168.6.2] [TXN_ID 65e5] [SRCPORT 63180]
kern.warn kernel: [STA_TRACKER] DNS request timed out; [STA: 12:f3:85:dd:92:95][QUERY: connectivitycheck.gstatic.com.] [DNS_SERVER :192.168.6.2] [TXN_ID 65e5] [SRCPORT 63180]
kern.warn kernel: [STA_TRACKER] DNS request timed out; [STA: 12:f3:85:dd:92:95][QUERY: connectivitycheck.gstatic.com.] [DNS_SERVER :192.168.6.2] [TXN_ID b304] [SRCPORT 55085]
daemon.info stahtd: stahtd[28828]: [STA-TRACKER].stahtd_dump_event(): {"message_type":"STA_ASSOC_TRACKER","mac":"ab:cd:ef:ab:cd:ef","vap":"ath4","event_type":"dns timeout","assoc_status":"0","query_0":"play.googleapis.com.","query_server_0":"192.168.6.2","query_1":"www.google.com.","query_server_1":"192.168.6.2","query_2":"play.googleapis.com.","query_server_2":"192.168.6.2","query_3":"www.google.com.","query_server_3":"192.168.6.2"}

Is it DNS?

DNS is looking awfully suspicious right now and it is frequently blamed for network issues. However, if I check Pi-Hole (running at 192.168.6.2 mentioned above), it shows that it successfully resolved queries in the query log and sent a response.

tcpdump everything

Time to start packet capturing everything. My network looks like this:

DNS queries are sent from my phone through the AP through the router to the Pi-hole. My pi-hole is being via out of Kubernetes, but the responses go directly to the sender because my server is on two networks. It’s complicated, but it does not break any Ethernet standards.

I ran tcpdump on the Wi-Fi access point, on the router, and on the server. All three points show the DNS queries going to the server, but no responses back. The access point shows this cpature:

1
2
3
4
5
6
7
8

13:54:11.331369 ath4  P   IP 192.168.2.62.32550 > 192.168.6.2.53: 12773+ AAAA? example.com. (35)
13:54:11.331918 eth0  B   ARP, Request who-has 192.168.2.62 tell 192.168.2.3, length 46
13:54:11.331953 ath4  Out ARP, Request who-has 192.168.2.62 tell 192.168.2.3, length 46
13:54:11.331953 ath4  P   IP 192.168.2.62.1558 > 192.168.6.2.53: 41230+ A? example.com. (35)
13:54:11.331986 ath2  Out ARP, Request who-has 192.168.2.62 tell 192.168.2.3, length 46
13:54:11.331993 eth0  Out IP 192.168.2.62.1558 > 192.168.6.2.53: 41230+ A? example.com. (35)
13:54:11.332003 ath0  Out ARP, Request who-has 192.168.2.62 tell 192.168.2.3, length 46
13:54:11.332020 br0   B   ARP, Request who-has 192.168.2.62 tell 192.168.2.3, length 46

The server is trying to figure out how to send the response back to the client by performing an ARP Request to say where is this IP address? The switch knows where the MAC address is and forwards the packet to the AP. The AP seems like it sends it to the phone, but the phone either doesn’t respond or doesn’t receive it even though the packet capture shows it going out the ath4 interface which is the Wi-Fi adapter.

Proxy ARP

That gave me the idea to check the Wi-Fi network settings. Unifi exposes a number of different options and I already turned off the more problematic settings to see, but didn’t turn off Proxy ARP.

Unifi says Proxy ARP Reduces airtime usage by allowing APs to “proxy common broadcast frames as unicast. This can improve latency, but may cause connectivity issues in some networks.

Clearly, the “may cause connectivity issues” is right. This is unusual because I had the setting on for all my U6 Lites without any issue, but as soon as I get a U7 Lite, it breaks.

It’s now been over a week after disabling this option without any issues with my phone now. For reference, I’m running firmware v8.0.49 in case they fix it later.

Comments

To give feedback, send an email to adam [at] this website url.

Donate

If you've found these posts helpful and would like to support this work directly, your contribution would be appreciated and enable me to dedicate more time to creating future posts. Thank you for joining me!