There’s this Polish Aero2 mobile network provider that offers a free, unmetered 512 kbps data plan, which is perfect for use as a backup link for your home router/server. And while my rather obscure, $10 Huawei USB LTE modem works out-of-the-box with Linux and NetworkManager, I struggled for quite a while to get it running under the FreeBSD-based OPNsense firewall. Turns out the modem’s peculiarities required a deep dive into tracing USB communication and its protocols to eventually get it initialized with a one-line command sending a raw USB message to device.
The LTE USB modem
A Huawei E3131 USB LTE modem
The LTE modem in question, the Huawei E3131, is a USB stick that was popular in the early to mid-2010s with local mobile operators in Europe and Asia. They can now be had for next to nothing and offer decent LTE speeds, especially considering that the free mobile service I mentioned is limited to just 512 kbps, perfectly fine for emergency purposes.
LTE modems typically offer two connectivity modes: a traditional, serial port-based PPP mode and a more modern NCM CDC Ethernet mode, which creates an Ethernet interface with DHCP support — marketed as “HiLink” by Huawei. The former is speed-limited, which is why manufacturers default to the second mode, as does NetworkManager under Linux when configuring the device. The modes exposed by the modem are changed using the AT^SETPORT command, which is well documented, so I’ll skip that part. For reference, I configured my modem as follows:
AT^SETPORT?
^SETPORT:;2,3,16
, which correspond to:
AT^SETPORT=?
^SETPORT:A1: CDROM
^SETPORT:A2: SD
^SETPORT:A: BLUE TOOTH
^SETPORT:B: FINGER PRINT
^SETPORT:D: MMS
^SETPORT:E: PC VOICE
^SETPORT:1: MODEM
^SETPORT:2: PCUI
^SETPORT:3: DIAG
^SETPORT:4: PCSC
^SETPORT:5: GPS
^SETPORT:6: GPS CONTROL
^SETPORT:16: NCM
Notable here is having serial-based PPP MODEM disabled, which I could not get working at all, not even under user-friendly Linux and NetworkManager combo.
The above results in following devices created under Linux:
[ 1334.805515] option 1-6:1.0: GSM modem (1-port) converter detected
[ 1334.807113] usb 1-6: GSM modem (1-port) converter now attached to ttyUSB0
[ 1334.807367] option 1-6:1.1: GSM modem (1-port) converter detected
[ 1334.807532] usb 1-6: GSM modem (1-port) converter now attached to ttyUSB1
[ 1334.811642] usbcore: registered new interface driver cdc_ether
[ 1334.813193] usbcore: registered new interface driver cdc_ncm
[ 1334.814079] usbcore: registered new interface driver cdc_wdm
[ 1334.892354] huawei_cdc_ncm 1-6:1.2: MAC-Address: aa:bb:cc:dd:ee:ff
[ 1334.892555] huawei_cdc_ncm 1-6:1.2: setting rx_max = 16384
[ 1334.895359] huawei_cdc_ncm 1-6:1.2: setting tx_max = 16384
[ 1334.898360] huawei_cdc_ncm 1-6:1.2: NDP will be placed at end of frame for this device.
[ 1334.898524] huawei_cdc_ncm 1-6:1.2: cdc-wdm0: USB WDM device
[ 1334.898766] huawei_cdc_ncm 1-6:1.2 wwan0: register 'huawei_cdc_ncm' at usb-0000:0c:00.0-6, Huawei CDC NCM device, 58:2c:80:13:92:63
[ 1334.898999] usbcore: registered new interface driver huawei_cdc_ncm
[ 1334.903013] huawei_cdc_ncm 1-6:1.2 wwx582c80139263: renamed from wwan0
The wwan0 Ethernet interface is provided by huawei_cdc_ncm kernel module, which notably also creates cdc-wdm character device under /dev/cdc-wdm0. This will get important later on, so bear with me.
The ttyUSB0 and ttyUSB1 GSM modem serial ports correspond to the DIAG and PCUI interfaces, the former of which supports AT communication:
root@proxmox:~ # cu -l /dev/ttyUSB0
AT
OK
Initializing connection
While Huawei typically uses some proprietary software for maintaining connectivity under Windows, presumably using the PCUI serial interface, it can also be used as a standard CDC NCM modem supporting the NDIS command set and can be initialized with a simple AT^NDISUP=1,1,"APN" command sent to one of its serial ports. This is evidenced by NetworkManager running in debug mode:
ModemManager[4118]: <debug> [1739448832.930575] [modem0] user request to connect modem
ModemManager[4118]: <info> [1739448832.933460] [modem0] simple connect started...
ModemManager[4118]: <debug> [1739448832.933510] [modem0] profile ID: unspecified
ModemManager[4118]: <debug> [1739448832.933521] [modem0] PIN: unspecified
ModemManager[4118]: <debug> [1739448832.933528] [modem0] operator ID: unspecified
ModemManager[4118]: <debug> [1739448832.933535] [modem0] allowed roaming: yes
ModemManager[4118]: <debug> [1739448832.933542] [modem0] APN: darmowy
ModemManager[4118]: <debug> [1739448832.933548] [modem0] APN type: unspecified
ModemManager[4118]: <debug> [1739448832.933561] [modem0] IP family: ipv4
ModemManager[4118]: <debug> [1739448832.933579] [modem0] allowed authentication: none, pap, chap, mschap, mschapv2, eap
ModemManager[4118]: <debug> [1739448832.933659] [modem0] user: unspecified
ModemManager[4118]: <debug> [1739448832.933669] [modem0] password: unspecified
ModemManager[4118]: <debug> [1739448832.933675] [modem0] multiplex: unspecified
ModemManager[4118]: <info> [1739448832.933682] [modem0] simple connect state (6/10): register
ModemManager[4118]: <debug> [1739448832.933699] [modem0] already registered automatically in network '26001', automatic registration not launched...
ModemManager[4118]: <info> [1739448832.933776] [modem0] simple connect state (7/10): wait to get packet service state attached
ModemManager[4118]: <info> [1739448832.933809] [modem0] simple connect state (8/10): bearer
ModemManager[4118]: <debug> [1739448832.933847] [modem0] creating new bearer...
ModemManager[4118]: <debug> [1739448832.933921] [modem0] (huawei) ^NDISDUP supported, creating huawei bearer...
ModemManager[4118]: <debug> [1739448832.934026] [cdc-wdm0/at] device open count is 2 (open)
ModemManager[4118]: <info> [1739448832.934207] [modem0] simple connect state (9/10): connect
ModemManager[4118]: <debug> [1739448832.934319] [modem0/bearer1] connecting...
ModemManager[4118]: <info> [1739448832.934348] [modem0] state changed (registered -> connecting)
ModemManager[4118]: <debug> [1739448832.934457] [modem0/bearer1] launching 3GPP connection attempt
ModemManager[4118]: <debug> [1739448832.934491] [cdc-wdm0/at] device open count is 3 (open)
ModemManager[4118]: <debug> [1739448832.934508] [cdc-wdm0/at] device open count is 2 (close)
ModemManager[4118]: <debug> [1739448832.934564] [cdc-wdm0/at] --> 'AT^NDISDUP=1,1,"darmowy"<CR><LF>'
ModemManager[4118]: <debug> [1739448832.941729] [cdc-wdm0/at] <-- '<CR><LF>OK<CR><LF>'
ModemManager[4118]: <debug> [1739448832.941823] [cdc-wdm0/at] device open count is 3 (open)
ModemManager[4118]: <debug> [1739448832.941866] [cdc-wdm0/at] device open count is 2 (close)
ModemManager[4118]: <debug> [1739448832.941883] [cdc-wdm0/at] --> 'AT^NDISSTATQRY?<CR><LF>'
ModemManager[4118]: <debug> [1739448832.945667] [cdc-wdm0/at] <-- '<CR><LF>^NDISSTATQRY: 0,,,"IPV4"<CR><LF><CR><LF>OK<CR><LF>'
ModemManager[4118]: <debug> [1739448832.945796] [cdc-wdm0/at] device open count is 1 (close)
ModemManager[4118]: <debug> [1739448833.964254] [cdc-wdm0/at] device open count is 2 (open)
ModemManager[4118]: <debug> [1739448833.964434] [cdc-wdm0/at] --> 'AT^NDISSTATQRY?<CR><LF>'
ModemManager[4118]: <debug> [1739448833.970162] [cdc-wdm0/at] <-- '<CR><LF>^NDISSTATQRY: 0,,,"IPV4"<CR><LF><CR><LF>OK<CR><LF>'
ModemManager[4118]: <debug> [1739448833.970508] [cdc-wdm0/at] device open count is 1 (close)
ModemManager[4118]: <debug> [1739448834.963393] [cdc-wdm0/at] device open count is 2 (open)
ModemManager[4118]: <debug> [1739448834.963704] [cdc-wdm0/at] --> 'AT^NDISSTATQRY?<CR><LF>'
ModemManager[4118]: <debug> [1739448834.968189] [cdc-wdm0/at] <-- '<CR><LF>^NDISSTATQRY: 0,,,"IPV4"<CR><LF><CR><LF>OK<CR><LF>'
ModemManager[4118]: <debug> [1739448834.968559] [cdc-wdm0/at] device open count is 1 (close)
ModemManager[4118]: <debug> [1739448835.323316] [cdc-wdm0/at] <-- '<CR><LF>^MODE: 5,9<CR><LF>'
ModemManager[4118]: <debug> [1739448835.323670] [modem0] access technology changed (umts -> hspa-plus)
ModemManager[4118]: <debug> [1739448835.328439] [cdc-wdm0/at] <-- '<CR><LF>^NDISSTAT:1,,,"IPV4"<CR><LF>'
ModemManager[4118]: <debug> [1739448835.329040] [modem0] (huawei) NDIS status: IPv4 connected, IPv6 not available
ModemManager[4118]: <debug> [1739448835.329219] [modem0/bearer0] (huawei) received spontaneous ^NDISSTAT (connected)
ModemManager[4118]: <debug> [1739448835.961357] [cdc-wdm0/at] device open count is 2 (open)
ModemManager[4118]: <debug> [1739448835.961734] [cdc-wdm0/at] --> 'AT^NDISSTATQRY?<CR><LF>'
ModemManager[4118]: <debug> [1739448835.966173] [cdc-wdm0/at] <-- '<CR><LF>^NDISSTATQRY: 1,,,"IPV4"<CR><LF><CR><LF>OK<CR><LF>'
ModemManager[4118]: <debug> [1739448835.966560] [cdc-wdm0/at] device open count is 3 (open)
ModemManager[4118]: <debug> [1739448835.966622] [cdc-wdm0/at] device open count is 2 (close)
ModemManager[4118]: <debug> [1739448835.966656] [cdc-wdm0/at] --> 'AT^DHCP?<CR><LF>'
ModemManager[4118]: <debug> [1739448835.972426] [cdc-wdm0/at] <-- '<CR><LF>^DHCP: 8EF45121,FCFFFFFF,8DF45364,8DF45364,FE7F02D4,346002D4,21600000,21600000<CR><LF><CR><LF>OK<CR><LF>'
ModemManager[4118]: <debug> [1739448835.972828] [modem0/wwx582c80139263/net] port now connected
ModemManager[4118]: <debug> [1739448835.972902] [modem0/bearer1] connected
ModemManager[4118]: <info> [1739448835.973091] [modem0] state changed (connecting -> connected)
The AT^NDISSTATQRY? commands, which follow the AT^NDISDUP, query the ECM/NDIS connection status. The eventual ^NDISSTAT:1,,,"IPV4" response signifies the modem has successfully established a connection and IPV4 address is available for DHCP to obtain on CDC Ethernet interface, while the ^MODE: 5,9 callback message informs that modem connected to LTE network and
Unfortunately, I spent days trying to replicate this without using NetworkManager automation, with bare network interface and AT communication only. My commands sent to ttyUSB0, while acknowledged by the modem exactly as when sent by NetworkManager in the excerpt above, were not getting the modem to connect, retuning ^NDISSTATQRY: 0,,,"IPV4" in perpetuity. I also tried enabling a regular MODEM functionality with AT^SETPORT and using its additional, dedicated serial port to initialize communication, but that also didn’t work.
The solution hint was in the debug log, after all: NDIS AT commands are actually sent to cdc-wdm0 interface, not the ttyUSBx, and simply issuing echo -e 'AT^NDISDUP=1,1,"APN"\r' > /dev/cdc-wdm0 made it connect within seconds. In my defence, this goes against all the instructions for the CDC modems out there which suggest sending AT commands to diagnostics/modem serial port. After digging a bit deeper I saw number of people reporting this particular modem having this quirk.
As a side note, in the process I also found out that either “APN”, “darmowy” or no APN at all (AT^NDISDUP=1,1) will get the modem connected.
The USB CDC WDM interface
WDM (WCM Device Management) provides an interface for controlling Wireless Mobile Communication (WCM) devices such as wireless modems and phones. Any such device connected to Linux box will have a /dev/cdc-wdmX character device created for it, which userspace software can then use to talk to using one of the protocols. Modern devices typically use dedicated protocols, like the Qualcomm’s proprietary QMI or the USB Implementers Forum’s standardized MBIM. Older models, like my Huawei E3131, rely on traditional AT commands — which is why sending a simple AT^NDISDUP command establishes a connection.
The problem now is that FreeBSD, which the OPNSense is based on, does not implement WCM Device Management driver so there’s no way to send the AT^NDISDUP as easily as under Linux. The only devices created there are the two serial ports (u3g0), which I already established cannot be used for that, and the ue0 Ethernet interface:
ugen8.2: <HUAWEI HUAWEI Mobile> at usbus8
u3g0 on uhub0
u3g0: <HUAWEI HUAWEI Mobile, class 0/0, rev 2.00/1.02, addr 1> on usbus8
u3g0: Found 2 ports.
cdce0 on uhub0
cdce0: <HUAWEI HUAWEI Mobile, class 0/0, rev 2.00/1.02, addr 1> on usbus8
cdce0: faking MAC address
ue0: <USB Ethernet> on cdce0
ue0: Ethernet address: gg:hh:ii:jj:kk:00
Sniffing up the USB communication
The next step was to see if I could replicate sending the AT command to cdc-wdm0 by addressing the USB device directly. To start with, I used an usbmon module under Linux to sniff on the communication on USB bus 1, where my modem sits (as inspected by lsusb -t):
root@proxmox:~# modprobe usbmon
root@proxmox:~# cat /sys/kernel/debug/usb/usbmon/1u
This opens the trace stream of URBs (USB Request Blocks) in the 1u format. Issuing an echo -e 'AT^NDISDUP=1,1,"APN"\r' > /dev/cdc-wdm0 will show in the trace as:
ffff88e146b706c0 1616543058 S Ii:1:020:3 -115:16 64 <
ffff88e146b70000 1616543085 S Co:1:020:0 s 21 00 0000 0002 0016 22 = 41545e4e 44495344 55503d31 2c312c22 41504e22 0d0a
ffff88e146b70000 1616545678 C Co:1:020:0 0 22 >
ffff88e146b706c0 1616545996 C Ii:1:020:3 -2:16 0
ffff88e146b706c0 1634336245 S Ii:1:020:3 -115:16 64 <
The highligted line of interest, which appears once the AT^NDISDUP command is sent to cdc-wdm0, breaks down to:
ffff88e146b70000: URB tag,1616543085: Timestamp in microseconds,S: Event Type: S - submission, C - callback, E - submission error,Co:1:020:0: “Address” word. Consists of four fields:Co: URB type and direction, in this case ‘Control output’,1: Bus number,020: Device address,0: Endpoint number,
s: a Setup Tag, expected here as this is a ‘Control’ message,21 00 0000 0002 0016: Setup packet, consisting of 5 hexadecimal words:21: bmRequestType.21is in the reserved range, and belongs to Microsoft’s RNDIS (Remote NDIS) specification,00: bRequest,0000: wValue,0002: wIndex,0016: length of the data transferred (number of bytes). 0x16 equals 22 decimal. This may be larger than the actual number of bytes transferred, but not smaller than.
22 =: the actual length of the data transferred, this time in decimal.41545e4e 44495344 55503d31 2c312c22 41504e22 0d0a: Data words in big endian hexadecimal format, i.e. the actual message content, which decodes toAT^NDISDUP=1,1,"APN"\r\n. Note the extra\nnewline feed character — presumably appended by theecho— which is why the length of the data is 22, not 21.
What’s interesting here is the endpoint number 0, sitting at interface 0, which is explicitly reserved for USB Control communication. Both are mandaded by the USB spec as the bare minimum of any USB device and as such are not reported by the lsusb -v, which otherwise lists all the interfaces and endpoints.
Replaying the command
All the information above provided by the usbmon is enough to replay the command onto the device directly. I used a pyusb library, itself a wrapper around libusb, in a simple script:
import usb.core
import usb.util
# Find the USB device
dev = usb.core.find(idVendor=0x12d1, idProduct=0x1506)
# Claim the interface 0 of device
usb.util.claim_interface(dev, 0)
# Prepare the AT command as a control transfer
command = 'AT^NDISDUP=1,1,"APN"\r\n'
command_bytes = command.encode('utf-8')
# Send the command using a control transfer
dev.ctrl_transfer(
bmRequestType=0x21,
bRequest=0x00,
wValue=0x0000,
wIndex=0x0002,
data_or_wLength=command_bytes
)
# Release the interface
usb.util.release_interface(dev, 0)
usb.util.dispose_resources(dev)
Linux failure
I first tried running the script under Linux to reduce number of variables. Before I could, though, the device needed to be unbound from the drivers in use, otherwise it resulted in usb.core.USBError: [Errno 16] Resource busy:
echo -n "1-6:1.1" > /sys/bus/usb/drivers/option/unbind
echo -n "1-6:1.2" > /sys/bus/usb/drivers/huawei_cdc_ncm/unbind
With that done, the command got issued to the device and appeared in usbmon trace:
ffff8f9cca7b7680 1202174990 C Co:1:012:0 0 0
ffff8f9cca920b40 1206191657 S Co:1:012:0 s 21 00 0000 0002 0016 22 = 41545e4e 44495344 55503d31 2c312c22 41504e22 0d0a
ffff8f9cca920b40 1206194025 C Co:1:012:0 0 22 >
The control command itself is identical as if sent to /dev/cdc-wdm character device. Unfortunately, after re-binding the devices to their respective modules, I could not obtain a DHCP addres. I suspect this is because re-binding to modules has them re-do the device initialization, as evidenced by substantial chatter on usbmon upon rebinding. If anyone has any idea how to get the command sent under Linux without unbinding the device, feel free to leave a commonet.
FreeBSD success
Running the above script under OPNSense would likely yield a similar result, but I couldn’t even confirm this as the default installation does not come with py311-pyusb package. Sideloding it from the FreeBSD’s repositories (which OPNSense is based on) proved annoyingly problematic and time-consuming.
However, turns out FreeBSD comes with the usbconfig tool, capable of performing a synchronous control request on a given device without unbinding it from the modules:
root@gw:~ # usbconfig -h
usbconfig - configure the USB subsystem
usage: usbconfig -d [ugen]<busnum>.<devaddr> [-i <ifaceindex>] [-v] [cmds...]
commands:
(...)
do_request <bmReqTyp> <bReq> <wVal> <wIdx> <wLen> <data...>
All the fields are pretty self-explenatory in light of the previous findings, except maybe for the scarce documentation on how to pass the data — and the ppular AI agents will totally throw you under a bus suggesting to use plain ASCII string:
root@gw:~ # usbconfig -d 8.2 -i 0 do_request 0x21 0 0 2 16 'AT^NDISDUP=1,1\r\n'
usbconfig: request data missing: No error: 0
The resulting error message is fairly ambigous, but cross-checking using a usbdump -d 8.2 -s 256 -vv, more or less equivlent of Linux’s usbmon, would confirm that no request was being sent.
Looking into the sourcecode, it was clear the tool expects the data to be provided as separate bytes in a format readable by strtoul C function, so the correct syntax is actually:
root@gw:~ # usbconfig -d 8.2 -i 0 do_request 0x21 0 0 2 16 0x41 0x54 0x5e 0x4e 0x44 0x49 0x53 0x44 0x55 0x50 0x3d 0x31 0x2c 0x31 0x0d 0x0a
REQUEST = <OK>
The usbdump output reflects this as:
C23:15:56.508476 usbus8.2 SUBM-CTRL-EP=00000000,SPD=HIGH,NFR=2,SLEN=24,IVAL=0
frame[0] WRITE 8 bytes
0000 21 00 00 00 02 00 10 00 -- -- -- -- -- -- -- -- |!....... |
frame[1] WRITE 16 bytes
0000 41 54 5E 4E 44 49 53 44 55 50 3D 31 2C 31 0D 0A |AT^NDISDUP=1,1..|
flags 0x2 <SHORT_XFER_OK|0>
status 0xca1a3 <OPEN|TRANSFERRING|STARTED|CONTROL_XFR|CONTROL_HDR|BDMA_ENABLE|BDMA_SETUP|CAN_CANCEL_IMMED|DOING_CALLBACK|0>
23:15:56.510703 usbus8.2 DONE-INTR-EP=00000083,SPD=HIGH,NFR=1,SLEN=8,IVAL=2,ERR=0
frame[0] READ 8 bytes
0000 A1 01 00 00 02 00 00 00 -- -- -- -- -- -- -- -- |........ |
flags 0x8a <SHORT_XFER_OK|PIPE_BOF|NO_PIPE_OK|0>
status 0xeb021 <OPEN|STARTED|SHORT_XFER_OK|BDMA_ENABLE|BDMA_SETUP|CURR_DMA_SET|CAN_CANCEL_IMMED|DOING_CALLBACK|0>
23:15:56.510706 usbus8.2 SUBM-INTR-EP=00000083,SPD=HIGH,NFR=1,SLEN=0,IVAL=2
frame[0] READ 32 bytes
flags 0x8a <SHORT_XFER_OK|PIPE_BOF|NO_PIPE_OK|0>
status 0xcb023 <OPEN|TRANSFERRING|STARTED|SHORT_XFER_OK|BDMA_ENABLE|BDMA_SETUP|CAN_CANCEL_IMMED|DOING_CALLBACK|0>
23:15:56.511120 usbus8.2 DONE-CTRL-EP=00000000,SPD=HIGH,NFR=2,SLEN=0,IVAL=0,ERR=0
frame[0] WRITE 8 bytes
frame[1] WRITE 16 bytes
flags 0x2 <SHORT_XFER_OK|0>
status 0xca1a1 <OPEN|STARTED|CONTROL_XFR|CONTROL_HDR|BDMA_ENABLE|BDMA_SETUP|CAN_CANCEL_IMMED|DOING_CALLBACK|0>
This made the modem establish communication, allowing DHCP to obtain the addresses:
root@gw:~ # dhclient ue0
DHCPREQUEST on ue0 to 255.255.255.255 port 67
DHCPACK from 100.xx.yyy.zz
bound to 100.xx.yyy.zz -- renewal in 259200 seconds.
Discussions around the web
Article is not submitted to any of Reddit, Lobste.rs or Hacker News.
Powered by discu.eu