Another OpenBTS YouTube Video – this time from Dario Igor Flores Fernandez. I’m not sure why the duplexer is required. Also I guess the dog is sensitive to feedback … I am!
A handy reference. UmTRXv2 RF Power Output Pout typically 20dBm / 100mW.
I had nearly forgotten about those Bitcoins I bought nearly two years ago:
UPDATE 29/03/2013: SMS delivery issues resolved with a workaround.
We were recently looking into a bug when MT-SMS delivery was interrupted with an unexpected UA frame on L2 layer. We’ve fixed this in our branch by ignoring a second UA in Established L2 state , but the real issue lies in the filler table coupled with delays in processing in higher layers.
If you are new to Genesys then this video is a good place to start.
I’ve been wearing a Pebble (http://getpebble.com/) “electronic tag” (keep reading) for the last month and thought it was about time to blog about my experiences to date.
In general I am very happy with the Pebble.
Battery life is good and I only have to charge once or twice a week. The lack of a visible battery indicator on the watchface or a low battery notification is a bit of an oversight.
Music control on iOS devices is pants! At best it is only of any use for pause and play.
Vibrating alerts are good but sometimes it feels as though you are getting an electric shock (hence “electronic tag” comment). I’m quite glad that email notifications do not seem to be working on iOS correctly since constant email notifications would be a pain – especially if you wear your watch at night like I do.
Male early adopters need to be aware of the wet shoes hazard. Basically receipt of a phone call or text message and the associated vibrating alert shock factor during a visit to the restroom can take you by surprise!
Finally on iOS the constant “Pebble would like to communicate …” popups are driving me mad.
I’m off to Dubai in a few weeks to lets see how the Pebble holds up to sun and water.
Having moved OpenBTS on to a new dedicated server I am pleased to report that all is well again!
I picked up a cheap (£25) HP Compaq DC7100 on eBay and installed Ubuntu 12.04. The build of OpenBTS and UHD went like a dream unlike on Centos 5 where it took ages to sort out the build dependencies.
Now I have no frame errors:
RSSI (Received Signal Strength Indication) figures all look good. I ran the “noise” command in the OpenBTS CLI several times to get the worst (=biggest) value of noise RSSI. It should be a negative value, like -44dB.
The MS RSSI Target should be higher than “noise” by a healthy number usually something like 4-6dB margin. Hence we need to add 4-6dB to the noise RSSI value to get the RSSI level we need to receive phone signal on the BTS without errors. This is called “target RSSI” in OpenBTS and is set with “GSM.Radio.RSSITarget” value in the config. If noise RSSI is -16dB, then you would set target RSSI to -10dB:
OpenBTS> config GSM.Radio.RSSITarget -10
I also ran the “chans” command in the OpenBTS and used the active channel between the mobile station (MS) and the base station (BS) to estimate link quality and tune Rx gain and Tx attenuation.
The easiest way to estimate reception quality is to use the “UPFER” value. Zero “UPFER” means good reception, non-zero “UPFER” means errors on UL channel:
A Wireshark capture shows lots of interesting GSMTAP packets captured (analysis of which I will save for another blog post):
Finally an unexpected surprise – an iPhone 5 registered successfully!
Strangely even though I am running an unencrypted GSM network (since encryption is only supported in the commercial “C” release of OpenBTS and it is a pain to configure GSM A5/1 encryption, obtaining SIM cards with a known authentication key / Ki etc. etc.) I did not see the “Unsecured Call” warning introduced in IOS 5 which I had been expecting:
UPDATE: Fairwaves implemented encryption in the open-source version of OpenBTS and it is maintained in a special branch. It has issues with some phones and thus it has not been merged into Fairwaves master branch yet.
It turns out that even if the GSM handset supports indications of the ciphering status of a call, these indications can be disabled by the carrier through settings in the SIM. Most carriers choose to do disable this indication feature. Hence even though the indications sound good in principle, they are often useless in practice!
Happy days. I’ll put a full demo on YouTube soon.
I added a USB 2.0 to 10/100/1000 Gigabit Ethernet LAN Wired Network Adapter based on ASIX AX88178 Chipset to my OpenBTS Server, configured it as eth0 with static IP address 192.168.1.9 and configured UmTRX with IP address 192.168.1.10.
To force the new NIC to be used (since the old NIC is on the same 192.168.1.0/24 subnet) I needed to add a static route: route add -host 192.168.1.10 dev eth0
After doing that guess what – the OpenBTS server and UmTRX can no longer talk (ping fails)! Looks like an ARP problem …
ARP is used to locate the Ethernet address associated with a desired IP address. When a machine has a packet bound for another IP on a locally connected Ethernet network, it will send a broadcast Ethernet frame containing an ARP request onto the Ethernet. All machines with the same Ethernet broadcast address will receive this packet. If a machine (such as UmTRX) receives the ARP request and it hosts the IP requested, it will respond with the link layer address on which it will receive packets for that IP address. Once the requestor receives the response packet, it associates the MAC address and the IP address. This information is stored in the ARP cache. In simplest terms, the ARP cache is a stored mapping of IP addresses with link layer addresses. An ARP cache obviates the need for an ARP request/reply conversation for each IP packet exchanged.
Running “arp -n” shows that the ARP cache entry state for 10.168.1.10 is “incomplete” which means we did not get an ARP reply to the broadcast ARP “Who is …”:
However through the existing eth1 with static IP address 192.168.1.11 all is OK:
Strangely when the UmTRX is power cycled I do see Gratuitous ARP reply frames on both eth0 and eth1 which are ARP replies to a question never asked. This sort of ARP is common in failover solutions and also for nefarious sorts of purposes:
Adding a static ARP entry did not seem to help at all:
arp -i eth0 192.168.1.10 00:1f:11:02:19:0d
When a Linux box is connected to a network segment with multiple network cards, a potential problem with the link layer address to IP address mapping can occur. The Linux server may respond to ARP requests from both Ethernet interfaces. On the machine creating the ARP request, these multiple answers can cause confusion, or worse yet, non-deterministic population of the ARP cache. Known as ARP flux and can lead to the possibly puzzling effect that an IP migrates non-deterministically through multiple link layer addresses.
In general, the arp_filter solution sufficiently solves the ARP flux problem. First, hosts do not generate ARP requests for networks to which they do not have a direct route and second, when such a route exists, the host normally chooses a source address in the same network as the destination.
sysctl -w net.ipv4.conf.default.arp_filter=1
sysctl -w net.ipv4.conf.all.arp_filter=1
sysctl -w net.ipv4.conf.eth0.arp_filter=1
sysctl -w net.ipv4.conf.eth1.arp_filter=1
Made no difference! At this point I’m running out of steam so back to basics ….
Changed the Gigabit switch – no difference.
Took eth1 (original NIC) down – no difference.
Is this an issue with the ASIX AX88178 Chipset or Driver or possibly the UmTRX. This is more likely to be an UmTRX issue since it works on eth1 but not on eth0! We know that the ARP “Who is …” gets sent. Does the UmTRX receive it? Does the UmTRX discard it? Does the UmTRX send and “is at” reply which is not received due to a framing error ….
Clear the ARP cache:
arp -d 192.168.1.10
eth0 (not OK):
So nothing much different about the request frame … I’m confused ……
Check the source code from https://github.com/fairwaves/UHD-Fairwaves/blob/fairwaves/umtrx/firmware/zpu/lib/net_common.c
Seems to be nothing wrong here. More testing …
eth0 statistics before test and after test:
After a bit eth0 seems to stop sending any packets regardless of what Wireshark says! Hence probably a dongle issue caused by interrupt conflicts!
chkconfig –level 2345 acpid off
You can also disable ACPI completely by appending acpi=off to the kernel boot command line in the grub.conf file. This seems to kill USB keyboard and mouse!!!!
I give up – time to build a new OpenBTS using proper hardware and running Ubuntu!
After my initial success getting OpenBTS up and running with a UmTRXv2 hardware Transceiver I seem to have gone backwards while trying to optimise my data network with a bit of Linux tuning and swapping out network hubs and cables!
The OpenBTS transceiver process communicates with UmTRXv2 using UDP over a LAN connection and hence network performance is vital in terms of GSM network stability. One of the most common causes of lost UDP datagrams is an undersized receive buffer on the socket. Hence I implemented some standard Linux tuning of buffers and made the changes permanent in /etc/sysctl.conf:
sysctl -w net.core.rmem_max=50000000
sysctl -w net.core.rmem_max=50000000
sysctl -w net.core.wmem_max=524288
sysctl -w net.core.wmem_max=524288
The Ethernet Chip in UmTRXv2 is LSI 1GbE PHY (ET1011C2-CI-DT) which supports 10Mbps/100Mbps/1Gbps. However, the Ettus UHD framework adopted for the UmTRX only supports GigE(1Gbps) although this may be changing. Hence the UmTRX needs to be connected to a switch that supports 1000baseT.
For my OpenBTS server I am using a Shuttle XS35-703 V2 in which the Ethernet Chip is a JMicron JMC250 and *should* support 1000baseT. After a kernel update and much playing about with JME drivers I could find no way of getting a 1000baseT link established. I even tried with auto negotiation off:
ethtool -s eth1 speed 1000 duplex full autoneg off
After much digging I found out the following: “There are two known 1000baseT link establishment issues with JMC25x. If the full mask revision number of JMC25x controller is less than or equal to 4 and the link partner enabled the IEEE 802.3az Energy Efficient Ethernet feature, the controller will not be able to establish a 1000baseT link. The known workaround for these issues is to force manual link configuration with 100baseTX instead of relying on auto-negotiation. The full mask revision number of controller can be checked with the verbose kernel boot option. Use the lower nibble of the chip revision number to get the full mask revision of the controller.”
And guess what – my Shuttle has a version 3 chip:
>> jme 0000:02:00.5: eth0: JMC250 Gigabit Ethernet chipver:23 pcirev:3
Time to give up (for now) and workout what has changed to the effect that I could not longer get the OpenBTS transceiver to start reliably and except for some short lived success straight after a UmTRX power cycle always got one of two errors:
terminate called after throwing an instance of ‘uhd::runtime_error’
what(): RuntimeError: no control response
EMERG 1101519168 OpenBTS.cpp:134:startTransceiver: Transceiver quit with status
EMERG 1104513344 OpenBTS.cpp:134:startTransceiver: Transceiver quit with status 256. Exiting.
Running ./transceiver on its own always worked fine. I even reset the OpenBTS configuration back to the example configuration just in case there was something if there causing problems (which I have seen) – no success.
Surely this had to be something to do with network configuration?
Time to dig deeper (and send a call help out on the UmTRX mailing list) …
Looking more closely I notice framing errors showing on the interface:
Not being a networking person it was time to do a bit of research – what does “frame” really mean? The answer is: “RX errors mean that your NIC is receiving malformed frames from the transmitting switchport. Frame errors mean CRC failures on receipt of a frame. The root cause of this could be a bad cable, or a bad interface on either the machine or the switch.”
Duh! – that’s what I suspected and have already swapped over cables and switch. What other tangent can I go off on now … I know “Jumbo Frames”!
Jumbo frames are Ethernet frames with more than 1500 bytes of payload. Many Gigabit Ethernet switches and Gigabit Ethernet network interface cards support jumbo frames. Some Fast Ethernet switches and Fast Ethernet network interface cards also support jumbo frames. Most modern Linux distros support frames larger than 1500 bytes. This can improve the performance.
The JMC250 also supports Jumbo Frames (up to 9216 bytes), which can be configured via the interface MTU setting. Mmm … sounds interesting but the largest packet I can see in a Wireshark trace is 1514 bytes:
Still, seems worth a go:
ifconfig eth1 mtu 9000
;-( – still no success.
Alexander replies (again) on the UmTRX mailing list: “We temporarily solved issues with this JMC chip by putting a switch between it and UmTRX, but not all models of switches worked and even after that stability was not perfect.”
Reconfigured my network and it works (sort of). UmTRX now hanging off TP-Link TL-SG1005D Gigabit switch and Shuttle XS35-703 V2 running OpenBTS hanging off Netgear FS605 10/100Mbps switch.
Root cause: JMicron JMC250 connecting to a Gigabit switch and the link partner enabled the IEEE 802.3az Energy Efficient Ethernet feature
I’m still getting (lots) of framing errors which is not good.
Next step – Since I’m limited on what I can install in the Shuttle I’m waiting for Mr. Amazon to deliver a USB 2.0 to 10/100/1000 Gigabit Ethernet LAN Wired Network Adapter based on ASIX AX88178 Chipset which has Linux drivers. I know USB 2.0 supports theoretical speeds up to 480Mbs which is not Gigabit speeds but lets see …
PS: Thanks again to Alexander and Stephane on the mailing list for their help.
My test GSM network is now up and running! Here are some pictures for your amusement.
A little case hacked together to keep things safe (and cool):
Network details from a survey on a Sequoia GSM tester. Note MCC/MNC set to 234/99 and ARFCN set to 512:
Network registration on an old Nokia MS:
Watch out for more blog posts over the next few days. PS: Calls into FreeSWITCH are working fine 🙂