Back again

I postponed the second part of killing roaming fees for almost three years, meanwhile I moved into another apartment, I changed job, I worked on other small projects - maybe I’ll write about them - and my pbx setup is still in place and working. I am really impressed by the resilience of the two pi board. Now it’s time to write the last two parts of the post that I started in 2013.

Updates on roaming in Europe

When I first wrote the first post, it seemed that EU was on the verge of erasing roaming surcharges from history, as reported on the Wikipedia article about roaming regulations:

In 2013 the Commission proposed to establish a single market for electronic communications within the Union and to abolish roaming charges. The proposal was approved by the European Parliament on 3 April 2014, by a margin of 534 votes to 25. As drafted it would have ended roaming charges from 15 December 2015. The Council of the European Union has to approve legislation before it can take effect,and ended up rejecting the specifics of the proposed legislation.

For whatever reason, the council rejected that particular law, but in November 2015 a different law was adopted. The new law introduces a gradual reduction of the roaming charges plus max capping on roaming surcharges. According to it, roaming surcharges can be applied until 14 Jun 2017. After that date and until 30 Jun 2022, no roaming surcharges can be applied.

Well, as often happens, devil is in the details, and in this case he’s hiding within article 6b, “Fair use”:

Roaming providers may apply in accordance with this Article and the implementing acts referred to in Article 6d a ‘fair use policy’ to the consumption of regulated retail roaming services provided at the applicable domestic retail price level, in order to prevent abusive or anomalous usage of regulated retail roaming services by roaming customers, such as the use of such services by roaming customers in a Member State other than that of their domestic provider for purposes other than periodic travel. Any fair use policy shall enable the roaming provider’s customers to consume volumes of regulated retail roaming services at the applicable domestic retail price that are consistent with their respective tariff plans.

Basically, if I understand it correctly, after 14 Jun 2017, operators are allowed to apply roaming surcharges (up to a point disciplined by this EU law) IF the customer is violating the fair user policies.

I am not able to find an official definition of Fair use, - someone could even think that they don’t want to define it, so operators can define they own criterias - but I think it’s related to how many days you stay in a country different from the one were your mobile phone operator is from.

Despite the effort made by some EU politicians, it seems that some roaming surcharges could be applied at discretion of the operator, so we still need a way to work around them.

Some considerations after 31 months of usage

My original system is still in place after 31 months, the two raspberry boards are still working, same for the two 3g dongles. Calling system works mostly ok, except for these minor problems:

• When the call is going all the way trough the two 3G dongles, I have some delay issue. This is a bit annoying when the conversation starts, because every partecipant in the conversation needs to adapt and get used to the delay.
• When my country B sim runs flat without me noticing, whoever calls me from country A gets the call answered by an automatic voice. This is something that can be fixed with a small script that periodically checks how much credit I have, and sends me an email when the credit is below some threshold.
• Some sms messages are encoded in PDU format and for some reasons they are not correctly decoded by the chan module. To decode them I need to copy the encoded string and convert it manually with a pdu tool. Something that can be fixed, but needs work on chan module side. Doesn’t happen often to me, but it is still a problem if for some reason you still receive lot of SMSs in 2016.
• If someone press a key during a conversation, the DTMF tone gets replicated on each side, and it is impossible to continue the conversation. This is something annoying, I think that it could be related to inband DTMF, but I don’t have a clear idea here. Suggestions are welcome.
• IVR quality with Festival is not the best, speech synthesis works, but it’s barely intelligible.
• My setup is based on three different machines, but this could be done with only two raspberry pi, without a central pbx. I think that it is potentially a better setup, and currently I am testing it using the central machine on the cloud only as a OpenVPN server.

OpenVpn

I am using OpenVpn with client certificates. Install the server on the central pbx, and use your tunnel interface when configuring Asterisk and other services.

For more information on installing OpenVpn on Debian:

• Openvpn Debian wiki
• Digital Ocean tutorial

Installing and configuring festival (optional)

When you call from country B and you want to reach country A, you can have (like me) an IVR on the central PBX that on request will read to you all the call options available. You can skip completely the speech synthesis and - for example - just remember the available extensions.

Installation on Debian is fairly easy:

sudo apt-get update
sudo apt-get install festival

Then proceed to create a configuration file vim /etc/festival.scm

/etc/festival.scm

; Log file location
(set! server_log_file "/var/log/festival/festival.log")

; Server access list (hosts)
; Secure default:
(set! server_access_list '("[^.]+" "127.0.0.1" "localhost"))

; Server deny list (hosts)

(set! NewStyle 'my_slow 100 24 1.0)
(set! style_default 'my_slow)
(set! voice_default 'voice_lp_diphone)
(define (tts_textasterisk string mode)
"(tts_textasterisk STRING MODE)
Apply tts to STRING. This function is specifically designed for
use in server mode so a single function call may synthesize the string.
This function name may be added to the server safe functions."
(let ((wholeutt (utt.synth (eval (list 'Utterance 'Text string)))))
(utt.wave.resample wholeutt 10000)
(utt.wave.rescale wholeutt 0.09)
(utt.send.wave.client wholeutt)))

I am using rescale and resample to slow down the voice. We want to run festival as a daemon, so you need to create a unit file for systemd: vim /lib/systemd/system/festival.service

/lib/systemd/system/festival.service

[Unit]
Description=Festival Daemon
After=syslog.target

[Service]
Type=forking
ExecStart=/sbin/start-stop-daemon --start --chuid festival --background --exec /usr/bin/festival -- --server -b /etc/festival.scm
ExecStop=/sbin/start-stop-daemon --stop --oknodo --exec /usr/bin/festival

[Install]
WantedBy=multi-user.target

Now you can start festival with systemctl sudo systemctl start festival.service, If everything is ok, you should see festival process running.

Now tell systemd to start festival daemon at boot: sudo systemctl enable festival.service

Asterisk needs now to be configured to use Festival, vim /etc/asterisk/festival.conf:

; Festival Configuration
;
[general]
;
; Host which runs the festival server (default : localhost);
;
host=localhost
;
; Port on host where the festival server runs (default : 1314)
;
port=1314
;
; Use cache (yes, no - defaults to no)
;
usecache=no
;
; If usecache=yes, a directory to store waveform cache files.
; The cache is never cleared (yet), so you must take care of cleaning it
; yourself (just delete any or all files from the cache).
; THIS DIRECTORY *MUST* EXIST and must be writable from the asterisk process.
; Defaults to /tmp/
;
cachedir=/var/lib/asterisk/festivalcache/
;
; Festival command to send to the server.
; Defaults to: (tts_textasterisk "%s" 'file)(quit)\n
; %s is replaced by the desired text to say. The command MUST end with a
; (quit) directive, or the cache handling mechanism will hang. Do not
; forget the \n at the end.
;
festivalcommand=(tts_textasterisk "%s" 'file)(quit)\n

Country A essential setup

Country A is your home country, probably is the place where you were born.

This instance of Asterisk on the raspberry needs to:

• Answer to all calls received via the dongle and redirect them to a reserved extension on central pbx;
• Call local numbers using chan_dongle when instructed by central pbx;
• Get all SMS, forward them via email
• Send SMS when received

Let’s see how we can manage the calls:

/etc/asterisk/extensions.conf/ excerpt

[from_trunk]
	exten => YOURCOUNTRYAPHONENUMBER,1,Verbose(Incoming call from ${CALLERID(num)})
  		  same => n,Dial(IAX2/central/dialcountrybgsm,60)
  		  same => n,Hangup()
  		  
[incoming_central]
	exten => dialsomeone,1,Dial(Dongle/dongle0/NUMBEROFSOMEONE)
	exten => _XXXXXXXXXX,1,Answer()
  		  same => n,Dial(Dongle/dongle0/${EXTEN})
  		  same => n,Hangup()

Check your context names to correctly apply the dialplan. In my case, on my dongle.conf I set the dongle calling context as ‘from_trunk’ and on the iax.conf I have the incoming_central context for calls that are coming from the central pbx trunk. For more information on chan_dongle, go here.

Inside the from_trunk context, I created an extension that has the same phone number as the phone number belonging the sim card currenly inside that dongle. If your dongle.conf is correctly configured, every call made to that number will trigger that extension in asterisk and in turn it will trigger another extension on the central pbx (dialcountrybgsm).

Besides, inside the incoming_central context, I created a kind-of favorite extension (dialsomeone). In my configuration I have a lot of hard encoded extensions like that one. I have one for my mother, one for my father and so on. I can dial these extensions from my central pbx. Later on I’ll try to explain better this last part.

So far, we created the configuration for managing calls. Let’s see how we can manage the received SMSs:

/etc/asterisk/extensions.conf/ excerpt

exten => sms,1,Verbose(Incoming SMS from ${CALLERID(num)} ${BASE64_DECODE(${SMS_BASE64})})
  		same => n,System(/usr/local/bin/mailsend -from youremail@whatever.com -to yourdestination@whatever.com -starttls -port 587 -auth -smtp smtp.whatever.com -sub ${CALLERID(num)} +cc +bc -M "${BASE64_DECODE(${SMS_BASE64})}" -user youremail@whatever.com -pass "yourpassword"

The extension name ‘sms’ is reserved by chan_dongle. More info about chan_dongle are here. I’m using mailsend as an email client, and then BASE64_DECODE to correctly parse the received sms. This won’t work with multipart (PDU) sms, tipically, when I get a PDU sms I get a set of chars inside the email.

Now let’s see how we can build a primitive email-to-sms gateway. To do this I’m using:

• Fechmail to, well, check if there are new emails and fetch email from mailserver every minute
• Procmail as an MDA, it will parse received emails and send SMS through a script

Install fetchmail with apt-get install fetchmail, then write a configuration file inside the home of the user that will run fetchmail from cron:

/home/user/.fetchmailrc

 poll imap.whatever.com protocol IMAP
   user "user" is asterisk here
   password 'password'
   folder 'SMS'
   keep
   ssl
   mda "/usr/bin/procmail -f %F -d %T"; 

Fetchmail will refuse to read the .fetchmailrc unless you set correct permissions (600). This is just an example, I am checking email inside folder SMS from imap.whatever.com with user user and password password. Then I am calling procmail and asking it to deliver the message.

Install procmail with apt-get install procmail, then create the .procmailrc if doesn’t exist:

/home/user/.procmailrc

MAILDIR=$HOME/mail
LOGFILE=$MAILDIR/log
VERBOSE=on

SUBJECT=`formail -zxSubject:`
BODY=`formail -I ""`
:0
* ^Subject:.*smsto\:.*
| /var/lib/asterisk/smsend.rb $SUBJECT "$BODY" 

I am assuming that the email will have a subject like: smsto:+34XXXXXXX or smsto:daddycool with the body containing the text of the message. If the email has this subject, procmail will call the smsend.rb script passing to it subject and the body of the email.

Finally, this is the ruby script that I am using to send the sms:

/var/lib/asterisk/smsend.rb

#!/usr/bin/ruby
subject = ARGV[0]
body = ARGV[1]


def sendsms number,text
  puts "Number:[#{number}] Text:[#{text}]"
  command = "/usr/sbin/asterisk -rx 'dongle sms dongle0 #{number} #{text}'"
  system(command)
end

unless (subject.nil? or subject.empty?) or (body.nil? or body.empty?)
  number = /\Asmsto:(.{3,13})\z/.match(subject)[1]
  unless number.nil?
    #This is how you can create some kind of shortcuts
    case number
      when "someone" then sendsms "+xxxxxxxxxxx",body
      when "someoneelse" then sendsms "+xxxxxxxxxxx",body
      else sendsms number,body
    end
  end
end

The case block contains a raw and ugly ‘favorites’ list of numbers. In this case someone and someonelse. I am just passing to asterisk the command dongle sms dongle0 #{number} #{text}. Setup on country A pbx is done, check out part 3 for country B and ending notes.