Kamailio and SIP training: notes from the field

Being one of the leading companies involved in Kamailio and open-source SIP infrastructure implementation for VoIP service providers in North America, we run our Kamailio and SIP fundamentals training curriculum a fair bit. It’s a distinctly secondary line of business for us, but since 2011, we’ve done it somewhere around 15 times by now, mostly here in the USA and occasionally internationally. That’s enough repetitions and customer feedback cycles for us to draw some conclusions and generalise about what we ourselves have learned.

It’s usually a two or three-day affair. The first day consists of SIP fundamentals training, which we consider prerequisite to anything else (if you know a bit about what’s required to configure Kamailio effectively, you’ll agree). There are customers who want that portion only, and depending on how in-depth they want to go, that can last two days. Otherwise, the second day is usually focused on Kamailio, and, when there is a third day, it’s usually filled with hands-on “lab” activities and applied exploration of things the customer is specifically interested in.

We generally tweak the structure to emphasise what the customer most hopes to get out of it, and this varies a lot. Some customers are impatiently laser-focused on applied use of Kamailio to solve very immediate needs. Some customers are focused more on shoring up a general understanding of SIP among their support staff to improve troubleshooting outcomes. Some customers’ primary goal is to get people with a fairly “standard” IT background more conversant with the exotic vocabulary and history of IP telephony and real-time communications. Some customers are highly technical developer types and are looking to reach into more rarefied knowledge of some APIs, or really niche aspects of SIP standards and protocol formalities. We have experience catering to that entire spectrum.

All told, teaching SIP and Kamailio is not so different to teaching most other niche software systems, tools, or frameworks. Most lessons we’ve learned seem to apply to all technical training of a 2-3 day introductory format (as opposed to an intensive and more long-term course). I’ll share a few:

Training is not a conference talk

There’s a point of view out there that slide content should be minimal, to serve only as speaking prompts, akin to a speaker’s private note-cards, or to hold illustrations. Certainly, walls of text aren’t useful, and nobody likes speakers who just read their lengthy slides verbatim. Pedagogically useful slides are by nature somewhat abstemious.

However, we’re yet to have a customer who did not ask to keep our slide deck for their own reference. That alone means that the slides have to have some standalone informational value, and can’t be too minimal.

Some hipster slide deck with five slides of faux-“Zen” rhetorical questions,

or vacuous treacle like:

will be of zero value to anyone. In situations where there is a declared intention to use the slides as reference material, they have to strike a balance between walls of text on the other hand, and an utter paucity on the other. They’re a document of sorts.

More generally:

I have observed that many people, when asked to teach a training course or a seminar of some kind, go to drink from a common well of “public speaking skills” they may have deployed in other contexts, such as presentations to management, conference talks, etc. The skills for conference talks seem to be an especially common departure point, where the focus is on keeping the audience engaged.

While all public speaking is a performance that makes demands upon one’s artistry, and there is no question that the challenge of keeping the audience engaged falls into your purview as a trainer, training is not the same as giving a conference talk.

For one, two days is not thirty minutes. More importantly, one’s purpose in being there is specifically to convey non-trivial information as a specialist, and the audience carries a greater responsibility to absorb it. It’s not a sales pitch. You’re not marketing your specialty. The business objectives of a half-hour conference talk given to a general audience are entirely different. It’s worthwhile to ponder that when wrestling with the temptation to pilfer the “performance art” of one and channel it into the other.

Your audience are mostly there because their boss said they have to be, so you don’t have to get them “amped” about the subject. You’re not a clown, and the primary purpose of your visit is not entertainment.

I’m not saying you should aim to be boring—no, by all means add earnestness, humour, wit and charisma to your presentation if you can, and good trainers do. However, if you feel like you have to make it “dynamite” enough for a bunch of ADHD hamsters who will move on to a different room/booth/track in 20 seconds if you don’t keep them on the edge of their seats, stop yourself. You’re optimising for the wrong problem. This is training; it’s their time and their dime.

Have a clear idea of the objective

Having a clear idea of an objective and mindfully allowing it to guide you is not the same as merely stating an objective or marketing an objective. Lots of folk do the latter without a dime of sincere thought capital invested in the former.

You’ve seen it in the facile syllabi of sundry curricula before:

By the end of the VoIP Bushido Expert Seminar 3XL, the student will have mastered the skills of real-world SIP aikido and H.323 jujitsu. The VoIP Bushido Expert Seminar 3XL will empower the student for maximum success in a fast-paced, ever-changing Ameriglobal VoIP marketplace that demands advanced expertise.

Yeah, okay. If you’re running anything remotely describable as a “seminar”, there is exactly 0.0% chance that anyone will come out of it with a mastery of anything. Either you’re teaching something utterly trivial and obvious, or you are abusing the concept of “mastery” in a way that is deeply fraudulentYour marketing department might say everyone’s doing it and it’s not meant literally, but this is the service you are rendering unto the use of language and the meaning of words:



It gives me no pleasure to say that it’s especially apropos in this very cultural moment.



But assuming we’ve been relieved of the notion that anyone stands to “master” anything from a seminar, or for that matter a 2 to 3-day training course, the entirely legitimate question arises: what, exactly, do you mean to accomplish?

Most people in pedagogy will agree, I think, that the primary goal of any “introductory” endeavour should be to leave everyone in the room with a greater level of knowledge and skills than you found them. In the case of the Kamailio side of our training, I like to abuse the cliché “teaching a man to fish”. Going from zero to a “distinguished and commercially viable skill set” with a system like Kamailio takes years. My intent is that everyone leave our training sessions:

  • With a better grasp of the ontologies surrounding Kamailio, and more especially a sense of the Kamailio idioms for various general concepts in SIP proxy behaviour and SIP routing;
    • Transactions;
    • Dialogs;
    • Initial vs. sequential requests and “loose routing”;
    • Hop-by-hop messages (CANCEL, 100 Trying, negative ACK) vs. end-to-end messages;
  • A clear high-level sense of where Kamailio is typically used in building large-scale SIP service provider architectures (e.g. registrar, load balancer, redirect server to add routing intelligence, and the rest);
  • Some familiarity and comfort level with the names of Kamailio concepts and the ideas to which they refer, e.g.
    • Core functions;
    • Modules;
      • Essential modules needed for almost any useful configuration; modules which are “good as core” (e.g. TM);
      • Ancillary modules to provide specific functionality (e.g. JANSSON);
    • Pseudo-variables;
    • Transformations.
  • A clear sense of where to find documentation and how it’s laid out, and some intuition of where to look for certain kinds of things;
  • Some visual and reflexive familiarity with the appearance and anatomy of the Kamailio configuration file
    • Core configuration directives;
    • Module parameters;
    • Subroutines (in essence, SIP event callbacks):
      • Request routes;
      • Reply routes (onreply_route);
      • Failure routes (failure_route);
      • Branch routes;
    • Specialised event routes (callbacks/event handlers exposed by modules).
    • Concepts analogous to general-purpose programming languages and runtimes:
      • String transformations (kind of like string methods in OO languages);
      • Variables
        • Ephemeral/scratch-pad variables ($var(…));
        • Transaction-persistent variables ($avp(…)/$xavp(…));
        • Dialog-persistent variables ($dlg_var(…));

This is not “mastery” of anything, including these very concepts. But the goal of the training is to expose these ideas and vocabulary to the audience so that they recognise them and can use them in the future to develop their knowledge toward their goals.

The “leave them better off than you found them” bit will have different results for different people and groups in our SIP and Kamailio training. People with some development background may go from having a loose-fitting acquaintance with these things already to a more buttoned-down one, allowing them to be more focused and efficient in building further knowledge and experimenting, or at least asking more focused questions of us or on mailing lists, leading to better and more useful answers. For others, it will simply mean putting these words on a mental map where they did not exist before, so that references to them in the future “ring a bell”, an improvement over total bewilderment. There is a notable difference in the nature of the leaps we can expect from developer sorts versus operations types.

That’s a realistic assessment of what we can hope to achieve, though hardly a guarantee. That’s what I tell management when they want to learn more. Some nod approvingly and appreciate my candour, while others, accustomed to viewing programming as a fancy form of typing, bristle at the notion that we can’t get the staff “trained up” so that they can just, you know, code up the product real quick. Regardless, honesty and specificity are the best policies.

Group dynamics

Technical proficiency among our audiences follows a fairly typical Pareto distribution. Due to our prevailing flat-fee structure for most engagements, management will send send five to ten people. One or two of them will have had deeper Kamailio and SIP experience internally and extract a lot of specific information from the training, while the rest are there to soak up “exposure” so that the activities of the other one or two will not be a completely opaque mystery to them.

Just about every group will have That One Guy. (I don’t mean that disparagingly; he’s just That One Guy for lack of a better name coming to mind. And doesn’t have to be male.)

He’ll already have come into contact with 40-70% of your material in some fashion, and is often keen to demonstrate that with pep and vigour. He’ll ask a lot of questions and generate a lot of tangents. The psychological motive is rarely to ingratiate himself to the trainer, who, after all, will pack up and leave soon, but the motives will vary, from genuine intellectual curiosity and affability at one end, to a more ulterior plot to position himself as the “go-to guy” for this subject matter in front of his colleagues. The latter is more common in large organisations, where ownership of projects, and the budgets and clout that come with them, is a contentious topic in the sizzling (or slowly marinating) “office politics” inevitable in any group of nontrivial size.

As in any other consulting project, so it goes in training: every active, invested participant has explicit and covert objectives—well, “covert” implies something nefarious in a way I don’t intend, so perhaps “tacit objectives” is better. Either way, a few walks around the consulting block will lead to the insight that identifying all of these—as best as one can—with sensitivity and perceptiveness is a very important “soft skill” and a key part of the value proposition to management stakeholders. Naturally, it’s a tacit one. This holds true in training as well.

Anyway, trainers and teachers seem divided on whether That One Guy is friend or foe from the point of view of maintaining structure. There are some trainers who believe that such people “hijack” the agenda and do a disservice to the rest of the group as well as the trainer’s efforts to bring things to a common denominator that everyone can access. And it’s true enough that I’ve had some training sessions with small groups, in earlier iterations of doing this, that seemed to turn into a conversation between me and That One Guy. It’s important to remember that the goal is to leave everyone better off than you found them.

But not everybody in a group of more than about 3-4 is going to get something out of the training, and one must accept that. Some attendees may have the potential, but instead zone into their laptops, fighting fires and paying half-attention. It’s their time and their dime, and you don’t need to slow the bus down for their benefit. If they tumble out, they tumble out.

I personally think that That One Guy is an asset. All interactions, even the overwhelmingly lopsided dynamic of lecturing to a group, are two-way. It’s still a conversation with the audience, whose temperature and tempo one must gauge. As long as That One Guy’s role is properly managed, he provides much-needed anchoring and telemetry for how to proceed, helps to generate good energy and convection around the topic, and, often, provides a window into the tacit objectives in the group.

Frequent breaks and atmosphere

There are some managers who would say taking a 10-15 minute break every 1 to 1.5 hours is too frequent. And to them I say: people simply cannot be bombarded with detailed information for two or three hours, even if it’s quite riveting. They will zone out. I would say it’s best practice to insist on it over any objections of management.

A related idea:

Darkened rooms are great for reading slides, but even better for inducing sleep. Windowless fluorescently-lit rooms and depressing dungeons seem to have a similar effect. Bright, sunny conference rooms with picturesque views of trees and park benches serve to grimly remind everyone that they could be having fun outside, but alas, are hearing about Via branch parameter GUIDs from some propeller-head instead. The effects of this on the mood of people in an hour-long meeting are different than on the mood of people stuck with you for two full days of training.

Adjust the implied sympathy of your approach accordingly, but ideally, find a venue that represents a happy medium.

Love letter to Vue

In the frustratingly fast-paced, ever-shifty and profoundly fashion-driven JavaScript web development ecosystem, it’s not easy to find something that one can even stand to use, to say nothing of love. And if you do find something, its obsolescence will be triumphantly announced on Hacker News in about three weeks.

MVC/MVVM frameworks in particular are a source of frustration. There’s AngularJS (often known as Angular 1.x), which, despite being fundamentally meritorious (indeed, I got started with modern JS web frameworks on it), is clearly subject to a strong effort at obsolescence.

AngularJS is also notorious for being highly opinionated about how your entire application should be structured, forcing many competent developers into stifling vocabularies of design patterns — things like “factories”, “services”, “providers” — that are neither wanted nor needed. I understand that this is sometimes viewed as a selling point because it imposes discipline and more homogenous, shared vocabularies on front-end teams with an entry-level skill set, but it is incredibly stifling and bureaucratic to people who know what they’re doing.

Angular 2.x (itself now obsolete!) went completely off the rails with the boilerplate, complexity, build tooling, and Byzantine structure required just to get started. I understand what the Angular people are trying to do, catering to the sensibilities of large enterprise projects. However, in the course of doing it, I fear they’ve lost their minds. “Make it more Enterprise™” is a common trap in the “evolution” of libraries and tools. Angular 2/4 is a completely over-engineered trainwreck.

It was with this realisation that I went looking for something new on which to standardise our ambitious internal portal project, which throws off reusable components that are cycled into the new CSRP UI. I considered React, Riot.js, and one or two others.

In that research, in late 2016 I stumbled upon Vue. It was an incredible breath of fresh air. I don’t mean to be melodramatic, but it’s not an exaggeration to say that it lifted me from an overwhelming depression about the future of front-end development in this company and got me coding enthusiastically again. They say you can’t look for love; it just happens. It doesn’t happen very often in the mess that is the JavaScript web ecosystem. In this case, it did.

I’m a back-end developer, systems person and telecoms nerd by trade; if you’ve got me loving UI development, you’ve achieved the certifiably impossible.

Reasons why Vue is amazing, in my eyes:

Not too much, not too little!

For an experienced software engineer, it is refreshing to be left in charge of one’s own architectural decisions about the application as a whole without being forced into contrived vernaculars and unwanted ontologies. Vue liberates one to architect the application as one chooses, providing only the UI bits and not a whole “pattern”.

At the same time — and this is critical — there’s plenty “in the box” with Vue. One doesn’t have to cobble together all the necessary pieces of a typical web application from a dizzying array of third-party components, as with React. It’s got an official, batteries-included router (vue-router) and a central state store (Vuex). The declarative templates are feature-complete and rich, requiring no additional plumbing, at least for typical CRUD business application use-cases. Vue really gets this intricate balance right in ways that have innate appeal to an experienced developer. It has everything you need to build such applications, including optional features that foresee considerable complexities and nuances of large-scale projects, but not a smidgeon more than truly necessary to do its job.

Vue achieves elusive simplicity in a realm where simplicity is seldom found, except in such forms as to require the passengers to build the plane themselves, right there on the tarmac.

Component-centric design

As in React, everything in Vue is a component. That’s it. You simply build self-contained components and compose them. You’re not bombarded with half a dozen different kinds of abstractions and related esoterica. You don’t have to master exotic vocabulary like “transclusion” and the fine points of scopes. Vue has alternatives for all of this functionality, of course, but they are much more succinct.

There are of course a few other constructs, such as filters and custom directives, to which you may need to resort. However, fundamentally, components are the only important first-class citizens of Vue.

Added business bonus: while the declarative template syntax, in essence identical to AngularJS’s, allows one to have meaningful Vue conversations with developers with an AngularJS background, the component-orientated focus of Vue allows one to have equally meaningful conversations with React developers.

If you hate declarative template logic and have an insatiable twitch for JSX and custom render functions, Vue has got you covered. And more fundamentally, Vue is also based on the idea of passively reactive data plumbing, so you don’t have to litter your code with imperative watchers.

So, although I don’t know React nearly as well, I believe Vue accommodates the habits of mind of both Angular and React developers.

Amazing documentation

Vue documentation is the gold standard of documentation, in my opinion. I’ve never read such clear, complete and easy-to-understand documentation for anything in my life.

Characteristically, it strikes a great balance between giving adequate conceptual background on the one hand for those who want to learn more, and instant gratification and quick examples to those with an applied, hands-on motive.

I don’t know how they got the documentation so right, but they did. Their claim that one can get started developing non-trivial things in Vue in about a day isn’t frivolous; I got started developing non-trivial things in Vue after about a day (although I was coming off an AngularJS background). Obviously, it takes time to learn to use anything fluently and come into better ways of doing things, but the learning curve is really short. I attribute that largely to the excellent documentation.


You can use ES6 constructs freely in Vue. The only major restriction I’ve run into is that Vue can’t observe changes inside ES6 keyed Collections. There are also a few places where traditional functions, rather than arrow functions, are required to preserve the appropriate scope of this, as for example in watcher callbacks.

However, arrow functions, native Promisesdestructuringasync/await (ES2017), and other modern goodies are good to go, and our Vue projects use them everywhere without a care. The Vuex store docs actually recommend the use of stage-3 spread syntax. That’s pretty modern!

Scales up and down, to large and small tasks alike

You can build a complex application architecture in Vue, making full use of Flux/Redux-type state-keeping patterns using the Vuex store. Or you can just attach a single Vue component into a single DOM element for a niche purpose, much as you’d do with jQuery. Although you certainly can build your Vue project with a Webpack-driven monstrosity, you can also inject it for that kind of niche purpose via a single <script> tag.

If you’re stuck with maintenance on a legacy web application or possibly even a pre-SPA, and want to inject some modern new widgetry, this sort of thing can be a godsend.

Vue can be as minimalistic or as expansive as you like in that sense, and that flexibility is one of my favourite things about it. You don’t have to load half a billion dependencies, rewrite half the code base, or structure complicated scaffoldings to use it for small tasks.

Plays well with others

A familiar problem to those with an AngularJS background is trying to get other DOM-impacting JS libraries to work within it and not make manipulations invisible to its dirty checking / digest cycle. I’ve never really had this problem with Vue (though I don’t doubt there are some edge cases). I’ve injected a little jQuery here and there for some effects and that, and, thanks to the clean reactivity model behind Vue, the changes pick up just fine.

More generally, Vue doesn’t force you to use a portfolio of native componentry for things unrelated to its core mission. It’s common to use a module like axios for XHR/AJAX/REST operations, and Vue plays ball. That’s because it operates on plain old JavaScript properties and doesn’t introduce a large out-of-band wrapper superstructure to effect its data binding and reactivity.

Clear and distinct project vision

Thanks to the author’s prudent leadership, Vue does not appear to suffer from existential confusion about what it is or wants to be. It has a clearly articulated philosophy of what it does and doesn’t aim to provide, and sticks to it. I hope that never changes.


The Vue that I am using at the time of this writing, in summer 2018, is the same Vue I picked up in autumn of 2016. With the way things work in the JavaScript web ecosystem, that alone says a lot about the project discipline and leadership.

From the perspective of a company whose core business is not web UI development, this is really, really important. Few things are safe investments in the JavaScript web ecosystem, often here today, completely rebuilt and backward-incompatible 2.0 tomorrow — no, I literally mean tomorrow. That’s how the Valley web hipsters of Hacker News do things. Not just the hipsters, actually: how long did it take for Angular 2 to be “obsoleted” by Angular 4 again?

As always, Vue manages to strike an elegant balance between staying current and keeping pace on the one hand, and providing a solid and credible technology platform on the other. Combined with Vue’s other aforementioned virtues, and more especially its strongly centred philosophical identity, this stability and seeming longevity puts JavaScript web development within reach of non-web-economy stakeholders like us.

We love Vue so much that we support Evan on Patreon, earning me a small mention in the Github repository’s backers list. We hope to increase our contribution in the near future to reflect the incredible utility and satisfaction, both business-level and psychological, that we’ve got out of Vue.