Just a few days ago we released a new Janus plugin, one that will allow users to write custom plugins in Lua rather than using C. It is currently available as a pull request on the Janus Github repo, as it’s still work in progress.
Rather than explaining how the plugin works or what it does (please refer to the lengthy description on the pull request for that), the purpose of this blog post is to show how easy it can be to write a new plugin from scratch using Lua: in particular, we’ll create a simple video call plugin, one that will allow two WebRTC endpoints to communicate with each other via Janus. To make things easier, we’ll mimic the API of the already existing VideoCall plugin: this will allow us to simply re-use the existing demo page to test our new plugin, by only changing a single line in the JavaScript.
Configuring the plugin
Assuming you correctly installed the Janus Lua branch, configuring the plugin is quite easy. We’ll need to edit the janus.plugin.lua.cfg configuration file and have the script property point at where our Lua script will be. Assuming Janus is installed in /opt/janus and that we’ll create our Lua script in /home/lminiero/videocall.lua, this is how the new configuration file might look like:
[general] path = /opt/share/janus/lua script = /home/lminiero/videocall.lua
The path property points to where the sample Lua scripts have been installed and you should not changed it, as it allows you, for instance, to take advantage of a couple of helper Lua libraries we wrote for the purpose (namely for logging and SDP munging). In case you have other libraries installed somethere that you want to use, you can add those to the provided path as well. Anyway,
this is not really relevant to this tutorial, as we won’t use any sample here (apart from a dependency that we’ll address later).
That said, nothing else will be needed: at startup, the Janus Lua plugin will automatically load our script.
Starting to write our plugin
A Janus plugin written in Lua must pretty much behave as a C plugin would: this means implementing the same callbacks to be notified by the core, for instance, when Janus starts or shuts down. In Lua, this means creating two functions called init() and destroy():
function init()
print("Lua videocall initialized")
end
function destroy()
print("Lua videocall deinitialized")
end
Two other important callbacks we’ll need to implement are createSession() and destroySession(): as the name suggests, these will be invoked by the core respectively whenever a user attaches to our plugin, and whenever the user detaches from our plugin instead. As such, they’ll allow us to keep track of users accessing our plugin. Since users are identified, within the C code, by unique numeric identifiers that are passed as arguments to our functions, we can use a hashtable to track users and keep their information up-to-date. As it will be clearer later, we add a hashtable to keep track of usernames as well. Let’s try and do just that:
sessions = {}
usernames = {}
function createSession(id)
print("Created new session: " .. id)
sessions[id] = { id = id }
end
function destroySession(id)
print("Destroyed session: " .. id)
hangupMedia(id)
local s = sessions[id]
if s == nil then return end
if s.username ~= nil then
usernames[s.username] = nil
end
sessions[id] = nil
end
We’re basically adding an object to track the new user in createSession(), and destroying the object when destroySession() is called on the same user instead. You may have noticed the hangupMedia() call there: we’re basically saying that, if a user’s session has been destroyed, we should get rid of its media resources as well. This hangupMedia() is not a request, though, but another callback the core will invoke and one we’ll need to implement, as we’ll explain in the next section.
Tracking a user’s PeerConnection life cycle
The core will notify us whenever a PeerConnection for a user comes up and when it goes down: this is done via setupMedia() and hangupMedia() respectively. This is helpful to decide when we should start relaying media, for instance, and when we should stop instead. That’s why, when we get a destroySession(), we also manually invoke hangupMedia() ourselves: we’re just making sure that the code we might have to clean up media resources will be invoked anyway, whether the core told us to or not.
Let’s see how these two callbacks may be implemented in our simple plugin:
function setupMedia(id)
print("WebRTC PeerConnection is up for session: " .. id)
local s = sessions[id]
if s == nil then return end
s.started = true
configureMedium(id, "audio", "in", true)
configureMedium(id, "audio", "out", true)
configureMedium(id, "video", "in", true)
configureMedium(id, "video", "out", true)
configureMedium(id, "data", "in", true)
configureMedium(id, "data", "out", true)
addRecipient(id, s.peerId)
sendPli(s.peerId)
end
function hangupMedia(id)
print("WebRTC PeerConnection is down for session: " .. id)
local s = sessions[id]
if s == nil then return end
s.started = false
removeRecipient(id, s.peerId)
if s.peerId ~= nil then
hangupPeer(s.peerId)
s.peerId = nil
s.peerUsername = nil
end
end
These two callbacks are a bit more complex than the others we’ve gone through so far: let’s check what they’re doing.
As anticipated, setupMedia() tells us when a PeerConnection for a user becomes available. As such, what we do is look for the session object, set its started property to true just for fun, specify we’re ok in letting media through via configureMedium() (we’ll see this request again in the next sections, so just hold on to it for now), and finally invoke a new method called addRecipient(). This method is one of the most important when developing Janus plugins in Lua, as it allows you to decide who should receive the media streams (audio, video and/or data) from a session. In this case, we’re saying that the session that should receive this user’s media is the one identified by the unique ID s.peerId: we haven’t seen where we set this value yet, but we’ll show later that this will derive from the user’s messaging and their willingness to call (or be called by) another user. To anticipate things, we’re saying “now that the PeerConnection is ready, send this user’s media to the person they’re in call with”. The same will happen for the other user as soon as their PeerConnection will be up, thus allowing for a bidirectional communication. As a last step, we’re also asking our peer to send us a keyframe via sendPli(): this might be needed in case, for instance, our peer’s PeerConnection was ready before ours, and so the first keyframe they sent may have got lost.
Of course, hangupMedia() notifies us about the opposite event, the user’s PeerConnection going down or becoming unavailable. When that happens, the first thing we do is invoking a method called removeRecipient(), which unsurprisingly does exactly the opposite of what the previous method did. We also reset the started property, and invoke a stub method called hangupPeer() to tell the person the user was in call with that the session is now over. We’ll see later, when addressing the messaging, what this method might look like.
Admin API and plugin queries
Before addressing the messaging and how to handle incoming requests, though, it’s important to introduce one more session-related callback our plugin will need to implement: querySession(). This function is invoked whenever the core is interested in getting plugin-specific information related to a user’s handle, which typically happens in response to Admin API requests. Since our plugin is actively responsible for a user when they attach, it’s our plugin responsibility to return this piece of information.
Since the information we have to return must be JSON encoded, we’ll need a way to encode/decode JSON blobs in our Lua script. One easy way to do that is relying on a library: just as an example, we’ll use lua-json, although there are of course other alternatives in case you’d rather use something else. Here’s how our function might look like:
json = require('json')
function querySession(id)
print("Queried session: " .. id)
local s = sessions[id]
if s == nil then return nil end
info = { id = s.id, username = s.username, started = s.started,
peerId = s.peerId, peerUsername = s.peerUsername }
infojson = json.encode(info)
return infojson
end
After requiring the JSON library, we build a table containing some information on the user’s session (e.g., the user’s ID, the peer’s ID, whether the WebRTC PeerConnection has been started or not). Then we encode the table to a JSON string, and we return that to the core. We’re free to put here whatever we like, so when writing your plugin you’ll be able to decide what to return here: just remember that the more info you return, the better, as it might help debugging problems related to your plugin via the Admin API.
Handling incoming requests
Now that we know how to track sessions and PeerConnections, we need to take care of the most important part of our plugin: its API. This will fundamentally decide how our plugin will interact with users, which requests it will expose, their syntax, what responses will look like, whether methods will be synchronous or asynchronous and so on. You have complete freedom, which is one of the key points when it comes to the flexibility Janus provides.
For the sake of this simple example, we’ll mimic the API of the existing VideoCall plugin, which is documented here. This is motivated by the fact that this way we can re-use an already existing UI for playing with our new plugin, which is always nice! We’ll basically need APIs to register a username, call another user or accept an incoming call, change some media properties and hangup an ongoing call.
Of course, though, the very first thing we need to address is how we can get incoming messages from users in the first place. In Lua, this is made possible by a callback called handleMessage(). This method is called by the core whenever there’s a message from a user meant for our plugin, and so this is where we’ll need to place logic to decide how to handle it. As we’ll see, we’ll need our JSON library again, here, as the core will pass us JSON strings, and we’ll need to send JSON strings back as either responses (synchronous requests) or events (asynchronous requests).
Let’s see how our function will look like, and we’ll explain in detail how it works:
tasks = {}
function handleMessage(id, tr, msg, jsep)
print("Handling message for session: " .. id)
local s = sessions[id]
if s == nil then
return -1, "Session not found"
end
local msgT = json.decode(msg)
local jsepT = nil
if jsep ~= nil then
jsepT = json.decode(jsep)
end
local request = msgT["request"]
if request == "list" then
local list = {}
for username,userId in pairs(usernames) do
list[#list+1] = username
end
local response = { videocall = "success", list = list }
responsejson = json.encode(response)
return 0, responsejson
else
async = coroutine.create(function(id, tr, comsg, cojsep)
processRequest(id, tr, comsg, cojsep)
end)
tasks[#tasks+1] = { co = async, id = id, tr = tr, msg = msgT, jsep = jsepT }
pokeScheduler()
return 1, nil
end
end
As you can see, once we find the session the message refers to we parse the JSON content to a table, so that we can evaluate it. If an SDP offer/answer is available, we parse the related JSON content to a table too. As soon as we have the content(s) available, we can start processing it, and so we first check what request we received.
Just to demonstrate how synchronous requests differ from asynchronous requests (as the original VideoCall plugin only implements asynchronous messaging), we check if the request we received is called “list”: if it is, we prepare a list of all the registered users, that we JSON encode and return back (with a 0 value to indicate a synchronous response), and that settles things. In fact, synchronous requests result in immediate responses, and as such are more suited for quick or simple requests that won’t block or keep the plugin busy for too long.
All other requests, instead, are handled asynchronously. In Lua lingo, this means creating a coroutine that will have the task of actually processing the request, and then pushing a notification/event back to the user with a response/confirmation on the result of the request itself. As you can see from the code, we create a coroutine, delegate the message processing to a method called processRequest(), and then add the coroutine itself to a list of tasks. We then invoke a method from the C side of the Lua plugin, called pokeScheduler(): as it will be clearer in the next section, this is only needed to tell the scheduler in the C code that there is a coroutine, the one we just created, that will need to be resumed later on. At this point, all we’re left to do is return 1 to tell the core this is just an ack, and that the actual results will come later.
Now, coming to the actual message processing, in the case of our plugin we handle “register”, “call”, “accept”, “set” and “hangup” asynchronously, as shown in the following code which implements the processRequest() stub we met earlier. For the sake of simplicity and readability, we delegate the processing of each of those requests in a separate function, which will make it easier to understand how we handle them: in case we don’t recognize the request, we generate an error instead.
function processRequest(id, tr, comsg, cojsep)
local s = sessions[id]
if s == nil then return end
local request = comsg["request"]
if request == "register" then
processRegister(s, tr, comsg)
elseif request == "call" then
processCall(s, tr, comsg, cojsep)
elseif request == "accept" then
processAccept(s, tr, comsg, cojsep)
elseif request == "set" then
processSet(s, tr, comsg)
elseif request == "hangup" then
processSet(s, tr, comsg)
else
local event = { videocall = "event", error_code = 472, error = "Invalid request" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
end
end
“register”
We handle a “register” request in processRegister(), whose only purpose is updating the usernames hashtable so that we know the mapping between a specific username and a specific session identifier. It’s not an actual registration (there are no credentials involved), as the only argument we require is the username the user wants to reserve. We return an error if no username was provided or if it’s already in use, and a success otherwise:
function processRegister(s, tr, comsg)
local username = comsg["username"]
if username == nil then
local event = { videocall = "event", error_code = 475, error = "Missing username" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
if usernames[username] ~= nil then
local event = { videocall = "event", error_code = 476, error = "Username taken" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
if s.username ~= nil then
local event = { videocall = "event", error_code = 477, error = "Already registered" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
s.username = username
usernames[username] = s.id
local event = { videocall = "event", result = { event = "registered", username = s.username } }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
end
“call”
When a user wants to call another user registered in the plugin, instead, a “call” request is used, which is when the processCall() gets invoked. Apart from some basic error checking, what we do is simply finding the session object associated with the callee, and notify them about the incoming call, by passing the SDP offered by the caller, and sending an ack back to the user. The caller and callee objects are also updated to keep track of each other.
function processCall(s, tr, comsg, cojsep)
if cojsep == nil or cojsep.type ~= "offer" then
local event = { videocall = "event", error_code = 482, error = "Missing offer" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
if s.peerUsername ~= nil then
local event = { videocall = "event", error_code = 480, error = "Already in a call" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
local username = comsg["username"]
if username == nil then
local event = { videocall = "event", error_code = 475, error = "Missing username" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
local peerId = usernames[username]
if peerId == nil then
local event = { videocall = "event", error_code = 478, error = "No such username" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
local p = sessions[peerId]
if p == nil then
local event = { videocall = "event", error_code = 478, error = "No such username" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
if p.peerId ~= nil or p.peerUsername ~= nil then
local event = { videocall = "event", result = { event = "hangup", reason = "User busy", username = username } }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
s.peerId = p.id
s.peerUsername = p.username
p.peerId = s.id
p.peerUsername = s.username
local eventPeer = { videocall = "event", result = { event = "incomingcall", username = s.username } }
local eventjson = json.encode(eventPeer)
local offer = { type = "offer", sdp = cojsep.sdp }
local offerjson = json.encode(offer)
pushEvent(p.id, nil, eventjson, offerjson)
local event = { videocall = "event", result = { event = "calling" } }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
end
Notice that, for the sake of simplicity, we’re just forwarding the same exact SDP the user sent, rather than doing any manipulation. We could actually mangle and munge the SDP to our liking, e.g., to force a specific codec, remove a medium, or something else: the repo currently provides a simple SDP library written for the purpose, so check the existing samples to see how they take advantage of that feature.
“accept”
A user can accept an incoming call by issuing an “accept”, which is handled by our processAccept() method. This basically notifies the caller, by providing them with the SDP answer, and sending an ack back to the user.
function processAccept(s, tr, comsg, cojsep)
if cojsep == nil or cojsep.type ~= "answer" then
local event = { videocall = "event", error_code = 482, error = "Missing answers" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
if s.username == nil then
local event = { videocall = "event", error_code = 481, error = "No incoming call to accept" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
local p = sessions[s.peerId]
if p == nil then
local event = { videocall = "event", error_code = 478, error = "No peer" }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
return
end
local eventPeer = { videocall = "event", result = { event = "accepted", username = s.username } }
local eventjson = json.encode(eventPeer)
local answer = { type = "answer", sdp = cojsep.sdp }
local answerjson = json.encode(answer)
pushEvent(p.id, nil, eventjson, answerjson)
local event = { videocall = "event", result = { event = "accepted" } }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
end
Again, no custom manipulation of the SDP, here, but just forwarding back the answer, to let the two users agree on what the session will look like.
“set”
Some media properties (e.g., whether video should be disabled, the bitrate changed, and so on) can be modified during the call via a “set” request. The processSet() method is where we handle such a request, and we’ll see how those translate to directives to the C side of the plugin, to handle the respective media accordingly. A simple ack is sent back to the user.
function processSet(s, tr, comsg)
if comsg["audio"] == true then
configureMedium(s.id, "audio", "out", true)
elseif comsg["audio"] == false then
configureMedium(s.id, "audio", "out", false)
end
if comsg["video"] == true then
configureMedium(s.id, "video", "out", true)
sendPli(s.id)
elseif comsg["video"] == false then
configureMedium(s.id, "video", "out", false)
end
if comsg["data"] == true then
configureMedium(s.id, "data", "out", true)
elseif comsg["data"] == false then
configureMedium(s.id, "data", "out", false)
end
if comsg["bitrate"] ~= nil then
setBitrate(s.id, comsg["bitrate"])
end
local event = { videocall = "event", result = { event = "set" } }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
end
You can see how the multiple properties we may be asked to manipulate result in different calls to functions the C side of the Lua plugin exposes, as an easy way to customize the media management. Enabling or disabling audio,
for instance, results in a call to configureMedium() with “out” set to true or false, which basically tells the plugin “don’t forward any of the audio packets this user is sending”. Enabling video also results in a keyframe request via sendPli(), as otherwise the peer may end up receiving something they cannot decode. Changing the bitrate of the video for the user is as simple as calling setBitrate(), which basically results in custom REMB feedback being sent back to the user. While there are other things you can ask the C code to do for you (e.g., record a conversation), we’ll skip it for brevity.
“hangup”
Finally, a “hangup” request is what the plugin uses to both reject an incoming call, or hangup an ongoing one. The management of this request is done in the processHangup() helper. As we can see from the code, we send a “hangup” event to the peer, if available, and then we force a disconnection of the PeerConnection of both via closePc(); this is usually not needed (an event on the client side to let them close the PeerConnection by themselves is often enough)
but is useful for the tutorial nature of this example.
function processHangup(s, tr, comsg)
local reason = comsg["reason"]
if reason == nil then
reason = "User did the hangup"
end
local p = sessions[peerId]
if p ~= nil then
local eventPeer = { videocall = "event", result = { event = "hangup", reason = reason, username = s.username } }
local eventjson = json.encode(eventPeer)
pushEvent(p.id, nil, eventjson, nil)
closePc(p.id)
end
local event = { videocall = "event", result = { event = "hangup", reason = reason, username = s.username } }
eventjson = json.encode(event)
pushEvent(s.id, tr, eventjson, nil)
closePc(s.id)
end
If you remember, in hangupMedia() before we had a hangupPeer() method to notify the peer about the user’s PeerConnection going away: now that we know how to notify hangups, we can implement that method too:
function hangupPeer(peerId)
local p = sessions[peerId]
if p == nil then return end
local eventPeer = { videocall = "event", result = { event = "hangup", reason = "Remote hangup", username = p.username } }
local eventjson = json.encode(eventPeer)
pushEvent(p.id, nil, eventjson, nil)
closePc(p.id)
end
After a hangup, both users are free to start a new call, with each other again or with someone else.
Forgot anything? Oh yeah, the scheduler!
Before we can test our plugin, we’ll need to implement our coroutine scheduler. As explained in the text that describes the new plugin, in fact, scheduling is taken care of by the C plugin by invoking a Lua function called resumeScheduler(): it will be up to our plugin to implement this function, so that it resumes our coroutines.
We’ve seen in a previous paragraph how we created asynchronous tasks when handling incoming messages. A simple way to resume those tasks with the help of the scheduler is implementing the function like this:
function resumeScheduler()
print("Resuming coroutines")
for index,task in ipairs(tasks) do
coroutine.resume(task.co, task.id, task.tr, task.msg, task.jsep)
end
print("Coroutines resumed")
tasks = {}
end
We’re basically iterating on the list of pending tasks so far, and resuming them all. Smarter or more efficient mechanisms may be envisioned here, but for the sake of this tutorial this is simple and effective enough.
Testing our new plugin
Now that our script is ready, and that we configured the Lua plugin to use it, we can see if it’s working as expected. Since we tried to mimic the existing VideoCall plugin API, we can use the existing demo page to test it. The only thing we’ll need to change in the JavaScript code is the name of the plugin to attach to, i.e.:
[..]
janus.attach(
{
plugin: "janus.plugin.lua", // was "janus.plugin.videocall"
[..]
At this point, we just need to open the VideoCall demo page from two different browsers, register a couple of usernames, and have one try and call the other. Let’s see how this would look like with a few snapshots from the real thing. In particular, the following picture is where we’d be asked for a username, which would translate to a “register” request:
When registered, we’d be able to invite another registered user in a new call (“call” request), or wait for someone to call us (“incomingcall” event):
Assuming we called someone else, this is what they’d get from a UI perspective, e.g., to allow them to either answer (“accept”) or reject (“hangup”) the call:
Once a call has been accepted, the SDPs exchanged, and a PeerConnection become ready on both ends, the users (in this case, just me talking with myself in an awesome Double Fine™ shirt!) will be able to start exchanging audio, video and text:
Remember the Admin API and the querySession() function we implemented? If we checked the handles for the two users, we’d find the info we added there. The following snapshot shows a snippet of the handle for the user “Lorenzo” while in a call with “Ciccio”:
As you can see, as part of the Admin API info returned when querying the handle, we see the handle is attached to the janus.plugin.lua plugin, and that the plugin itself has returned some context-specific information, like the registration info, and some details on the peer we’re in call with.
What’s next?
This example should have given you a fairly good idea of how you can implement a Janus plugin logic in Lua, and how easy it should be to route media. Now that you know about the different callbacks and functions that are involved in the process, on the repo you’ll find two additional examples you may want to look at, namely echotest.lua and videoroom.lua, that mimic respectively the existing EchoTest and VideoRoom plugins. Of course, the flexibility of the Lua approach is more apparent when doing new stuff rather than mimicing existing APIs, and so I’d say the real next step would be: YOUR plugin!
I guess that’s all. Please notice that, while experimenting with the plugin, you might encounter weird behaviours or bugs: this is to be expected since, as anticipated, this is a brand new effort, and very much work in progress. Should you spot anything strange, please don’t hesitate to contribute to the discussion on the pull request!
That said, any question or feedback? Let us know, and good work!
