Status, readiness and reports framework in OpenSIPS 3.3

We already know how many things you can do with OpenSIPS and what is its true value in providing services. But the value of a software is not to be see from the perspective of how good it is in doing its job, but also from the perspective of how manageable it is under operational conditions. And what “manageable” means in our days got a different weight considering the heavy migration to virtual machines and container based environments (like Kubernetes). And there, “manageable” means to be able automatically spin up, terminate or monitor new instances, on demand, with zero human intervention.

So, OpenSIPS 3.3 evolves, together with the world around it. And to address this automatic “manageability”, the 3.3 is bringing the Status-Report framework.

The Status-Report Framework

The Status-Report framework aims to provide visibility inside the OpenSIPS’s internals, so external parties may understand what is the status (readiness) of various OpenSIPS components and may access reports (to understand the in-time evolution, or the history) of such components.

The fundamental entity in this framework is an “identifier” – an OpenSIPS part (a module, a functionality) that wants to share its status or reports with the outer world. For manageability reasons, such identifiers are bundled in groups.

Typical example here, is the drouting (Dynamic Routing) module holding and caching a lot of routing data, data organized in partitions (different data sets that can be individually used). The drouting module advertises the drouting group with one identifier per partitions (as each data set may its own status or history):

• the status of a partition means if the cached data is available – during OpenSIPS startup, data caching may take a while (after OpenSIPS started as application) before having the drouting able to route
• the reports on the partitions are used to provide access to the history of the events related to the partitions. Like when the partition was last time reloaded from DB, what was the outcomes of that reload, etc.

The Status

The concept of status is to tell if the identifier is ready for operations. What this readiness means, depends on the nature of the identifier. For example, on the drouting module, the readiness is about the available cached data; in the clusterer module, the readiness is about data being synced across the cluster; or in the cgrates module, the readiness may be about being connected to the cgrates daemon.

What this status is bring in addition here? Well, OpenSIPS as application may start really fast, but that does not mean it is fully operational (to handle the traffic). The data caching (from DB) or sync’ing (via cluster) may take additional time (after OpenSIPS fully started), so you need to be sure that such operations (caching or sync’ing) are completed before moving OpenSIPS under production traffic.

Shortly said, the Status tells you when OpenSIPS as a whole (application / data / external connections ) is ready to fully handle your service. The querying for status may be done per identifier or per group – if per group, the result will be a logical “AND” over all the identifiers in the group (all identifiers must be ready for the group to be also ready).

The status may be checked from script level via the sr_check_status() core function, so you can reject OPTIONS pinging if not fully ready.

For external querying, the sr_get_status() MI command may be used to fetching the status:

$ opensips-cli -x mi sr_get_status drouting default
{
    "Readiness": true,
    "Status": 1,
    "Details": "data available"
}
$ opensips-cli -x mi sr_get_status drouting all
{
    "Readiness": true,
    "Status": 1,
    "Details": "aggregated"
}
$ opensips-cli -x mi sr_get_status core
{
    "Readiness": true,
    "Status": 1,
    "Details": "running"
}

Also see the sr_list_status MI command for bulk access to the available status’s in OpenSIPS.

The Reports

The reports are basically logs related to the past activity of the identifier. Each identifier may decide when to generate a log, and what info to log, depending on its nature. The dialplan module may generate reports upon data reloading (with the outcome); the clusterer module may generate reports upon nodes joining in or out, about list connection to nodes, about changing the status of the sharing tags.

Only a limiter numbers (per identifier) are kept in history. The oldest ones are discarded as new ones are produced, the framework storing only the last N (usually 20) reports.

The reports are available only to external parties, via the sr_list_reports MI command.

$ opensips-cli -x mi sr_list_reports core
[
    {
        "Name": "main",
        "Reports": [
            {
                "Timestamp": 1647526996,
                "Date": "Thu Mar 17 16:23:16 2022",
                "Log": "initializing"
            },
            {
                "Timestamp": 1647526996,
                "Date": "Thu Mar 17 16:23:16 2022",
                "Log": "initialization completed, ready now"
            }
        ]
    }
]
$ opensips-cli -x mi sr_list_reports drouting
[
    {
        "Name": "Default",
        "Reports": [
            {
                "Timestamp": 1647526996,
                "Date": "Thu Mar 17 16:23:16 2022",
                "Log": "starting DB data loading"
            },
            {
                "Timestamp": 1647526996,
                "Date": "Thu Mar 17 16:23:16 2022",
                "Log": "DB data loading successfully completed"
            },
            {
                "Timestamp": 1647526996,
                "Date": "Thu Mar 17 16:23:16 2022",
                "Log": "2 gateways loaded (0 discarded), 2 carriers loaded (0 discarded), 1 rules loaded (0 discarded)"
            },
            {
                "Timestamp": 1647527212,
                "Date": "Thu Mar 17 16:26:52 2022",
                "Log": "starting DB data loading"
            },
            {
                "Timestamp": 1647527212,
                "Date": "Thu Mar 17 16:26:52 2022",
                "Log": "DB data loading successfully completed"
            },
            {
                "Timestamp": 1647527212,
                "Date": "Thu Mar 17 16:26:52 2022",
                "Log": "2 gateways loaded (0 discarded), 2 carriers loaded (0 discarded), 1 rules loaded (0 discarded)"
            }
        ]
    }
]

Status-Report module

Status-report groups/identifiers may be defined also at script level, by using the status_report module. So you can set status or do reporting from script, depending on the specifics of your routing logic.

loadmodule "status_report.so"
modparam("status_report", "script_sr_group", "security")
...
sr_add_report("security","IP $si detected as attacker");

Conclusions

The status aspect is essential to automatize the spinning and monitoring of OpenSIPS instances, while the reports are providing a focused history on what happened with certain parts of OpenSIPS. Both, ideal when comes to operating OpenSIPS.

Even if the framework is in place, there is still work ahead for extending this status-report support in more modules – see the module README to check which ones are already doing it – the drouting, sql_cacher, pike, dialplan and of course the core 🙂